Topic-GAN01: BasicUnderstanding
Generative Adversarial Networks (GAN) is one of the most promising recent developments in Deep Learning. GAN, introduced by Ian Goodfellow in 2014, attacks the problem of unsupervised learning by training two deep network .In this blog i am trying to give a brief understanding about working of GAN and the idea behind GAN.
What is Generative Model?
If any system is able to generate new data which can be believed as real data then its use case is numerous . A generative model is a model which generates new data /image samples from the real distribution set which was never seen before yet fitting into the target distribution .
The intent of these generative models is to generate new data .As show in the image above , if we have a trained a generative model which can generate new data samples from the probability distribution of available real data a new data samples which are real in appearance then i have achieved the balance of data , which is area of interest in the industry.
Taxonomy of GAN
The below image gives the taxonomy or the development of GAN , since probability distribution of data is considered while generating new data/ images sample having the maximum log likelihood of the data is equivalent to minimizing the data distribution between generated data samples and real data samples, which is KL divergence (Kullback-Leibler Divergence) a good measure to find the distance between the distribution of data.
The generative models can be categorized into two groups based on the distributions:
- Explicit Distribution: Varriational Auto Encoder (VAE) is an example of explicit distribution generation method where the parameters are explicitly defined and estimated by the assumption of the prior data and likelihood distribution samples , by using varriational approximation between these components they estimate the performance.
- Implicit Distribution : GAN is an example of implicit distribution generative method where the models train by indirectly sampling data from their distribution since they don’t need any explicit definition of the model distribution.
GAN understanding:
GAN’s have two components Generator which is referred to as G-network and Discriminator which is referred to as D-network. The below figure shows the block level overview of GAN. The G-network, generates new data instances, while the other, the D-network, evaluates them for authenticity; i.e. the discriminator decides whether each instance of data it reviews belongs to the actual training dataset or not.
The main philosophy is both G and D starts from zero/scratch and learn of it self .They are always against each other but learning together by growing together and eventually learns to create object similar to the real world objects. Eg: We can imagine a GAN as the combination of a counterfeiter and a cop in a game , where the counterfeiter is learning to create a perfect false notes, and the cop is learning to detect them. Both the process are dynamic ,the cop is in training so that he can identify better false and counterfeiter is also training to prepare better false note to fool the cops,and each side comes to learn the other’s methods in a constant escalation.
GAN Math:
In GAN , the G network and D network have the goal of finding Nash equilibrium as defined by game theory by solving an minimax equation.
The equation is show above , in which both G network and D network try to find equilibrium by jointly optimizing their parameters. The equation can be considered as two halves.
1.From D network side it wants to maximize its output when new data comes in by differentiating it into real and fake which is on the right side of the equation
2.As counter to it , G network side is trying to minimize by trying to trick D network to maximize the fake data
By actions of both G and D network maximizing and minimizing the term minmax has been obtained.
In the next series we shall understand more in-depth working of GAN and its application into different domains. Since I might not be an expert on the topic, if you find any mistakes in the article, or have any suggestions for improvement, please mention in comments.