Generative Adversarial Network

Generative Adversarial Network
Presented by Mohammad Khalooei
PhD student of Amirkabir University of Technology (Tehran Polytecnic)
Under supervision of Prof. Mohammad Mehdi Homayounpour & Dr. Maryam Amirmazlaghani
Laboratory of Intelligence and Multimedia Processing (LIMP)
http://ceit.aut.ac.ir/~khalooei
khalooei [at] aut.ac.ir
Generative Adversarial Network Mohammad Khalooei | khalooei@aut.ac.ir 1

Presented by Mohammad Khalooei
PhD student of Amirkabir University of Technology (Tehran Polytecnic)
Under supervision of Prof. Mohammad Mehdi Homayounpour & Dr. Maryam Amirmazlaghani
Laboratory of Intelligence and Multimedia Processing (LIMP)
http://ceit.aut.ac.ir/~khalooei
khalooei [at] aut.ac.ir

GAN Zoo!
https://github.com/hindupuravinash/the-gan-zoo

Best paper statistic from CVPR 2018
Are GANs the new Deep?
http://jponttuset.cat/are-gans-the-new-deep/

Best paper statistic from CVPR 2018
The most important one, in my opinion, is adversarial
training (also called GAN for Generative Adversarial
Networks).
https://medium.com/syncedreview/cvpr-2018-kicks-off-best-papers-announced-d3361bcc6984
Yann LeCun
More than eight percent of CVPR 2018’s
accepted papers include “GANs”
in their titles,
doubling the frequency at CVPR 2017.
Google AI Research Scientist Jordi Pont-Tuset suggested
in his blog that Generative Adversarial Networks (GANs)
might catch up with deep learning someday. Jordi Pont-Tuset

A brief Applications of GAN :: overview on CVPR18 paper
• Perceptual Fidelity
• Data Augmentation
• Adversarial Attack
• Domain Adaptation
• Improved GAN
• Metric Learning
Categories:

GAN !
https://goo.gl/oCdBRj
https://goo.gl/ibYzBr

Supervised learning
• Find deterministic function f
Introduction
x : data
y : label
f : y = f(x)

Supervised learning
• Challenges:
- Image is high dimensional data
Introduction
x : data
y : label
f : y = f(x)
3×224×224
224 px
224 px
R
G
B
= 150528

Supervised learning
• Challenges:
- Many variations
Viewpoint, illumination, deformation,
occlusion, background clutter,
intraclass variation
Introduction
x : data
y : label
f : y = f(x)

Supervised learning
• Challenges:
- Many variations
Introduction
x : data
y : label
f : y = f(x)
All pixels change when the camera moves !
http://cs231n.stanford.edu/slides/2018/cs231n_2018_lecture02.pdf

Supervised learning
• Challenges:
- Many variations
Introduction
x : data
y : label
f : y = f(x)

Supervised learning
• Challenges:
- Many variations
Introduction
x : data
y : label
f : y = f(x)
• Solution:
- Feature Vector
3×224×224
224 px
224 px
R
G
B
= 150528 2048
Feature extractor

Supervised learning
• Challenges:
- Many variations
Introduction
x : data
y : label
f : y = f(x)
• Solution:
- Feature Vector :: Synonyms
Latent Vector
Hidden Vector
Unobservable Vector
Feature
Representation

Supervised learning
Introduction
f : y = f(x)
Good Bad

Supervised learning
Introduction
f : y = f(x)
Good features:
Less redundancy
Similar features for similar data
High fidelity
Good Bad

Supervised learning
• More flexible solution
Introduction
x : data
y : label
f : y = f(x)
Cat

Supervised learning
Introduction
x : data
y : label
f : y = f(x)
0.87 Cat
0.22 Dog
0.01 Cake

UnSupervised learning
Introduction
x : data
z : latent
f : z = f(x)
Similaritymeasure

• More challenging than supervised learning !
• No label or curriculum → self learning
• Some NN solutions :
• Boltzmann machine
• Auto-encoder or Variational Inference
• Generative Adversarial Network
Introduction
x : data
z : latent
f : z = f(x)

Generative model
• Find generation function g
Introduction
x : data
z : latent
g : x = g(z)

Generative model
Introduction
x : data
z : latent
g : x = g(z)
x : data
z : latent
f : z = f(x)
VS.

Generative model
Introduction
x : data
z : latent
g : x = g(z)
x : data
z : latent
f : z = f(x)
VS.
P(z|x)
P(x|z)

Generative model
Introduction
x : data
z : latent
g : x = g(z)
x : data
z : latent
f : z = f(x)
VS.
P(z|x)
P(x|z)
Encod
er
Decoder
(Generator)

Generative Modeling
Sample GeneratorTraining Data
Training Data Density function
Sample Generation
Density Estimation

AutoEncoder
http://curiousily.com/data-science/2017/02/02/what-to-do-when-data-is-missing-part-2.html

AutoEncoder
• Family of AE:
• Stack AutoEncoder (SAE) :: Use data itself as label / z = f(x), x=g(z) → x = g(f(x))
• Denoising autoencoder (DAE) :: Add random noise to input data
• Variational autoencoder (VAE) :: Generative Model + Stacked Autoencoder
• …

AutoEncoder
• Family of AE:
• …
Sample Code:
https://github.com/buriburisuri/sugartensor/blob/master/sugart
ensor/example/mnist_sae.py

AutoEncoder
• Family of AE:
• …
Sample Code:
https://github.com/buriburisuri/sugartensor/blob/master/sugart
ensor/example/mnist_dae.py

Mohammad Khalooei | khalooei@aut.ac.ir 35
AutoEncoder
• Family of AE:
• … • Based on Variational approximation
• Kingma et al, “Auto-Encoding Variational Bayes”, 2013
Train
ing
phases

AutoEncoder
• Family of AE:
Generating phases

AutoEncoder
• Family of AE:
• Reparameterization trick
• Enable back propagation
• Reduce variances of gradients

AutoEncoder
• Family of AE:

AutoEncoder
• Family of AE:
• …
(Namjukim – 2017) (Namjukim – 2017)

Review …

Review:: Generative Model
https://www.slideshare.net/BrianKim244/dcgan-77452250

Distribution of the actual images

Contents
o Machine learning
o Supervised learning
o Unsupervised learning
o Generative vs. Discriminative models
o Generative Adversarial Network
o Introduction
o Definition
o Challenges
o Applications
o Tricks for training

Adversarial Nets :: Introduction
Ian Goodfellow et al, “Generative
Adversarial Networks”, 2014

Adversarial Nets :: Definition
https://techcrunch.com/2017/06/20/gangogh

Tries to
generates
more real-
likefake bills
Tries tocatchfake
bills
Penalty if failure

Tries to
generates
more real-
likefake bills

Tries tocatchfake
bills
Penalty if failure

https://github.com/dmonn/GAN-face-generator

Adversarial Nets :: Framework
(Goodfellow et al., 2014)

AutoEncoder :

• Different distance metrics :
KL-divergence
JS-divergence
Earth-mover distance (Wasserstein distance)
Total variation distance
Hellinger distance
Mahalanobis distance
Bhattacharyya distance
Energy distance
…

• Pure GAN’s measure for difference ::

Namjukim - 2017

GAN ideas (review intuitive papers)
https://carpedm20.github.io/faces/ https://github.com/carpedm20/DCGAN-tensorflow
DCGAN Deep convolutional generative adversarial network (DCGAN)

Vector space arithmetic

Super-Resolution

Super-Resolution
https://www.youtube.com/watch?v=9c4z6YsBGQ0

Conditional Generative Adversarial Network

Invertible Conditional GANs for image editing

Image to Image translation with conditional generative networks
https://phillipi.github.io/pix2pix/

Cycle GAN
F(G(X)) ≈ X
G: X → Y
F: Y → X

Unsupervised cross-domain image generation

Denoising GAN

Review:: Super resolution (SRGAN)
https://github.com/zsdonghao/SRGAN

Text to Image

MoCoGAN: Decomposing Motion and Content for Video Generation
https://github.com/sergeytulyakov/mocogan

ALOCC :: Adversarially Learned One-Class Classifier for Novelty Detection
https://github.com/khalooei/ALOCC-CVPR2018

• Converging
• Mode collapse
• Counting
…
GAN challenges

GAN’s Applications

3.5 Years of Progress on Faces
(Brundage et al, 2018) (Goodfellow 2018)

(Brundage et al, 2018) (Goodfellow 2018)
< 2 Years of Progress on Faces

(Zhang et al., 2018) (Goodfellow 2018)
Self-Attention GAN

(Goodfellow 2018)
Some intuitive:
Depth and Convolution
Class-conditional generation
Spectral Normalization
Hinge loss
Two-timescale update rule
Self-attention

(Goodfellow 2018)
Some intuitive:
Hinge loss
Self-attention
No Convolution Needed to Solve Simple Tasks
Original GAN, 2014

(Goodfellow 2018)
Some intuitive:
Hinge loss
Self-attention
Class-Conditional GANs
(Mirza and Osindero, 2014)

(Goodfellow 2018)
Some intuitive:
Hinge loss
Self-attention
(Odena et al, 2016)
AC-GAN: Specialist Generators

(Goodfellow 2018)
Some intuitive:
Hinge loss
Self-attention
(Miyato et al, 2017)
SN-GAN: Shared Generator

(Goodfellow 2018)
Some intuitive:
Hinge loss
Self-attention
(Miyato et al 2017, Lim and Ye 2017, Tran et al 2017)
Hinge Loss

(Goodfellow 2018)
Some intuitive:
Hinge loss
Self-attention

Thank you!
Mohammad Khalooei
Mkhalooei [at] gmail.com
Khalooei [at] aut.ac.ir
https://ceit.aut.ac.ir/~khalooei

Generative Adversarial Network

Recommended

Recommended

More Related Content

Similar to Generative Adversarial Network

Similar to Generative Adversarial Network (6)

Recently uploaded

Recently uploaded (20)

Generative Adversarial Network