LSGAN - SIMPle(Simple Idea Meaningful Performance Level up)
2 likes596 views
LSGAN은 기존의 GAN loss가 아닌 MSE loss를 사용하여, 더욱 realistic한 데이터를 생성함. LSGAN 논문 리뷰 및 PyTorch 기반의 구현. [참고] Mao, Xudong, et al. "Least squares generative adversarial networks." Proceedings of the IEEE International Conference on Computer Vision. 2017.
]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx
DG zdata
Smart D
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
Real case
Fake case
1
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
0
should be 0
should be 0
1
Log(x)
cf.
Stupid D
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
Real case
Fake case
0
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
1
should be negative infinity
should be negative infinity
D perspective,
it should be maximum.](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-9-320.jpg)
]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx
DG zdata
Generator
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
1
should be negative infinity
1
Log(x)
cf.
G perspective,
it should be minimum.
Smart G
Stupid G )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
0
should be 0](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-10-320.jpg)
![Introduction
2019-04-09
15
• InfoGAN - Latent Code
GNoise
Latent code
0.001
0.008
1.000
0.007
…
0.005
? : 실제 latent code의 구조는 복잡하여
해석이 어려움(entangled).
Let's make the latent code simple.
The proper generation is difficult.
[0.001, 0.008, …, 0.005] c
Latent code
0.001
0.008
1.000
0.007
…
0.005
0
0
0
0
…
1
Z C
Z C : Condition
How about adding latent code?
해석이 가능한 Condition을 제공.
Idea](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-15-320.jpg)
![Introduction
2019-04-09
16
• InfoGAN - Latent Code
G
Latent code
Z C
“뭐야? 그러면 C를 Z 옆에 바로
붙이면 되는 거야?”
[0.001, 0.008, …, 005 | 0, 0, … 1]
z c
[0.001, 0.008, …, 005 | 1, 0, … 0]
z c
[0.001, 0.008, …, 005 ]
z
[0.001, 0.008, …, 005 ]
z
Ignore the additional latent code c
Cost function을 수정하여 c의 영향을 만듦.),(maxmin GDV
DG
(Mutual Information)](https://image.slidesharecdn.com/lsgan-190521075423/85/LSGAN-SIMPle-Simple-Idea-Meaningful-Performance-Level-up-16-320.jpg)
![Objects as points (CenterNet) review [CDM]](https://cdn.slidesharecdn.com/ss_thumbnails/objectsaspointscenternetreviewcdm-200327113331-thumbnail.jpg?width=560&fit=bounds)
![[부스트캠프 Tech Talk] 신원지_Wandb Visualization](https://cdn.slidesharecdn.com/ss_thumbnails/boostcampaitechtechtalkshinwonji-211210113635-thumbnail.jpg?width=560&fit=bounds)
![[DL輪読会] off-policyなメタ強化学習](https://cdn.slidesharecdn.com/ss_thumbnails/20190405journalclub-190415082841-thumbnail.jpg?width=560&fit=bounds)
![[PR12] intro. to gans jaejun yoo](https://cdn.slidesharecdn.com/ss_thumbnails/pr12intro-170416162251-thumbnail.jpg?width=560&fit=bounds)
![論文読み会@AIST (Deep Virtual Stereo Odometry [ECCV2018])](https://cdn.slidesharecdn.com/ss_thumbnails/dvsoslideshare-181104042256-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to LSGAN - SIMPle(Simple Idea Meaningful Performance Level up) (20)
![[系列活動] Data exploration with modern R](https://cdn.slidesharecdn.com/ss_thumbnails/dataexplorationwithmodernr1221-161219044516-thumbnail.jpg?width=560&fit=bounds)
![[AWS Dev Day] 인공지능 / 기계 학습 | 개발자를 위한 수백만 사용자 대상 기계 학습 서비스 확장 하기 - 윤석찬 AWS 수석테...](https://cdn.slidesharecdn.com/ss_thumbnails/scalableawsmlservicefromzerotomillions-190927015651-thumbnail.jpg?width=560&fit=bounds)
![[CB20] DeClang: Anti-hacking compiler by Mengyuan Wan](https://cdn.slidesharecdn.com/ss_thumbnails/mengyuanwancb20-210112133029-thumbnail.jpg?width=560&fit=bounds)
![VitaFlow | Mageswaran Dhandapani [Pramati]](https://cdn.slidesharecdn.com/ss_thumbnails/vitaflow-190430092022-thumbnail.jpg?width=560&fit=bounds)
Ad
More from Hansol Kang (20)
Ad
Recently uploaded (20)
LSGAN - SIMPle(Simple Idea Meaningful Performance Level up)
- 1. SIMPLe : Simple Idea Meaningful Performance Level up* ISL Lab Seminar Hansol Kang * Mao, Xudong, et al. "Least squares generative adversarial networks." Proceedings of the IEEE International Conference on Computer Vision. 2017.
- 2. Contents 2019-04-09 2 Introduction Research Trend Review LSGAN Concept Objective Function Global Optimality Parameter Selection Results Experiment Source Code Celeb-A(DCGAN, LSGAN) Summary
- 3. I. Introduction Introduction Research Trend, Review(Concept, Vanilla GAN, DCGAN, InfoGAN)
- 5. Introduction • Research Trend 2019-04-09 5 • 70,000 high-quality PNG images at 1024×1024 resolution • Considerable variation in terms of age, ethnicity and image background • Good coverage of accessories such as eyeglasses, sunglasses, hats, etc. Flickr-Faces-HQ (FFHQ)
- 7. Introduction • Concept of GAN 2019-04-09 7 VsD GF1 F1 F1 F1 FakeR1
- 8. Introduction • Concept of GAN 2019-04-09 8 VsD G Fake?R1 @ F1
- 9. Introduction • Vanilla GAN : Adversarial Nets 2019-04-09 9 )))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx DG zdata Smart D )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata Real case Fake case 1 )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 0 should be 0 should be 0 1 Log(x) cf. Stupid D )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata Real case Fake case 0 )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 1 should be negative infinity should be negative infinity D perspective, it should be maximum.
- 10. Introduction • Vanilla GAN : Adversarial Nets 2019-04-09 10 )))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx DG zdata Generator )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 1 should be negative infinity 1 Log(x) cf. G perspective, it should be minimum. Smart G Stupid G )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata 0 should be 0
- 11. Introduction 2019-04-09 11 • Vanilla GAN : Mathematical Proof 1) Global Optimality of datag pp 2) Convergence of Algorithm D GVs x )(xpdata “Generative Adversarial Networks” Goal Method
- 12. Introduction 2019-04-09 12 • DCGAN : Network D G “쟤들 뭐하냐?” “CNN이 MLP보다 훨씬 낫지롱” D “우리가 짱이야” G Vanilla GAN DCGAN “VAE 죽어요 ㅠㅠ”
- 13. Introduction 2019-04-09 13 • DCGAN : Latent Space 0 1 0.1 0.15 0.18 0.143 0.5 0.45 0.47 0.473 0.9 0.95 0.96 0.937 0.607± 고차원(Image)에서 의미 x 저차원(Latent code)에서 의미 o
- 14. Introduction 2019-04-09 14 • InfoGAN - Network D “우리가 짱이야” G D DCGAN InfoGAN Gz“나도 신경 써줘…” zc z “우리는 Z(Latent code)를 더 세분화 해서 조작이 가능해!”
- 15. Introduction 2019-04-09 15 • InfoGAN - Latent Code GNoise Latent code 0.001 0.008 1.000 0.007 … 0.005 ? : 실제 latent code의 구조는 복잡하여 해석이 어려움(entangled). Let's make the latent code simple. The proper generation is difficult. [0.001, 0.008, …, 0.005] c Latent code 0.001 0.008 1.000 0.007 … 0.005 0 0 0 0 … 1 Z C Z C : Condition How about adding latent code? 해석이 가능한 Condition을 제공. Idea
- 16. Introduction 2019-04-09 16 • InfoGAN - Latent Code G Latent code Z C “뭐야? 그러면 C를 Z 옆에 바로 붙이면 되는 거야?” [0.001, 0.008, …, 005 | 0, 0, … 1] z c [0.001, 0.008, …, 005 | 1, 0, … 0] z c [0.001, 0.008, …, 005 ] z [0.001, 0.008, …, 005 ] z Ignore the additional latent code c Cost function을 수정하여 c의 영향을 만듦.),(maxmin GDV DG (Mutual Information)
- 17. Introduction 2019-04-09 17 • InfoGAN - Latent Code : Generator와 c 사이의 연관성을 cost로 정의 ),(;),(),(maxmin czGcIGDVGDVI DG Maximize Hard to maximize directly as it requires access to the posterior )|( xcP )(||)|()(|log),,( )|( zpxzqKLzgxExL xzq ),,(min xL Reconstruction Error Regularization VAE Seminar (18.07.23)
- 18. Introduction 2019-04-09 18 • InfoGAN - Results
- 19. Introduction 2019-04-09 19 GAN Concept Performance Manipulability Stability Concept • GAN Performance • DCGAN, LSGAN Stability • DCGAN, LSGAN Manipulability • DCGAN, InfoGAN
- 20. II. LSGAN LSGAN Concept, Objective Function, Global Optimality, Parameter Selection, Results
- 21. LSGAN 2019-04-09 21 • Concept D학습이 잘되었다 (=50:50) 학습이 잘되었다 (=Good representation) G Real Fake F5 R1 R2 R5 F1 F5 R1 F1 여전히 너무나도 가짜 같은 데이터가 존재 If 60:40 then stupid G If 40:60 then stupid D
- 22. LSGAN 2019-04-09 22 • Concept G Real Fake F5 R1 R2 R5 F1 F5 R1 F1 경계 근처 ≈ Real? Fake? Fake -> 경계 근처 F5 F1 훨씬 헷갈리는(Real에 가까운) 데이터 생성
- 24. LSGAN • Objective function 2019-04-09 24 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata Smart D 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata Real case Fake case 1 0 (b=1), should be 0 (a=0), should be 0 Stupid D D perspective, it should be minimum. 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )( azGDEbxDE zpxpx zxdata Real case Fake case 0 1 (b=1), should be 1 (a=0), should be 1 a : fake label. b : real label. c : G wants to make D believe for fake data 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G
- 25. LSGAN • Objective function 2019-04-09 25 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata Generator 2 ~ ))(( 2 1 )( czGDE zzpz Smart G Stupid G 1 (c=1), should be 0 (c=1), should be 1 2 ~ ))(( 2 1 )( czGDE zzpz 0 G perspective, it should be minimum. 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G a : fake label. b : real label. c : G wants to make D believe for fake data
- 26. LSGAN • Objective function 2019-04-09 26 a : fake label. b : real label. c : G wants to make D believe for fake data 조금 더 직관적으로 생각해보면, D = Classifier 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G Prediction - Label
- 27. LSGAN 2019-04-09 27 • Global Optimality – Vanilla GAN LSGAN : 𝝌 𝑷𝒆𝒂𝒓𝒔𝒐𝒏 𝟐 를 통한 증명 Vanilla GAN : JSD를 통한 증명
- 28. LSGAN 2019-04-09 28 • Global Optimality – LSGAN 2* ~ 2* ~ )()()(2 cxDEcxDEGC gd pxpx 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G )()( )()( )(* xpxp xapxbp xD gdata gdata 2 ~ 2 ~ )()( )()( )()( )()( c xpxp xapxbp Ec xpxp xapxbp E gdata gdata px gdata gdata px gd 2 ~ )( 2 1 )( cxDE xdatapx This term does not contain parameters of G
- 29. LSGAN 2019-04-09 29 • Global Optimality – LSGAN 22 )()( )()()()( )( )()( )()()()( )( xpxp xpcaxpcb xpdx xpxp xpcaxpcb xp gdata gdata x g gdata gdata x data 2 ~ 2 ~ )()( )()( )()( )()( c xpxp xapxbp Ec xpxp xapxbp E gdata gdata px gdata gdata px gd gdata gdata gdata gdata gdata gdata pp xpcapcb xpxp xcpxcp xpxp xapxbp )()()( )()( )()( )()( )()( x gdata gdata dx xpxp xpcaxpcb )()( )()()()( 2 x gdata gdata gdata dx xpxp xpcaxpcb xpxp 2 )()( )()()()( )()( x gdata ggdata dx xpxp xpabxpxpcb )()( )()()()()( 2
- 30. LSGAN 2019-04-09 30 • Global Optimality – LSGAN x gdata ggdata dx xpxp xpabxpxpcb )()( )()()()()( 2 dx xpxp xpxpxp GC x gdata gdatag )()( ))()(()(2 )(2 2 )2||(2 ggdataPearson ppp datag pp )( )()( 2 2 xp xpxq Pearson 21 abandcbIf we set If minimum
- 31. LSGAN 2019-04-09 31 • Parameters Selection 21 abandcb 01,1 candba 2 )(~ 2 ~ ))(( 2 1 )( 2 1 )(min )( azGDEbxDEDV zpxpxLSGAN D zxdata 2 ~ ))(( 2 1 )(min )( czGDEGV zzpzLSGAN G 2 )(~ 2 ~ 1))(( 2 1 1)( 2 1 )(min )( zGDExDEDV zpxpxLSGAN D zxdata i) 2 ~ ))(( 2 1 )(min )( zGDEGV zzpzLSGAN G bcandba 1,0 2 )(~ 2 ~ 1))(( 2 1 1)( 2 1 )(min )( zGDExDEDV zpxpxLSGAN D zxdata ii) 2 ~ ))(( 2 1 )(min )( zGDEGV zzpzLSGAN G ->조건을 따르지 않는 경우 성능은 비슷하며, 큰 차이가 없음!
- 34. LSGAN 2019-04-09 34 • Results Example of mode collapse
- 36. III. Experiment Experiment Source Code, Celeb-A, Korean Idol
- 39. Experiment • Celeb-A(without BN) 2019-04-09 39 DCGAN LSGAN ep=1 ep=2 ep=3 ep=5
- 40. Experiment 2019-04-09 40 • Korean Idol(DCGAN) ep 1 ep 6 ep 51ep 21 ep 201ep 101
- 41. Experiment 2019-04-09 41 • Korean Idol(LSGAN) ep 1 ep 6 ep 51ep 21 ep 201ep 101
- 42. Experiment 2019-04-09 42 • Gueess#1 Optimizer problem?
- 43. Experiment 2019-04-09 43 • Gueess#2 Domain problem?
- 44. Experiment • Celeb-A 2019-04-09 44 https://www.slideshare.net/NaverEngineering/1-gangenerative-adversarial-network “저자들이 source code를 공개했으면 한다.” “같은 구조라면, LSGAN이 훨씬 잘 동작한다.”
- 45. IV. Summary Summary Summary, Future Work
- 46. Summary 2019-04-09 46 • 기존의 GAN보다 Real에 가까운 데이터를 생성하고, 안정성도 확보함. • Pearson Chi square divergence으로 global optimality를 증명함. (기존 GAN은 JSD로 증명) • 클래스가 많은 데이터에 대해서도 정상적으로 데이터를 생성함. • 기존의 코드에서 단순히 loss만을 변경하기에 손쉽게 적용이 가능함.
- 47. GAN Research Vanilla GAN DCGAN InfoGAN LSGAN BEGAN Cycle GAN Style GAN SRGAN Tools Document Programming PyTorch Python executable & UI I Know What You Did Last Faculty C++ Coding Standard Mathematical theory LSM applications Other Research Level Processor Ice Propagation Future work 2019-04-09 47
- 48. &