[논문 소개] Deep Unsupervised Clustering with Gaussian Mixture Variational AutoEncoders

https://arxiv.org/pdf/1611.02648.pdf

간단히 설명하자면, 기존의 auto-encoder는 다차원 데이터를 적은 차원의 데이터로 압축시키는 것을 의미하고, variatioinal auto-encoder는 그 압축시킨 데이터를 일종의 확률로써 생각하여 정규분포로 나타낸다.

이 논문의 핵시 contribution은 그 압축시킨 distribution을 mixture model로 fitting하여 이를 clustring에 이용한 것이다.

 

해당 컨셉을 나 혼자 생각해서 "아 이거다!" 했는데, 찾아보니 역시 존재하는 아이디어였다ㅠ

 

 

 

Abstract

We study a variant of the variational autoencoder model (VAE) with a Gaussian mixture as a prior distribution, with the goal of performing unsupervised clustering through deep generative models. We observe that the known problem of over-regularisation that has been shown to arise in regular VAEs also manifests itself in our model and leads to cluster degeneracy. We show that a heuristic called minimum information constraint that has been shown to mitigate this effect in VAEs can also be applied to improve unsupervised clustering performance with our model. Furthermore we analyse the effect of this heuristic and provide an intuition of the various processes with the help of visualizations. Finally, we demonstrate the performance of our model on synthetic data, MNIST and SVHN, showing that the obtained clusters are distinct, interpretable and result in achieving competitive performance on unsupervised clustering to the state-of-the-art results.

 

 

 

Experiments

 

 

 

참고

http://ruishu.io/2016/12/25/gmvae/

Gaussian Mixture VAE: Lessons in Variational Inference, Generative Models, and Deep Nets - Rui Shu

 

https://stats.stackexchange.com/questions/350921/variational-autoencoder-with-gaussian-mixture-model

Variational autoencoder with Gaussian mixture model