Auto-Encoders
By Eric Antoine ScuccimarraIn one of my classes last semester we had to make a variational auto-encoder for the MNIST dataset. MNIST is a pretty simple and small dataset so that wasn't very difficult. Looking for something more challenging I decided to try to make a face autoencoder.
The first challenge was finding a suitable dataset. There are several public datasets, but they are inconsistent in terms of image size, shape and other important details. I ended up using several datasets - the celebA was the easiest one to use without having to do much work on it. But it is relatively small, consisting of only 200,000 images. I ended up using the ETHZ dataset from Wikipedia and IMDB, which are quite large but the images are all different shapes and sizes. So I had to do some pre-processing of the data to remove unusable images. I removed any images small than the input size I was using of 162x190, and I also removed any images that were wider than they were higher or bigger than 500x500. This dataset also contains some images which have been stretched out at the edges to bizarre proportions. I removed these by deleting any images where the 10th row or column was identical to the first row or column. Finally I resized the large images down to a more reasonable size. This resulted in a dataset of about 390,000 faces, all of which were roughly the right size and shape.
I decided to train my autoencoder as a normal autoencoder rather than a variational one, mostly due to the extra overhead required for the variational layers. I used a latent space of size 4096, and after training for 12 hours a day for a few weeks on Google CoLab the results were surprisingly accurate. Once the model seemed to start overfitting the training data I stopped training it so I could play around with it.
I wanted to try to do interpolation between faces, which was when I realized what the advantage of making the auto-encoder variational was. When I tried to interpolate between faces, because the latent space was not continuous, rather than working as one would expect it was more like adding the faces together. Training the autoencoders as variational forces the latent space to be continuous which makes interpolation possible, so I am currently trying to retrain the model as variational.
Since the non-variational autoencoder had started to overfit the training data I wanted to try to find other ways to improve the quality, so I added an discriminative network which I am also currently training as a GAN, using the autoencoder as the generator. I will update with results of that when I have results worth reporting.
The notebooks used are available on GitHub, and the datasets I used are on Google Cloud Storage, although due to their size and the cost of downloading them they are not publicly available.
Labels: python, pytorch, autoencoders