Ever wondered how it feels like getting old photos back to new all bright and shiny? Well you’ll say that it is something very tedious and time taking; old photos cannot be restored to new completely.
However it has been made possible by the combined efforts of Ziyu Wan (City University of Hong Kong), Bo Zhang (Microsoft Research Asia), Dongdong Chen (Microsoft Cloud+AI) and Pan Zhang (University of Science and Technology of China).
In this project, it is proposed that by following a deep learning approach one can restore old, damaged photos which have been degraded to such an extent that even after using different techniques can’t be obtained, but this project confirms that such degraded photos can be restored by the technique developed by them. A novel triplet domain translation network by leveraging real photos along with massive synthetic image pairs has been proposed. Two variational-auto-encoders (VAEs) are trained to transform and clean photos into two latent spaces which is learned with the help of synthetic paired data. This closes the domain gap in the compact latent space in real photos.
For multiple degradations in a single old photo, a global branch with partial non local block is used to target structured defects. Two branches are fused in the latent space which then leads to improvised capabilities to restore old photos from multiple defects. This method is considered even better than state-of-the-art methods when compared on the basis of visual quality for old photos restoration.
Two VAEs are trained
VAE 1 is for images in real and synthetic images.
By training an adversarial discriminator, their domain gap is closed.
VAE 2 is trained for cleaning the images which are further transformed to compact latent space.
With the help of mapping, corrupted images are restored to clean images in that latent space with partial non-local blocks.