DeOldify: Visualizing Image Colorization with Hyper-Realistic Artwork

What is DeOldify?

Image colorization may have been reserved for those with artistic talent in the past, but now anyone with basic programming knowledge can get in on the action. Thanks to projects like DeOldify, we can colorize black and white images and video with minimal setup! The project even includes pre-trained weights, so you don’t need to spend time training on your machine.

As an example of what we can do with DeOldify, here’s a colorization I made of the last known Tasmanian Tiger, which went extinct in 1936. This appears to be the first time we can see what the Tasmanian Tiger might have looked like in color:

You can get started coloring your own images by visiting the DeOldify repository and following the instructions for setup. I will also leave a short tutorial on how to set everything up at the bottom of this article.

Visualizing the colorization process

While DeOldify is primarily meant to colorize real images, we can colorize artwork as well. In my experience, the quality of the colorization relies on how realistic the object and perspective in your artwork appear to be. I assume this is a result of the dataset used to produce these pre-trained weights. It might be cool to see the results we could get if trained on only artwork.

Because of its ability to colorize art, we can actually visualize how DeOldify recognizes and colors certain attributes in an image. This can be achieved by giving DeOldify a time lapse of a hyper-realistic face being drawn. As the face is being created, we can see where the colorization is applied at different stages. Everything from a general face shape to basic eyes, a nose, and so on.

So what does this actually look like? I made an example using a time lapse of Cara Delevingne being drawn by U.N.I.C.O on YouTube:

As mentioned above, this video gives us a good look into how colorization of the image is being accomplished. While this example features a face, I think it would also be neat to see how a landscape might look as well.

How to setup DeOldify

For those looking to try this themselves, here is a short tutorial on how to get things setup.

First, cd into the directory you would like to place DeOldify, then clone it:

Now cd into the newly created DeOldify directory and create a new “models” directory for your pre-trained weights:

You can now download the pre-trained weights through the command line:

If you don’t have wget, you can find links to manually download these pre-trained weights in the DeOldify README.

We also need to install dependencies:

Once the dependencies are installed, open the project in a Jupyter Notebook:

At this point, you should be able to open the ImageColorizer.ipynb file and continue from there. I hope this helped! Happy colorizing!