Through Fields
Through Fields is a meditative short film I created by combining 3D animation, GauGAN, and a lot of hard work. It is a flow through a field as interpreted by the creative AI landscape generator. I also did sound design for the video, which I think adds a whole new dimension to the experience.
Depending on what information I give it, GauGAN can produce surreal or photo-real images. It is up to me to control it with the segmentation maps I feed it (below).
Each color in the map corresponds to a different feature that GauGAN's neural network has been trained to produce -- mountain, cloud, tree, for instance. GauGAN can produce realistic images if you give it a map with naturalistic relationships between features, sky at the top, ground at the bottom, for example.
Unexpected results appear when you give it an unusual segmentation map -- sky at the bottom, or even a completely upside down composition:
I started experimenting with turning these images -- the intended output of GauGAN -- into videos by doing simple keyframe animation of the segmentation maps in After Effects, such as in this test "Cloud Mountain Tree."
Through a process of trial and error I found that lower frame rate animation works well for these videos; there's less image noise distracting from the overall picture. I also found that given the right input, GauGAN can create fairly stable results. If the segmentation map is stable the output is also fairly stable. This gave me the idea to create maps using 3D animation -- in that case I could ensure an accurate and steady movement of objects over time. I asked myself whether GauGAN could produce immersive worlds, rather than just images -- and images that feel fairly flat at that.
Here's what the segmentation map looks like in 3D:
The results exceeded my expectations. Here are a few of my favorite images from the video:
