Stable Diffusion Cat Animator (Rotoscoping)

This project uses a base kitten video to drive animations of a cartoon cat, an owl, and a racoon.

This video is produced using Stable Diffusion with the EbSynth tool. The top left is the original video of the kitten, the top right is an animation of a cartoon cat. At the bottom left is an owl that is drawn based on a few selected frames from the original video. At the bottom right is a raccoon that is produced similarly.

The cartoon, the owl and the racoon is drawn using Stable Diffusion using a mixture of control nets (Softedge, Canny Edge and Depth Map) with selected frames extracted from the original video. A total of nine keyframes are produced for each animation and these are fed into the EbSynth tool to interpolate between the frames.