Stable Diffusion Cat Animator (Rotoscoping)

This project uses a base kitten video to drive animations of a cartoon cat, an owl, and a racoon.

Stable Diffusion Cat Animator

This video is produced using Stable Diffusion with the EbSynth tool. The top left is the original video of the kitten, the top right is an animation of a cartoon cat. At the bottom left is an owl that is drawn based on a few selected frames from the original video. At the bottom right is a raccoon that is produced similarly.

Original - Top Left
Cartoon Cat - Top Right
Owl - Bottom Left
Raccoon - Bottom Right

The cartoon, the owl and the racoon is drawn using Stable Diffusion using a mixture of control nets (Softedge, Canny Edge and Depth Map) with selected frames extracted from the original video. A total of nine keyframes are produced for each animation and these are fed into the EbSynth tool to interpolate between the frames.

Cat Retouch/Cartoon
Owl Beak