FPS (frames per second) or frame rate in a video usually denotes the frequency at which consecutive images called frames to appear on the display. This means every video is composed of numbers of still images or frames that are displayed every second. The human visual system can process 10 to 12 images per second and perceive them individually, while higher rates are perceived as motion. So higher the frame rate more smooth the motion we perceive. This means higher frame rates videos can be useful in creating slow-motion videos.

Early silent films had frame rates anywhere from 16 to 24 FPS. Nowadays the normal frame rate of films has around 24-30 FPS. Practically, can we increase the frame rate or add new frames within the original frames of any existing video to make it more smooth and real?

The AI-based approach called Depth-Aware Video Frame Interpolation (DAIN) gives a rigorous answer to the above question that basically aims to synthesize nonexistent frames in-between the original frames.

Depth-Aware Video Frame Interpolation Approach Overview

Architecture of the proposed depth-aware video frame interpolation mode from DAIN paper

According to the DAIN paper, the proposed model consists of the following submodules: flow estimation, depth estimation, context extraction, kernel estimation, and frame synthesis networks as shown in the architecture above. Given two input frames, their method first estimates the optical flows and depth maps and use the proposed depth-aware flow projection layer to generate intermediate flows. The adaptive warping layer warps the input frames, depth maps, and contextual features based on the flows and spatially varying interpolation kernels. Finally, a frame synthesis network is used to generate the output frame. For further details, you can go through this paper.

The youtube video below shows briefly explains the papers about how to convert videos into 60fps.

Furthermore the source code in python for DAIN implementation is in this GitHub repository which was used to create the above starting upscaled video of classic B&W movie.

In Conclusion

If you want to convert your low frame rate video to a higher frame rate just by clicking then, you can download the app from this Dain app and take a look at this video on how to use the app.


Please enter your comment!
Please enter your name here