Video Frame Interpolation via Cyclic Fine-Tuning and Asymmetric Reverse Flow

Morten Hannemose*, Janus Nørtoft Jensen, Gudmundur Einarsson, Jakob Wilm, Anders Bjorholm Dahl, Jeppe Revall Frisvad

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

13 Downloads (Pure)

Abstract

The objective in video frame interpolation is to predict additional in-between frames in a video while retaining natural motion and good visual quality. In this work, we use a convolutional neural network (CNN) that takes two frames as input and predicts two optical flows with pixelwise weights. The flows are from an unknown in-between frame to the input frames. The input frames are warped with the predicted flows, multiplied by the predicted weights, and added to form the in-between frame. We also propose a new strategy to improve the performance of video frame interpolation models: we reconstruct the original frames using the learned model by reusing the predicted frames as input for the model. This is used during inference to fine-tune the model so that it predicts the best possible frames. Our model outperforms the publicly available state-of-the-art methods on multiple datasets.

Original languageEnglish
Title of host publicationImage Analysis - Proceedings of the 21st Scandinavian Conference, SCIA 2019
EditorsMichael Felsberg, Per-Erik Forssén, Jonas Unger, Ida-Maria Sintorn
PublisherSpringer VS
Publication date2019
Pages311-323
ISBN (Print)9783030202040
DOIs
Publication statusPublished - 2019
Event21st Scandinavian Conference on Image Analysis, SCIA 2019 - Norrköping, Sweden
Duration: 11. Jun 201913. Jun 2019

Conference

Conference21st Scandinavian Conference on Image Analysis, SCIA 2019
Country/TerritorySweden
CityNorrköping
Period11/06/201913/06/2019
SeriesLecture Notes in Computer Science
Volume11482
ISSN0302-9743

Keywords

  • Convolutional neural networks
  • Slow motion
  • Video frame interpolation

Fingerprint

Dive into the research topics of 'Video Frame Interpolation via Cyclic Fine-Tuning and Asymmetric Reverse Flow'. Together they form a unique fingerprint.

Cite this