User:Naugam/sandbox

Video Super-Resolution (VSR) is the process of generating high-resolution video frames from the given low-resolution ones. Unlike single image super-resolution (SISR), the main goal is not only to restore more fine details while saving coarse ones, but also to preserve motion consistensy.

There are many approaches for this task, but this problem still remains to be popular and challenging.

Math explanation
The most research works consider degradation process of frames as
 * $$\{y\} = (\{x\} * k)\downarrow{_s} + \{n\} $$

where $$\{x\}$$ — original high-resolution frame sequence,

$$k$$ — blur kernel,

$$*$$ — convolution operation,

$$\downarrow{_s}$$ — downscaling operation,

$$\{n\}$$ — additive noise,

$$\{y\}$$ — low-resolution frame sequence

Super-resolution is an inverse operation, so its problem is to estimate frame sequence $$\{\overline{x}\}$$ from frame sequence $$\{y\}$$ so that $$\{\overline{x}\}$$ is close to original $$\{x\}$$. Blur kernel, downscaling operation and additive noise should be estimated for given input to achieve better results.

Video super-resolution approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension. Complex designs are not uncommon. Some most essential components for VSR are guided by four basic functionalities: Propagation, Alignment, Aggregation, and Upsampling.


 * Propagation refers to the way in which features are propagated temporally
 * Alignment concerns on the spatial transformation applied to misaligned images/features
 * Aggregation defines the steps to combine aligned features
 * Upsampling describes the method to transform the aggregated features to the final output image

Methods
When working with video, temporal information could be used to improve upscaling quality. Single image super-resolution methods could be used too, generating high-resolution frames independently from their neighbours, but it's less effective and introduces temporal instability. There are a few traditional methods, which consider the video super-resolution task as an optimization problem. Last years deep learning based methods for video upscaling outperform traditional ones.