Video encoding, how it works (part 1)
The effective compression of video information is based on two main ideas: the suppression of small details of the spatial distribution of individual frames that are insignificant to visual perception, and the elimination of temporal redundancy in the sequence of these frames. Consequently, we speak of spatial and temporal compression.
The first one uses the experimentally established low sensitivity of human perception to distortions of small image details. The eye notices a non-uniform background more quickly than the curvature of a thin edge or a change in brightness and color of a small area. Two equivalent representations of the image are known from mathematics: the familiar spatial distribution of brightness and color and the so-called frequency distribution associated with the spatial Discrete Cosine Transform (DCT). In theory, they are equivalent and reversible, but they store information about the image structure in completely different ways: the transmission of smooth background changes is provided by low-frequency (center) values of the frequency distribution, and the high-frequency coefficients. They are often responsible for the fine details of spatial distribution. This allows the following compression algorithm to be used. The frame is divided into 16×16 blocks (720×576 corresponds to 45×36 blocks), each of which is converted to DCT in the frequency domain. Then the corresponding frequency coefficients are quantized (rounding of values with a given interval). If the DCT itself does not lead to data loss, the quantization of the coefficients obviously causes a thickening of the image. The quantization operation is performed with a variable interval: low-frequency information is transmitted more precisely, while many high-frequency coefficients take zero values. This provides significant compression of the data stream, but leads to a decrease in effective resolution and the possible appearance of minor spurious details (particularly at block boundaries). Obviously
For attentive readers, we repeat that this algorithm came from digital photography, where, under the name JPEG, it was developed to efficiently compress individual frames (JPEG is an abbreviation of the name of the Joint Photographic Experts Group, which endorsed it). It was then successfully applied to frame video sequences (each processed completely independently) and renamed MJPEG (Motion-JPEG). It should also be noted that the DV encoding of the DV / DVCAM / DVCPRO digital standards is essentially based on the same algorithm, but uses a more flexible scheme with adaptive selection of quantization tables. The compression ratio for different blocks, unlike MJPEG, varies with the image: for non-informational blocks (for example, at the edges of the image) it increases, and for blocks with a large number of small details, it decreases relative to the middle level of the image. As a result, with the same quality, the data volume is reduced by approximately 15% (or vice versa, with the same flow, the quality of the output signal is higher).
Temporal MPEG compression uses a high redundancy of information in images separated by small intervals. In fact, between adjacent images, usually only a small part of the scene changes; for example, there is a smooth movement of a small object on the background of a fixed background. In this case, the complete information about the scene should be saved only selectively, for reference images. For the rest, it is enough to transmit only difference information: about the position of the object, the direction and magnitude of its displacement, about new background elements (which open behind the object as it moves). In addition, these differences can form not only in comparison with the previous images, but also with the later ones (since it is in them, as the object moves, the part of the background that was previously hidden behind the object is revealed). Note that mathematically the most difficult element is the search for displaced blocks, but little change in structure, (16×16) and the determination of the corresponding vectors of their displacement. However, this element is the most essential as it can significantly reduce the amount of information required. It is the efficiency of the real-time execution of this “smart” element that distinguishes various MPEG encoders.
