Category: Audio and video

Description of the compression formats MPEG-2 and MPEG-4 part 2

MPEG-4 and HDTV

Digital satellite television uses the MPEG-2 format, where at a frame resolution of 720×576 pixels, the information flow is 30 frames / sec. it is about 12 Mbit / s, in practice, the throughput is about 3 Mbit / s. With a standard 54 MHz bandwidth, a single satellite transponder typically supports 18 channels. When transmitting in HDTV, the image resolution is 1920 x 1080 pixels, which is 5 times higher than conventional SD TV, and to transmit an HDTV channel in the MPEG-2 standard, the operator would have to rent almost a third of the transponder .

The next round in the development of video compression algorithms was the MPEG-4 standard. It was originally intended to stream video in real time over low-speed channels, but it has also found application in digital television.

Video compression in MPEG-4 format is done in the same way as in MPEG-2. When encoding the original video image, the codec finds and saves the most significant frames, generally those in which the scene changes. Instead of storing intermediate frames, the algorithm processes and saves data about changes in the current frame relative to the previous one, that is, differentially. At the same time, in the image processing process, the codec operates with arbitrarily shaped objects, in contrast to the MPEG-2 format, which could only operate with rectangular areas of the image. As a result, a person moving around the room will be perceived by MPEG-4 as a separate object moving relative to a stationary object – the background.

The idea behind the MPEG-4 standard is to combine 22 sub-standards from which providers can choose the one that best suits their needs.

Let’s select the most important sub-standards of them:

ISO 14496-3 – Audio – A set of codecs for voice and audio compression, including advanced audio coding (AAC)
ISO 14496-10 – Video: Technically identical Advanced Video Coding (AVC) known as H.264 codec
With the transition of satellite television operators to the DVB-S2 standard and the compression of data in MPEG-4 with the H.264 codec, it was possible to place 8 to 10 HDTV channels on the trunk of a transponder.

Description of the MPEG-2 and MPEG-4 compression formats

A distinctive feature of video data is its extremely large volume.

Data compression specialists have been working for many years to improve the efficiency of video compression algorithms. At the beginning of the 21st century, with the advent of HDTV, there was an urgent need to transmit large amounts of video information over satellite and cable networks, and the task of optimizing video data encoding methods arose.

Today, MPEG-2 is a digital encoding standard for audio and video signals that is used by most satellite television operators to transmit signals to subscribers. This standard was developed by Moving Pictures Experts Group and has been approved by the International Organization for Standardization.

Technical aspects of the MPEG-2 standard
The MPEG working group described the general principles of audio and video compression and left the details to the codec manufacturers. The compression algorithm was based on the human eye perception model of video images and the structural characteristics of the human eye: its ability to perceive color variations and brightness gradations. For example, the human eye can better perceive gradations of brightness than chromaticity.

The task is reduced to determining a stationary background and moving objects on the screen, on the basis of this, it is possible to select and transmit information about the base frame, and then transmit frames with information about moving objects. Insignificant information is discarded in the data transmission process, similar to the principles used in the JPEG graphic format. The process is done by dividing the video information stream into groups of video images, each group consists of 3 types of video frames. Sequences of 30 frames per second are typically used.

Due to the constant improvement of MPEG-2 video codecs, cable and satellite transmission operators have been able to transmit twice as much information with the same channel bandwidth as at the dawn of the evolution of digital transmission. An increasing number of different video codecs began to appear, but they no longer corresponded to the existing MPEG-2 format. A further unification of the norm is necessary.

MPEG-1 How it works Part 2

Mpeg-1 video

In this case, part of the information is lost due to the alignment of strongly differentiated areas that do not obey the correlation. After this procedure, the quantization algorithm comes into play: quantization, which forms the quantization matrix. The quantization matrix is ​​a quantization matrix whose elements are continuous to discrete data, that is, numbers representing the amplitude values ​​of the Fourier frequency components. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. whose elements are data continuously converted to discrete, that is, numbers representing the amplitude values ​​of the Fourier frequency components. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. whose elements are data continuously converted to discrete, that is, numbers representing the amplitude values ​​of the Fourier frequency components. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape.

All these transformations concern only the image. But in addition to the image in almost any video clip, there is also sound. Sound encoding is done by a separate audio encoder. As the MPEG format has evolved, audio encoders have been modified several times to be more efficient. At the time of the final standardization of the MPEG-1 format, three audio encoders of this family were created: MPEG-1 Layer I, Layer II and Layer 3 (the same famous MP3). The coding principles of all these codecs are based on the psychoacoustic model, which was further refined and reached its apotheosis for the MPEG-1 family in Layer-3 algorithms.

Many articles have been written on the psychoacoustic model and the principles of lossy audio data compression, in particular the article “Description of MP3 audio compression format”, which you can read on this site, so it can be skipped the description of the audio encoders, mentioning only the synchronization of audio and video data and audio track formats.

The synchronization of audio and video data is carried out using a specially dedicated data stream called a system stream. This sequence contains a built-in timer that operates at a speed of 90 KHz and contains 2 layers: a system layer with a timer and service information to synchronize frames with an audio track and a compression layer with audio and video sequences.

The service information refers to various kinds of marks, the most important of which are the marks SCR (System Clock Reference), the codec time counter increment and PDS (Presentation Data Stamp), the beginning mark of the video frame. or audio frame playback.

Well, the story about MPEG-1 has almost come to an end, it only remains to name a few parameters of the audio tracks that are used in this format.

The quality of audio tracks in MPEG-1 can vary widely, from high quality to ugly. Finally, all audio compression formats were standardized in 1992 by the European Commission for ISO standards.

MPEG-1 How it works

Mpeg-1 video

After the division of frames into different types is completed, the process of preparing for encoding begins.

With I-frames, the process of preparing for encoding is quite simple: the frame is divided into blocks. In MPEG-1, the blocks are 8×8 pixels.

But for type P and B personnel, the preparation is much more complicated. To compress frames of the indicated types more strongly, a motion prediction algorithm is used.

As input, the motion prediction algorithm receives an 8×8 pixel block from the current frame and similar blocks from previous frames (type I or P). In the output of this algorithm, we have the following information about the previous block:

The motion vector of the current block relative to previous ones
The difference between the current and previous blocks, which will, in fact, be further encoded.
All redundant information must be removed, which is why such a high compression ratio is achieved that it is impossible with lossless compression.

But the motion prediction algorithm has limitations. Often in movies, there are still scenes where there is little or no movement and full blocks or frames appear where it is impossible to use the motion prediction algorithm. I think you have noticed that MPEG-1 compressed videos have noticeably better quality in scenes with few moving objects than in high traffic scenes. This is due to the fact that in still scenes the P and B-frames are in fact copies of the I-frames, there is practically no loss, but the compression of the information is negligible.

In the case of the correct operation of the motion prediction algorithm, the frame volumes of different types in bytes are related to each other roughly as follows: I: P: B as 15: 5: 2. As you can see from this relationship , the reduction in the amount of video information is already evident in the preparation stage for encoding.

At the end of this stage, the encoding itself begins. The encoding process includes 3 stages:

Discrete Cosine Transformation: DTC, Discrete Cosine Transform, Fourier Transform.
Quantization – quantification. Conversion of data from continuous to discontinuous, discrete.
Convert the received data blocks into a sequence, that is, convert from a matrix to a linear form.
When encoding, the pixel blocks or the calculated difference between blocks are processed by the first of the transformation algorithms: DTC (Discrete Cosine Transform). Usually the pixels in a block and the image blocks themselves are related to each other in some way, for example a solid background, a uniform lighting gradient, a repeating pattern, and so on. This relationship is called correlation. The DTC algorithm uses correlated effects to transform the blocks into frequency Fourier components.

A family of MPEG formats. Part One: MPEG-1 – 2

Information compression principles in MPEG-1.

As an example, consider the reduced PAL / SECAM format, which is more common than SIF, although both formats are not different from each other with the exception of resolution and frame rate.

The simplified version of the PAL / SECAM format contains 352 ppl (point per line – points per line), 288 lpf (line per frame – lines per frame) and 25 fps (frame per second – frames per second). I must say that the full PAL / SECAM standard has parameters 4 times higher than MPEG-1 (except for fps). Therefore, it is often said that VIDEO-CD has four times worse clarity compared to normal video.

As for the color depth, it is not as simple as in computer graphics, where a certain fixed number of bits is assigned for each pixel. MPEG-1 uses the YCbCr color scheme, where Y is the luminance plane, Cb and Cr are the color planes. These planes are encoded in different resolutions. There are several encoding options, which can be represented as follows:
Encoding option Resolution ratio Сb / Y (Сr / Y) horizontally Resolution ratio Сb / Y (Сr / Y) vertically
4: 4: 4 1: 1 1: 1
4: 2: 2 1: 2 1: 1
4: 2: 0 1: 2 1: 2
4: 1: 1 1: 4 1: 1
4: 1: 0 1: 4 1: 4
As you can see from the table, Cb and Cr are almost always encoded at a lower resolution than Y. The lower the resolution of the color planes, the coarser and more unnatural the color reproduction in video is. Of course, the last option will be the lowest quality, but also the most compact.

Before the start of encoding, the video information is analyzed, the keyframes that will not change during compression are selected, as well as the frames during encoding from which some of the information will be removed. In total, there are three types of frames:

Type I frames – Intra frame. Unchanged compressed keyframes.
P-type frames – Predirected frame. When these frames are encoded, some of the information is removed. When playing a P-frame, the information from the previous I or P-frames is used.
Type B frames: bidirectional frame. When these frames are encoded, the loss of information is even more significant. When playing back a B-frame, information from the previous two I or P-frames is used. The presence of B-frames in a video is the factor by which MPEG-1 has a high compression ratio (but not a very high quality either).
When encoding, a chain of frames of different types is formed. The most typical sequence might look like this: IBBPBBPBBIBBPBBPBB … Consequently, the play queue by frame numbers will look like this: 1423765 …

A family of MPEG formats. Part One: MPEG-1

The history of the MPEG family of formats, to which the rapidly gaining popularity MPEG-4 belongs, began in 1988.

It was in this year that the committee of the Group of Experts on Moving Pictures was founded, which roughly translates into Russian as the Expert Group on Cinematography (Moving Pictures), the abbreviation of which is MPEG is now known to anyone who deals with multimedia: computers or digital. TV. In the same year, development of the MPEG-1 format began, which was finally released in 1993. Despite all the obvious shortcomings of this format, MPEG-1 remains one of the most widespread video compression formats.
I must say that almost all the innovative developments at that time that formed the basis of the MPEG-1 format, in one way or another, are in more advanced formats of this series, therefore, having considered in detail the first representative of this family. of video compression formats, you can get a general idea of ​​how MPEG algorithms work.

MPEG-1 format. Old, but not yet defeated.
The MPEG-1 format began to develop in those hard-to-imagine times when high-volume media was not widely available, while video data, even compressed, was occupying absolutely colossal volumes by the late 1980s, the average length of a la movie was over a gigabyte. If anyone does not remember, then it was the era of the 286 and 386 processors, 4 MB of RAM and a 250 MB hard disk were considered a luxury, not a misery, since now Windows was a gadget for DOS, and not the reverse. and SONY’s newly released 5 “floppy disks and 3.5” floppy disks dominated as easily portable media. In such conditions, it was necessary to find a means

And such a medium was found. Just in those years, for the first time on the PC platform, a new type of storage medium appeared, such as CD-ROM discs, which could provide the necessary amount of information. It is true that a movie in MPEG-1 format did not fit on a disc, but what prevented it from being recorded on 2 CDs, especially since the novelty was very cheap? Of course, the first CD-ROM players were single speed, so it should come as no surprise that the maximum bit stream rate in MPEG-1 format is limited to 150 Kb / s, which corresponds to a CD speed. -ROM.

I must say that the possibilities of MPEG-1 are not limited to the low resolution that everyone saw when watching VIDEO-CD. The format itself allowed the compression and playback of video information with a resolution of up to 4095×4095 and a frame rate of up to 60 Hz. But due to the fact that the data transfer stream was limited to 150 Kb / s, it is i.e. the so-called restricted parameter bitstream (CPB) – the fixed width of the data transfer stream, the developers of the format and later the creators of codecs based on it were forced to use optimized frame resolutions for the given CPB. The most widely used are two of these optimized formats: the SIF 352×240 format,

Well, we have already decided the resolution, now you can see how everything is compressed.