Description of the compression formats MPEG-2 and MPEG-4 part 2

MPEG-4 and HDTV

Digital satellite television uses the MPEG-2 format, where at a frame resolution of 720×576 pixels, the information flow is 30 frames / sec. it is about 12 Mbit / s, in practice, the throughput is about 3 Mbit / s. With a standard 54 MHz bandwidth, a single satellite transponder typically supports 18 channels. When transmitting in HDTV, the image resolution is 1920 x 1080 pixels, which is 5 times higher than conventional SD TV, and to transmit an HDTV channel in the MPEG-2 standard, the operator would have to rent almost a third of the transponder .

The next round in the development of video compression algorithms was the MPEG-4 standard. It was originally intended to stream video in real time over low-speed channels, but it has also found application in digital television.

Video compression in MPEG-4 format is done in the same way as in MPEG-2. When encoding the original video image, the codec finds and saves the most significant frames, generally those in which the scene changes. Instead of storing intermediate frames, the algorithm processes and saves data about changes in the current frame relative to the previous one, that is, differentially. At the same time, in the image processing process, the codec operates with arbitrarily shaped objects, in contrast to the MPEG-2 format, which could only operate with rectangular areas of the image. As a result, a person moving around the room will be perceived by MPEG-4 as a separate object moving relative to a stationary object – the background.

The idea behind the MPEG-4 standard is to combine 22 sub-standards from which providers can choose the one that best suits their needs.

Let’s select the most important sub-standards of them:

ISO 14496-3 – Audio – A set of codecs for voice and audio compression, including advanced audio coding (AAC)
ISO 14496-10 – Video: Technically identical Advanced Video Coding (AVC) known as H.264 codec
With the transition of satellite television operators to the DVB-S2 standard and the compression of data in MPEG-4 with the H.264 codec, it was possible to place 8 to 10 HDTV channels on the trunk of a transponder.

Description of the MPEG-2 and MPEG-4 compression formats

A distinctive feature of video data is its extremely large volume.

Data compression specialists have been working for many years to improve the efficiency of video compression algorithms. At the beginning of the 21st century, with the advent of HDTV, there was an urgent need to transmit large amounts of video information over satellite and cable networks, and the task of optimizing video data encoding methods arose.

Today, MPEG-2 is a digital encoding standard for audio and video signals that is used by most satellite television operators to transmit signals to subscribers. This standard was developed by Moving Pictures Experts Group and has been approved by the International Organization for Standardization.

Technical aspects of the MPEG-2 standard
The MPEG working group described the general principles of audio and video compression and left the details to the codec manufacturers. The compression algorithm was based on the human eye perception model of video images and the structural characteristics of the human eye: its ability to perceive color variations and brightness gradations. For example, the human eye can better perceive gradations of brightness than chromaticity.

The task is reduced to determining a stationary background and moving objects on the screen, on the basis of this, it is possible to select and transmit information about the base frame, and then transmit frames with information about moving objects. Insignificant information is discarded in the data transmission process, similar to the principles used in the JPEG graphic format. The process is done by dividing the video information stream into groups of video images, each group consists of 3 types of video frames. Sequences of 30 frames per second are typically used.

Due to the constant improvement of MPEG-2 video codecs, cable and satellite transmission operators have been able to transmit twice as much information with the same channel bandwidth as at the dawn of the evolution of digital transmission. An increasing number of different video codecs began to appear, but they no longer corresponded to the existing MPEG-2 format. A further unification of the norm is necessary.

MPEG-1 How it works Part 2

Mpeg-1 video

In this case, part of the information is lost due to the alignment of strongly differentiated areas that do not obey the correlation. After this procedure, the quantization algorithm comes into play: quantization, which forms the quantization matrix. The quantization matrix is ​​a quantization matrix whose elements are continuous to discrete data, that is, numbers representing the amplitude values ​​of the Fourier frequency components. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. whose elements are data continuously converted to discrete, that is, numbers representing the amplitude values ​​of the Fourier frequency components. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. whose elements are data continuously converted to discrete, that is, numbers representing the amplitude values ​​of the Fourier frequency components. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape. After the formation of the quantization matrix, the frequency coefficients are divided into a specific number of values. The precision of the frequency coefficients is fixed at 8 bits. After quantization, many of the coefficients in the matrix are set to zero. And as a final stage, the matrix is ​​transformed into a linear shape.

All these transformations concern only the image. But in addition to the image in almost any video clip, there is also sound. Sound encoding is done by a separate audio encoder. As the MPEG format has evolved, audio encoders have been modified several times to be more efficient. At the time of the final standardization of the MPEG-1 format, three audio encoders of this family were created: MPEG-1 Layer I, Layer II and Layer 3 (the same famous MP3). The coding principles of all these codecs are based on the psychoacoustic model, which was further refined and reached its apotheosis for the MPEG-1 family in Layer-3 algorithms.

Many articles have been written on the psychoacoustic model and the principles of lossy audio data compression, in particular the article “Description of MP3 audio compression format”, which you can read on this site, so it can be skipped the description of the audio encoders, mentioning only the synchronization of audio and video data and audio track formats.

The synchronization of audio and video data is carried out using a specially dedicated data stream called a system stream. This sequence contains a built-in timer that operates at a speed of 90 KHz and contains 2 layers: a system layer with a timer and service information to synchronize frames with an audio track and a compression layer with audio and video sequences.

The service information refers to various kinds of marks, the most important of which are the marks SCR (System Clock Reference), the codec time counter increment and PDS (Presentation Data Stamp), the beginning mark of the video frame. or audio frame playback.

Well, the story about MPEG-1 has almost come to an end, it only remains to name a few parameters of the audio tracks that are used in this format.

The quality of audio tracks in MPEG-1 can vary widely, from high quality to ugly. Finally, all audio compression formats were standardized in 1992 by the European Commission for ISO standards.

MPEG-1 How it works

Mpeg-1 video

After the division of frames into different types is completed, the process of preparing for encoding begins.

With I-frames, the process of preparing for encoding is quite simple: the frame is divided into blocks. In MPEG-1, the blocks are 8×8 pixels.

But for type P and B personnel, the preparation is much more complicated. To compress frames of the indicated types more strongly, a motion prediction algorithm is used.

As input, the motion prediction algorithm receives an 8×8 pixel block from the current frame and similar blocks from previous frames (type I or P). In the output of this algorithm, we have the following information about the previous block:

The motion vector of the current block relative to previous ones
The difference between the current and previous blocks, which will, in fact, be further encoded.
All redundant information must be removed, which is why such a high compression ratio is achieved that it is impossible with lossless compression.

But the motion prediction algorithm has limitations. Often in movies, there are still scenes where there is little or no movement and full blocks or frames appear where it is impossible to use the motion prediction algorithm. I think you have noticed that MPEG-1 compressed videos have noticeably better quality in scenes with few moving objects than in high traffic scenes. This is due to the fact that in still scenes the P and B-frames are in fact copies of the I-frames, there is practically no loss, but the compression of the information is negligible.

In the case of the correct operation of the motion prediction algorithm, the frame volumes of different types in bytes are related to each other roughly as follows: I: P: B as 15: 5: 2. As you can see from this relationship , the reduction in the amount of video information is already evident in the preparation stage for encoding.

At the end of this stage, the encoding itself begins. The encoding process includes 3 stages:

Discrete Cosine Transformation: DTC, Discrete Cosine Transform, Fourier Transform.
Quantization – quantification. Conversion of data from continuous to discontinuous, discrete.
Convert the received data blocks into a sequence, that is, convert from a matrix to a linear form.
When encoding, the pixel blocks or the calculated difference between blocks are processed by the first of the transformation algorithms: DTC (Discrete Cosine Transform). Usually the pixels in a block and the image blocks themselves are related to each other in some way, for example a solid background, a uniform lighting gradient, a repeating pattern, and so on. This relationship is called correlation. The DTC algorithm uses correlated effects to transform the blocks into frequency Fourier components.

A family of MPEG formats. Part One: MPEG-1 – 2

Information compression principles in MPEG-1.

As an example, consider the reduced PAL / SECAM format, which is more common than SIF, although both formats are not different from each other with the exception of resolution and frame rate.

The simplified version of the PAL / SECAM format contains 352 ppl (point per line – points per line), 288 lpf (line per frame – lines per frame) and 25 fps (frame per second – frames per second). I must say that the full PAL / SECAM standard has parameters 4 times higher than MPEG-1 (except for fps). Therefore, it is often said that VIDEO-CD has four times worse clarity compared to normal video.

As for the color depth, it is not as simple as in computer graphics, where a certain fixed number of bits is assigned for each pixel. MPEG-1 uses the YCbCr color scheme, where Y is the luminance plane, Cb and Cr are the color planes. These planes are encoded in different resolutions. There are several encoding options, which can be represented as follows:
Encoding option Resolution ratio Сb / Y (Сr / Y) horizontally Resolution ratio Сb / Y (Сr / Y) vertically
4: 4: 4 1: 1 1: 1
4: 2: 2 1: 2 1: 1
4: 2: 0 1: 2 1: 2
4: 1: 1 1: 4 1: 1
4: 1: 0 1: 4 1: 4
As you can see from the table, Cb and Cr are almost always encoded at a lower resolution than Y. The lower the resolution of the color planes, the coarser and more unnatural the color reproduction in video is. Of course, the last option will be the lowest quality, but also the most compact.

Before the start of encoding, the video information is analyzed, the keyframes that will not change during compression are selected, as well as the frames during encoding from which some of the information will be removed. In total, there are three types of frames:

Type I frames – Intra frame. Unchanged compressed keyframes.
P-type frames – Predirected frame. When these frames are encoded, some of the information is removed. When playing a P-frame, the information from the previous I or P-frames is used.
Type B frames: bidirectional frame. When these frames are encoded, the loss of information is even more significant. When playing back a B-frame, information from the previous two I or P-frames is used. The presence of B-frames in a video is the factor by which MPEG-1 has a high compression ratio (but not a very high quality either).
When encoding, a chain of frames of different types is formed. The most typical sequence might look like this: IBBPBBPBBIBBPBBPBB … Consequently, the play queue by frame numbers will look like this: 1423765 …

A family of MPEG formats. Part One: MPEG-1

The history of the MPEG family of formats, to which the rapidly gaining popularity MPEG-4 belongs, began in 1988.

It was in this year that the committee of the Group of Experts on Moving Pictures was founded, which roughly translates into Russian as the Expert Group on Cinematography (Moving Pictures), the abbreviation of which is MPEG is now known to anyone who deals with multimedia: computers or digital. TV. In the same year, development of the MPEG-1 format began, which was finally released in 1993. Despite all the obvious shortcomings of this format, MPEG-1 remains one of the most widespread video compression formats.
I must say that almost all the innovative developments at that time that formed the basis of the MPEG-1 format, in one way or another, are in more advanced formats of this series, therefore, having considered in detail the first representative of this family. of video compression formats, you can get a general idea of ​​how MPEG algorithms work.

MPEG-1 format. Old, but not yet defeated.
The MPEG-1 format began to develop in those hard-to-imagine times when high-volume media was not widely available, while video data, even compressed, was occupying absolutely colossal volumes by the late 1980s, the average length of a la movie was over a gigabyte. If anyone does not remember, then it was the era of the 286 and 386 processors, 4 MB of RAM and a 250 MB hard disk were considered a luxury, not a misery, since now Windows was a gadget for DOS, and not the reverse. and SONY’s newly released 5 “floppy disks and 3.5” floppy disks dominated as easily portable media. In such conditions, it was necessary to find a means

And such a medium was found. Just in those years, for the first time on the PC platform, a new type of storage medium appeared, such as CD-ROM discs, which could provide the necessary amount of information. It is true that a movie in MPEG-1 format did not fit on a disc, but what prevented it from being recorded on 2 CDs, especially since the novelty was very cheap? Of course, the first CD-ROM players were single speed, so it should come as no surprise that the maximum bit stream rate in MPEG-1 format is limited to 150 Kb / s, which corresponds to a CD speed. -ROM.

I must say that the possibilities of MPEG-1 are not limited to the low resolution that everyone saw when watching VIDEO-CD. The format itself allowed the compression and playback of video information with a resolution of up to 4095×4095 and a frame rate of up to 60 Hz. But due to the fact that the data transfer stream was limited to 150 Kb / s, it is i.e. the so-called restricted parameter bitstream (CPB) – the fixed width of the data transfer stream, the developers of the format and later the creators of codecs based on it were forced to use optimized frame resolutions for the given CPB. The most widely used are two of these optimized formats: the SIF 352×240 format,

Well, we have already decided the resolution, now you can see how everything is compressed.

What is an MPEG file?

An MPEG file (pronounced “em-peg”) is an MPEG (Moving Picture Expert Group) video file.

Videos in this format are compressed using MPEG-1 or MPEG-2 compression. This makes MPEG files popular for online distribution – they can be uploaded and downloaded faster than other video formats.

The MPEG format can store compressed data at such a high rate because instead of storing each frame of video, it only stores the changes that occur between each frame.

Important information about MPEG
Note that “MPEG” refers not only to the file extension (eg .MPEG), but also to the type of compression.

The specific file may be an MPEG file, but it doesn’t actually use the MPEG file extension. Below is a more detailed description, but for now, keep in mind that an MPEG video or audio file does not need to use the MPEG, MPG, or MPE file extension to be considered MPEG.

For example, an MPEG2 video file may use the MPG2 file extension, while audio files compressed with the MPEG-2 codec generally use MP2. Usually an MPEG-4 video file ends with the extension MP4. Both file extensions point to an MPEG file, but neither actually uses the .MPEG file extension.

Some other MPEG standards include MPEG-7 (Multimedia Content Description Interface), MPEG-MAR (Augmented and Mixed Reality Reference Model), and MPEG-DASH (Adaptive HTTP Dynamic Streaming).

More information on MPEG

There are many different file formats that can use MPEG-1, MPEG-2, MPEG-3, or MPEG-4 compression to store audio and / or video. You can read more about these specific standards on the MPEG Wikipedia page.

Therefore, these compressed MPEG files do not use the file extension MPEG, MPG, or MPE, but rather the extension that you are probably most familiar with. Some types of MPEG video and audio files include MP4V, MP4, XVID, M4V, F4V, AAC, MP1, MP2, MP3, MPG2, M1V, M1A, M2A, MPA, MPV, M4A, and M4B.

If you follow these links, you will see that, for example, M4V files are MPEG-4 video files, that is, they comply with the MPEG-4 compression standard. They do not use the MPEG file extension because they have a specific use with Apple products and are therefore easier to identify with the M4V file extension and can be opened by programs assigned to use that particular suffix. However, they are still MPEG files.

Still can’t open the file?
This can be confusing when it comes to audio and video file codecs and their corresponding file extensions. If your file doesn’t open with the suggestions above, you may be reading the file extension incorrectly or not sure what MPEG file it is. you’re trying.

Let’s use the M4V example again. If you are trying to convert or open an MPEG video file downloaded from the iTunes Store, it probably uses the M4V file extension. At first glance, you might say that you are trying to open an MPEG video file because it is true, but it is also true that the particular MPEG video file you have is a protected video that can only be opened if your computer is authorized to play the file.

However, saying you only have a regular MPEG video file that you need to open doesn’t necessarily mean a lot. It can be M4V, as we’ve seen, or it can be something completely different, like MP4, which doesn’t have the same playback protection as M4V files.

The point is to pay close attention to what the file extension says. If it’s MP4, treat it like this and use an MP4 player, but be sure to do the same with everything you might have, be it an MPEG video or audio file.

Another thing to consider if your file won’t open with your media player is that you misread the file extension and instead have a file that looks like an MPEG file.

Make sure the file extension reads like a video or audio file, or that it actually uses the MPEG or MPG file extension, and not something written like a MEGA data file (.MEG), an object file of multiple images (.MPO) or Megacubo. XML Metafile (.MEGA): These files open in MEGA, StereoPhotoView, and Megacubo, respectively.

MPEG formats

Motion Picture Experts Group

Motion Picture Experts Group, abbreviated as MPEG, is in charge of inventing, promoting and developing MPEG formats. The specialists of the MPEG expert group are subordinate to the international organization ISO (International Organization for Standardization), which develops standards for digital video and audio compression.

MPEG-1
MPEG-1 video was originally developed with the aim of achieving acceptable video quality at 1.5 Mbit / s streams, at a resolution of 352 × 240. Although MPEG-1 is used for low resolution encoding and low bit rate, the standard allows you to use any resolution up to 4095 × 4095. Most implementations are designed with restricted parameter bit stream in mind.

Currently, MPEG-1 is the most compatible format in the MPEG family; plays on almost all computers with VCD / DVD players. The biggest disadvantage of MPEG-1 video is that it only supports progressive scan. This deficiency at one time aided in faster acceptance of the more universal MPEG-2 standard.

MPEG-2
MPEG-2 is the name of a group of digital coding standards for digital video and audio signals approved by ISO – International Organization for Standardization / IEC Moving Picture Experts Group (MPEG). The MPEG-2 standard is used primarily to encode video and audio in broadcasting, including satellite broadcasting and cable television. MPEG-2 with some modifications is also actively used as a standard for DVD compression. The use of MPEG-2 requires the payment of royalties to patent holders through the MPEG Licensing Association.

MPEG-4
MPEG-4 is an international standard that is used primarily to compress digital video and audio. It appeared in 1998 and includes a group of audio and video compression standards and related technologies approved by ISO – International Organization for Standardization / IEC Moving Picture Experts Group (MPEG). The MPEG-4 standard is used primarily for streaming (video streaming), movie CD recording, video telephony (videophone), and streaming, which make heavy use of digital video and audio compression. MPEG-4 includes many of the features of MPEG-1, MPEG-2, and other similar standards, adding features such as VRML support for displaying 3D objects, object-oriented files, rights management support, and various types of interactive media.

AAC (Advanced Audio Codec) was standardized as a complement to MPEG-2 (part 7), it was also expanded and included in MPEG-4. MPEG-4 is still under development and is divided into several parts. The key parts of the MPEG-4 standard are Part 2 (MPEG-4 part 2, including the advanced simple profile used by codecs such as DivX, Xvid, Nero Digital and 3ivx, as well as Quicktime 6) and Part 10 (MPEG-4 part 10 / MPEG-4 AVC / H.264 or advanced video encoding, used by codecs like x264, Nero Digital AVC, Quicktime 7, and also used in next-generation DVD formats like HD DVD and Blu-ray Disc.)

MPEG format: specifications and capabilities Part 5

Frame sequence can have any frame arrangement

I, P and B. In industrial practice, it is common to have a fixed sequence (such as IBBPBBPBBPBBPBB), however more powerful encoders can optimize frame type selection based on the context and global characteristics of the footage.
Each type of framing has its own advantages depending on the characteristics of the image (movement activity, temporary masking effects, …).
For example, if the sequence of images changes little from frame to frame, it makes sense to encode more B-frames than P. Since B-frames are not used in the subsequent decoding process, they can be further compressed without affecting the overall video. . quality.
Specific application requirements also affect the choice of frame type: keyframes, channel switching, program indexing, error recovery, etc.
The following statistical characteristics are used for video compression:
1. Spatial correlation: 8 × 8 point discrete cosine transform.

MPEG format

2. Characteristics of human vision – immunity to high frequency components: scalar quantification of DCT coefficients with loss of quality.

MPEG format

3. Large spatial correlation of the entire image: prediction of the first low-frequency transformation coefficient in an 8×8 block (mean value of the entire block).

4. Statistics of occurrence of syntactic elements in the most probable coding stream: optimal coding of motion vectors, DCT coefficients, types of macroblocks, and so on.

5. Sparse matrix of quantized DCT coefficients: encoding of repeated zero elements with the designation of the end of the block.

6. Spatial masking: the degree of quantization of the macroblock.

7. Coding of areas taking into account the content of the scene: the degree of quantization of the macroblock.

8. Adaptation to the characteristics of the local image: block coding, macroblock type, adaptive quantization.

9. Constant adaptive quantization step size: The new quantization degree is set only by a special macroblock type and is not transmitted by default.

10. Temporal redundancy: 16×16 pixel macroblock level forward and backward motion vectors.

11. Coding of the macroblock prediction error taking into account perception: adaptive quantization and quantization of transformation coefficients.

12. Small prediction error: no error can be reported for a macroblock.

13. Macroblock level prediction error fine encoding: Each of the blocks within the macroblock can be encoded or omitted.

14. Motion vectors: slow motion of a fragment of an image with a complex pattern: predicting motion vectors.

15. Appearances and disappearances: forward and backward prediction in B-frames.

16. Interprediction precision: bilinear interpolated (filtered) block differences. In the real world, the movement of objects from one frame to another rarely falls within the bounds of the points. Interpolation allows you to find out the actual position of the object, which often increases the compression efficiency by 1 dB.

MPEG format

17. Motion activity limited in P-frames: macroblocks skipped. When the motion vector and the prediction error are zero. Skipped macroblocks are very desirable in the codestream, as they take no bits except in the header of the next macroblock.

18. Coplanar movement in B-frames: macroblocks skipped. When the motion vector is the same and the prediction error is zero.