Data compression

The process of encoding information using fewer bits than the original representation

In computer science and information theory, data compression or source coding is the process of representing information with fewer data bits (or other information-related units) than if it were not encoded, according to an encoding mechanism specific . For example, if we encode “compression” as “comp”, the item can be represented with fewer data bits. A common example is the ZIP archive format, which not only provides compression but also acts as an archiver, capable of storing many files in the same archive.

We can use data consistency (represented by information entropy, entropy), regularity, and predictability to achieve data compression. The compression technology first developed by humans is actually natural language. Generally speaking, if a thing can be described in a relatively simplified natural language, then it will be better able to compress such things. The more consistent the data, the more concentrated its statistical features. Take image compression as an example, which centrally accounts for the time domain and frequency domain of the Fourier transform, the histogram, and the eigenvalues.

Data compression is possible because most real-world data has statistical redundancy. For example, the letter “e” is more commonly used in English than the letter “z”, and it is very unlikely that the letter “q” will be followed by a “z”. Non-destructive data compression generally exploits statistical redundancy so that the sender’s data can be represented more succinctly, but fully.

The compression ratio of non-destructive data compression is not sufficient to handle the large volume of audio and video data, but if some loss of fidelity is allowed, higher compression can be achieved. For example, when people look at photographs or television images, they may not realize that some details are not perfect. Similarly, two audio recording sample streams may sound the same, but they are not actually exactly the same. Destructive data compression uses fewer bits to represent images, video, or audio with acceptable or imperceptible numbers.

However, there are often files that cannot be compressed using destructive data compression, and in fact cannot be compressed using any compression algorithm for data that does not contain discernible patterns. Also, trying to compress already compressed data often results in data bloat.

In fact, destructive data compression will eventually get to the point where it won’t work. For example, an extreme example: the compression algorithm deletes the last byte of the file every time, and after this algorithm continues to compress until the file is empty, the compression algorithm will not continue to work.

Compression is important because it helps reduce the consumption of expensive resources such as hard drive space and connection bandwidth, however, compression requires information processing resources, which can also be expensive. Therefore, the design of the data compression mechanism requires a compromise between the compression capacity, the degree of distortion, the computing resources required, and various other factors that must be taken into account.

As with any form of communication, compressed data communication only works if both the sender and receiver of the information understand the encryption mechanism. For example, the article only makes sense if the recipient knows that the article is to be interpreted in Chinese characters. Also, the compressed data can only be understood by the receiver if he knows the encoding method.