Digital compression: what does this revolutionary technology mean and how does it work?

These days "Digital Compression" has become a familiar term for anyone connected with television production. We all agree that it is the most revolutionary technology of the 90s, because it allows us to achieve things that a few years ago seemed impossible such as sending (and receiving) 180 satellite television channels, or recording good quality images on 4 mm wide tape. It even makes it possible to record entire movies on compact discs, and watch unstable video clips on our home computers.

But what exactly does this revolutionary technology mean, and how does it work? Compressing is reducing the size, volume, concentration, or density of an object. An internal combustion engine compresses a mixture of gases to achieve an explosion. A brick maker compresses a clay mixture into a mold to form a homogeneous block. Very different is when a digital video or audio system compresses your information to better take advantage of the "space" of your hard drives. In the physical sense, compressing is related to applying pressure to achieve change, but this is not what happens with digital compression. The "free space" of a hard disk is a virtual space made up of a large number of immovable "cells". It is impossible to accommodate two bytes where only one fits. Digital information is made up of discrete units, and that is why it behaves differently than clay.

Digital video is a representation of moving images as digital information, information that can be "understood" and manipulated by computers, and described by bits and bytes. Digitizing the video has many advantages, because once the images are converted into a "package" of digital information, the mechanical, magnetic and electronic problems that cause the analog video to deteriorate inexorably disappear. In addition, digitized video can be stored on more stable media than magnetic tape and is more easily manipulated than analog video, if the right equipment is available.

But the digitization of video has a serious drawback: To digitally represent a single image requires a very large number of bits, approximately five and a half million, which describe the conformation of almost 350,000 pixels or image elements. And it is necessary to repeat the same operation 25 or 30 times per second, adding the corresponding information to one or more digitized sound channels.

Handling digital video on a computer represents a storage problem of transmitting information, especially if the computer is expected to capture and play the video in real time. This implies that the computer is able to deliver digital information converted into images at the same rate at which these images are generated, at the rate of 150 or 180 million bits per second.

Meeting these requirements is technically possible. In fact, there are currently several systems on the market that handle digital video without using compression. However, these are very expensive equipment and in most cases have a very limited storage capacity.

To render one second of uncompressed digital video takes about 20 megabytes. If it is to play this video, the computer must mobilize 1.2 gigabytes of information per minute. And for a computer it is not easy to sustain a flow of digital information of this volume continuously. To achieve this, you need a large processing capacity, a very efficient design and special hard drives.

One of the options to facilitate this task is to process the flow of data by applying some system to reduce its volume, in such a way that the demand for computer capacity to store and deliver digital video is reduced. This is precisely the basic principle of digital compression. It is about encoding in some way the digital information that represents the video to achieve a more efficient use of storage space and computer resources. This makes it necessary to compress the information at the time of capturing the video and decompress it again every time you want to see it, which implies an additional effort of data processing because the computer must be constantly decompressing the video while it is used. However, it is usually easier to increase processing capacity than storage capacity.

But how can an effective reduction in the volume of information be achieved without affecting image quality? Fortunately for software engineers a basic feature of moving image sequences is redundancy. Images are redundant in several ways. On the one hand it is possible to find uniform colored areas within a single video frame. And it is also possible that these areas are preserved unchanged for several frames. We can then speak of spatial redundancy, which occurs within space that describes a single frame, and of temporal redundancy, which is given by the persistence of parts of the image in several frames. A simple form of digital compression can occur by manipulating information that describes a single chart. If we are digitizing a general plane with a large area of blue sky we can store a code that represents a certain number of points or blue pixels, and this will be more efficient than saving the same number of blue pixels. This is a linear coding system whose efficiency varies dramatically depending on the amount of detail found in the image. If the pixel map that represents the image has a great diversity, effective compression cannot be achieved. If the image does not have many details it can be significantly compressed.

If we take the option to compress the video based on the temporal variations we find a serious drawback: To determine what the changes of the image are we need to constantly compare the image we try to compress with the previous ones, and then with the following ones.

In some cases this makes it difficult to design procedures to apply this type of compression, as it requires a large capacity for processing and manipulating data. However, compression processes based on temporary redundancy are much more efficient than others, so they are especially suitable for cases where storage space is especially critical.

When applying any digital compression procedure to an information stream, an additional problem appears. If we have as a premise that our digital video system must capture and deliver images in real time, then there is a limited time to compress and decompress each video frame. The whole process should take 1/25 or 1/30 of a second. So far it has not been possible to develop compression systems that allow optimal video compression within these time limits.

In theory it is perfectly possible to achieve optimal compression of the video while keeping intact all the elements that describe the original image. However, this type of compression would present several problems. On the one hand, the result of compression is unpredictable, as it is directly related to the complexity of the image. If an attempt were made to process an image composed of completely different pixels from each other, it would not be possible to compress it. In addition, the complexity of the image would affect the time in which the compression process would be carried out. And when faced with a task that would require varying levels of processing power, it would be difficult to ensure stable operation of the computers dedicated to this task.

In order to implement viable video compression/decompression systems with the current limitations of time and processing capacity, it has been necessary to develop procedures that achieve a valid "translation" of the image even if part of the elements that make it up must be discarded. These compression schemes try to achieve images of acceptable quality by storing as little digital information as possible. This implies that they must discard some image elements to reach the desired level of compression in the necessary time.

It is clear that the most diverse images will suffer some deterioration during this process, as it will be necessary to discard more image elements to achieve the established goals of reducing data flow. In practice, the more required of the compression procedure, the more the images deteriorate.

In order to increase the effectiveness of digital compression procedures, several techniques have been implemented, some of them external to the compression process itself. For example, some digital video systems apply "filters" to the analog video you want to process. In this way, its contrast is reduced, details are eliminated and the diversity of colors is reduced to achieve a more efficient compression. Another option is to reduce the resolution of the video, reducing the number of pixels that describe each image. In some cases, the amount of information contained in each pixel is reduced, especially that related to color variations. Some of these procedures significantly alter the appearance of the video before it is compressed. However, it is impossible to objectively assess the quality of the image. In fact, images that may seem of unacceptable quality to a television engineer can be very pleasing to an unsuspecting viewer. And the same goes for compressed images: Although in some cases the deterioration caused by the compression process is evident, the final images can be considered suitable for many applications.

On the other hand, compressed information coding systems have been developed that allow a more efficient sampling of the diversity of the pixels that make up the image. For example, a variable encoding system can be used, in which bit sequences are not used uniformly to describe pixels. In this way, fewer bits can be used to describe the most frequently occurring pixels, and thus it is possible to reconstruct the original image with greater fidelity using less storage space.

It is also feasible to use block analysis systems that allow to accurately determine the frequency of appearance of each pixel in each sector of the frame, generating a specific "compression language" for each image. Even spatial and temporal compression can be dynamically combined to achieve greater efficiency. The most frequently used compression systems today use these types of resources to achieve better image quality. These software developments have made it possible to develop compression systems in which there is no direct relationship between the volume of compressed information and the quality of the decompressed image. For example, the use of 2:1 compression does not necessarily imply that 50% of the elements that made up the original image have been discarded. This has allowed the use of destructive compression systems to thrive, which although they affect the quality of the image do so within levels that may be acceptable for many uses.

It is undeniable that production and post-production systems based on compressed digital video have reached a level of functionality that makes them practical for many applications. More recent developments, especially in the field of software, have led to the emergence of non-linear editors, graphics and video distribution systems that can be extremely productive while maintaining a level of image quality at least acceptable. According to some manufacturers of digital video equipment, it is possible to use compression levels of 4:1 or 5:1 to achieve broadcast quality images and without a visible deterioration of the image being perceived. However, when evaluating compressed video images, several factors must be taken into account. First, the quality of the compressed image can never surpass that of the original image. If the material has originated with household equipment, an excellent result cannot be expected, although paradoxically this lower resolution video can respond better to the compression process. Using better quality video will make the effects of compression more apparent, especially for those who have learned to recognize them. Second, do not confuse the disappearance of the characteristic problems of analog video with a real "improvement" of the video. It is possible that the disappearance of jumps and stripes makes the video more pleasant, but this does not prevent the appearance of the defects introduced by compression. Digital video will have greater stability, but may present deterioration in the definition of the edges, loss of detail in the dark parts and even loss of contrast. More sophisticated algorithms tend to discard information precisely in the parts of the image where variations are less noticeable. Third, the quality of the compressed video should be evaluated based on the application. There is a tendency to assimilate the degrees of compression with the characteristics of different analog video formats, and this is not valid in all cases. It cannot be categorically stated that a compression of 16:1 is equivalent to a VHS recording, or that a level of 4:1 is similar to BETACAM SP. Factors such as the use of optical filters, certain light levels or the characteristics of a camera can vary the response of compression systems. For example, low-contrast images tend to lose more detail when compressed, regardless of the format in which they originated. Finally, we must take into account the purpose of digital video systems and their suitability for specific tasks. It's silly, to say the least, to want to end programs in a non-linear editor designed for off-line work or to pretend to edit a 45-minute documentary in a special effects system with 3 minutes of storage. In addition, as many of us have seen, it is necessary to adapt work systems to the advantages and limitations of new technology. Those who turn their editing sessions into endless hours of tape review won't be able to easily adapt to non-linear editors with limited storage. And those who don't understand the reach of digital video systems won't be able to use them productively.

Digital video compression puts at our disposal a new repertoire of tools of relatively low cost and great capacity, but we can not expect to use them successfully if we do not make an effort to generate a professional culture appropriate to their peculiarities.