Reference frame (video)
Encyclopedia
Reference frames are frames of a compressed video that are used to define future frames. As such, they are only used in inter-frame compression techniques. In older video encoding standards, such as MPEG-2
, only one reference frame – the previous frame – was used for P-frames. Two reference frames (one past and one future) were used for B-frames.
in the next frame. While the best frame for this purpose is usually the previous frame, the extra reference frames can improve compression efficiency and/or video quality
. Note that different reference frames can be chosen for different macroblocks in the same frame. The maximum number of concurrent reference frames supported by H.264 is 16. Different reference frames can be chosen for each 8x8 partition of a macroblock. Another video format that supports multiple reference frames is Snow, which can handle up to eight. The Theora
codec provides a limited form of multiple reference frames, allowing references to both the preceding frame and the most recent intra frame.
, that are ordinarily carried out only on one reference frame have to be repeated on all of the reference frames. Heuristics can be used to reduce this speed cost at the cost of quality. Very high numbers of reference frames are rarely useful in terms of quality for live-action material because frames from farther back in time generally have less and less correlation with the current frame. This is not as true for animated
sources, where repetitive motion can make high numbers of reference frames more useful.
usage of the decoder for videos with large numbers of reference frames. The use of several reference frames also decreases locality of reference
, which might cause a speed impact.
MPEG-2
MPEG-2 is a standard for "the generic coding of moving pictures and associated audio information". It describes a combination of lossy video compression and lossy audio data compression methods which permit storage and transmission of movies using currently available storage media and transmission...
, only one reference frame – the previous frame – was used for P-frames. Two reference frames (one past and one future) were used for B-frames.
Multiple reference frames
Some modern video encoding standards, such as H.264/AVC, allow the use of multiple reference frames. This allows the video encoder to choose among more than one previously decoded frame on which to base each macroblockMacroblock
Macroblock is an image compression component and technique based on discrete cosine transform used on still images and video frames. Macroblocks are usually composed of two or more blocks of pixels. In the JPEG standard macroblocks are called MCU blocks....
in the next frame. While the best frame for this purpose is usually the previous frame, the extra reference frames can improve compression efficiency and/or video quality
Video quality
Video quality is a characteristic of a video passed through a video transmission/processing system, a formal or informal measure of perceived video degradation...
. Note that different reference frames can be chosen for different macroblocks in the same frame. The maximum number of concurrent reference frames supported by H.264 is 16. Different reference frames can be chosen for each 8x8 partition of a macroblock. Another video format that supports multiple reference frames is Snow, which can handle up to eight. The Theora
Theora
Theora is a free lossy video compression format. It is developed by the Xiph.Org Foundation and distributed without licensing fees alongside their other free and open media projects, including the Vorbis audio format and the Ogg container....
codec provides a limited form of multiple reference frames, allowing references to both the preceding frame and the most recent intra frame.
Encoding
Multiple reference frames can considerably increase encoding time because many of the decisions, such as motion estimationMotion estimation
Motion estimation is the process of determining motion vectors that describe the transformation from one 2D image to another; usually from adjacent frames in a video sequence. It is an ill-posed problem as the motion is in three dimensions but the images are a projection of the 3D scene onto a 2D...
, that are ordinarily carried out only on one reference frame have to be repeated on all of the reference frames. Heuristics can be used to reduce this speed cost at the cost of quality. Very high numbers of reference frames are rarely useful in terms of quality for live-action material because frames from farther back in time generally have less and less correlation with the current frame. This is not as true for animated
Animation
Animation is the rapid display of a sequence of images of 2-D or 3-D artwork or model positions in order to create an illusion of movement. The effect is an optical illusion of motion due to the phenomenon of persistence of vision, and can be created and demonstrated in several ways...
sources, where repetitive motion can make high numbers of reference frames more useful.
Decoding
When decoding, reference frames must be stored in memory until they are no longer needed for further decoding. This can considerably raise the memoryRam
-Animals:*Ram, an uncastrated male sheep*Ram cichlid, a species of freshwater fish endemic to Colombia and Venezuela-Military:*Battering ram*Ramming, a military tactic in which one vehicle runs into another...
usage of the decoder for videos with large numbers of reference frames. The use of several reference frames also decreases locality of reference
Locality of reference
In computer science, locality of reference, also known as the principle of locality, is the phenomenon of the same value or related storage locations being frequently accessed. There are two basic types of reference locality. Temporal locality refers to the reuse of specific data and/or resources...
, which might cause a speed impact.