Hardware overlay
Encyclopedia
In computing
, hardware overlay, a type of video overlay, provides a method of rendering an image to a display screen with a dedicated memory buffer inside computer video
hardware
. The technique aims to improve the display of a fast-moving video image
— such as a computer game, a DVD
, or the signal from a TV card. Most video cards manufactured since about 1998 and most media player
s support hardware overlay.
A computer draws on its display by writing a bitmap
ped representation of the graphics into a special portion of its memory known as video memory
. Without any hardware overlays, only one chunk of video memory exists which all applications must share - and the location of a given application's video memory moves whenever the user changes the position of the application's window. With shared video memory, an application must constantly check that it is only writing to memory that belongs to that application.
When running a high-bandwidth video application such as a movie player or some games, the computing power and complexity needed to perform constant clipping and checking negatively impacts performance and compatibility. To escape these limitations, the hardware overlay was invented. In addition, the graphics processing unit
(GPU) provides an efficient way to scale the video in size and often performs color-format conversions (such as MPEG-2's YCbCr
into RGB).
An application using a hardware overlay gets a completely separate section of video memory that belongs only to that application. Because nothing else uses it, the program never needs to waste resources in checking whether a given piece of the memory belongs to it, nor does it need to monitor whether the user moves the window and changes the location of the video memory. To get the image from the separate video memory to display in tandem with the remaining shared elements on the display, the graphical subsystem associates a certain attribute (for example, a particular color) as a "mask" for that overlay, which the graphics card understands to mean that it is to draw from the separate overlay buffer onto the screen. (This technique has become known as "chroma key
".)
As an example, assume that the graphical subsystem defines the color "purple" as the mask color that triggers the overlay to appear. An application that plays a DVD will draw a solid purple rectangle on the shared screen, and then "play" the DVD into the special region of memory dedicated to the overlay. The graphics card will render the DVD playback only inside the purple area. If another window moves over top of the purple area and obscures a part of it, then the graphics hardware performs the clipping by itself. In practice, the actual color "purple" is not used - rather, either a nondescript near-black color is used (as the system often has thousands of such shades to choose from), or the mask region is denoted using some similar masking method not involving colors.
program (for example, the one automatically built into Windows that activates when a user presses the PrtSc key) often does not capture the content appearing in the hardware overlay window. Rather, a blank region containing only the special mask color is captured. This is because the screen capture routine doesn't consider the special video memory regions dedicated to overlays - it simply captures the shared main screen as rendered by the software's graphical subsystem. Some Digital Rights Management
schemes use hardware overlay to display protected content on the screen, taking advantage of this quirk to prevent the copying of protected documents by way of screen capture. Disabling the support for overlays causes the normal overlay-using software to fallback to the shared memory, thus enabling screenshot capture.
Secondary displays require both hardware and driver support; some graphics cards may support overlay on the second display while their drivers may not yet support it (note: graphics chipset driver bugs can cause most video formats apart from mpeg2 to work on both monitors, and mpeg2 only on the primary with most players).
Some users note that DVD movies display properly on a laptop screen but don't display on a TV connected to the laptop; in these cases it may be possible to designate the TV as the primary display. Sometimes, the use of hardware overlays may have to be disabled in the media player. However, some graphics cards have the option to completely redirect hardware overlay to the TV screen. In this case, starting a DVD player on the main screen with overlay enabled would result in video being displayed on the attached TV screen.
's enhanced graphics capabilities replace the basic concept of hardware overlays with full hardware compositing
for every application window running on the system, not just movie players or games, through the Desktop Window Manager
. Mac OS X
has used hardware compositing since the introduction of Quartz Extreme
into Mac OS X 10.2
. To improve performance, each program draws to its own independent memory buffer instead of to a slow graphical subsystem. (In Windows Vista, each hardware overlay is more correctly known as a Direct3D surface). Then the system's GPU assembles each of the windows into a single display screen in real time. With enhanced GPUs on the market capable of stunning 3D graphics (as a consequence of the video game industry), operating systems can apply impressive motion, scaling, and lighting effects to normal 2D windows.
In the X Window System
, the windowing system of most Unix
operating systems, the XVideo extension can allow applications to employ hardware overlays. Compositing is also used, with compiz
and Beryl
compositing window manager
s being the most prominent examples since 2006. They are able to take advantage of OpenGL
(through glx extension) for 3D and 2D overlay visual effects. Other implementations like Metacity
and xfwm have been available since 2004.
The Amiga
supported a form of hardware compositing through the use of a coprocessor called copper
. The copper ran a very simple program ("copperlist") synchronized with the video-display hardware. The copper could execute a WAIT instruction which caused its program execution to pause until the video beam reached a particular position on the screen. Then the copper would execute the next instruction, a MOVE instruction, which would reprogram the display hardware's memory pointer. This would cause the display hardware to immediately begin reading from a new memory location and displaying the image held there. The copper could also change the video hardware's pixel clock, allowing the display of multiple resolutions simultaneously.
Some dedicated hardware overlay devices use embedded Linux
as an operating system, for example the video logger http://www.videovbox.co.uk by Racelogic uses a Texas Instruments
DM355 micro-controller to blend a frame buffer containing graphics onto a live video feed, and then store the result in MPEG-4
format on a flash card.
Computing
Computing is usually defined as the activity of using and improving computer hardware and software. It is the computer-specific part of information technology...
, hardware overlay, a type of video overlay, provides a method of rendering an image to a display screen with a dedicated memory buffer inside computer video
Video
Video is the technology of electronically capturing, recording, processing, storing, transmitting, and reconstructing a sequence of still images representing scenes in motion.- History :...
hardware
Hardware
Hardware is a general term for equipment such as keys, locks, hinges, latches, handles, wire, chains, plumbing supplies, tools, utensils, cutlery and machine parts. Household hardware is typically sold in hardware stores....
. The technique aims to improve the display of a fast-moving video image
Image
An image is an artifact, for example a two-dimensional picture, that has a similar appearance to some subject—usually a physical object or a person.-Characteristics:...
— such as a computer game, a DVD
DVD
A DVD is an optical disc storage media format, invented and developed by Philips, Sony, Toshiba, and Panasonic in 1995. DVDs offer higher storage capacity than Compact Discs while having the same dimensions....
, or the signal from a TV card. Most video cards manufactured since about 1998 and most media player
Portable media player
A portable media player or digital audio player, is a consumer electronics device that is capable of storing and playing digital media such as audio, images, video, documents, etc. the data is typically stored on a hard drive, microdrive, or flash memory. In contrast, analog portable audio...
s support hardware overlay.
Overview
The use of a hardware overlay is important for several reasons:- in a graphical user interfaceGraphical user interfaceIn computing, a graphical user interface is a type of user interface that allows users to interact with electronic devices with images rather than text commands. GUIs can be used in computers, hand-held devices such as MP3 players, portable media players or gaming devices, household appliances and...
(GUI) operating systemOperating systemAn operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
such as WindowsMicrosoft WindowsMicrosoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, one display-device can typically display multiple applications simultaneously. - consider how a display works without a hardware overlay. When each application draws to the screen, the operating system's graphical subsystem must constantly check to ensure that the objects being drawn appear on the appropriate location on the screen, and that they don't collide with overlapping and neighboring windows. The graphical subsystem must clipClipping (computer graphics)Any procedure which identifies that portion of a picture which is either inside or outside a picture is referred to as a clipping algorithm or clipping.The region against which an object is to be clipped is called clipping window.-Examples:...
objects while they are being drawn when a collision occurs. This constant checking and clipping ensures that different applications can cooperate with one another in sharing a display, but also consumes a significant proportion of computing power.
A computer draws on its display by writing a bitmap
Bitmap
In computer graphics, a bitmap or pixmap is a type of memory organization or image file format used to store digital images. The term bitmap comes from the computer programming terminology, meaning just a map of bits, a spatially mapped array of bits. Now, along with pixmap, it commonly refers to...
ped representation of the graphics into a special portion of its memory known as video memory
Video memory
Video memory is a term generally used in computers to describe some form of writable memory, usually RAM, dedicated to the purpose of holding the information necessary for a graphics card to drive a display device...
. Without any hardware overlays, only one chunk of video memory exists which all applications must share - and the location of a given application's video memory moves whenever the user changes the position of the application's window. With shared video memory, an application must constantly check that it is only writing to memory that belongs to that application.
When running a high-bandwidth video application such as a movie player or some games, the computing power and complexity needed to perform constant clipping and checking negatively impacts performance and compatibility. To escape these limitations, the hardware overlay was invented. In addition, the graphics processing unit
Graphics processing unit
A graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
(GPU) provides an efficient way to scale the video in size and often performs color-format conversions (such as MPEG-2's YCbCr
YCbCr
YCbCr or Y′CbCr, sometimes written or , is a family of color spaces used as a part of the color image pipeline in video and digital photography systems. Y′ is the luma component and CB and CR are the blue-difference and red-difference chroma components...
into RGB).
An application using a hardware overlay gets a completely separate section of video memory that belongs only to that application. Because nothing else uses it, the program never needs to waste resources in checking whether a given piece of the memory belongs to it, nor does it need to monitor whether the user moves the window and changes the location of the video memory. To get the image from the separate video memory to display in tandem with the remaining shared elements on the display, the graphical subsystem associates a certain attribute (for example, a particular color) as a "mask" for that overlay, which the graphics card understands to mean that it is to draw from the separate overlay buffer onto the screen. (This technique has become known as "chroma key
Chroma key
Chroma key compositing is a technique for compositing two images together. A color range in the top layer is made transparent, revealing another image behind. The chroma keying technique is commonly used in video production and post-production...
".)
As an example, assume that the graphical subsystem defines the color "purple" as the mask color that triggers the overlay to appear. An application that plays a DVD will draw a solid purple rectangle on the shared screen, and then "play" the DVD into the special region of memory dedicated to the overlay. The graphics card will render the DVD playback only inside the purple area. If another window moves over top of the purple area and obscures a part of it, then the graphics hardware performs the clipping by itself. In practice, the actual color "purple" is not used - rather, either a nondescript near-black color is used (as the system often has thousands of such shades to choose from), or the mask region is denoted using some similar masking method not involving colors.
Screenshots
As a consequence of hardware overlay use, a screenshotScreenshot
A screenshot , screen capture , screen dump, screengrab , or print screen is an image taken by a computer to record the visible items displayed on the monitor, television, or another visual output device...
program (for example, the one automatically built into Windows that activates when a user presses the PrtSc key) often does not capture the content appearing in the hardware overlay window. Rather, a blank region containing only the special mask color is captured. This is because the screen capture routine doesn't consider the special video memory regions dedicated to overlays - it simply captures the shared main screen as rendered by the software's graphical subsystem. Some Digital Rights Management
Digital rights management
Digital rights management is a class of access control technologies that are used by hardware manufacturers, publishers, copyright holders and individuals with the intent to limit the use of digital content and devices after sale. DRM is any technology that inhibits uses of digital content that...
schemes use hardware overlay to display protected content on the screen, taking advantage of this quirk to prevent the copying of protected documents by way of screen capture. Disabling the support for overlays causes the normal overlay-using software to fallback to the shared memory, thus enabling screenshot capture.
Secondary displays
Many graphics cards can support more than one monitor and/or a TV screen as output devices. Typically one of these output devices has to be declared the "primary" device, and only the primary device can display hardware overlays. There are exceptions: Intel writes in the FAQ for their Embedded Graphics Drivers that the overlay can be attached to either one of the displays but not to both (note: the Intel 945, G33-Q965 chipsets have dual hardware overlay, and are capable of good quality mpeg2 on secondary monitors when appropriate software is installed), and some newer Matrox graphics card support overlay on both displays (for example, the Parhelia Series).Secondary displays require both hardware and driver support; some graphics cards may support overlay on the second display while their drivers may not yet support it (note: graphics chipset driver bugs can cause most video formats apart from mpeg2 to work on both monitors, and mpeg2 only on the primary with most players).
Some users note that DVD movies display properly on a laptop screen but don't display on a TV connected to the laptop; in these cases it may be possible to designate the TV as the primary display. Sometimes, the use of hardware overlays may have to be disabled in the media player. However, some graphics cards have the option to completely redirect hardware overlay to the TV screen. In this case, starting a DVD player on the main screen with overlay enabled would result in video being displayed on the attached TV screen.
Implementations in various operating systems
Windows VistaWindows Vista
Windows Vista is an operating system released in several variations developed by Microsoft for use on personal computers, including home and business desktops, laptops, tablet PCs, and media center PCs...
's enhanced graphics capabilities replace the basic concept of hardware overlays with full hardware compositing
Compositing
Compositing is the combining of visual elements from separate sources into single images, often to create the illusion that all those elements are parts of the same scene. Live-action shooting for compositing is variously called "chroma key", "blue screen", "green screen" and other names. Today,...
for every application window running on the system, not just movie players or games, through the Desktop Window Manager
Desktop Window Manager
Desktop Window Manager is the window manager in Windows Vista and Windows 7 that enables the Windows Aero graphical user interface and visual theme. The Desktop Window Manager requires video cards supporting DirectX 9.0 and Shader Model 2.0. DWM is not included with Windows Vista Starter edition...
. Mac OS X
Mac OS X
Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
has used hardware compositing since the introduction of Quartz Extreme
Quartz Compositor
Quartz Compositor is the windowing system in Mac OS X. It is responsible for presenting and maintaining rasterized, rendered graphics from the rest of the Core Graphics framework and other renderers in the Quartz technologies family...
into Mac OS X 10.2
Mac OS X v10.2
Mac OS X version 10.2 "Jaguar" is the third major release of Mac OS X, Apple's desktop and server operating system. It superseded Mac OS X v10.1 code name Puma and preceded Mac OS X Panther...
. To improve performance, each program draws to its own independent memory buffer instead of to a slow graphical subsystem. (In Windows Vista, each hardware overlay is more correctly known as a Direct3D surface). Then the system's GPU assembles each of the windows into a single display screen in real time. With enhanced GPUs on the market capable of stunning 3D graphics (as a consequence of the video game industry), operating systems can apply impressive motion, scaling, and lighting effects to normal 2D windows.
In the X Window System
X Window System
The X window system is a computer software system and network protocol that provides a basis for graphical user interfaces and rich input device capability for networked computers...
, the windowing system of most Unix
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
operating systems, the XVideo extension can allow applications to employ hardware overlays. Compositing is also used, with compiz
Compiz
Compiz is one of the first compositing window managers for the X Window System that uses 3D graphics hardware to create fast compositing desktop effects for window management. The effects, such as a minimization effect and a cube workspace are implemented as loadable plugins...
and Beryl
Beryl (window manager)
Beryl was a compositing window manager for the X Window System which forked from Compiz in September 2006 and was re-merged in 2007, under the name of Compiz Fusion.-Origin:...
compositing window manager
Compositing window manager
A compositing window manager is a type of window manager. A window manager is software that draws a graphical user interface on a computer display – it positions windows, draws additional elements on windows , and controls how windows interact with each other, and with the rest of the desktop...
s being the most prominent examples since 2006. They are able to take advantage of OpenGL
OpenGL
OpenGL is a standard specification defining a cross-language, cross-platform API for writing applications that produce 2D and 3D computer graphics. The interface consists of over 250 different function calls which can be used to draw complex three-dimensional scenes from simple primitives. OpenGL...
(through glx extension) for 3D and 2D overlay visual effects. Other implementations like Metacity
Metacity
Metacity was the window manager used by default in the GNOME desktop environment until GNOME 3, where it was replaced by Mutter. The development of Metacity was started by Havoc Pennington and it is released under the GNU General Public License....
and xfwm have been available since 2004.
The Amiga
Amiga
The Amiga is a family of personal computers that was sold by Commodore in the 1980s and 1990s. The first model was launched in 1985 as a high-end home computer and became popular for its graphical, audio and multi-tasking abilities...
supported a form of hardware compositing through the use of a coprocessor called copper
Original Amiga chipset
The Original Chip Set was a chipset used in the earliest Commodore Amiga computers and defined the Amiga's graphics and sound capabilities...
. The copper ran a very simple program ("copperlist") synchronized with the video-display hardware. The copper could execute a WAIT instruction which caused its program execution to pause until the video beam reached a particular position on the screen. Then the copper would execute the next instruction, a MOVE instruction, which would reprogram the display hardware's memory pointer. This would cause the display hardware to immediately begin reading from a new memory location and displaying the image held there. The copper could also change the video hardware's pixel clock, allowing the display of multiple resolutions simultaneously.
Some dedicated hardware overlay devices use embedded Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
as an operating system, for example the video logger http://www.videovbox.co.uk by Racelogic uses a Texas Instruments
Texas Instruments
Texas Instruments Inc. , widely known as TI, is an American company based in Dallas, Texas, United States, which develops and commercializes semiconductor and computer technology...
DM355 micro-controller to blend a frame buffer containing graphics onto a live video feed, and then store the result in MPEG-4
MPEG-4
MPEG-4 is a method of defining compression of audio and visual digital data. It was introduced in late 1998 and designated a standard for a group of audio and video coding formats and related technology agreed upon by the ISO/IEC Moving Picture Experts Group under the formal standard ISO/IEC...
format on a flash card.