Real-time computer graphics
Encyclopedia
Real-time computer graphics is the subfield of computer graphics
focused on producing and analyzing images in real time
. The term is most often used in reference to interactive 3D computer graphics
, typically using a GPU
, with video games the most noticeable users. The term can also refer to anything from rendering an application
's GUI
to real-time image processing
and image analysis
.
Although computers have been known from the beginning to be capable of generating 2D images involving simple lines, images and polygons in real-time (e.g. Bressenham's line drawing algorithm
), the creation of 3D computer graphics and the speed necessary for generating fast, good quality 3D images onto a display screen has always been a daunting task for traditional Von Neumann architecture
-based systems. The rest of this article concentrates on this widely-accepted aspect of real-time graphics rather than expanding on the principles of real-time 2D computer graphics.
. How fast these images or frames are generated in a given second determines the method's real-timeliness.
One interesting aspect of real-time computer graphics is the way in which it differs from traditional off-line rendering systems (and hence, these are the non-real-time graphics systems); non-real-time graphics typically rely on ray-tracing where the expensive operation of tracing rays from the camera to the world is allowed and can take as much as hours or even days for a single frame. On the other hand, in the case of real-time graphics, the system has less than 1/30th of a second per image. In order to do that, the current systems cannot afford shooting millions or even billions of rays; instead, they rely on the technique of z-buffer triangle rasterization. In this technique, every object is decomposed into individual primitives — the most popular and common one is the triangle. These triangles are then 'drawn' or rendered onto the screen one by one. Each of these triangles get positioned, rotated and scaled on the screen and a special hardware (or in the case of an emulator, the software rasterizer) called rasterizer generates the pixels inside each of these triangles. These triangles are then decomposed into further smaller atomic units called pixels (or in computer graphics terminology, aptly called fragments)
that are suitable for displaying on a display screen. The pixels are then drawn on the screen using a certain color; current systems are capable of deciding the color that results in these triangles — for e.g. a texture
can be used to 'paint' onto a triangle, which is simply deciding what color to output at each pixel based on a stored picture; or in a more complex case, at each pixel, one can compute if a certain light is being seen or not resulting in very good shadows (using a technique called shadow mapping
).
Thus, real-time graphics is oriented toward providing as much performance as possible for the lowest quality possible for a given class of hardware. Most video-games and simulators fall in this category of real-time graphics. As mentioned above, real-time graphics is currently possible due to the significant recent advancements in these special hardware components called graphics processing units (GPUs). These GPUs are capable of handling millions of triangles per frame and within each such triangle capable of handling millions or even billions of pixels (i.e. generating these pixel colors). Current DirectX 11/OpenGL
4.x class hardware is capable of generating complex effects on the fly (i.e. in real-time) such as shadow volume
s, motion blur
ring, real-time triangle generation among many others. Although the gap in quality between real-time graphics and traditional off-line graphics is narrowing, the accuracy is still far below the accuracy of offline rendering .
In the case of real-time interactive computer graphics, usually a user is in control of what is about to be drawn on the display screen; the user typically uses an input device to provide feedback to the system — for example, wanting to move a character on the screen — and the system decides the next frame based on this particular instance of action. Usually the display is far slower (in terms of the number of frames per second) in responsiveness than the input device (in terms of the input device's response time measured in ms
). In a way this is justified due to the immense difference between the infinitesimal response time generated by a human-being's motion and the very slow perspective speed of the human-visual system
; this results in significant advancements in computer graphics, whereas the advancements in input devices typically take a much longer time to achieve the same state of fundamental advancement (e.g., the current Wii controller), as these input devices have to be extremely fast in order to be usable.
Another important factor controlling real-time computer graphics is the combination of physics
and animation
. These techniques largely dictate what is to be drawn on the screen — or more precisely, where to draw certain objects (deciding their position) on the screen. These techniques imitate the behavior (the temporal dimension
, not the spatial dimensions) seen in real-world to a degree that is far more realistic than and compensating computer-graphics' degree of realism.
Computer graphics
Computer graphics are graphics created using computers and, more generally, the representation and manipulation of image data by a computer with help from specialized software and hardware....
focused on producing and analyzing images in real time
Real-time computing
In computer science, real-time computing , or reactive computing, is the study of hardware and software systems that are subject to a "real-time constraint"— e.g. operational deadlines from event to system response. Real-time programs must guarantee response within strict time constraints...
. The term is most often used in reference to interactive 3D computer graphics
3D computer graphics
3D computer graphics are graphics that use a three-dimensional representation of geometric data that is stored in the computer for the purposes of performing calculations and rendering 2D images...
, typically using a GPU
Graphics processing unit
A graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
, with video games the most noticeable users. The term can also refer to anything from rendering an application
Application software
Application software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...
's GUI
Graphical user interface
In computing, a graphical user interface is a type of user interface that allows users to interact with electronic devices with images rather than text commands. GUIs can be used in computers, hand-held devices such as MP3 players, portable media players or gaming devices, household appliances and...
to real-time image processing
Image processing
In electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...
and image analysis
Image analysis
Image analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques...
.
Although computers have been known from the beginning to be capable of generating 2D images involving simple lines, images and polygons in real-time (e.g. Bressenham's line drawing algorithm
Bresenham's line algorithm
The Bresenham line algorithm is an algorithm which determines which points in an n-dimensional raster should be plotted in order to form a close approximation to a straight line between two given points...
), the creation of 3D computer graphics and the speed necessary for generating fast, good quality 3D images onto a display screen has always been a daunting task for traditional Von Neumann architecture
Von Neumann architecture
The term Von Neumann architecture, aka the Von Neumann model, derives from a computer architecture proposal by the mathematician and early computer scientist John von Neumann and others, dated June 30, 1945, entitled First Draft of a Report on the EDVAC...
-based systems. The rest of this article concentrates on this widely-accepted aspect of real-time graphics rather than expanding on the principles of real-time 2D computer graphics.
Principles of real-time 3D computer graphics
The goal of computer graphics is to generate a computer generated image using certain desired metrics. This image is often called a frameFilm frame
In filmmaking, video production, animation, and related fields, a film frame or video frame is one of the many still images which compose the complete moving picture...
. How fast these images or frames are generated in a given second determines the method's real-timeliness.
One interesting aspect of real-time computer graphics is the way in which it differs from traditional off-line rendering systems (and hence, these are the non-real-time graphics systems); non-real-time graphics typically rely on ray-tracing where the expensive operation of tracing rays from the camera to the world is allowed and can take as much as hours or even days for a single frame. On the other hand, in the case of real-time graphics, the system has less than 1/30th of a second per image. In order to do that, the current systems cannot afford shooting millions or even billions of rays; instead, they rely on the technique of z-buffer triangle rasterization. In this technique, every object is decomposed into individual primitives — the most popular and common one is the triangle. These triangles are then 'drawn' or rendered onto the screen one by one. Each of these triangles get positioned, rotated and scaled on the screen and a special hardware (or in the case of an emulator, the software rasterizer) called rasterizer generates the pixels inside each of these triangles. These triangles are then decomposed into further smaller atomic units called pixels (or in computer graphics terminology, aptly called fragments)
Fragment (computer graphics)
In computer graphics, a fragment is the data necessary to generate a single pixel's worth of a drawing primitive in the frame buffer.This data may include, but is not limited to:* raster position* depth...
that are suitable for displaying on a display screen. The pixels are then drawn on the screen using a certain color; current systems are capable of deciding the color that results in these triangles — for e.g. a texture
Texture mapping
Texture mapping is a method for adding detail, surface texture , or color to a computer-generated graphic or 3D model. Its application to 3D graphics was pioneered by Dr Edwin Catmull in his Ph.D. thesis of 1974.-Texture mapping:...
can be used to 'paint' onto a triangle, which is simply deciding what color to output at each pixel based on a stored picture; or in a more complex case, at each pixel, one can compute if a certain light is being seen or not resulting in very good shadows (using a technique called shadow mapping
Shadow mapping
Shadow mapping or projective shadowing is a process by which shadows are added to 3D computer graphics. This concept was introduced by Lance Williams in 1978, in a paper entitled "Casting curved shadows on curved surfaces"...
).
Thus, real-time graphics is oriented toward providing as much performance as possible for the lowest quality possible for a given class of hardware. Most video-games and simulators fall in this category of real-time graphics. As mentioned above, real-time graphics is currently possible due to the significant recent advancements in these special hardware components called graphics processing units (GPUs). These GPUs are capable of handling millions of triangles per frame and within each such triangle capable of handling millions or even billions of pixels (i.e. generating these pixel colors). Current DirectX 11/OpenGL
OpenGL
OpenGL is a standard specification defining a cross-language, cross-platform API for writing applications that produce 2D and 3D computer graphics. The interface consists of over 250 different function calls which can be used to draw complex three-dimensional scenes from simple primitives. OpenGL...
4.x class hardware is capable of generating complex effects on the fly (i.e. in real-time) such as shadow volume
Shadow volume
Shadow volume is a technique used in 3D computer graphics to add shadows to a rendered scene. They were first proposed by Frank Crow in 1977 as the geometry describing the 3D shape of the region occluded from a light source...
s, motion blur
Motion blur
Motion blur is the apparent streaking of rapidly moving objects in a still image or a sequence of images such as a movie or animation. It results when the image being recorded changes during the recording of a single frame, either due to rapid movement or long exposure.- Photography :When a camera...
ring, real-time triangle generation among many others. Although the gap in quality between real-time graphics and traditional off-line graphics is narrowing, the accuracy is still far below the accuracy of offline rendering .
Why real-time computer graphics?
Another interesting difference between real-time and non-real-time graphics is the interactivity desired in real-time graphics. Feedback is typically the main motivation for pushing real-time graphics to its furore. In cases like films, the director has the complete control and determinism of what has to be drawn on each frame, typically involving weeks or even years of decision-making involving a number of people.In the case of real-time interactive computer graphics, usually a user is in control of what is about to be drawn on the display screen; the user typically uses an input device to provide feedback to the system — for example, wanting to move a character on the screen — and the system decides the next frame based on this particular instance of action. Usually the display is far slower (in terms of the number of frames per second) in responsiveness than the input device (in terms of the input device's response time measured in ms
Millisecond
A millisecond is a thousandth of a second.10 milliseconds are called a centisecond....
). In a way this is justified due to the immense difference between the infinitesimal response time generated by a human-being's motion and the very slow perspective speed of the human-visual system
Persistence of vision
Persistence of vision is the phenomenon of the eye by which an afterimage is thought to persist for approximately one twenty-fifth of a second on the retina....
; this results in significant advancements in computer graphics, whereas the advancements in input devices typically take a much longer time to achieve the same state of fundamental advancement (e.g., the current Wii controller), as these input devices have to be extremely fast in order to be usable.
Another important factor controlling real-time computer graphics is the combination of physics
Game physics
Computer animation physics or game physics involves the introduction of the laws of physics into a simulation or game engine, particularly in 3D computer graphics, for the purpose of making the effects appear more real to the observer...
and animation
Computer animation
Computer animation is the process used for generating animated images by using computer graphics. The more general term computer generated imagery encompasses both static scenes and dynamic images, while computer animation only refers to moving images....
. These techniques largely dictate what is to be drawn on the screen — or more precisely, where to draw certain objects (deciding their position) on the screen. These techniques imitate the behavior (the temporal dimension
Spacetime
In physics, spacetime is any mathematical model that combines space and time into a single continuum. Spacetime is usually interpreted with space as being three-dimensional and time playing the role of a fourth dimension that is of a different sort from the spatial dimensions...
, not the spatial dimensions) seen in real-world to a degree that is far more realistic than and compensating computer-graphics' degree of realism.
See also
- Augmented realityAugmented realityAugmented reality is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is...
- DemosceneDemosceneThe demoscene is a computer art subculture that specializes in producing demos, which are non-interactive audio-visual presentations that run in real-time on a computer...
- Mixed realityMixed realityMixed reality refers to the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real time...
- Optical feedbackOptical feedbackOptical feedback is the optical equivalent of acoustic feedback. A simple example is the feedback that occurs when a loop exists between an optical input, e.g., a video camera, and an optical output, e.g., a television screen or monitor...
- Video artVideo artVideo art is a type of art which relies on moving pictures and comprises video and/or audio data. . Video art came into existence during the 1960s and 1970s, is still widely practiced and has given rise to the widespread use of video installations...
External links
- Real-Time Rendering resources page - links to many useful resources for 3D computer graphics
- RTR Portal - a trimmed-down "best of" set of links to resources