Volume rendering - AbsoluteAstronomy.com

Scientific visualization

Scientific visualization is an interdisciplinary branch of science according to Friendly "primarily concerned with the visualization of three-dimensional phenomena , where the emphasis is on realistic renderings of volumes, surfaces, illumination sources, and so forth, perhaps...

and computer graphics

Computer graphics

Computer graphics are graphics created using computers and, more generally, the representation and manipulation of image data by a computer with help from specialized software and hardware....

, volume rendering is a set of techniques used to display a 2D projection of a 3D discretely sampled

Sampling (signal processing)

In signal processing, sampling is the reduction of a continuous signal to a discrete signal. A common example is the conversion of a sound wave to a sequence of samples ....

data set

Data set

A data set is a collection of data, usually presented in tabular form. Each column represents a particular variable. Each row corresponds to a given member of the data set in question. Its values for each of the variables, such as height and weight of an object or values of random numbers. Each...

.

A typical 3D data set is a group of 2D slice images acquired by a
CT, MRI

Magnetic resonance imaging

Magnetic resonance imaging , nuclear magnetic resonance imaging , or magnetic resonance tomography is a medical imaging technique used in radiology to visualize detailed internal structures...

, or MicroCT

Microtomography

Microtomography , like tomography, uses x-rays to create cross-sections of a 3D-object that later can be used to recreate a virtual model without destroying the original model....

scanner

Image scanner

In computing, an image scanner—often abbreviated to just scanner—is a device that optically scans images, printed text, handwriting, or an object, and converts it to a digital image. Common examples found in offices are variations of the desktop scanner where the document is placed on a glass...

.
Usually these are acquired in a regular pattern (e.g., one slice every millimeter) and
usually have a regular number of image pixel

Pixel

In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....

s in a regular pattern. This is an example
of a regular volumetric grid, with each volume element, or voxel

Voxel

A voxel is a volume element, representing a value on a regular grid in three dimensional space. This is analogous to a pixel, which represents 2D image data in a bitmap...

represented by
a single value that is obtained by sampling the immediate area surrounding the voxel.

To render a 2D projection of the 3D data set, one first needs to define a camera in space
relative to the volume. Also, one needs to define the opacity

Opacity (optics)

Opacity is the measure of impenetrability to electromagnetic or other kinds of radiation, especially visible light. In radiative transfer, it describes the absorption and scattering of radiation in a medium, such as a plasma, dielectric, shielding material, glass, etc...

and color of every voxel.
This is usually defined using an RGBA

RGBA color space

RGBA stands for Red Green Blue Alpha. While it is sometimes described as a color space, it is actually simply a use of the RGB color model, with extra information. The color is RGB, and may belong to any RGB color space, but an integral alpha value as invented by Catmull and Smith between 1971 and...

(for red, green, blue, alpha) transfer function
that defines the RGBA value for every possible voxel value.

For example, a volume may be viewed by extracting isosurface

Isosurface

An isosurface is a three-dimensional analog of an isoline. It is a surface that represents points of a constant value within a volume of space; in other words, it is a level set of a continuous function whose domain is 3D-space.Isosurfaces are normally displayed using computer graphics, and are...

s (surfaces of equal values) from the volume and rendering them as polygonal meshes

Polygon mesh

A polygon mesh or unstructured grid is a collection of vertices, edges and faces that defines the shape of a polyhedral object in 3D computer graphics and solid modeling...

or by rendering the volume directly as a block of data. The marching cubes

Marching cubes

Marching cubes is a computer graphics algorithm, published in the 1987 SIGGRAPH proceedings by Lorensen and Cline, for extracting a polygonal mesh of an isosurface from a three-dimensional scalar field...

algorithm is a common technique for extracting an isosurface from volume data. Direct volume rendering is a computationally intensive task that may be performed in several ways.

Direct volume rendering

A direct volume renderer requires every sample value to be mapped to opacity and a color. This is done with a "transfer function" which can be a simple ramp, a piecewise linear function or an arbitrary table. Once converted to an RGBA (for red, green, blue, alpha) value, the composed RGBA result is projected on correspondent pixel of the frame buffer. The way this is done depends on the rendering technique.

A combination of these techniques is possible. For instance, a shear warp implementation could use texturing hardware to draw the aligned slices in the off-screen buffer.

Volume ray casting

The technique of volume ray casting can be derived directly from the rendering equation

Rendering equation

In computer graphics, the rendering equation is an integral equation in which the equilibrium radiance leaving a point is given as the sum of emitted plus reflected radiance under a geometric optics approximation. It was simultaneously introduced into computer graphics by David Immel et al. and...

. It provides results of very high quality, usually considered to provide the best image quality. Volume ray casting is classified as image based volume rendering technique, as the computation emanates from the output image, not the input volume data as is the case with object based techniques. In this technique, a ray is generated for each desired image pixel. Using a simple camera model, the ray starts at the center of projection of the camera (usually the eye point) and passes through the image pixel on the imaginary image plane floating in between the camera and the volume to be rendered. The ray is clipped by the boundaries of the volume in order to save time. Then the ray is sampled at regular or adaptive intervals throughout the volume. The data is interpolated at each sample point, the transfer function applied to form an RGBA sample, the sample is composited onto the accumulated RGBA of the ray, and the process repeated until the ray exits the volume. The RGBA color is converted to an RGB color and deposited in the corresponding image pixel. The process is repeated for every pixel on the screen to form the completed image.

Splatting

This is a technique which trades quality for speed. Here, every volume element is splatted

Texture Splatting

In computer graphics, texture splatting is a method for combining different textures. The method works by applying an alphamap to the higher levels, revealing the layers underneath where the alphamap is partially or completely transparent...

, as Lee Westover said, like a snow ball, on to the viewing surface in back to front order. These splats are rendered as disks whose properties (color and transparency) vary diametrically in normal (Gaussian) manner. Flat disks and those with other kinds of property distribution are also used depending on the application.

Shear warp

The shear warp approach to volume rendering was developed by Cameron and Undrill, popularized by Philippe Lacroute and Marc Levoy

Marc Levoy

Marc Levoy is a computer graphics researcher and Professor of Computer Science and Electrical Engineering at Stanford University. He is noted for pioneering work in volume rendering....

. In this technique, the viewing transformation is transformed such that the nearest face of the volume becomes axis aligned with an off-screen image buffer with a fixed scale of voxels to pixels. The volume is then rendered into this buffer using the far more favorable memory alignment and fixed scaling and blending factors. Once all slices of the volume have been rendered, the buffer is then warped into the desired orientation and scaled in the displayed image.

This technique is relatively fast in software at the cost of less accurate sampling and potentially worse image quality compared to ray casting. There is memory overhead for storing multiple copies of the volume, for the ability to have near axis aligned volumes. This overhead can be mitigated using run length encoding.

Texture mapping

Many 3D graphics systems use texture mapping

Texture mapping

Texture mapping is a method for adding detail, surface texture , or color to a computer-generated graphic or 3D model. Its application to 3D graphics was pioneered by Dr Edwin Catmull in his Ph.D. thesis of 1974.-Texture mapping:...

to apply images, or textures, to geometric objects. Commodity PC graphics cards are fast at texturing and can efficiently render slices of a 3D volume, with real time interaction capabilities. Workstation

Workstation

A workstation is a high-end microcomputer designed for technical or scientific applications. Intended primarily to be used by one person at a time, they are commonly connected to a local area network and run multi-user operating systems...

GPUs are even faster, and are the basis for much of the production volume visualization used in medical imaging

Medical imaging

Medical imaging is the technique and process used to create images of the human body for clinical purposes or medical science...

, oil and gas, and other markets (2007). In earlier years, dedicated 3D texture mapping systems were used on graphics systems such as Silicon Graphics

Silicon Graphics

Silicon Graphics, Inc. was a manufacturer of high-performance computing solutions, including computer hardware and software, founded in 1981 by Jim Clark...

InfiniteReality

InfiniteReality refers to a 3D graphics hardware architecture and a family of graphics systems that implemented the aforementioned hardware architecture that was developed and manufactured by Silicon Graphics from 1996 to 2005...

, HP

Hewlett-Packard

Hewlett-Packard Company or HP is an American multinational information technology corporation headquartered in Palo Alto, California, USA that provides products, technologies, softwares, solutions and services to consumers, small- and medium-sized businesses and large enterprises, including...

Visualize FX graphics accelerator, and others. This technique was first described by Bill Hibbard

Bill Hibbard

Bill Hibbard is a scientist at the University of Wisconsin–Madison Space Science and Engineering Center working on visualization and machine intelligence. He is principal author of the Vis5D, Cave5D and VisAD open source visualization systems. Vis5D was the first open source 3-D visualization...

and Dave Santek.

These slices can either be aligned with the volume and rendered at an angle to the viewer, or aligned with the viewing plane and sampled from unaligned slices through the volume. Graphics hardware support for 3D textures is needed for the second technique.

Volume aligned texturing produces images of reasonable quality, though there is often a noticeable transition when the volume is rotated.

Maximum intensity projection

As opposed to direct volume rendering, which requires every sample value to be mapped to opacity and a color, maximum intensity projection picks out and projects

3D projection

3D projection is any method of mapping three-dimensional points to a two-dimensional plane. As most current methods for displaying graphical data are based on planar two-dimensional media, the use of this type of projection is widespread, especially in computer graphics, engineering and drafting.-...

only the voxel

Voxel

A voxel is a volume element, representing a value on a regular grid in three dimensional space. This is analogous to a pixel, which represents 2D image data in a bitmap...

s with maximum intensity that fall in the way of parallel rays traced from the viewpoint to the plane of projection.

This technique is computationally fast, but the 2D results do not provide a good sense of depth of the original data. To improve the sense of 3D, animations are usually rendered of several MIP frames in which the viewpoint is slightly changed from one to the other, thus creating the illusion of rotation

Rotation

A rotation is a circular movement of an object around a center of rotation. A three-dimensional object rotates always around an imaginary line called a rotation axis. If the axis is within the body, and passes through its center of mass the body is said to rotate upon itself, or spin. A rotation...

. This helps the viewer's perception

Perception

Perception is the process of attaining awareness or understanding of the environment by organizing and interpreting sensory information. All perception involves signals in the nervous system, which in turn result from physical stimulation of the sense organs...

to find the relative 3D positions of the object components. This implies that two MIP renderings from opposite viewpoints are symmetrical images, which makes it impossible for the viewer to distinguish between left or right, front or back and even if the object is rotating clockwise

Clockwise

Circular motion can occur in two possible directions. A clockwise motion is one that proceeds in the same direction as a clock's hands: from the top to the right, then down and then to the left, and back to the top...

or counterclockwise even though it makes a significant difference for the volume being rendered.

MIP imaging was invented for use in nuclear medicine

Nuclear medicine

In nuclear medicine procedures, elemental radionuclides are combined with other elements to form chemical compounds, or else combined with existing pharmaceutical compounds, to form radiopharmaceuticals. These radiopharmaceuticals, once administered to the patient, can localize to specific organs...

by Jerold Wallis, MD, in 1988, and subsequently published in IEEE Transactions in Medical Imaging.

Surprisingly, an easy improvement to MIP is Local maximum intensity projection

Local maximum intensity projection

In scientific visualization, a Local Maximum Intensity projection is a volume rendering method for 3D data, that is proposed as an improvement to the Maximum Intensity Projection...

. In this technique we don't take the global maximum value, but the first maximum value that is above a certain threshold. Because - in general - we can terminate the ray earlier this technique is faster and also gives somehow better results as it approximates occlusion.

Hardware-accelerated volume rendering

Due to the extremely parallel nature of direct volume rendering, special purpose volume rendering hardware was a rich research topic before GPU volume rendering became fast enough. The most widely cited technology was VolumePro, which used high memory bandwidth and brute force to render using the ray casting algorithm.

A recently exploited technique to accelerate traditional volume rendering algorithms such as ray-casting is the use of modern graphics cards. Starting with the programmable pixel shaders, people recognized the power of parallel operations on multiple pixels and began to perform general-purpose computing on (the) graphics processing units

GPGPU

General-purpose computing on graphics processing units is the technique of using a GPU, which typically handles computation only for computer graphics, to perform computation in applications traditionally handled by the CPU...

(GPGPU). The pixel shaders are able to read and write randomly from video memory and perform some basic mathematical and logical calculations. These SIMD

SIMD

Single instruction, multiple data , is a class of parallel computers in Flynn's taxonomy. It describes computers with multiple processing elements that perform the same operation on multiple data simultaneously...

processors were used to perform general calculations such as rendering polygons and signal processing. In recent GPU generations, the pixel shaders now are able to function as MIMD

MIMD

In computing, MIMD is a technique employed to achieve parallelism. Machines using MIMD have a number of processors that function asynchronously and independently. At any time, different processors may be executing different instructions on different pieces of data...

processors (now able to independently branch) utilizing up to 1 GB of texture memory with floating point formats. With such power, virtually any algorithm with steps that can be performed in parallel, such as volume ray casting

Volume ray casting

Volume ray casting, sometimes called volumetric ray casting, volumetric ray tracing, or volume ray marching, is an image-based volume rendering technique. It computes 2D images from 3D volumetric data sets...

or tomographic reconstruction

Tomographic reconstruction

The mathematical basis for tomographic imaging was laid down by Johann Radon. It is applied in computed tomography to obtain cross-sectional images of patients...

, can be performed with tremendous acceleration. The programmable pixel shaders can be used to simulate variations in the characteristics of lighting, shadow, reflection, emissive color and so forth. Such simulations can be written using high level shading language

Shading language

A shading language is a special programming language adapted to map on shader programming. Those kind of languages usually have special data types like color and normal...

Optimization techniques

The primary goal of optimization is to skip as much of the volume as possible. A typical medical data set can be 1 GB in size. To render that at 30 frame/s requires an extremely fast memory bus. Skipping voxels means that less information needs to be processed.

Empty space skipping

Often, a volume rendering system will have a system for identifying regions of the volume containing no visible material. This information can be used to avoid rendering these transparent regions.

Early ray termination

This is a technique used when the volume is rendered in front to back order. For a ray through a pixel, once sufficient dense material has been encountered, further samples will make no significant contribution to the pixel and so may be neglected

Antiportal

In computer-generated imagery and real-time 3D computer graphics, antiportal rendering is a way to reduce overdraw , and in this way to optimize draw speed...

Octree and BSP space subdivision

The use of hierarchical structures such as octree

Octree

An octree is a tree data structure in which each internal node has exactly eight children. Octrees are most often used to partition a three dimensional space by recursively subdividing it into eight octants. Octrees are the three-dimensional analog of quadtrees. The name is formed from oct + tree,...

and BSP

Binary space partitioning

In computer science, binary space partitioning is a method for recursively subdividing a space into convex sets by hyperplanes. This subdivision gives rise to a representation of the scene by means of a tree data structure known as a BSP tree.Originally, this approach was proposed in 3D computer...

-tree could be very helpful for both compression of volume data and speed optimization of volumetric ray casting process.

Volume segmentation

By sectioning out large portions of the volume that one considers uninteresting before rendering, the amount of calculations that have to be made by ray casting or texture blending can be significantly reduced. This reduction can be as much as from O(n) to O(log n) for n sequentially indexed voxels. Volume segmentation also has significant performance benefits for other ray tracing algorithms.

Multiple and adaptive resolution representation

By representing less interesting regions of the volume in a coarser resolution, the data input overhead can be reduced. On closer observation, the data in these regions can be populated either by reading from memory or disk, or by interpolation

Interpolation

In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points....

. The coarser resolution volume is resampled to a smaller size in the same way as a 2D mipmap image is created from the original. These smaller volume are also used by themselves while rotating the volume to a new orientation.

Pre-integrated volume rendering

Pre-integrated volume rendering is a method that can reduce sampling artifacts by pre-computing much of the required data. It is especially useful in hardware-accelerated applications because it improves quality without a large performance impact. Unlike most other optimizations, this does not skip voxels. Rather it reduces the number of samples needed to accurately display a region of voxels. The idea is to render the intervals between the samples instead of the samples themselves. This technique captures rapidly changing material, for example the transition from muscle to bone with much less computation.

Image-based meshing

Image-based meshing is the automated process of creating computer models for computational fluid dynamics and finite element analysis from 3D image data...

is the automated process of creating computer models from 3D image data (such as MRI, CT, Industrial CT

Industrial CT Scanning

Industrial CT scanning is a process which uses X-ray equipment to produce three-dimensional representations of components both externally and internally. Industrial CT scanning has been used in many areas of industry for internal inspection of components...

or microtomography

Microtomography

Microtomography , like tomography, uses x-rays to create cross-sections of a 3D-object that later can be used to recreate a virtual model without destroying the original model....

) for computational analysis and design, e.g. CAD, CFD, and FEA.

Temporal reuse of voxels

For a complete display view, only one voxel per pixel (the front one) is required to be shown (although more can be used for smoothing the image), if animation is needed, the front voxels to be shown can be cached and their location relative to the camera can be recalculated as it moves. Where display voxels become too far apart to cover all the pixels, new front voxels can be found by ray casting or similar, and where two voxels are in one pixel, the front one can be kept.

External links

Vaa3D is a free and open source 3D visualization software suite designed for large-scale volumetric image rendering and analysis.
The Visualization Toolkit – VTK is a free open source toolkit, which implements several CPU and GPU volume rendering methods in C++ using OpenGL, and can be used from python, tcl and java wrappers.
Linderdaum Engine is a free open source rendering engine with GPU raycasting capabilities.
Open Inventor by VSG is a commercial 3D graphics toolkit for developing scientific and industrial applications.
Avizo
Avizo (software)
Avizo is a general-purpose commercial software application for scientific and industrial data visualization and analysis....

is a general-purpose commercial software application for scientific and industrial data visualization and analysis.
HiVE (Hierarchical Volume Encapsulation) – open source library developed for storage and processing of large, boundless, sparse volumes

Direct volume rendering

Volume ray casting

Splatting

Shear warp

Texture mapping

Maximum intensity projection

Hardware-accelerated volume rendering

Optimization techniques

Empty space skipping

Early ray termination

Octree and BSP space subdivision

Volume segmentation

Multiple and adaptive resolution representation

Pre-integrated volume rendering

Image-based meshing

Temporal reuse of voxels

External links

See also