Camera resectioning
Encyclopedia
Camera resectioning is the process of finding the true parameters of the camera that produced a given photograph or video. Usually, the camera parameters are represented in a 3 × 4 matrix called the camera matrix
.
This process is often called camera calibration, but "camera calibration" can also mean photometric camera calibration.
to represent a 2D point position in Pixel coordinates. 
is used to represent a 3D point position in World coordinates.Note: they were expressed in augmented notation of Homogeneous coordinates
which is most common notation in robotics
and rigid body
transforms.
Referring to the pinhole camera model
, a camera matrix
is used to denote a projective mapping from World coordinates to Pixel coordinates.
The intrinsic matrix containing 5 intrinsic parameters. These parameters encompass focal length
, image format, and principal point
.
The parameters
and 
represent focal length in terms of pixels, where 
and 
are the scale factor
s relating pixels to distance.
represents the skew coefficient between the x and the y axis, and is often 0.
and 
represent the principal point, which would be ideally in the centre of the image.
Nonlinear intrinsic parameters such as lens distortion are also important although they cannot be included in the linear camera model described by the intrinsic parameter matrix. Many modern camera calibration algorithms estimate these intrinsic parameters as well.
are the extrinsic parameters which denote the coordinate system transformations from 3D world coordinates to 3D camera coordinates. Equivalently, the extrinsic parameters define the position of the camera center and the camera's heading in world coordinates. 
is not the position of the camera. It is the position of the origin of the world coordinate system expressed in coordinates of the camera-centered coordinate system. The positon, 
, of the camera expressed in world coordinates is 
(since 
is a rotation matrix).
Camera calibration is often used as an early stage in computer vision
and especially in the field of augmented reality
.
When a camera
is used, light from the environment is focused on an image plane and captured. This process reduces the dimensions of the data taken in by the camera from three to two (light from a 3D scene is stored on a 2D image). Each pixel
on the image plane therefore corresponds to a shaft of light from the original scene. Camera resectioning determines which incoming light is associated with each pixel on the resulting image. In an ideal pinhole camera
, a simple projection matrix is enough to do this. With more complex camera systems, errors resulting from misaligned lenses and deformations in their structures can result in more complex distortions in the final image.
The camera projection matrix is derived from the intrinsic and extrinsic parameters of the camera, and is often represented by the series of transformations; e.g., a matrix of camera intrinsic parameters, a 3 × 3 rotation matrix, and a translation vector. The camera projection matrix can be used to associate points in a camera's image space with locations in 3D world space.
Camera resectioning is often used in the application of stereo vision where the camera projection matrices of two cameras are used to calculate the 3D world coordinates of a point viewed by both cameras.
Some people call this camera calibration, but many restrict the term camera calibration for the estimation of internal or intrinsic parameters only.
that maps points 
on a "probe plane" 
to points 
on the image.
The circular points
lie on both our probe plane 
and on the absolute conic 
. Lying on 
of course means they are also projected onto the image of the absolute conic (IAC) 
, thus 
and 
. The circular points project as
.
We can actually ignore
while substituting our new expression for 
as follows:
X-ray camera systems, consisting of the X-ray generating tube and a solid state detector can be modelled as pinhole camera systems, comprising 9 intrinsic and extrinsic camera parameters.
Intensity based registration based on an arbitrary X-ray image and a reference model (as a tomographic dataset) can then be used to determine the relative camera parameters without the need of a special calibration body or any ground-truth data.
Camera matrix
In computer vision a camera matrix or projection matrix is a 3 \times 4 matrix which describes the mapping of a pinhole camera from 3D points in the world to 2D points in an image....
.
This process is often called camera calibration, but "camera calibration" can also mean photometric camera calibration.
Parameters of camera model
Often, we use to represent a 2D point position in Pixel coordinates. is used to represent a 3D point position in World coordinates.Note: they were expressed in augmented notation of Homogeneous coordinatesHomogeneous coordinates
In mathematics, homogeneous coordinates, introduced by August Ferdinand Möbius in his 1827 work Der barycentrische Calcül, are a system of coordinates used in projective geometry much as Cartesian coordinates are used in Euclidean geometry. They have the advantage that the coordinates of points,...
which is most common notation in robotics
Robotics
Robotics is the branch of technology that deals with the design, construction, operation, structural disposition, manufacture and application of robots...
and rigid body
Rigid body
In physics, a rigid body is an idealization of a solid body of finite size in which deformation is neglected. In other words, the distance between any two given points of a rigid body remains constant in time regardless of external forces exerted on it...
transforms.
Referring to the pinhole camera model
Pinhole camera model
The pinhole camera model describes the mathematical relationship between the coordinates of a 3D point and its projection onto the image plane of an ideal pinhole camera, where the camera aperture is described as a point and no lenses are used to focus light...
, a camera matrix
Camera matrix
In computer vision a camera matrix or projection matrix is a 3 \times 4 matrix which describes the mapping of a pinhole camera from 3D points in the world to 2D points in an image....
is used to denote a projective mapping from World coordinates to Pixel coordinates.
Intrinsic parameters
The intrinsic matrix containing 5 intrinsic parameters. These parameters encompass focal length
Focal length
The focal length of an optical system is a measure of how strongly the system converges or diverges light. For an optical system in air, it is the distance over which initially collimated rays are brought to a focus...
, image format, and principal point
Principal Point
Principal Point is a prominent ice-covered headland lying 4 nautical miles east of Cape Errera and forming the southeast end of Wiencke Island, in the Palmer Archipelago. First charted by the French Antarctic Expedition under Charcot, 1903-05. The name, applied by the Argentine Antarctic...
.
The parameters
and represent focal length in terms of pixels, where and are the scale factorScale factor
A scale factor is a number which scales, or multiplies, some quantity. In the equation y=Cx, C is the scale factor for x. C is also the coefficient of x, and may be called the constant of proportionality of y to x...
s relating pixels to distance.
represents the skew coefficient between the x and the y axis, and is often 0. and represent the principal point, which would be ideally in the centre of the image.Nonlinear intrinsic parameters such as lens distortion are also important although they cannot be included in the linear camera model described by the intrinsic parameter matrix. Many modern camera calibration algorithms estimate these intrinsic parameters as well.
Extrinsic parameters
are the extrinsic parameters which denote the coordinate system transformations from 3D world coordinates to 3D camera coordinates. Equivalently, the extrinsic parameters define the position of the camera center and the camera's heading in world coordinates. is not the position of the camera. It is the position of the origin of the world coordinate system expressed in coordinates of the camera-centered coordinate system. The positon, , of the camera expressed in world coordinates is (since is a rotation matrix).Camera calibration is often used as an early stage in computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
and especially in the field of augmented reality
Augmented reality
Augmented reality is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is...
.
When a camera
Camera
A camera is a device that records and stores images. These images may be still photographs or moving images such as videos or movies. The term camera comes from the camera obscura , an early mechanism for projecting images...
is used, light from the environment is focused on an image plane and captured. This process reduces the dimensions of the data taken in by the camera from three to two (light from a 3D scene is stored on a 2D image). Each pixel
Pixel
In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....
on the image plane therefore corresponds to a shaft of light from the original scene. Camera resectioning determines which incoming light is associated with each pixel on the resulting image. In an ideal pinhole camera
Pinhole camera
A pinhole camera is a simple camera without a lens and with a single small aperture – effectively a light-proof box with a small hole in one side. Light from a scene passes through this single point and projects an inverted image on the opposite side of the box...
, a simple projection matrix is enough to do this. With more complex camera systems, errors resulting from misaligned lenses and deformations in their structures can result in more complex distortions in the final image.
The camera projection matrix is derived from the intrinsic and extrinsic parameters of the camera, and is often represented by the series of transformations; e.g., a matrix of camera intrinsic parameters, a 3 × 3 rotation matrix, and a translation vector. The camera projection matrix can be used to associate points in a camera's image space with locations in 3D world space.
Camera resectioning is often used in the application of stereo vision where the camera projection matrices of two cameras are used to calculate the 3D world coordinates of a point viewed by both cameras.
Some people call this camera calibration, but many restrict the term camera calibration for the estimation of internal or intrinsic parameters only.
Algorithms
There are many different approaches to calculate the intrinsic and extrinsic parameters for a specific camera setup.- Direct linear transformation (DLT) method
- A classical approach is "Roger Y. Tsai Algorithm".It is a 2-stage algorithm, calculating the pose (3D Orientation, and x-axis and y-axis translation) in first stage. In second stage it computes the focal length, distortion coefficients and the z-axis translation.
- Zhengyou Zhang's "a flexible new technique for camera calibration" based on a planar chess board. It is based on constrains on homographyHomographyHomography is a concept in the mathematical science of geometry.A homography is an invertible transformation from a projective space to itself that maps straight lines to straight lines...
Zhang's method
Zhang's camera calibration method employs abstract concepts like the image of the absolute conic and circular points.Derivation
Assume we have a homography that maps points on a "probe plane" to points on the image.The circular points
lie on both our probe plane and on the absolute conic . Lying on of course means they are also projected onto the image of the absolute conic (IAC) , thus and . The circular points project as.We can actually ignore
while substituting our new expression for as follows:Selby's method (for X-ray cameras)
Selby's camera calibration method addresses the auto-calibration of X-ray camera systems.X-ray camera systems, consisting of the X-ray generating tube and a solid state detector can be modelled as pinhole camera systems, comprising 9 intrinsic and extrinsic camera parameters.
Intensity based registration based on an arbitrary X-ray image and a reference model (as a tomographic dataset) can then be used to determine the relative camera parameters without the need of a special calibration body or any ground-truth data.
See also
- Augmented realityAugmented realityAugmented reality is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is...
- Augmented virtualityAugmented virtualityAugmented virtuality refers to the merging of real world objects into virtual worlds.As an intermediate case in the Virtuality Continuum, it refers to predominantly virtual spaces, where physical elements, e.g. physical objects or people, are dynamically integrated into, and can interact with the...
- Mixed realityMixed realityMixed reality refers to the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real time...
- Pinhole camera modelPinhole camera modelThe pinhole camera model describes the mathematical relationship between the coordinates of a 3D point and its projection onto the image plane of an ideal pinhole camera, where the camera aperture is described as a point and no lenses are used to focus light...
- Camera auto-calibrationCamera auto-calibrationCamera auto-calibration is the process of determining internal camera parameters directly from multiple uncalibrated images. In contrast to classic camera calibration, auto-calibration does not require any special calibration objects in the scene....
External links
- C++ Camera Calibration Toolbox with source code
- Camera Calibration Toolbox for Matlab
- Zhang's Camera Calibration Method with Software
- Zhang's Camera Calibration and Tsai's Calibration Softwares on LGPL licence
- Camera Calibration - Augmented reality lecture at TU Muenchen, Germany
- Tsai's Approach
- Camera calibration (using ARToolKitARToolKitARToolKit is a computer tracking library for creation of strong augmented reality applications that overlay virtual imagery on the real world. To do this, it uses video tracking capabilities that calculate the real camera position and orientation relative to square physical markers in real time...
) - A Four-step Camera Calibration Procedure with Implicit Image Correction
- Free software for distortion correction