REDCODE
Encyclopedia
REDCODE RAW is a proprietary multimedia audio/video file format
File format
A file format is a particular way that information is encoded for storage in a computer file.Since a disk drive, or indeed any computer storage, can store only bits, the computer must have some way of converting information to 0s and 1s and vice-versa. There are different kinds of formats for...

 owned by Red Digital Cinema Camera Company and featuring lossy compression for video contents and lossless for audio contents. It is used as native recording format of the Red One 4K digital camera and recorded on proprietary Hard Disk Drives or CompactFlash
CompactFlash
CompactFlash is a mass storage device format used in portable electronic devices. Most CompactFlash devices contain flash memory in a standardized enclosure. The format was first specified and produced by SanDisk in 1994...

 cards, but can be extracted from there and be trivially handled in any file-based audio/video, IT or home environments.

Overview

All Red cameras record only in the REDCODE RAW
REDCODE
REDCODE RAW is a proprietary multimedia audio/video file format owned by Red Digital Cinema Camera Company and featuring lossy compression for video contents and lossless for audio contents...

 codec
Codec
A codec is a device or computer program capable of encoding or decoding a digital data stream or signal. The word codec is a portmanteau of "compressor-decompressor" or, more commonly, "coder-decoder"...

. This codec has a constant-bitrate wavelet codec with a compression ratio from 18:1 to 3:1 which allows raw Bayer sensor data to be compressed for on-camera recording. For the Red One two variants were offered initially, one with a maximum data rate of 28 MB/s (224 Mbit/s), and one with a maximum data rate of 36 MB/s (288 Mbit/s). Firmware upgrades allowed the camera to record an additional data rate of 42 MB/s (336 Mbit/s). These bit rates represent compression ratios of about 12:1, 9:1, and 8:1 respectively.

Unlike cameras that record RGB data, Red cameras record compressed bayer data. Recording compressed raw data allows white balance, gamma and other image processing parameters like sharpening to be set during post production. Adjusting these settings directly on camera does not impact the raw data that is actually recorded, but rather modifies metadata recorded alongside recorded images, which is accessible to software utilizing the camera's SDK. These meta-settings, called "looks" can be saved and applied to the camera's outputs to allow on-set crew to see an image more closely representing the director of photography's vision for the picture rather than the more "flat" look of the raw image data that comes from the sensor by default, while retaining the flexibility of the sensor data for post production.

Compression algorithm

Video streams within a R3D file are stored using a constant bit-rate, intraframe, wavelet-based codec
Codec
A codec is a device or computer program capable of encoding or decoding a digital data stream or signal. The word codec is a portmanteau of "compressor-decompressor" or, more commonly, "coder-decoder"...

 with four channels (1 for red, 2 for green, 1 for blue), which is a variant of the well-known JPEG2000 algorithm for digital still images.

Decompression is not the only coding the video stream is subject to in order to be playable: since the video data recorded from the camera sensor (the Mysterium sensor in the case of the Red One camera) are stored in the geometrical order they are transferred, a Bayer-patterned frame sequence is generated in the first place, which needs later transcoding (deBayering) to obtain a visible RGB stream. For that reason, R3D is a raw image format
RAW image format
A camera raw image file contains minimally processed data from the image sensor of either a digital camera, image scanner, or motion picture film scanner. Raw files are so named because they are not yet processed and therefore are not ready to be printed or edited with a bitmap graphics editor...

 for video. A similar file format (or wrapper) is the Adobe CinemaDNG
CinemaDNG
CinemaDNG is the result of an Adobe-led initiative to define an industry-wide open file format for digital cinema files. CinemaDNG caters for sets of movie clips, each of which is a sequence of raw video images, accompanied by audio and metadata. CinemaDNG supports stereoscopic cameras and multiple...

.

Audio streams (mono, stereo or 4-channels) are coded, uncompressed, in plain 48 kHz, 24-bit PCM.

REDCODE is a mathematically lossy codec, meaning that decompression does not fully restore the original image data captured by the camera. Red claims the codec is "visually lossless", suggesting that the information loss is not visible to the naked eye when images are viewed. However, sample images detailing noticeable artifacts have been posted on the manufacturer's forum. Because Redcode is a wavelet
Wavelet
A wavelet is a wave-like oscillation with an amplitude that starts out at zero, increases, and then decreases back to zero. It can typically be visualized as a "brief oscillation" like one might see recorded by a seismograph or heart monitor. Generally, wavelets are purposefully crafted to have...

 codec, similar to CineForm RAW and JPEG2000, the nature of these artifacts is different from "blocking", characteristic of traditional compression algorithms.

Naming convention

R3D files, as generated by RED's digital cameras, have a symbolic naming convention whose aim is to both uniquely and unambiguously reference each clip/take which has ever been shot every day by each camera, as well as aiding during the online editing
Online editing
Online editing is an older post-production linear video editing process that is performed in the final stage of a video production. It occurs after offline editing...

 and post-production phases to discern which reel, camera and shooting date combinations that clip "belongs" to.

RED Digital Magazines

R3D files are first arranged into former-level folders, called RED Digital Magazines (RDMs); a RDM folder is created for each storage medium initialized (i.e. soft-formatted
Disk formatting
Disk formatting is the process of preparing a hard disk drive or flexible disk medium for data storage. In some cases, the formatting operation may also create one or more new file systems...

) and used by a RED digital camera and for every day of shooting (according to the camera's own internal clock).
The RDM directory associated to camera (cam.) X (where X ranges from A to Z), equivalent of a traditional 3-digit reel number NNN, shot on the ddth (2-digit) day of the mmth (2-digit) month of any year is named X NNN_dd mm HH.RDM. In this case HH is just a hexadecimal
Hexadecimal
In mathematics and computer science, hexadecimal is a positional numeral system with a radix, or base, of 16. It uses sixteen distinct symbols, most often the symbols 0–9 to represent values zero to nine, and A, B, C, D, E, F to represent values ten to fifteen...

 hash code to make that clip universally unique.

RED Digital Clip

Inside each RDM, lower-level subfolders are generated for each take shot by the camera (i.e. sub-directory creation is triggered by the camera's record start/stop button): those are the
RED Digital Clipss (RDCs). Takes are chronologically referenced by an increasing number within each RDM such that, apart from the hash code (which is RDM-independent), the TTTth (3-digit) take from the above RDM is named X NNN_C TTT_dd mm HH.RDC.

R3D digital contents

Each take is stored in one R3D stream, by one or more R3D file with the same name as the RDC containing it, but with a .R3D file extension. Since R3D files are limited to a maximum size of 2GiB
Gibibyte
The gibibyte is a standards-based binary multiple of the byte, a unit of digital information storage. The gibibyte unit symbol is GiB....

, upon reaching this limit, a new R3D file (with an appended _ppp tail for the pppth, 3-digit part) is generated within the same folder, whose contents are the raw continuation of the former one.

Other files

A RDC folder always stores, apart from the R3D files of that specific take, four wrapper
Wrapper
Wrapper generally refers to a type of packaging, such as a flat sheet made out of paper, cellophane or plastic to enclose an object.Wrapper may also refer to:* Wrapper , a woman's garment which is worn over nightwear or lingerie...

 QuickTime
QuickTime
QuickTime is an extensible proprietary multimedia framework developed by Apple Inc., capable of handling various formats of digital video, picture, sound, panoramic images, and interactivity. The classic version of QuickTime is available for Windows XP and later, as well as Mac OS X Leopard and...

 files. These are just metadata containers without stream contents (they are a few kilobytes large) and must reside together with associated R3D file(s) in the same folder, otherwise they are useless. Once opened by a QuickTime Player utility with a R3D codec installed/binded, they point to the R3D file and allow normal playback of the latter file(s) as if they were normal compressed-video files. The four QuickTime proxies also automatically set the playback frame size to either full-resolution (_F filename suffix), halved, one-fourth or one-eighth resolutions (_H, _M and _P suffixes respectively).

There is no real need of these four QuickTime containers though, if the R3D files are to be processed by software natively supporting the REDCODE codec.

Other than that, additional files may be present inside a RDC folder, as generated by either the camera and intermediate software as well. Among these there might be content-indexing ASCII
ASCII
The American Standard Code for Information Interchange is a character-encoding scheme based on the ordering of the English alphabet. ASCII codes represent text in computers, communications equipment, and other devices that use text...

 or XML
XML
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....

 files, as well as RSX files (same name as the RDC with .rsx extension), automatically generated by Red's proprietary software and keeping shooting-time and offline colour-correction / pre-grading information. Such metadata may be later used for either later previewing, offline editing and DI conforming.

R3D metadata file format specifications

Apart from the audio/video codec itself, which is proprietary and closed-source but made public via a SDK licensed by RED, some details about the container and metadata within R3D files generated by a Red One camera have been partially reverse-engineered by several, independent projects.

First of all, metadata are recorded at both the beginning and ending of the R3D data stream, that is as the file header in the first R3D file, as well as a file footer in the last R3D file making the stream.

The file format's magic number
Magic number (programming)
In computer programming, the term magic number has multiple meanings. It could refer to one or more of the following:* A constant numerical or text value used to identify a file format or protocol; for files, see List of file signatures...

 is "016016116DRED1", which also defines the beginning of the file header on the first R3D file. The stream tag "REO" marks the end of the whole R3D stream (the end of the clip recording, 52 bytes from the EOF
EOF
EOF may refer to:*End-of-file, the computing term for an end-of-file condition or its tangible indication*Empirical orthogonal functions, a statistical technique for simplifying a dataset*Enterprise Objects Framework, a product from Apple Computer...

), i.e. the file footer of the last R3D file. The header stores a priori information on the clip like camera/storage settings/serials, firmware versions, start TimeCode
Time code
A timecode is a sequence of numeric codes generated at regular intervals by a timing system.- Video and film timecode :...

s (read below), plus frame-, audio- and file-format settings, including colorimetry as well as on-camera colour-correction parameters. The footer stores a fortiori information instead like, most notably, the overall clip duration in frames; from the latter and the knowledge of the record frame-rate, the clip's ending TimeCodes can be easily recovered.

A R3D file is internally organized into substreams, mainly containing small chunks of audio or video contents (as well as other metadata). These chunks are sequentially stored within the R3D file and initiated by specific tags (REDV for video chunks, REDA for audio chunks) for synchronization purposes during file playback. The initial video chunk in a R3D stream (the first REDV chunk in the first R3D file of a RDC) also happens to include the file header itself.

Most notable metadata are recovered at absolute positions within the header, according to this table (offsets are expressed in 0-based bytes from the start of the first R3D file):
Offset Type Field name Description
14 Word Sample rate Rate of audio samples (Hz
HZ
Hz is the International Standard symbol for Hertz, the unit of frequencyHZ may also stand for:* Habitable zone, the distance from a star where a planet can maintain Earth-like life* Hamilton Zoo, in New Zealand...

)
54 Word Width Clip's original frame width (pixels)
60 Word Height Clip's original frame height (pixels)
62 Word Framerate's # of frames Read next field
66 Word Framerate's # of seconds Framerate being measured in frames per second, it is computable dividing the above field by this one.
67 32 bytes Filename Original R3D file name as created by the camera.


Within the file header, each metadata starts with a 16-bit identifier, which is a "1016" character followed by the metadata's ordinal number ("0016" being the first one). Metadata are typically written in order, although not all are consecutively defined (e.g. those whose ID starts with "10016" refer to TimeCode-based chrono-timing, those starting with "10316" are information about recording media, and so on). This format is reminiscent of JPEG
JPEG
In computing, JPEG . The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and image quality. JPEG typically achieves 10:1 compression with little perceptible loss in image quality....

 standard, used for both JFIF and Exif metadata encoding, as well as binary internal tagging used for TIFF file format's metadata.

Metadata are written as either plain ASCII
ASCII
The American Standard Code for Information Interchange is a character-encoding scheme based on the ordering of the English alphabet. ASCII codes represent text in computers, communications equipment, and other devices that use text...

 strings, words, IEEE floating-point values, or other proprietary codings; strings can be either fixed- or variable-length (metadata IDs allow the header's elastic syntax), but they end in both cases with either a NULL
Null character
The null character , abbreviated NUL, is a control character with the value zero.It is present in many character sets, including ISO/IEC 646 , the C0 control code, the Universal Character Set , and EBCDIC...

 ("0016") byte or "000F16".

The following table contains some of the reverse-engineered metadata within a R3D file header. First column contains the (decimal) metadata ID: for example, metadata ID #42 (hexadecimal
Hexadecimal
In mathematics and computer science, hexadecimal is a positional numeral system with a radix, or base, of 16. It uses sixteen distinct symbols, most often the symbols 0–9 to represent values zero to nine, and A, B, C, D, E, F to represent values ten to fifteen...

 "2A16") is coded, within the file header, just after the string "102A16".
Metadata ID Length Field name Description
0 11 bytes Start EdgeCode Start EdgeCode, i.e. clip TimeCode
Time code
A timecode is a sequence of numeric codes generated at regular intervals by a timing system.- Video and film timecode :...

, as hh:mm:ss:ff
1 11 bytes Start TimeCode Start Time-of-Day TimeCode
Time code
A timecode is a sequence of numeric codes generated at regular intervals by a timing system.- Video and film timecode :...

 (ToD) of the clip, as hh:mm:ss:ff
2 14 bytes Date #1 Auxiliary date, coded as YYYY_MM_DD_timezone
3 14 bytes Date #2 Auxiliary date (as ID #2)
4 14 bytes Date #3 Auxiliary date (as ID #2)
5 14 bytes Record date/time Record date and time, coded as YYYYMMDDhhmmss
6 8 + 24 bytes Camera model + S/N Camera model name + serial number
25 1 byte Camera number Camera abbreviated name (ASCII
ASCII
The American Standard Code for Information Interchange is a character-encoding scheme based on the ordering of the English alphabet. ASCII codes represent text in computers, communications equipment, and other devices that use text...

 A through Z)
26 3 bytes Reel number Digital Reel (REDMag) number as 1-based, 3-digit string
26 3 bytes Reel number Take/clip number within a REDMag, as 1-based, 3-digit string
35 8 bytes Original shooting date Original shooting date in YYYYmmdd format.
36 6 bytes Original shooting time Original shooting time in HHMMSS format.
37 variable Camera firmware Camera firmware string, usually coded in major/minor/revision/Build hierarchy, as MM.m.R#BB (Build being usually the firmware's major version).
41 11 bytes Reel-start ToD TimeCode Starting ToD TimeCode of the REDMag, coded as hh:mm:ss:ff.
42 16 bytes Storage name Storage medium Name (should begin with RED if a RED-proprietary storage).
48 8 bytes Storage up-date Date in which the storage medium (REDMag) was formatted or re-initialized, coded as YYYYMMDD.
49 6 bytes Storage up-time ToD in which the storage medium (REDMag) was formatted or re-initialized, coded as hhmmss.
50 20 bytes (?) Storage S/N Storage medium Serial Number.
51 10 bytes (?) Storage Model Storage medium Model.
54 variable Aspect ratio Name of the frame's aspect ratio (either FULL, 1.85, 2.35,16:9, etc.).

From the file footer (i.e. the final part of the last R3D file of a clip) the number of frames can be simply recovered as being a word (16 bits) 30 bytes from EOF
EOF
EOF may refer to:*End-of-file, the computing term for an end-of-file condition or its tangible indication*Empirical orthogonal functions, a statistical technique for simplifying a dataset*Enterprise Objects Framework, a product from Apple Computer...

.

Workflow

A consequence of the Red's raw-based workflow is that footage must be processed through a demosaicing
Demosaicing
A demosaicing algorithm is a digital image process used to reconstruct a full color image from the incomplete color samples output from an image sensor overlaid with a color filter array...

 algorithm before it can be viewed. Red has provided a QuickTime
QuickTime
QuickTime is an extensible proprietary multimedia framework developed by Apple Inc., capable of handling various formats of digital video, picture, sound, panoramic images, and interactivity. The classic version of QuickTime is available for Windows XP and later, as well as Mac OS X Leopard and...

 component which allows a fast demosaic to occur in real time so the footage can be used in applications that support QuickTime without transcoding. Higher-quality output can be achieved by transcoding the footage through Red's RedCine-X or RedCine-X Pro desktop software. Using the Red Rocket card allows for real-time processing. Currently the QuickTime component is only available for Intel-based Mac OS X and not Windows.

Playing back the high-resolution 4K files directly is extremely processor-intensive and outside the capabilities of most current computers. However, since REDCODE
REDCODE
REDCODE RAW is a proprietary multimedia audio/video file format owned by Red Digital Cinema Camera Company and featuring lossy compression for video contents and lossless for audio contents...

 is a wavelet codec, the files contain several lower resolution versions of the video; the codec uses each in sequence to build the next higher resolution version. That means a 4K file can supply 2K, 1K, or even 0.5K footage directly without decoding the full 4K resolution data followed by scaling. For QuickTime and other programs without support for this feature, reference movies can be made. These are small files without video data that contain pointers into the original video file. For example, in Final Cut Pro, one may import QuickTime reference files that only have pointers to the parts of the 4K file that contain the lower-resolution version. This way work is done with speedy low-resolution video without need for a separate low-resolution copy.

Application compatibility

Apple's Final Cut Pro
Final Cut Pro
Final Cut Pro is a non-linear video editing software developed by Macromedia Inc. and then Apple Inc. The most recent version, Final Cut Pro X, runs on Mac personal computers powered by Mac OS X version 10.6.7 or later and using Intel processors...

 can use REDCODE
REDCODE
REDCODE RAW is a proprietary multimedia audio/video file format owned by Red Digital Cinema Camera Company and featuring lossy compression for video contents and lossless for audio contents...

 files (via Quicktime reference movies) when the Red QuickTime codecs are installed. Assimilate Scratch was one of the earliest applications to support the raw camera data off the natively generated REDCODE files, and has continued to do so ever since.

Adobe
Adobe
Adobe is a natural building material made from sand, clay, water, and some kind of fibrous or organic material , which the builders shape into bricks using frames and dry in the sun. Adobe buildings are similar to cob and mudbrick buildings. Adobe structures are extremely durable, and account for...

's video products (Premiere and After Effects
Adobe After Effects
Adobe After Effects is a digital motion graphics and compositing software published by Adobe Systems, used in the post-production process of filmmaking and television production. Its main uses are the origination of 2D and 2.5D animation, visual effects compositing and finishing...

) have been able to work with the native REDCODE
REDCODE
REDCODE RAW is a proprietary multimedia audio/video file format owned by Red Digital Cinema Camera Company and featuring lossy compression for video contents and lossless for audio contents...

 files (.R3D file extension) as of version CS4, and the plug-ins to do so are officially supported by Red.

Sony
Sony
, commonly referred to as Sony, is a Japanese multinational conglomerate corporation headquartered in Minato, Tokyo, Japan and the world's fifth largest media conglomerate measured by revenues....

's video editing product Vegas Pro
Sony Vegas
Sony Vegas is a professional video editing software package for non-linear editing systems originally published by Sonic Foundry, now owned and run by Sony Creative Software. Originally developed as an audio editor, it eventually developed into an NLE for video and audio from version 2.0...

 can natively edit .R3D files as of version 9.0, and allows full access to raw decode metadata with real-time preview.

Autodesk
Autodesk
Autodesk, Inc. is an American multinational corporation that focuses on 3D design software for use in the architecture, engineering, construction, manufacturing, media and entertainment industries. The company was founded in 1982 by John Walker, a coauthor of the first versions of the company's...

's 2010 releases of Smoke
IFF (software)
Inferno, Flame and Flint is a series of compositing and visual effects applications made by Autodesk Media and Entertainment. They share the same software base, but differ mainly in hardware configuration. Traditionally Inferno ran on the SGI Onyx series, while Flame and Flint ran on the SGI...

 and Flame
IFF (software)
Inferno, Flame and Flint is a series of compositing and visual effects applications made by Autodesk Media and Entertainment. They share the same software base, but differ mainly in hardware configuration. Traditionally Inferno ran on the SGI Onyx series, while Flame and Flint ran on the SGI...

 have a tool, WireTap Central, that converts .R3D files into a native media form. The 2011 releases of Smoke and Flame have native R3D import.

Avid Media Composer v5 has native REDCODE support via AMA (Avid Media Access). AMA allows for the direct linking to the R3D files and immediate editing of the sources. Avid Media Composer also allows direction interaction on a per clip or batch process for the color metadata. RLS, RSX, and RMD files are all supported. The RedRocket card is also supported for accelerated transcoding of the R3D files into Avid DNxHD and other editing codecs.

Avid DS v10.1.1 has native REDCODE import and linking support. Avid also has a tool called MetaFuze that will convert .R3D files into a native Avid MXF
MXF
Material eXchange Format is a container format for professional digital video and audio media defined by a set of SMPTE standards.- A brief summary of MXF :...

 files for use in Media Composer and Symphony (v4 and earlier). Media Composer v5 and Symphony v5 have AMA support for REDCODE, allowing for direct editing R3D media. Avid DS and MetaFuze also support use of the RedRocket card.

Eyeon Fusion
Eyeon Fusion
Fusion is an image compositing software program created by eyeon Software Inc. It is typically used to create visual effects and digital compositing for film, HD and commercials...

 v6.0 and later includes native .R3D support.

Nuke
Nuke (software)
Nuke is a node-based compositor produced by The Foundry, and used for film and television post-production. Nuke is cross-platform, and is available for Microsoft Windows, Mac OS X, and Linux...

5.2v2 (The Foundry) includes updated .R3D support (SDK v2).

External links

The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
 
x
OK