NILFS
Encyclopedia
NILFS is a log-structured file system
implementation for Linux
. It is being developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories and released under the terms of the GNU General Public License
(GPL).
Version 2 of the filesystem, known as NILFS2, is included in Linux kernel
2.6.30. A separate, BSD licensed implementation, currently with read-only support, is included in NetBSD
.
technique known as "nothing in life is free", NILFS records all data in a continuous log-like format that is only appended to, never overwritten, an approach that is designed to reduce seek times, as well as minimize the kind of data loss
that occurs after a crash with conventional Linux filesystems. For example, data loss occurs on ext3
filesystems when the system crashes
during a write operation. When the system reboots
, the journal
notes that the write did not complete, and any partial data writes are lost.
Some filesystems, like the legacy UFS filesystem used by the Solaris operating system, provide a snapshot
feature that prevents such data loss, but filesystem operation must be suspended to use the feature, reducing performance. NILFS, in contrast, can "continuously and automatically [save] instantaneous states of the file system without interrupting service", according to NTT Labs.
The "instantaneous states" that NILFS continuously saves can actually be mounted, read-only, at the same time that the actual filesystem is mounted read-write — a capability useful for data recovery after hardware failures and other system crashes. The "lscp" (list checkpoint) command of an interactive NILFS "inspect" utility is first used to find the checkpoint's address, in this case "2048":
# inspect /dev/sda2
...
nilfs> listcp
1 6 Tue Jul 12 14:55:57 2005 MajorCP|LogiBegin|LogiEnd
2048 2352 Tue Jul 12 14:55:58 2005 MajorCP|LogiEnd
...
nilfs> quit
The checkpoint address is then used to mount the checkpoint:
# mount -t nilfs -r -o cp=2048 /dev/sda2 /nilfs-cp
# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 70332412 8044540 62283776 12% /nilfs
/dev/sda2 70332412 8044540 62283776 12% /nilfs-cp
Log-structured file system
A log-structured filesystem is a file system design first proposed in 1988 by John K. Ousterhout and Fred Douglis. Designed for high write throughput, all updates to data and metadata are written sequentially to a continuous stream, called a log...
implementation for Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
. It is being developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories and released under the terms of the GNU General Public License
GNU General Public License
The GNU General Public License is the most widely used free software license, originally written by Richard Stallman for the GNU Project....
(GPL).
Version 2 of the filesystem, known as NILFS2, is included in Linux kernel
Linux kernel
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
2.6.30. A separate, BSD licensed implementation, currently with read-only support, is included in NetBSD
NetBSD
NetBSD is a freely available open source version of the Berkeley Software Distribution Unix operating system. It was the second open source BSD descendant to be formally released, after 386BSD, and continues to be actively developed. The NetBSD project is primarily focused on high quality design,...
.
Features
Using a copy-on-writeCopy-on-write
Copy-on-write is an optimization strategy used in computer programming. The fundamental idea is that if multiple callers ask for resources which are initially indistinguishable, they can all be given pointers to the same resource...
technique known as "nothing in life is free", NILFS records all data in a continuous log-like format that is only appended to, never overwritten, an approach that is designed to reduce seek times, as well as minimize the kind of data loss
Data loss
Data loss is an error condition in information systems in which information is destroyed by failures or neglect in storage, transmission, or processing. Information systems implement backup and disaster recovery equipment and processes to prevent data loss or restore lost data.Data loss is...
that occurs after a crash with conventional Linux filesystems. For example, data loss occurs on ext3
Ext3
The ext3 or third extended filesystem is a journaled file system that is commonly used by the Linux kernel. It is the default file system for many popular Linux distributions, including Debian...
filesystems when the system crashes
Crash (computing)
A crash in computing is a condition where a computer or a program, either an application or part of the operating system, ceases to function properly, often exiting after encountering errors. Often the offending program may appear to freeze or hang until a crash reporting service documents...
during a write operation. When the system reboots
Booting
In computing, booting is a process that begins when a user turns on a computer system and prepares the computer to perform its normal operations. On modern computers, this typically involves loading and starting an operating system. The boot sequence is the initial set of operations that the...
, the journal
Journal (computing)
In computer storage, a journal is a chronological record of data processing operations that may be used to construct or reinstate an historical or alternative version of a computer system or computer file....
notes that the write did not complete, and any partial data writes are lost.
Some filesystems, like the legacy UFS filesystem used by the Solaris operating system, provide a snapshot
Snapshot (computer storage)
In computer systems, a snapshot is the state of a system at a particular point in time. The term was coined as an analogy to that in photography. It can refer to an actual copy of the state of a system or to a capability provided by certain systems....
feature that prevents such data loss, but filesystem operation must be suspended to use the feature, reducing performance. NILFS, in contrast, can "continuously and automatically [save] instantaneous states of the file system without interrupting service", according to NTT Labs.
The "instantaneous states" that NILFS continuously saves can actually be mounted, read-only, at the same time that the actual filesystem is mounted read-write — a capability useful for data recovery after hardware failures and other system crashes. The "lscp" (list checkpoint) command of an interactive NILFS "inspect" utility is first used to find the checkpoint's address, in this case "2048":
# inspect /dev/sda2
...
nilfs> listcp
1 6 Tue Jul 12 14:55:57 2005 MajorCP|LogiBegin|LogiEnd
2048 2352 Tue Jul 12 14:55:58 2005 MajorCP|LogiEnd
...
nilfs> quit
The checkpoint address is then used to mount the checkpoint:
# mount -t nilfs -r -o cp=2048 /dev/sda2 /nilfs-cp
# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 70332412 8044540 62283776 12% /nilfs
/dev/sda2 70332412 8044540 62283776 12% /nilfs-cp
Additional features
- Fast write and recovery times
- Minimal damage to file data and system consistency on hardware failure
- 32-bit checksumChecksumA checksum or hash sum is a fixed-size datum computed from an arbitrary block of digital data for the purpose of detecting accidental errors that may have been introduced during its transmission or storage. The integrity of the data can be checked at any later time by recomputing the checksum and...
s (CRC32) on data and metadata for integrity assurance (per block group, in segment summary) - Correctly ordered data and meta-data writes
- Redundant superblockSuperblockSuperblock may refer to:* A type of city block that is much larger than a traditional city block* A segment of metadata describing the file system on a block device...
- 32-bit checksum
- File and inode blocks are managed by a B-treeB-treeIn computer science, a B-tree is a tree data structure that keeps data sorted and allows searches, sequential access, insertions, and deletions in logarithmic time. The B-tree is a generalization of a binary search tree in that a node can have more than two children...
structure - Internal data is processed in 64-bit wide word size
- Can create and store huge files (8 EiBExbibyteThe exbibyte is a standards-based binary multiple of the byte, a unit of digital information storage. The exbibyte unit symbol is EiB....
) - Block sizes smaller than page size (e.g. 1 KB or 2 KB)
- Online filesystem resizing
See also
- ZFSZFSIn computing, ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include data integrity verification against data corruption modes , support for high storage capacities, integration of the concepts of filesystem and volume management,...
- BtrfsBtrfsBtrfs is a GPL-licensed copy-on-write file system for Linux.Development began at Oracle Corporation in 2007....
- List of file systems
- Comparison of file systemsComparison of file systems-General information:-Limits:-Metadata:-Features:-Allocation and layout policies:-Supporting operating systems:-See also:* Comparison of archive formats* Comparison of file archivers* List of archive formats* List of file archivers...
- Log-structured File System (BSD)
- Sprite operating systemSprite operating systemSprite was an experimental Unix-like distributed operating system developed at the University of California, Berkeley by John Ousterhout's research group between 1984 and 1992. Its notable features included support for single system image on computer clusters and for the introduction of the...