HPFS
Encyclopedia
HPFS or High Performance File System is a file system
created specifically for the OS/2
operating system
to improve upon the limitations of the FAT
file system. It was written by Gordon Letwin
and others at Microsoft
and added to OS/2 version 1.2, at that time still a joint undertaking of Microsoft and IBM.
Among its improvements are:
HPFS also can keep 64 KiB
of metadata ("extended attributes") per file.
IBM offers two kind of IFS
drivers for this file system:
HPFS386's cache is limited by the amount of available memory in OS/2's system memory arena and was implemented in 32-bit assembler
. HPFS386 is a ring 0
driver (allowing it direct hardware access and direct interaction with the kernel) with built-in SMB networking properties that are utilizable by various server daemons, whereas HPFS is a ring 3 driver. Thus, HPFS386 is faster than HPFS and highly optimized for server applications. It is also highly tunable by experienced administrators.
Though IBM still had rights to HPFS, their agreement with Microsoft to continue licensing the HPFS386 version is contingent upon them paying Microsoft a licensing fee for each copy sold. This was a result from the Microsoft and IBM collaboration that both IBM and Microsoft had right to use Windows and OS/2 technology. Microsoft used HPFS in Windows NT.
Because of the Microsoft dependence, limited partition size, file size limit of 2GiB and the long disk check times after a crash, IBM ported the journaling file system
JFS to OS/2 as a substitute.
DOS and Linux support HPFS via third-party drivers. Windows NT
versions 3.51 and earlier had native support for HPFS.
and its successors Windows 98
, Windows Me
can read/write HPFS only when mapped via a network share, but cannot read it from a local disk. They listed the NTFS
partitions of networked computers as "HPFS", because NTFS and HPFS share the same filesystem identification number in the partition table.
Windows NT 3.1
and 3.5
have native read/write support for local disks and can even be installed onto an HPFS partition. This is because NT was originally going to be a version of OS/2
.
Windows NT 3.51
can also read and write from local HPFS formatted drives. However, Microsoft discouraged using HPFS in Windows NT 4 and in subsequent versions despite upgrades to NT 4.1 operating satisfactorily with servers pre-formatted with HPFS. Microsoft even removed the ability of NT 3.51 to format an HPFS file system. Starting with Windows NT 4 the filesystem driver pinball.sys enabling the read/write access is not included in a default installation anymore. Later Windows versions do not ship with this driver.
Microsoft retained rights to OS/2 technologies, including the HPFS filesystem, after they ceased collaboration with IBM
. Since Windows NT 3.1 was designed for more rigorous (enterprise-class) use than previous versions of Windows, it included support for HPFS (and NTFS) giving it a larger storage capacity than FAT filesystems. However, since HPFS lacks a journal
, any recovery after an unexpected shutdown or other error state takes progressively longer as the filesystem grows. A utility such as CHKDSK
would need to scan each entry in the filesystem to ensure no errors are present, a problem which is vastly reduced on NTFS where the journal is simply replayed.
File system
A file system is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device which contain it. A file system organizes data in an efficient manner and is tuned to the...
created specifically for the OS/2
OS/2
OS/2 is a computer operating system, initially created by Microsoft and IBM, then later developed by IBM exclusively. The name stands for "Operating System/2," because it was introduced as part of the same generation change release as IBM's "Personal System/2 " line of second-generation personal...
operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
to improve upon the limitations of the FAT
File Allocation Table
File Allocation Table is a computer file system architecture now widely used on many computer systems and most memory cards, such as those used with digital cameras. FAT file systems are commonly found on floppy disks, flash memory cards, digital cameras, and many other portable devices because of...
file system. It was written by Gordon Letwin
Gordon Letwin
James Gordon Letwin is an American software developer and one of the original twelve Microsoft employees.Prior to joining Microsoft, he worked for Heathkit, on HDOS and Benton Harbor Basic....
and others at Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
and added to OS/2 version 1.2, at that time still a joint undertaking of Microsoft and IBM.
Among its improvements are:
- support for mixed case file names, in different code pages
- support for long file names (255 characters as opposed to FAT's 8+3 characters)
- more efficient use of disk space (files are not stored using multiple-sector clusters but on a per-sector basis)
- an internal architecture that keeps related items close to each other on the disk volume
- less fragmentationFile system fragmentationIn computing, file system fragmentation, sometimes called file system aging, is the inability of a file system to lay out related data sequentially , an inherent phenomenon in storage-backed file systems that allow in-place modification of their contents. It is a special case of data fragmentation...
of data - extent-based space allocation
- separate datestamps for last modification, last access, and creation (as opposed to FAT's last modification-only datestamp)
- a B+ treeB+ treeIn computer science, a B+ tree or B plus tree is a type of tree which represents sorted data in a way that allows for efficient insertion, retrieval and removal of records, each of which is identified by a key. It is a dynamic, multilevel index, with maximum and minimum bounds on the number of...
structure for directories - root directory located at the mid-point, rather than beginning of the disk, for faster average access
HPFS also can keep 64 KiB
Kibibyte
The kibibyte is a multiple of the unit byte for quantities of digital information. The binary prefix kibi means 1024; therefore, 1 kibibyte is . The unit symbol for the kibibyte is KiB. The unit was established by the International Electrotechnical Commission in 1999 and has been accepted for use...
of metadata ("extended attributes") per file.
IBM offers two kind of IFS
Installable File System
The Installable File System is a filesystem API in IBM OS/2 and Microsoft Windows that enables the operating system to recognize and load drivers for file systems...
drivers for this file system:
- the standard one with a cache limited to 2 MiB
- HPFS386 provided with certain server versions of OS/2, or as added component for the server versions that did not come with it
HPFS386's cache is limited by the amount of available memory in OS/2's system memory arena and was implemented in 32-bit assembler
Assembly language
An assembly language is a low-level programming language for computers, microprocessors, microcontrollers, and other programmable devices. It implements a symbolic representation of the machine codes and other constants needed to program a given CPU architecture...
. HPFS386 is a ring 0
Ring (computer security)
In computer science, hierarchical protection domains, often called protection rings, are a mechanism to protect data and functionality from faults and malicious behaviour . This approach is diametrically opposite to that of capability-based security.Computer operating systems provide different...
driver (allowing it direct hardware access and direct interaction with the kernel) with built-in SMB networking properties that are utilizable by various server daemons, whereas HPFS is a ring 3 driver. Thus, HPFS386 is faster than HPFS and highly optimized for server applications. It is also highly tunable by experienced administrators.
Though IBM still had rights to HPFS, their agreement with Microsoft to continue licensing the HPFS386 version is contingent upon them paying Microsoft a licensing fee for each copy sold. This was a result from the Microsoft and IBM collaboration that both IBM and Microsoft had right to use Windows and OS/2 technology. Microsoft used HPFS in Windows NT.
Because of the Microsoft dependence, limited partition size, file size limit of 2GiB and the long disk check times after a crash, IBM ported the journaling file system
Journaling file system
A journaling file system is a file system that keeps track of the changes that will be made in a journal before committing them to the main file system...
JFS to OS/2 as a substitute.
DOS and Linux support HPFS via third-party drivers. Windows NT
Windows NT
Windows NT is a family of operating systems produced by Microsoft, the first version of which was released in July 1993. It was a powerful high-level-language-based, processor-independent, multiprocessing, multiuser operating system with features comparable to Unix. It was intended to complement...
versions 3.51 and earlier had native support for HPFS.
Windows Native Support
Windows 95Windows 95
Windows 95 is a consumer-oriented graphical user interface-based operating system. It was released on August 24, 1995 by Microsoft, and was a significant progression from the company's previous Windows products...
and its successors Windows 98
Windows 98
Windows 98 is a graphical operating system by Microsoft. It is the second major release in the Windows 9x line of operating systems. It was released to manufacturing on 15 May 1998 and to retail on 25 June 1998. Windows 98 is the successor to Windows 95. Like its predecessor, it is a hybrid...
, Windows Me
Windows Me
Windows Millennium Edition, or Windows Me , is a graphical operating system released on September 14, 2000 by Microsoft, and was the last operating system released in the Windows 9x series. Support for Windows Me ended on July 11, 2006....
can read/write HPFS only when mapped via a network share, but cannot read it from a local disk. They listed the NTFS
NTFS
NTFS is the standard file system of Windows NT, including its later versions Windows 2000, Windows XP, Windows Server 2003, Windows Server 2008, Windows Vista, and Windows 7....
partitions of networked computers as "HPFS", because NTFS and HPFS share the same filesystem identification number in the partition table.
Windows NT 3.1
Windows NT 3.1
Windows NT 3.1 is the first release of Microsoft's Windows NT line of server and business desktop operating systems, and was released to manufacturing on 27 July 1993. The version number was chosen to match the one of Windows 3.1, the then-latest operating environment from Microsoft, on account of...
and 3.5
Windows NT 3.5
Windows NT 3.5 is the second release of the Microsoft Windows NT operating system. It was released on 21 September 1994.One of the primary goals during Windows NT 3.5's development was to increase the speed of the operating system; as a result, the project was given the codename "Daytona" in...
have native read/write support for local disks and can even be installed onto an HPFS partition. This is because NT was originally going to be a version of OS/2
OS/2
OS/2 is a computer operating system, initially created by Microsoft and IBM, then later developed by IBM exclusively. The name stands for "Operating System/2," because it was introduced as part of the same generation change release as IBM's "Personal System/2 " line of second-generation personal...
.
Windows NT 3.51
Windows NT 3.51
Windows NT 3.51 is the third release of Microsoft's Windows NT line of operating systems. It was released on 30 May 1995, nine months after Windows NT 3.5. The release provided two notable feature improvements; firstly NT 3.51 was the first of a short-lived outing of Microsoft Windows on the...
can also read and write from local HPFS formatted drives. However, Microsoft discouraged using HPFS in Windows NT 4 and in subsequent versions despite upgrades to NT 4.1 operating satisfactorily with servers pre-formatted with HPFS. Microsoft even removed the ability of NT 3.51 to format an HPFS file system. Starting with Windows NT 4 the filesystem driver pinball.sys enabling the read/write access is not included in a default installation anymore. Later Windows versions do not ship with this driver.
Microsoft retained rights to OS/2 technologies, including the HPFS filesystem, after they ceased collaboration with IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
. Since Windows NT 3.1 was designed for more rigorous (enterprise-class) use than previous versions of Windows, it included support for HPFS (and NTFS) giving it a larger storage capacity than FAT filesystems. However, since HPFS lacks a journal
Journaling file system
A journaling file system is a file system that keeps track of the changes that will be made in a journal before committing them to the main file system...
, any recovery after an unexpected shutdown or other error state takes progressively longer as the filesystem grows. A utility such as CHKDSK
CHKDSK
CHKDSK is a command on computers running DOS, OS/2 and Microsoft Windows operating systems that displays the file system integrity status of hard disks and floppy disk and can fix logical file system errors. It is similar to the fsck command in Unix.The command is implemented as an executable...
would need to scan each entry in the filesystem to ensure no errors are present, a problem which is vastly reduced on NTFS where the journal is simply replayed.