IOPS
Encyclopedia
IOPS is a common performance measurement used to benchmark
computer storage
devices like hard disk drives (HDD), solid state drives (SSD), and storage area networks
(SAN). As with any benchmark, IOPS numbers published by storage device manufacturers do not guarantee real-world application performance.
IOPS can be measured with applications, such as Iometer
(originally developed by Intel), as well as IOzone
and FIO and is primarily used with servers
to find the best storage configuration.
The specific number of IOPS possible in any system configuration will vary greatly, depending upon the variables the tester enters into the program, including the balance of read and write operations, the mix of sequential
and random
access patterns, the number of worker threads
and queue depth, as well as the data block sizes. There are other factors which can also affect the IOPS results including the system setup, storage drivers, OS background operations, etc. Also, when testing SSDs in particular, there are preconditioning considerations that must be taken into account.
The most common performance characteristics measured are sequential and random operations. Sequential operations access locations on the storage device in a contiguous manner and are generally associated with large data transfer sizes, e.g., 128 KB
. Random operations access locations on the storage device in a non-contiguous manner and are generally associated with small data transfer sizes, e.g., 4 KB.
The most common performance characteristics are as follows:
For HDDs and similar electromechanical storage devices, the random IOPS numbers are primarily dependent upon the storage device's random seek time, whereas for SSDs and similar solid state storage devices, the random IOPS numbers are primarily dependent upon the storage device's internal controller and memory interface speeds. On both types of storage devices the sequential IOPS numbers (especially when using a large block size) typically indicate the maximum sustained bandwidth that the storage device can handle. Often sequential IOPS are reported as a simple MB/s number as follows:
Some HDDs will improve in performance as the number of outstanding IO's (i.e. queue depth) increases. This is usually the result of more advanced controller logic on the drive performing command queuing and reordering commonly called either Tagged Command Queuing
(TCQ) or Native Command Queuing
(NCQ). Most commodity SATA
drives either cannot do this, or their implementation is so poor that no performance benefit can be seen. Enterprise class SATA drives, such as the Western Digital Raptor and Seagate Barracuda NL will improve by nearly 100% with deep queues. High-end SCSI
drives more commonly found in servers, generally show much greater improvement, with the Seagate
Savvio exceeding 400 IOPS—more than doubling its performance.
While traditional HDDs have about the same IOPS for read and write operations, most NAND flash-based
SSDs are much slower writing than reading due to the inability to rewrite directly into a previously written location forcing a procedure called garbage collection. This has caused hardware test sites to start to do IOPS testing, providing independently measured results.
Newer flash SSD drives such as the Intel X25-E have much higher IOPS than traditional hard disk drives. In a test done by Xssist, using IOmeter, 4 KB random transfers, 70/30 read/write ratio, queue depth 4, the IOPS delivered by the Intel X25-E 64 GB G1 started around 10000 IOPs, and dropped sharply after 8 minutes to 4000 IOPS, and continued to decrease gradually for the next 42 minutes. IOPS vary between 3000 to 4000 from around the 50th minutes onwards for the rest of the 8+ hours test run. Even with the drop in random IOPS after the 50th minute, the X25-E still has much higher IOPS compared to traditional hard disk drives. Some SSDs, including the OCZ RevoDrive 3 x2 PCIe using the SandForce
controller, have shown much higher sustained write performance that more closely matches the read speed.
Solid State Devices
Benchmark (computing)
In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it...
computer storage
Data storage device
thumb|200px|right|A reel-to-reel tape recorder .The magnetic tape is a data storage medium. The recorder is data storage equipment using a portable medium to store the data....
devices like hard disk drives (HDD), solid state drives (SSD), and storage area networks
Storage area network
A storage area network is a dedicated network that provides access to consolidated, block level data storage. SANs are primarily used to make storage devices, such as disk arrays, tape libraries, and optical jukeboxes, accessible to servers so that the devices appear like locally attached devices...
(SAN). As with any benchmark, IOPS numbers published by storage device manufacturers do not guarantee real-world application performance.
IOPS can be measured with applications, such as Iometer
Iometer
Iometer is an I/O subsystem measurement and characterization tool for single and clustered systems. It is used as a benchmark and troubleshooting tool and is easily configured to replicate the behaviour of many popular applications...
(originally developed by Intel), as well as IOzone
IOzone
IOzone is a file system benchmark utility. Originally made by William Norcott, further enhanced by Don Capps. Source code is available. It does mmap file I/O and uses POSIX Threads.It won the 2007 Infoworld Bossie Awards for Best file I/O tool....
and FIO and is primarily used with servers
Server (computing)
In the context of client-server architecture, a server is a computer program running to serve the requests of other programs, the "clients". Thus, the "server" performs some computational task on behalf of "clients"...
to find the best storage configuration.
The specific number of IOPS possible in any system configuration will vary greatly, depending upon the variables the tester enters into the program, including the balance of read and write operations, the mix of sequential
Sequential access
In computer science, sequential access means that a group of elements is accessed in a predetermined, ordered sequence. Sequential access is sometimes the only way of accessing the data, for example if it is on a tape...
and random
Random access
In computer science, random access is the ability to access an element at an arbitrary position in a sequence in equal time, independent of sequence size. The position is arbitrary in the sense that it is unpredictable, thus the use of the term "random" in "random access"...
access patterns, the number of worker threads
Thread (computer science)
In computer science, a thread of execution is the smallest unit of processing that can be scheduled by an operating system. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process...
and queue depth, as well as the data block sizes. There are other factors which can also affect the IOPS results including the system setup, storage drivers, OS background operations, etc. Also, when testing SSDs in particular, there are preconditioning considerations that must be taken into account.
Performance characteristics
Kilobyte
The kilobyte is a multiple of the unit byte for digital information. Although the prefix kilo- means 1000, the term kilobyte and symbol KB have historically been used to refer to either 1024 bytes or 1000 bytes, dependent upon context, in the fields of computer science and information...
. Random operations access locations on the storage device in a non-contiguous manner and are generally associated with small data transfer sizes, e.g., 4 KB.
The most common performance characteristics are as follows:
| Measurement | Description |
|---|---|
| Total IOPS | Total number of I/O operations per second (when performing a mix of read and write tests) |
| Random Read IOPS | Average number of random read I/O operations per second |
| Random Write IOPS | Average number of random write I/O operations per second |
| Sequential Read IOPS | Average number of sequential read I/O operations per second |
| Sequential Write IOPS | Average number of sequential write I/O operations per second |
For HDDs and similar electromechanical storage devices, the random IOPS numbers are primarily dependent upon the storage device's random seek time, whereas for SSDs and similar solid state storage devices, the random IOPS numbers are primarily dependent upon the storage device's internal controller and memory interface speeds. On both types of storage devices the sequential IOPS numbers (especially when using a large block size) typically indicate the maximum sustained bandwidth that the storage device can handle. Often sequential IOPS are reported as a simple MB/s number as follows:
(with the answer typically converted to MegabytesPerSec)
Some HDDs will improve in performance as the number of outstanding IO's (i.e. queue depth) increases. This is usually the result of more advanced controller logic on the drive performing command queuing and reordering commonly called either Tagged Command Queuing
Tagged Command Queuing
Tagged Command Queuing is a technology built into certain ATA and SCSI hard drives. It allows the operating system to send multiple read and write requests to a hard drive. ATA TCQ is not identical in function to the more efficient native command queuing used by SATA drives...
(TCQ) or Native Command Queuing
Native Command Queuing
Native Command Queuing is a technology designed to increase performance of SATA hard disks under certain conditions by allowing the individual hard disk to internally optimize the order in which received read and write commands are executed...
(NCQ). Most commodity SATA
Sata
Sata is a traditional dish from the Malaysian state of Terengganu, consisting of spiced fish meat wrapped in banana leaves and cooked on a grill.It is a type of Malaysian fish cake, or otak-otak...
drives either cannot do this, or their implementation is so poor that no performance benefit can be seen. Enterprise class SATA drives, such as the Western Digital Raptor and Seagate Barracuda NL will improve by nearly 100% with deep queues. High-end SCSI
SCSI
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, and electrical and optical interfaces. SCSI is most commonly used for hard disks and tape drives, but it...
drives more commonly found in servers, generally show much greater improvement, with the Seagate
Seagate Technology
Seagate Technology is one of the world's largest manufacturers of hard disk drives. Incorporated in 1978 as Shugart Technology, Seagate is currently incorporated in Dublin, Ireland and has its principal executive offices in Scotts Valley, California, United States.-1970s:On November 1, 1979...
Savvio exceeding 400 IOPS—more than doubling its performance.
While traditional HDDs have about the same IOPS for read and write operations, most NAND flash-based
Flash memory
Flash memory is a non-volatile computer storage chip that can be electrically erased and reprogrammed. It was developed from EEPROM and must be erased in fairly large blocks before these can be rewritten with new data...
SSDs are much slower writing than reading due to the inability to rewrite directly into a previously written location forcing a procedure called garbage collection. This has caused hardware test sites to start to do IOPS testing, providing independently measured results.
Newer flash SSD drives such as the Intel X25-E have much higher IOPS than traditional hard disk drives. In a test done by Xssist, using IOmeter, 4 KB random transfers, 70/30 read/write ratio, queue depth 4, the IOPS delivered by the Intel X25-E 64 GB G1 started around 10000 IOPs, and dropped sharply after 8 minutes to 4000 IOPS, and continued to decrease gradually for the next 42 minutes. IOPS vary between 3000 to 4000 from around the 50th minutes onwards for the rest of the 8+ hours test run. Even with the drop in random IOPS after the 50th minute, the X25-E still has much higher IOPS compared to traditional hard disk drives. Some SSDs, including the OCZ RevoDrive 3 x2 PCIe using the SandForce
SandForce
SandForce is an American "fabless" semiconductor company based in Milpitas, California, that designs and manufactures flash memory controllers for solid-state drives . On October 26th 2011 it was acquired by LSI Corporation....
controller, have shown much higher sustained write performance that more closely matches the read speed.
Examples
Some ballpark numbers:| Device | Type | IOPS | Interface | Notes |
|---|---|---|---|---|
| 7,200 rpm Revolutions per minute Revolutions per minute is a measure of the frequency of a rotation. It annotates the number of full rotations completed in one minute around a fixed axis... SATA Sata Sata is a traditional dish from the Malaysian state of Terengganu, consisting of spiced fish meat wrapped in banana leaves and cooked on a grill.It is a type of Malaysian fish cake, or otak-otak... drives |
HDD | ~75-100 IOPS | SATA 3 Gb/s | |
| 10,000 rpm SATA drives | HDD | ~125-150 IOPS | SATA 3 Gb/s | |
| 15,000 rpm SAS Serial Attached SCSI Serial Attached SCSI is a computer bus used to move data to and from computer storage devices such as hard drives and tape drives. SAS depends on a point-to-point serial protocol that replaces the parallel SCSI bus technology that first appeared in the mid 1980s in data centers and workstations,... drives |
HDD | ~175-210 IOPS | SAS | |
Solid State Devices
| Device | Type | IOPS | Interface | Notes |
|---|---|---|---|---|
| Simple SLC SSD Solid-state drive A solid-state drive , sometimes called a solid-state disk or electronic disk, is a data storage device that uses solid-state memory to store persistent data with the intention of providing access in the same manner of a traditional block i/o hard disk drive... |
SSD Solid-state drive A solid-state drive , sometimes called a solid-state disk or electronic disk, is a data storage device that uses solid-state memory to store persistent data with the intention of providing access in the same manner of a traditional block i/o hard disk drive... |
~400 IOPS | SATA 3 Gb/s | |
| Intel X25-M G2 (MLC Multi-level cell In electronics, a multi-level cell is a memory element capable of storing more than a single bit of information.MLC NAND flash is a flash memory technology using multiple levels per cell to allow more bits to be stored using the same number of transistors... ) |
SSD | ~8,600 IOPS | SATA 3 Gb/s | Intel's data sheet claims 6,600/8,600 IOPS (80 GB/160 GB version) and 35,000 IOPS for random 4 KB writes and reads, respectively. |
| Intel X25-E (SLC) | SSD | ~5,000 IOPS | SATA 3 Gb/s | Intel's data sheet claims 3,300 IOPS and 35,000 IOPS for writes and reads, respectively. 5,000 IOPS are measured for a mix. Intel X25-E G1 has around 3 times higher IOPS compared to the Intel X25-M G2. |
| G.Skill G.Skill G.Skill International Enterprise Co., Ltd. is a Taiwanese computer hardware manufacturing company. The company's main target is the overclocking community and produces a variety of products, particularly memory.-History:... Phoenix Pro |
SSD | ~20,000 IOPS | SATA 3 Gb/s | SandForce SandForce SandForce is an American "fabless" semiconductor company based in Milpitas, California, that designs and manufactures flash memory controllers for solid-state drives . On October 26th 2011 it was acquired by LSI Corporation.... -1200 based SSD drives with enhanced firmware, states up to 50,000 IOPS, but benchmarking shows for this particular drive ~25,000 IOPS for random read and ~15,000 IOPS for random write. |
| OCZ OCZ Technology OCZ Technology is a manufacturer of computer hardware based in San Jose, California, USA. Since entering the memory market in 2002, OCZ has targeted its products primarily at the computer hardware enthusiast market, first producing performance DDR RAM, Video Cards, USB drives, and various Cooling... Vertex 3 |
SSD | Up to 60,000 IOPS | SATA 6 Gb/s | Random Write 4 KB (Aligned) |
| Texas Memory Systems Texas Memory Systems Texas Memory Systems, Inc. is a privately held American corporation that designs and manufacturers solid-state disks and digital signal processors . TMS was founded in 1978 and that same year introduced their first solid-state drive, followed by their first digital signal processor... RamSan-20 |
SSD | 120,000+ Random Read/Write IOPS | PCIe | Includes RAM cache and UltraCapacitor |
| Fusion-io Fusion-io Fusion-io is a computer hardware and software systems company based in Cottonwood Heights, Utah, that designs and manufactures what it calls a new memory tier based on NAND Flash memory technology... ioDrive |
SSD | 140,000 Read IOPS, 135,000 Write IOPS | PCIe | |
| Virident Systems Virident Systems Virident Systems is a computer systems company headquartered in Milpitas, California, that designs and builds Storage Class Memory storage solutions and aims to redefine memory-storage hierarchy... tachIOn |
SSD | 320,000 sustained READ IOPS using 4KB blocks and 200,000 sustained WRITE IOPS using 4KB blocks | PCIe | NAND SLC Flash up to 800G capacity |
| OCZ RevoDrive 3 X2 | SSD | 200,000 Random Write 4K IOPS | PCIe | |
| Fusion-io ioDrive Duo | SSD | 250,000+ IOPS | PCIe | |
| Violin Memory Violin Memory Violin Memory is a private American company based in Mountain View, California, that designs and manufactures enterprise flash memory arrays that combine Toshiba NAND flash, DRAM, distributed processing, and software to create solutions like Network-attached storage , local clusters, and QFabric... Violin 3200 |
SSD | 250,000+ Random Read/Write IOPS | PCIe /FC/Infiniband/iSCSI | Flash Memory Array |
| WhipTail, XLR8r 6TB | SSD | 250,000+ Write/Read IOPS | FC, iscsi, Infiniband, NAS | Flash Based Storage Array |
| DDRdrive X1, | SSD | 300,000+ (512B Random Read IOPS) and 200,000+ (512B Random Write IOPS) | PCIe | |
| OCZ Single SuperScale Z-Drive R4 PCI-Express SSD | SSD | Up to 500,000 IOPS | PCIe | 3,800MB/s |
| Texas Memory Systems RamSan-70 | SSD | 600,000+ Random Read/Write IOPS | PCIe | Includes RAM cache and UltraCapacitor to enable backup to flash memory at power loss |
| Texas Memory Systems RamSan-630 Appliance | SSD | 1,000,000+ Random Read/Write IOPS | FC / InfiniBand | |
| OCZ 2x SuperScale Z-Drive R4 PCI-Express SSD | SSD | Up to 1,200,000 IOPS | PCIe | 5,600MB/s |
| Fusion-io ioDrive Octal (single PCI Express card) | SSD | 1,180,000+ Random Read/Write IOPS | PCIe |