Disk cloning
Encyclopedia
Disk cloning is the process of copying the contents of one computer hard disk
to another disk or to an "image" file. Often, the contents of the first disk are written to an image
file
as an intermediate step, and the second disk is loaded with the contents of the image. This procedure is also useful when moving to a larger capacity disk or to restore the disk to a previous state.
To provision the hard disk of a computer without using disk cloning software, the following steps are generally required for each computer:
With disk cloning, this is simplified to:
This can be referred to simply as a recovery disc.
, some computer manufacturers used hardware disk copying machines to copy software. This had the disadvantages of copying not just the used data
on the disk, but also unused sector
s, as the hardware used was not aware of the structures on the disks. A larger hard disk could not be copied to a smaller one, and copying a smaller one to a larger left the remaining space on the new disk unused. The two disks required identical geometries.
Other manufacturers and companies partitioned and formatted disks manually, then used file copy utilities or archiving utilities, such as tar
or zip
to copy files. It is not sufficient simply to copy all files from one disk to another, because there are special boot
files or boot tracks which must be specifically placed for an operating system to run, so additional manual steps were required.
Windows 95 compounded the problems because it was larger than earlier popular operating systems, and thus took more time to install. The long filenames added to the FAT
filesystem by Microsoft in Windows 95 were not supported by most copy programs, and the introduction of the FAT32
filesystem in 1997 caused problems for others. The growth of the personal computer market at this time also made a more efficient solution desirable.
Ghost
was introduced in 1996 by Binary Research
. It initially supported only FAT filesystems directly, but it could copy but not resize other filesystems by performing a sector copy on them. Ghost added support for the NTFS
filesystem later that year, and also provided a program to change the Security Identifier
(SID) which made Windows NT systems distinguishable from each other. Support for the ext2
filesystem was added in 1999.
Competitors to Ghost soon arose, and a features war has carried on to the present day. Many disk cloning programs now offer features which go beyond simple disk cloning, such as asset management and user settings migration.
On UNIX based computer systems, dd
was more commonplace due to the lack of filesystem support in Ghost.
and its successors, two machines with identical security IDs (SIDs, aka Security Identifier
) are said not to be allowed on the same Active Directory domain.The problem with duplicated SIDs in a Workgroup of computers running Windows NT/2K/XP is only related to different user accounts having the same SID. This could lead to unexpected access to shared files or files stored on a removable storage: If some ACLs (Access control list
s) are set on a file, the actual permissions can be associated with a user SID. If this user SID is duplicated on a cloned computer (because the computer SID is duplicated and because the user SIDs are built based on the computer SID + a sequential number), a user of a second computer (cloned from the first one) could have access to the files that the user of a first computer has protected. A disk cloning program should change these as part of copying the disk or restoring the image. Some operating systems are also not well suited to changes in hardware, so that a clone of Windows XP
for example may object to being booted on a machine with a different motherboard
, graphics card and network card
, especially if non-generic drivers are used. Microsoft
's solution to this is Sysprep
, a utility which runs hardware detection scans and sets the SID and computer name freshly when the machine boots. Microsoft recommends that Sysprep be set up on all machines before cloning, rather than allow third party programs to configure them. Similarly, Linux systems simply require the necessary kernel modules to be available (or compiled directly into the kernel), in order to support new hardware when the machine boots. However there are ways to help make images for cloning with Windows more portable. One such example would be a product called Universal Imaging Utility from Binary Research
(original developers of Symantec's Ghost) which incorporates a large number of hardware device drivers into the sysprep routine.
When it comes to "Domain SID", the Domain SID is recomputed each time a computer enters a domain. Thus, all the "post-cloning operations" that are based on "leave the domain and then rejoin the domain" will actually cause a re-creation of the Domain SID for the computer that joins the domain.
In other words, duplicated SIDs are usually not a problem with Microsoft Windows systems
There are files in some Microsoft operating systems (called BOOTSECT.*) which are copies of the Boot Partition Block (BPB) used by alternate operating systems that Microsoft Windows loader (NTLDR
) can load. BOOTSECT.* files may have to be altered if partition sizes or layouts are changed during the clone.
Linux systems usually boot using either the LILO
or GRUB bootloaders. These contain lists of absolute disk sectors in their MBR, which must be altered by the cloning program as the files they refer to are likely not to be in the same location of the destination disk. For example, if the original boot loader script points to the system being on a disk on channel 0 and the system being of the second partition, the target computer will need to have the same configuration.
different from the native operating system of the host computer, for example, MS-DOS
or an equivalent such as PC-DOS
or DR-DOS
, or Linux
. The computer is booted from this operating system, the cloning program is loaded and copies the Windows file system. Many programs (e.g. Acronis True Image
) can clone a disk, or make an image, from within Windows, with special provision for copying open files; but an image cannot be restored onto the Windows System Drive under Windows.
A disc cloning program running under non-Windows operating systems must have device drivers or equivalent for all devices used. The manufacturers of some devices do not provide suitable drivers, so the manufacturers of disk cloning software must write their own drivers, or include device access functionality in some other way. This applies to tape
drives, CD
and DVD
readers and writers, and USB
and FireWire drives. Cloning software contains its own TCP/IP stack
for multicast transfer
of data where required.
technology. This allows a single image to be sent simultaneously to many machines without putting greater stress on the network than sending an image to a single machine.
programs, they are sometimes used as such. A key feature of a backup program is to allow the retrieval of individual files without needing to restore the entire backup. Disk cloning programs either provide a Windows Explorer-like program to browse image files and extract individual files from them, or allow an image file to be mounted as a read-only filesystem within Windows Explorer.
Some such programs allow deletion of files from images, and addition of new files.
Hard disk
A hard disk drive is a non-volatile, random access digital magnetic data storage device. It features rotating rigid platters on a motor-driven spindle within a protective enclosure. Data is magnetically read from and written to the platter by read/write heads that float on a film of air above the...
to another disk or to an "image" file. Often, the contents of the first disk are written to an image
Disk image
A disk image is a single file or storage device containing the complete contents and structure representing a data storage medium or device, such as a hard drive, tape drive, floppy disk, CD/DVD/BD, or USB flash drive, although an image of an optical disc may be referred to as an optical disc image...
file
Computer file
A computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...
as an intermediate step, and the second disk is loaded with the contents of the image. This procedure is also useful when moving to a larger capacity disk or to restore the disk to a previous state.
Common Uses
There are a number of notable uses for disk cloning software. These include:- Reboot and restore – a technique in which the disk of a computer is automatically wiped and restored from a "clean", master image, which should be in full working order and should have been swept for viruses. This is used by some cybercafes and some training and educational institutes, and helps ensure that even if a user does misconfigure something, downloads inappropriate content or programs, or infects a computer with a virus, the computer will be restored to a clean, working state. The reboot and restore process can either take place irregularly when a computer shows signs of malfunctioning, on a regular basis (e.g., nightly) or even, in some cases, every time a user logs off, which is the safest approach (although that does involve some downtime).
- Provisioning new computers – Provisioning with a standard set of software so that a new user is ready to go straight away with a complete application suite and does not have to waste time installing individual applications. This is often done by original equipment manufacturers and larger companies.
- Hard drive upgrade – An individual user may use disk copying (cloning) to upgrade to a new, usually larger, hard disk.
- Full system backup – A user may create a comprehensive backupBackupIn information technology, a backup or the process of backing up is making copies of data which may be used to restore the original after a data loss event. The verb form is back up in two words, whereas the noun is backup....
of their operating system and installed software. - System recovery – An OEM can provide media that can restore a computer to its original factory software configuration
- Transfer to another user – A system sold or given to another person may be reset by reloading a known, previously-saved image that contains no personal files or information.
How it works
This article is specific to disk cloning on the x86 platform; specific details may not apply to other platforms.To provision the hard disk of a computer without using disk cloning software, the following steps are generally required for each computer:
- Create one or more partitions on the disk
- FormatDisk formattingDisk formatting is the process of preparing a hard disk drive or flexible disk medium for data storage. In some cases, the formatting operation may also create one or more new file systems...
each partition to create a file systemFile systemA file system is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device which contain it. A file system organizes data in an efficient manner and is tuned to the...
on it - Install the operating systemOperating systemAn operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
- Install device driverDevice driverIn computing, a device driver or software driver is a computer program allowing higher-level computer programs to interact with a hardware device....
s for the particular hardware - Install application softwareApplication softwareApplication software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...
With disk cloning, this is simplified to:
- Install the first computer, as above.
- Create an image of the hard disk (optional)
- Clone the first disk, or its image, to the remaining computers.
This can be referred to simply as a recovery disc.
History
Before Windows 95Windows 95
Windows 95 is a consumer-oriented graphical user interface-based operating system. It was released on August 24, 1995 by Microsoft, and was a significant progression from the company's previous Windows products...
, some computer manufacturers used hardware disk copying machines to copy software. This had the disadvantages of copying not just the used data
Data
The term data refers to qualitative or quantitative attributes of a variable or set of variables. Data are typically the results of measurements and can be the basis of graphs, images, or observations of a set of variables. Data are often viewed as the lowest level of abstraction from which...
on the disk, but also unused sector
Disk sector
In computer disk storage, a sector is a subdivision of a track on a magnetic disk or optical disc. Each sector stores a fixed amount of user data. Traditional formatting of these storage media provides space for 512 bytes or 2048 bytes of user-accessible data per sector...
s, as the hardware used was not aware of the structures on the disks. A larger hard disk could not be copied to a smaller one, and copying a smaller one to a larger left the remaining space on the new disk unused. The two disks required identical geometries.
Other manufacturers and companies partitioned and formatted disks manually, then used file copy utilities or archiving utilities, such as tar
Tar (file format)
In computing, tar is both a file format and the name of a program used to handle such files...
or zip
ZIP (file format)
Zip is a file format used for data compression and archiving. A zip file contains one or more files that have been compressed, to reduce file size, or stored as is...
to copy files. It is not sufficient simply to copy all files from one disk to another, because there are special boot
Booting
In computing, booting is a process that begins when a user turns on a computer system and prepares the computer to perform its normal operations. On modern computers, this typically involves loading and starting an operating system. The boot sequence is the initial set of operations that the...
files or boot tracks which must be specifically placed for an operating system to run, so additional manual steps were required.
Windows 95 compounded the problems because it was larger than earlier popular operating systems, and thus took more time to install. The long filenames added to the FAT
File Allocation Table
File Allocation Table is a computer file system architecture now widely used on many computer systems and most memory cards, such as those used with digital cameras. FAT file systems are commonly found on floppy disks, flash memory cards, digital cameras, and many other portable devices because of...
filesystem by Microsoft in Windows 95 were not supported by most copy programs, and the introduction of the FAT32
File Allocation Table
File Allocation Table is a computer file system architecture now widely used on many computer systems and most memory cards, such as those used with digital cameras. FAT file systems are commonly found on floppy disks, flash memory cards, digital cameras, and many other portable devices because of...
filesystem in 1997 caused problems for others. The growth of the personal computer market at this time also made a more efficient solution desirable.
Ghost
Ghost (software)
Ghost is a disk cloning program product sold by Symantec. Originally developed by Murray Haszard in 1995 for Binary Research, the technology was acquired in 1998 by Symantec. The name Ghost is an acronym for General Hardware-Oriented System Transfer.- History :Binary Research developed Ghost in...
was introduced in 1996 by Binary Research
Binary Research
Binary Research Ltd was a company founded in Auckland, New Zealand by Murray Haszard in 1991 after the sale of his previous company, B32 Software.Binary Research initially considered developing competitors to the file transfer programs Blast and Laplink...
. It initially supported only FAT filesystems directly, but it could copy but not resize other filesystems by performing a sector copy on them. Ghost added support for the NTFS
NTFS
NTFS is the standard file system of Windows NT, including its later versions Windows 2000, Windows XP, Windows Server 2003, Windows Server 2008, Windows Vista, and Windows 7....
filesystem later that year, and also provided a program to change the Security Identifier
Security Identifier
In the context of the Microsoft Windows NT line of operating systems, a Security Identifier is a unique name which is assigned by a Windows Domain controller during the log on process that is used to identify a subject, such as a user or a group of users in a network of NT/2000...
(SID) which made Windows NT systems distinguishable from each other. Support for the ext2
Ext2
The ext2 or second extended filesystem is a file system for the Linux kernel. It was initially designed by Rémy Card as a replacement for the extended file system ....
filesystem was added in 1999.
Competitors to Ghost soon arose, and a features war has carried on to the present day. Many disk cloning programs now offer features which go beyond simple disk cloning, such as asset management and user settings migration.
On UNIX based computer systems, dd
Dd (Unix)
In computing, dd is a common Unix program whose primary purpose is the low-level copying and conversion of raw data. According to the manual page for Version 7 Unix, it will "convert and copy a file". It is used to copy a specified number of bytes or blocks, performing on-the-fly byte order...
was more commonplace due to the lack of filesystem support in Ghost.
Post-cloning operations
Two machines with identical names are said not to be allowed on the same network, and, for Windows NTWindows NT
Windows NT is a family of operating systems produced by Microsoft, the first version of which was released in July 1993. It was a powerful high-level-language-based, processor-independent, multiprocessing, multiuser operating system with features comparable to Unix. It was intended to complement...
and its successors, two machines with identical security IDs (SIDs, aka Security Identifier
Security Identifier
In the context of the Microsoft Windows NT line of operating systems, a Security Identifier is a unique name which is assigned by a Windows Domain controller during the log on process that is used to identify a subject, such as a user or a group of users in a network of NT/2000...
) are said not to be allowed on the same Active Directory domain.The problem with duplicated SIDs in a Workgroup of computers running Windows NT/2K/XP is only related to different user accounts having the same SID. This could lead to unexpected access to shared files or files stored on a removable storage: If some ACLs (Access control list
Access control list
An access control list , with respect to a computer file system, is a list of permissions attached to an object. An ACL specifies which users or system processes are granted access to objects, as well as what operations are allowed on given objects. Each entry in a typical ACL specifies a subject...
s) are set on a file, the actual permissions can be associated with a user SID. If this user SID is duplicated on a cloned computer (because the computer SID is duplicated and because the user SIDs are built based on the computer SID + a sequential number), a user of a second computer (cloned from the first one) could have access to the files that the user of a first computer has protected. A disk cloning program should change these as part of copying the disk or restoring the image. Some operating systems are also not well suited to changes in hardware, so that a clone of Windows XP
Windows XP
Windows XP is an operating system produced by Microsoft for use on personal computers, including home and business desktops, laptops and media centers. First released to computer manufacturers on August 24, 2001, it is the second most popular version of Windows, based on installed user base...
for example may object to being booted on a machine with a different motherboard
Motherboard
In personal computers, a motherboard is the central printed circuit board in many modern computers and holds many of the crucial components of the system, providing connectors for other peripherals. The motherboard is sometimes alternatively known as the mainboard, system board, or, on Apple...
, graphics card and network card
Network card
A network interface controller is a computer hardware component that connects a computer to a computer network....
, especially if non-generic drivers are used. Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
's solution to this is Sysprep
Sysprep
Sysprep is the name of Microsoft's System Preparation Utility for Microsoft Windows operating system deployment.- History :Sysprep was originally introduced for use with Windows NT 4.0. Later versions introduced for Windows 2000 and Windows XP are available for download from Microsoft and included...
, a utility which runs hardware detection scans and sets the SID and computer name freshly when the machine boots. Microsoft recommends that Sysprep be set up on all machines before cloning, rather than allow third party programs to configure them. Similarly, Linux systems simply require the necessary kernel modules to be available (or compiled directly into the kernel), in order to support new hardware when the machine boots. However there are ways to help make images for cloning with Windows more portable. One such example would be a product called Universal Imaging Utility from Binary Research
Binary Research
Binary Research Ltd was a company founded in Auckland, New Zealand by Murray Haszard in 1991 after the sale of his previous company, B32 Software.Binary Research initially considered developing competitors to the file transfer programs Blast and Laplink...
(original developers of Symantec's Ghost) which incorporates a large number of hardware device drivers into the sysprep routine.
When it comes to "Domain SID", the Domain SID is recomputed each time a computer enters a domain. Thus, all the "post-cloning operations" that are based on "leave the domain and then rejoin the domain" will actually cause a re-creation of the Domain SID for the computer that joins the domain.
In other words, duplicated SIDs are usually not a problem with Microsoft Windows systems
There are files in some Microsoft operating systems (called BOOTSECT.*) which are copies of the Boot Partition Block (BPB) used by alternate operating systems that Microsoft Windows loader (NTLDR
NTLDR
NTLDR is the boot loader for all releases of Windows NT operating system up to and including Windows XP and Windows Server 2003. NTLDR is typically run from the primary hard disk drive, but it can also run from portable storage devices such as a CD-ROM, USB flash drive, or floppy disk...
) can load. BOOTSECT.* files may have to be altered if partition sizes or layouts are changed during the clone.
Linux systems usually boot using either the LILO
LILO (boot loader)
LILO is a generic boot loader for Linux.-Overview:LILO does not depend on a specific file system, and can boot an operating system from floppy disks and hard disks. One of up to sixteen different images can be selected at boot time. Various parameters, such as the root device, can be set...
or GRUB bootloaders. These contain lists of absolute disk sectors in their MBR, which must be altered by the cloning program as the files they refer to are likely not to be in the same location of the destination disk. For example, if the original boot loader script points to the system being on a disk on channel 0 and the system being of the second partition, the target computer will need to have the same configuration.
Operating environment
A disk cloning program needs to be able to read even protected operating system files on the source disk, and must guarantee that the system is in a consistent state at the time of reading. It must also overwrite any operating system already present on the destination disk. To simplify these tasks, most disk cloning programs can run under an operating systemOperating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
different from the native operating system of the host computer, for example, MS-DOS
MS-DOS
MS-DOS is an operating system for x86-based personal computers. It was the most commonly used member of the DOS family of operating systems, and was the main operating system for IBM PC compatible personal computers during the 1980s to the mid 1990s, until it was gradually superseded by operating...
or an equivalent such as PC-DOS
PC-DOS
IBM PC DOS is a DOS system for the IBM Personal Computer and compatibles, manufactured and sold by IBM from the 1980s to the 2000s....
or DR-DOS
DR-DOS
DR-DOS is an MS-DOS-compatible operating system for IBM PC-compatible personal computers, originally developed by Gary Kildall's Digital Research and derived from Concurrent PC DOS 6.0, which was an advanced successor of CP/M-86...
, or Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
. The computer is booted from this operating system, the cloning program is loaded and copies the Windows file system. Many programs (e.g. Acronis True Image
Acronis True Image
Acronis True Image is disk imaging software, developed by Acronis, that creates backups and recovers PC systems. The software allows a user to create an image of a disk while it is running Microsoft Windows or offline by booting from CD/DVD, USB flash drives, PXE, or other bootable media...
) can clone a disk, or make an image, from within Windows, with special provision for copying open files; but an image cannot be restored onto the Windows System Drive under Windows.
A disc cloning program running under non-Windows operating systems must have device drivers or equivalent for all devices used. The manufacturers of some devices do not provide suitable drivers, so the manufacturers of disk cloning software must write their own drivers, or include device access functionality in some other way. This applies to tape
Magnetic tape
Magnetic tape is a medium for magnetic recording, made of a thin magnetizable coating on a long, narrow strip of plastic. It was developed in Germany, based on magnetic wire recording. Devices that record and play back audio and video using magnetic tape are tape recorders and video tape recorders...
drives, CD
Compact Disc
The Compact Disc is an optical disc used to store digital data. It was originally developed to store and playback sound recordings exclusively, but later expanded to encompass data storage , write-once audio and data storage , rewritable media , Video Compact Discs , Super Video Compact Discs ,...
and DVD
DVD
A DVD is an optical disc storage media format, invented and developed by Philips, Sony, Toshiba, and Panasonic in 1995. DVDs offer higher storage capacity than Compact Discs while having the same dimensions....
readers and writers, and USB
Universal Serial Bus
USB is an industry standard developed in the mid-1990s that defines the cables, connectors and protocols used in a bus for connection, communication and power supply between computers and electronic devices....
and FireWire drives. Cloning software contains its own TCP/IP stack
Internet protocol suite
The Internet protocol suite is the set of communications protocols used for the Internet and other similar networks. It is commonly known as TCP/IP from its most important protocols: Transmission Control Protocol and Internet Protocol , which were the first networking protocols defined in this...
for multicast transfer
IP Multicast
IP multicast is a method of sending Internet Protocol datagrams to a group of interested receivers in a single transmission. It is often employed for streaming media applications on the Internet and private networks. The method is the IP-specific version of the general concept of multicast...
of data where required.
Image transfer
The simplest method of cloning a disk is to have both the source and destination disks present in the same machine, but this is often not possible. Disk cloning programs can link two computers by a parallel cable, or save and load images to an external USB drive or network drive. As disk images tend to be very large (usually a minimum of several hundred MB), performing several clones at a time puts excessive stress on a network. The solution is to use multicastMulticast
In computer networking, multicast is the delivery of a message or information to a group of destination computers simultaneously in a single transmission from the source creating copies automatically in other network elements, such as routers, only when the topology of the network requires...
technology. This allows a single image to be sent simultaneously to many machines without putting greater stress on the network than sending an image to a single machine.
Image manipulation
Although disk cloning programs are not primarily backupBackup
In information technology, a backup or the process of backing up is making copies of data which may be used to restore the original after a data loss event. The verb form is back up in two words, whereas the noun is backup....
programs, they are sometimes used as such. A key feature of a backup program is to allow the retrieval of individual files without needing to restore the entire backup. Disk cloning programs either provide a Windows Explorer-like program to browse image files and extract individual files from them, or allow an image file to be mounted as a read-only filesystem within Windows Explorer.
Some such programs allow deletion of files from images, and addition of new files.
See also
- List of disk cloning software
- Comparison of disk cloning software
- Disk mirroringDisk mirroringIn data storage, disk mirroring or RAID1 is the replication of logical disk volumes onto separate physical hard disks in real time to ensure continuous availability...
- Disk imageDisk imageA disk image is a single file or storage device containing the complete contents and structure representing a data storage medium or device, such as a hard drive, tape drive, floppy disk, CD/DVD/BD, or USB flash drive, although an image of an optical disc may be referred to as an optical disc image...
- Live USBLive USBA live USB is a USB flash drive or a USB external hard disk drive containing a full operating system that can be booted. Live USBs are closely related to live CDs, but sometimes have the ability to persistently save settings and permanently install software packages back onto the USB device...
- Recovery disc
- Security IdentifierSecurity IdentifierIn the context of the Microsoft Windows NT line of operating systems, a Security Identifier is a unique name which is assigned by a Windows Domain controller during the log on process that is used to identify a subject, such as a user or a group of users in a network of NT/2000...