VMScluster
Encyclopedia
A VMScluster is a computer cluster involving a group of computers running the OpenVMS
operating system. Whereas tightly coupled multiprocessor systems run a single copy of the operating system
, a VMScluster is loosely coupled: each machine runs its own copy of OpenVMS, but the disk storage, lock manager
, and security domain are all cluster-wide. Machines can join or leave a VMScluster without affecting the rest of the cluster. For enhanced availability, VMSclusters support the use of dual-ported disks connected to two machines or storage controllers simultaneously. With OpenVMS now ported to Alpha
and IA-64 machines, the facility originally named VAXclustering was renamed to VMSclustering.
first announced VAXclusters in May 1983. At this stage, clustering required specialised communications hardware, as well as some major changes to low-level subsystems in VMS. The software and hardware were designed jointly.
At the center of each cluster was a star coupler
, to which every node (computer) and data storage device in the cluster was connected by one or two pairs of CI cables. ("CI" stands for Computer Interconnect.) Each pair of cables had a transmission rate of 70 megabits per second, a high speed for that era. Using two pairs gave an aggregate transmission rate of 140 megabits per second, with redundancy in case one cable failed; the star couplers also had redundant wiring for better availability.
Each CI cable connected to its computer via a CI Port, which could send and receive packets without any CPU involvement. To send a packet, a CPU had only to create a small data structure in memory and append it to a "send" queue; similarly, the CI Port would append each incoming message to a "receive" queue. Tests showed that a VAX-11/780 could send and receive 3000 messages per second, even though it was nominally a 1-MIPS machine. The closely related Mass Storage Control Protocol
(MSCP) allowed similarly high performance from the mass storage subsystem. In addition, MSCP packets were very easily transported over the CI allowing remote access to storage devices.
VAXclustering was the first clustering system to achieve commercial success, and was a major selling point for VAX systems.
minicomputers, running over Ethernet
instead of special-purpose hardware. While not giving the high-availability
advantages of the CI hardware, these Local Area VAXclusters provided an attractive expansion path for buyers of low-end minicomputers.
Later versions of OpenVMS (V5.0 and later) supported "mixed interconnect" VAXclusters (using both CI and Ethernet), and VAXclustering over DSSI (Digital Systems and Storage Interconnect
) and FDDI, among other transports. Eventually, as high-bandwidth wide area networking became available, clustering was extended to allow satellite data links and long-distance terrestrial links. This allowed the creation of disaster-tolerant clusters; by locating the single VAXcluster in several diverse geographical areas, the cluster could survive infrastructure failures and natural disasters.
VAXclustering was greatly aided by the introduction of terminal server
s using the LAT
protocol. By allowing ordinary serial terminals to access the host nodes via Ethernet, it became possible for any terminal to rapidly and easily connect to any host node. This made it much simpler to accomplish fail over of the user terminals from one node of the cluster to another.
Eventually, VAXclusters reached the point where the cluster as a whole essentially never went down. Rolling upgrades even allowed the system operators to upgrade the OpenVMS system software, shutting down, upgrading, and rebooting individual nodes while the cluster as a whole continued processing. Cluster uptimes are frequently measured in years with the current longest uptime being at least twelve years.Uptimes Project breakdown for VMSclusters
As mentioned above, OpenVMS now also runs on Alpha
and IA-64 systems, so the term VAXcluster has been replaced by VMScluster. With Gigabit Ethernet
now common and 10-gigabit Ethernet being introduced, standard networking cables and cards are quite sufficient to support VMSclustering.
OpenVMS
OpenVMS , previously known as VAX-11/VMS, VAX/VMS or VMS, is a computer server operating system that runs on VAX, Alpha and Itanium-based families of computers. Contrary to what its name suggests, OpenVMS is not open source software; however, the source listings are available for purchase...
operating system. Whereas tightly coupled multiprocessor systems run a single copy of the operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
, a VMScluster is loosely coupled: each machine runs its own copy of OpenVMS, but the disk storage, lock manager
Distributed lock manager
A distributed lock manager provides distributed software applications with a means to synchronize their accesses to shared resources....
, and security domain are all cluster-wide. Machines can join or leave a VMScluster without affecting the rest of the cluster. For enhanced availability, VMSclusters support the use of dual-ported disks connected to two machines or storage controllers simultaneously. With OpenVMS now ported to Alpha
DEC Alpha
Alpha, originally known as Alpha AXP, is a 64-bit reduced instruction set computer instruction set architecture developed by Digital Equipment Corporation , designed to replace the 32-bit VAX complex instruction set computer ISA and its implementations. Alpha was implemented in microprocessors...
and IA-64 machines, the facility originally named VAXclustering was renamed to VMSclustering.
Initial release
Digital Equipment CorporationDigital Equipment Corporation
Digital Equipment Corporation was a major American company in the computer industry and a leading vendor of computer systems, software and peripherals from the 1960s to the 1990s...
first announced VAXclusters in May 1983. At this stage, clustering required specialised communications hardware, as well as some major changes to low-level subsystems in VMS. The software and hardware were designed jointly.
At the center of each cluster was a star coupler
Star coupler
A star coupler is a device that takes in an input signal and splits it into several output signals.In fiber optics, and especially in telecommunications, a star coupler is a passive optical device, used in network applications. An optical signal introduced into any input port is distributed to all...
, to which every node (computer) and data storage device in the cluster was connected by one or two pairs of CI cables. ("CI" stands for Computer Interconnect.) Each pair of cables had a transmission rate of 70 megabits per second, a high speed for that era. Using two pairs gave an aggregate transmission rate of 140 megabits per second, with redundancy in case one cable failed; the star couplers also had redundant wiring for better availability.
Each CI cable connected to its computer via a CI Port, which could send and receive packets without any CPU involvement. To send a packet, a CPU had only to create a small data structure in memory and append it to a "send" queue; similarly, the CI Port would append each incoming message to a "receive" queue. Tests showed that a VAX-11/780 could send and receive 3000 messages per second, even though it was nominally a 1-MIPS machine. The closely related Mass Storage Control Protocol
Mass Storage Control Protocol
The Mass Storage Control Protocol was a protocol designed by Digital Equipment Corporation of Maynard, Massachusetts for the purposes of controlling their high-end mass storage options....
(MSCP) allowed similarly high performance from the mass storage subsystem. In addition, MSCP packets were very easily transported over the CI allowing remote access to storage devices.
VAXclustering was the first clustering system to achieve commercial success, and was a major selling point for VAX systems.
Later developments
In 1986, DEC added VAXclustering support to their MicroVAXMicroVAX
The MicroVAX was a family of low-end minicomputers developed and manufactured by Digital Equipment Corporation . The first model, the MicroVAX I, was introduced in 1984...
minicomputers, running over Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....
instead of special-purpose hardware. While not giving the high-availability
High-availability cluster
High-availability clusters are groups of computers that support server applications that can be reliably utilized with a minimum of down-time. They operate by harnessing redundant computers in groups or clusters that provide continued service when system components fail...
advantages of the CI hardware, these Local Area VAXclusters provided an attractive expansion path for buyers of low-end minicomputers.
Later versions of OpenVMS (V5.0 and later) supported "mixed interconnect" VAXclusters (using both CI and Ethernet), and VAXclustering over DSSI (Digital Systems and Storage Interconnect
Digital Storage Systems Interconnect
The Digital Storage Systems Interconnect is a bus developed by Digital Equipment Corporation for connecting storage devices and clustering VAX systems and support was extended to MIPS based DECsystem and later to Alpha processor based AlphaServer systems.It was introduced in 1988 and has a...
) and FDDI, among other transports. Eventually, as high-bandwidth wide area networking became available, clustering was extended to allow satellite data links and long-distance terrestrial links. This allowed the creation of disaster-tolerant clusters; by locating the single VAXcluster in several diverse geographical areas, the cluster could survive infrastructure failures and natural disasters.
VAXclustering was greatly aided by the introduction of terminal server
Terminal server
A terminal server enables organizations to connect devices with an RS-232, RS-422 or RS-485 serial interface to a local area network . Products marketed as terminal servers can be very simple devices that do not offer any security functionality, such as data encryption and user authentication...
s using the LAT
Local Area Transport
Local Area Transport is a non-routable networking technology developed by Digital Equipment Corporation to provide connection between the DECserver 90, 100, 200, 300, 500, 700 and DECserver 900 terminal servers and Digital's VAX and Alpha and MIPS_architecture host computers via Ethernet, giving...
protocol. By allowing ordinary serial terminals to access the host nodes via Ethernet, it became possible for any terminal to rapidly and easily connect to any host node. This made it much simpler to accomplish fail over of the user terminals from one node of the cluster to another.
Eventually, VAXclusters reached the point where the cluster as a whole essentially never went down. Rolling upgrades even allowed the system operators to upgrade the OpenVMS system software, shutting down, upgrading, and rebooting individual nodes while the cluster as a whole continued processing. Cluster uptimes are frequently measured in years with the current longest uptime being at least twelve years.Uptimes Project breakdown for VMSclusters
As mentioned above, OpenVMS now also runs on Alpha
DEC Alpha
Alpha, originally known as Alpha AXP, is a 64-bit reduced instruction set computer instruction set architecture developed by Digital Equipment Corporation , designed to replace the 32-bit VAX complex instruction set computer ISA and its implementations. Alpha was implemented in microprocessors...
and IA-64 systems, so the term VAXcluster has been replaced by VMScluster. With Gigabit Ethernet
Gigabit Ethernet
Gigabit Ethernet is a term describing various technologies for transmitting Ethernet frames at a rate of a gigabit per second , as defined by the IEEE 802.3-2008 standard. It came into use beginning in 1999, gradually supplanting Fast Ethernet in wired local networks where it performed...
now common and 10-gigabit Ethernet being introduced, standard networking cables and cards are quite sufficient to support VMSclustering.
Further reading
- Nancy P. Kronenberg, Henry M. Levy, William D. Strecker, "VAXcluster: a closely-coupled distributed system". ACM Transactions on Computer Systems 4 (2), 1986
- DIGITAL Technical Journal 3 (3), 1991 This issue was devoted to VAXclusters and FDDI networking. (Archived as PDF files.)