Message-oriented middleware
Encyclopedia
Message-oriented middleware (MOM) is software or hardware
infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating system and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.
MOM provides software elements that reside in all communicating components of a client/server architecture and typically support asynchronous calls between the client and server applications. MOM reduces the involvement of application developers with the complexity of the master-slave nature of the client/server mechanism.
since the 1960s. This mainframe remained in heavy use and had undergone several upgrades.
Although groundbreaking in its day, the mainframe's usefulness to the bank’s staff diminished as the bank introduced new, separate applications based on personal computer
s (PCs), allowing the bank’s staff to offer customers new services that the mainframe could not support.
An ideal situation would allow the PC-based applications to link to the older mainframe application and allow the mainframe and the PCs to share each other's data
. Accessing the mainframe’s data offers two advantages:
Up until the late 1980s system integrator
s had no easy way to link these different applications together. Developers faced several challenges:
Such issues made integration between applications difficult. Much of such integration also required re-engineering every time two applications on disparate platforms needed linking together, as every situation differed to some extent. By devoting effort to linking together applications on different systems, IT departments started to spend an amount significantly greater than that spent on original development per sub-system.
Developers needed a separate piece of software that would sit in the middle of two or more applications and would handle all the ‘plumbing’ between the two systems. Such software needed the intelligence to handle different platform
s, different programming language
s, various network protocols and diverse hardware
. Developers wanted to remove themselves from the complexities of the underlying computing infrastructure
so that they could focus on functionality within actual applications.
Towards the end of the 1980s middleware began to emerge that attempted to address these issues. Initial middleware offerings addressed specific handfuls of platforms or languages and thus had limited usefulness. Over time, however, middleware products have become more and more advanced, supporting multiple platforms, languages and protocols.
The ability of middleware to link together disparate systems across a heterogeneous network
environment offers only one example of the benefits of this dominant technology. Middleware provides a whole raft of new functionality that augments and enhances the existing applications that it interconnects.
s while conveying them from sender to receiver(s).
communication software
that generally relies on asynchronous message-passing
, as opposed to a request-response
metaphor. In asynchronous systems, message queue
s provide temporary storage when the destination program is busy or not connected. In addition, most asynchronous MOM systems provide persistent storage to back up
the message queue. This means that the sender and receiver do not need to connect to the network at the same time (asynchronous delivery
), and solves problems with intermittent connectivity. It also means that should the receiver application fail for any reason, the senders can continue unaffected, as the messages they send will simply accumulate in the message queue
for later processing when the receiver restarts.
implementations depend on a message queue
system. Some implementations permit routing logic to be provided by the messaging layer itself, while others depend on client applications to provide routing information or allow for a mix of both paradigms. Some implementations make use of broadcast
or multicast
distribution paradigms.
messages en-route to match the requirements of the sender or of the recipient. In conjunction with the routing and broadcast/multicast facilities, one application can send a message in its own native format, and two or more other applications may each receive a copy of the message in their own native format. Many modern MOM systems provide sophisticated message transformation (or mapping) tools which allow programmer
s to specify transformation rules applicable to a simple GUI
drag-and-drop
operation.
, the message transfer agent (message broker
). As with any system
, adding another component can lead to reductions in performance
and reliability, and can also make the system as a whole more difficult and expensive to maintain
.
In addition, many inter-application communications have an intrinsically synchronous
aspect, with the sender specifically wanting to wait for a reply to a message before continuing (see real-time computing
and near-real-time
for extreme cases). Because message-based communication inherently functions asynchronously, it may not fit well in such situations. That said, most MOM systems have facilities to group a request and a response as a single pseudo-synchronous transaction.
governing the use of message oriented middleware has caused problems. All the major vendors have their own implementations, each with its own application programming interface
(API) and management tools.
The Java EE programming environment provides a standard API called JMS
(Java Message Service), which is implemented by most MOM vendors and aims to hide the particular MOM API implementations; however, JMS does not define the format of the messages that are exchanged, so JMS systems are not interoperable. Microsoft's MSMQ
doesn't support JMS, although there are third-party products that can offer this. WebSphere Message Broker, from IBM
, does provide JMS support, as well as a whole suite of modern functionality.
The Advanced Message Queuing Protocol
(AMQP) is an emerging standard that defines the protocol and formats used in the messaging server and client, so implementations are interoperable. AMQP is defined to provide flexible routing, including common messaging paradigms like point-to-point, fanout, publish/subscribe
, and request-response. It also supports transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support. Java applications that use AMQP are typically written in Java JMS. Other implementations provide APIs for C#, C++, PHP, Python, Ruby, and other languages.
Other protocols used for message-oriented middleware include XMPP
and Streaming Text Oriented Messaging Protocol.
Message-oriented messaging protocols under development include RestMS, a protocol similar in nature to AMQP but constructed over a RESTful HTTP transport, and SPB, a minimalist message framing protocol that can be used to carry higher-level MOM protocols.
An additional trend sees message-oriented middleware functions being implemented in hardware - usually FPGAs or other specialized silicon chips.
Hardware
Hardware is a general term for equipment such as keys, locks, hinges, latches, handles, wire, chains, plumbing supplies, tools, utensils, cutlery and machine parts. Household hardware is typically sold in hardware stores....
infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating system and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.
MOM provides software elements that reside in all communicating components of a client/server architecture and typically support asynchronous calls between the client and server applications. MOM reduces the involvement of application developers with the complexity of the master-slave nature of the client/server mechanism.
A requirements story
The bank had stored all its customer details on its large mainframeMainframe computer
Mainframes are powerful computers used primarily by corporate and governmental organizations for critical applications, bulk data processing such as census, industry and consumer statistics, enterprise resource planning, and financial transaction processing.The term originally referred to the...
since the 1960s. This mainframe remained in heavy use and had undergone several upgrades.
Although groundbreaking in its day, the mainframe's usefulness to the bank’s staff diminished as the bank introduced new, separate applications based on personal computer
Personal computer
A personal computer is any general-purpose computer whose size, capabilities, and original sales price make it useful for individuals, and which is intended to be operated directly by an end-user with no intervening computer operator...
s (PCs), allowing the bank’s staff to offer customers new services that the mainframe could not support.
An ideal situation would allow the PC-based applications to link to the older mainframe application and allow the mainframe and the PCs to share each other's data
Data
The term data refers to qualitative or quantitative attributes of a variable or set of variables. Data are typically the results of measurements and can be the basis of graphs, images, or observations of a set of variables. Data are often viewed as the lowest level of abstraction from which...
. Accessing the mainframe’s data offers two advantages:
- new front-end PC applications can replace the old user-unfriendly mainframe terminals
- PC-based systems can use the data from the mainframe in new ways — previously impractical due to the constraints of the mainframe’s software
Up until the late 1980s system integrator
System integrator
A systems integrator is a person or company that specializes in bringing together component subsystems into a whole and ensuring that those subsystems function together, a practice known as System Integration...
s had no easy way to link these different applications together. Developers faced several challenges:
- the developerSoftware developerA software developer is a person concerned with facets of the software development process. Their work includes researching, designing, developing, and testing software. A software developer may take part in design, computer programming, or software project management...
s would have to construct a separate software ‘adapter’ on each system to translate data from source applications into a format that the destination system could understand (and vice versa). - the processing speed of each system would constrain the other system. For example, if the mainframe ran slowly, the PC-based application would have to wait until the mainframe caught up, thereby slowing down the PC application. Conversely, processing that had been offloaded to distributed servers for cost reasons would run slowly and the mainframe would have to wait until the server caught up.
- communications programmers would need to install a network gateway system to form a bridge between the mainframe’s network and the PC network if the different systems used different network protocols. The gateway would translate the network packets from the source system and pass them on to the destination system using the destination system’s protocol.
Such issues made integration between applications difficult. Much of such integration also required re-engineering every time two applications on disparate platforms needed linking together, as every situation differed to some extent. By devoting effort to linking together applications on different systems, IT departments started to spend an amount significantly greater than that spent on original development per sub-system.
Developers needed a separate piece of software that would sit in the middle of two or more applications and would handle all the ‘plumbing’ between the two systems. Such software needed the intelligence to handle different platform
Platform (computing)
A computing platform includes some sort of hardware architecture and a software framework , where the combination allows software, particularly application software, to run...
s, different programming language
Programming language
A programming language is an artificial language designed to communicate instructions to a machine, particularly a computer. Programming languages can be used to create programs that control the behavior of a machine and/or to express algorithms precisely....
s, various network protocols and diverse hardware
Hardware
Hardware is a general term for equipment such as keys, locks, hinges, latches, handles, wire, chains, plumbing supplies, tools, utensils, cutlery and machine parts. Household hardware is typically sold in hardware stores....
. Developers wanted to remove themselves from the complexities of the underlying computing infrastructure
Infrastructure
Infrastructure is basic physical and organizational structures needed for the operation of a society or enterprise, or the services and facilities necessary for an economy to function...
so that they could focus on functionality within actual applications.
Towards the end of the 1980s middleware began to emerge that attempted to address these issues. Initial middleware offerings addressed specific handfuls of platforms or languages and thus had limited usefulness. Over time, however, middleware products have become more and more advanced, supporting multiple platforms, languages and protocols.
The ability of middleware to link together disparate systems across a heterogeneous network
Heterogeneous network
A heterogeneous network is a network connecting computers and other devices with different operating systems and/or protocols. For example, local area networks that connect Microsoft Windows and Linux based personal computers with Apple Macintosh computers are heterogeneous. The word...
environment offers only one example of the benefits of this dominant technology. Middleware provides a whole raft of new functionality that augments and enhances the existing applications that it interconnects.
Advantages
Central reasons for using a message-based communications protocol include its ability to store (buffer), route, or transform messageMessage
A message in its most general meaning is an object of communication. It is a vessel which provides information. Yet, it can also be this information. Therefore, its meaning is dependent upon the context in which it is used; the term may apply to both the information and its form...
s while conveying them from sender to receiver(s).
Synchronicity
MOM comprises a category of inter-applicationApplication software
Application software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...
communication software
Communication software
Communication software is used to provide remote access to systems and exchange files and messages in text, audio and/or video formats between different computers or user IDs...
that generally relies on asynchronous message-passing
Message passing
Message passing in computer science is a form of communication used in parallel computing, object-oriented programming, and interprocess communication. In this model, processes or objects can send and receive messages to other processes...
, as opposed to a request-response
Request-response
Request-response or request-reply is one of the basic methods computers use to talk to each other. When using request-response, the first computer requests some data and the second computer responds to the request. Usually there is a series of such interchanges until the complete message is sent. ...
metaphor. In asynchronous systems, message queue
Message queue
In computer science, message queues and mailboxes are software-engineering components used for interprocess communication, or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content...
s provide temporary storage when the destination program is busy or not connected. In addition, most asynchronous MOM systems provide persistent storage to back up
Backup
In information technology, a backup or the process of backing up is making copies of data which may be used to restore the original after a data loss event. The verb form is back up in two words, whereas the noun is backup....
the message queue. This means that the sender and receiver do not need to connect to the network at the same time (asynchronous delivery
Asynchrony
Asynchrony, in the general meaning, is the state of not being synchronized.* Asynchronous learning* Collaborative editing systemsIn specific terms of digital logic and physical layer of communication, an asynchronous process does not require a clock signal, in contrast with synchronous and...
), and solves problems with intermittent connectivity. It also means that should the receiver application fail for any reason, the senders can continue unaffected, as the messages they send will simply accumulate in the message queue
Message queue
In computer science, message queues and mailboxes are software-engineering components used for interprocess communication, or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content...
for later processing when the receiver restarts.
Routing
Many message-oriented middlewareMiddleware
Middleware is computer software that connects software components or people and their applications. The software consists of a set of services that allows multiple processes running on one or more machines to interact...
implementations depend on a message queue
Message queue
In computer science, message queues and mailboxes are software-engineering components used for interprocess communication, or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content...
system. Some implementations permit routing logic to be provided by the messaging layer itself, while others depend on client applications to provide routing information or allow for a mix of both paradigms. Some implementations make use of broadcast
Broadcasting (networks)
right|250pxIn telecommunication and information theory, broadcasting refers to a method of transferring a message to all recipients simultaneously...
or multicast
Multicast
In computer networking, multicast is the delivery of a message or information to a group of destination computers simultaneously in a single transmission from the source creating copies automatically in other network elements, such as routers, only when the topology of the network requires...
distribution paradigms.
Transformation
In a message-based middleware system, the recipient's message need not replicate the sender's message exactly. A MOM system with built-in intelligence can transformData transformation
In metadata and data warehouse, a data transformation converts data from a source data format into destination data.Data transformation can be divided into two steps:...
messages en-route to match the requirements of the sender or of the recipient. In conjunction with the routing and broadcast/multicast facilities, one application can send a message in its own native format, and two or more other applications may each receive a copy of the message in their own native format. Many modern MOM systems provide sophisticated message transformation (or mapping) tools which allow programmer
Software developer
A software developer is a person concerned with facets of the software development process. Their work includes researching, designing, developing, and testing software. A software developer may take part in design, computer programming, or software project management...
s to specify transformation rules applicable to a simple GUI
Graphical user interface
In computing, a graphical user interface is a type of user interface that allows users to interact with electronic devices with images rather than text commands. GUIs can be used in computers, hand-held devices such as MP3 players, portable media players or gaming devices, household appliances and...
drag-and-drop
Drag-and-drop
In computer graphical user interfaces, drag-and-drop is the action of selecting a virtual object by "grabbing" it and dragging it to a different location or onto another virtual object...
operation.
Disadvantages
The primary disadvantage of many message oriented middleware systems is that they require an extra component in the architectureSoftware architecture
The software architecture of a system is the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both...
, the message transfer agent (message broker
Message broker
Message broker is an intermediary program which translates the language of a system from one internationally recognized language to another by way of a telecommunications medium.-Pattern:...
). As with any system
System
System is a set of interacting or interdependent components forming an integrated whole....
, adding another component can lead to reductions in performance
Computer performance
Computer performance is characterized by the amount of useful work accomplished by a computer system compared to the time and resources used.Depending on the context, good computer performance may involve one or more of the following:...
and reliability, and can also make the system as a whole more difficult and expensive to maintain
Software maintenance
Software Maintenance in software engineering is the modification of a software product after delivery to correct faults, to improve performance or other attributes....
.
In addition, many inter-application communications have an intrinsically synchronous
Synchronization (computer science)
In computer science, synchronization refers to one of two distinct but related concepts: synchronization of processes, and synchronization of data. Process synchronization refers to the idea that multiple processes are to join up or handshake at a certain point, so as to reach an agreement or...
aspect, with the sender specifically wanting to wait for a reply to a message before continuing (see real-time computing
Real-time computing
In computer science, real-time computing , or reactive computing, is the study of hardware and software systems that are subject to a "real-time constraint"— e.g. operational deadlines from event to system response. Real-time programs must guarantee response within strict time constraints...
and near-real-time
Near real time
The term "near real-time" or "nearly real-time" , in telecommunications and computing, refers to the time delay introduced, by automated data processing or network transmission, between the occurrence of an event and the use of the processed data, such as for display or feedback and control purposes...
for extreme cases). Because message-based communication inherently functions asynchronously, it may not fit well in such situations. That said, most MOM systems have facilities to group a request and a response as a single pseudo-synchronous transaction.
Lack of standards
The lack of standardsStandardization
Standardization is the process of developing and implementing technical standards.The goals of standardization can be to help with independence of single suppliers , compatibility, interoperability, safety, repeatability, or quality....
governing the use of message oriented middleware has caused problems. All the major vendors have their own implementations, each with its own application programming interface
Application programming interface
An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
(API) and management tools.
The Java EE programming environment provides a standard API called JMS
Java Message Service
The Java Message Service API is a Java Message Oriented Middleware API for sending messages between two or more clients. JMS is a part of the Java Platform, Enterprise Edition, and is defined by a specification developed under the Java Community Process as JSR 914...
(Java Message Service), which is implemented by most MOM vendors and aims to hide the particular MOM API implementations; however, JMS does not define the format of the messages that are exchanged, so JMS systems are not interoperable. Microsoft's MSMQ
Microsoft Message Queuing
Microsoft Message Queuing or MSMQ is a Message Queue implementation developed by Microsoft and deployed in its Windows Server operating systems since Windows NT 4 and Windows 95. The latest Windows 7 also includes this component...
doesn't support JMS, although there are third-party products that can offer this. WebSphere Message Broker, from IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
, does provide JMS support, as well as a whole suite of modern functionality.
The Advanced Message Queuing Protocol
Advanced Message Queuing Protocol
The Advanced Message Queuing Protocol is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing , reliability and security.AMQP mandates the behaviour of the messaging provider and client to the extent...
(AMQP) is an emerging standard that defines the protocol and formats used in the messaging server and client, so implementations are interoperable. AMQP is defined to provide flexible routing, including common messaging paradigms like point-to-point, fanout, publish/subscribe
Publish/subscribe
Publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers. Published messages are characterized into classes, without knowledge of what, if any, subscribers there may be...
, and request-response. It also supports transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support. Java applications that use AMQP are typically written in Java JMS. Other implementations provide APIs for C#, C++, PHP, Python, Ruby, and other languages.
Trends
AMQP has been gaining adoption in applications that need an interoperable protocol for Message-oriented middleware.Other protocols used for message-oriented middleware include XMPP
Extensible Messaging and Presence Protocol
Extensible Messaging and Presence Protocol is an open-standard communications protocol for message-oriented middleware based on XML . The protocol was originally named Jabber, and was developed by the Jabber open-source community in 1999 for near-real-time, extensible instant messaging , presence...
and Streaming Text Oriented Messaging Protocol.
Message-oriented messaging protocols under development include RestMS, a protocol similar in nature to AMQP but constructed over a RESTful HTTP transport, and SPB, a minimalist message framing protocol that can be used to carry higher-level MOM protocols.
An additional trend sees message-oriented middleware functions being implemented in hardware - usually FPGAs or other specialized silicon chips.
See also
- Advanced Message Queuing ProtocolAdvanced Message Queuing ProtocolThe Advanced Message Queuing Protocol is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing , reliability and security.AMQP mandates the behaviour of the messaging provider and client to the extent...
- Enterprise messaging systemEnterprise messaging systemAn enterprise messaging system is a set of published Enterprise-wide standards that allows organizations to send semantically precise messages between computer systems. EMS systems promote loosely coupled architectures that allow changes in the formats of messages to have minimum impact on...
- Enterprise service busEnterprise service busAn enterprise service bus is a software architecture model used for designing and implementing the interaction and communication between mutually interacting software applications in Service Oriented Architecture...
- Flow-based programmingFlow-based programmingIn computer science, flow-based programming is a programming paradigm that defines applications as networks of "black box" processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes...