Enterprise application integration - AbsoluteAstronomy.com

Enterprise Application Integration (EAI) is defined as the use of software and computer systems architectural principles to integrate a set of enterprise computer applications.

Overview

Enterprise Application Integration (EAI) is an integration framework composed of a collection of technologies and services which form a middleware

Middleware

Middleware is computer software that connects software components or people and their applications. The software consists of a set of services that allows multiple processes running on one or more machines to interact...

to enable integration of systems and applications across the enterprise.

Supply chain management

Supply chain management

Supply chain management is the management of a network of interconnected businesses involved in the ultimate provision of product and service packages required by end customers...

applications (for managing inventory and shipping), customer relationship management

Customer relationship management

Customer relationship management is a widely implemented strategy for managing a company’s interactions with customers, clients and sales prospects. It involves using technology to organize, automate, and synchronize business processes—principally sales activities, but also those for marketing,...

applications (for managing current and potential customers), business intelligence

Business intelligence

Business intelligence mainly refers to computer-based techniques used in identifying, extracting, and analyzing business data, such as sales revenue by products and/or departments, or by associated costs and incomes....

applications (for finding patterns from existing data from operations), and other types of applications (for managing data such as human resources

Human resources

Human resources is a term used to describe the individuals who make up the workforce of an organization, although it is also applied in labor economics to, for example, business sectors or even whole nations...

data, health care, internal communications, etc.) typically cannot communicate with one another in order to share data or business rules. For this reason, such applications are sometimes referred to as islands of automation

Islands of automation

Islands of automation was a popular term used largely during the 1980s to describe how rapidly developing automation systems were at first unable to communicate easily with each other. Industrial communication protocols, network technologies, and system integration helped to improve this situation...

or information silo

Information silo

An information silo is a management system incapable of reciprocal operation with other, related management systems. A bank's management system, for example, is considered a silo if it cannot exchange information with other related systems within its own organization, or with the management systems...

s. This lack of communication leads to inefficiencies, wherein identical data are stored in multiple locations, or straightforward processes are unable to be automated.

Enterprise application integration (EAI) is the process of linking such applications within a single organization together in order to simplify and automate business processes to the greatest extent possible, while at the same time avoiding having to make sweeping changes to the existing applications or data structures. In the words of the Gartner Group, EAI is the “unrestricted sharing of data and business processes among any connected application or data sources in the enterprise.”

One large challenge of EAI is that the various systems that need to be linked together often reside on different operating system

Operating system

An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...

s, use different database

Database

A database is an organized collection of data for one or more purposes, usually in digital form. The data are typically organized to model relevant aspects of reality , in a way that supports processes requiring this information...

solutions and different computer languages, and in some cases are legacy system

Legacy system

A legacy system is an old method, technology, computer system, or application program that continues to be used, typically because it still functions for the users' needs, even though newer technology or more efficient methods of performing a task are now available...

s that are no longer supported by the vendor who originally created them. In some cases, such systems are dubbed "stovepipe system

Stovepipe system

In engineering and computing, a stovepipe system is a system that is an assemblage of inter-related elements that are so tightly bound together that the individual elements cannot be differentiated, upgraded or refactored. The stovepipe system must be maintained until it can be entirely replaced by...

s" because they consist of components that have been jammed together in a way that makes it very hard to modify them in any way.

Improving connectivity

If integration is applied without following a structured EAI approach, point-to-point connections grow across an organization. Dependencies are added on an impromptu basis, resulting in a tangled mess that is difficult to maintain. This is commonly referred to as spaghetti, an allusion to the programming equivalent of spaghetti code

Spaghetti code

Spaghetti code is a pejorative term for source code that has a complex and tangled control structure, especially one using many GOTOs, exceptions, threads, or other "unstructured" branching constructs. It is named such because program flow tends to look like a bowl of spaghetti, i.e. twisted and...

. For example:

The number of connections needed to have fully meshed point-to-point connections, with

points, is given by

. Thus, for ten applications to be fully integrated point-to-point,

, or 45 point-to-point connections are needed.

However, EAI is not just about sharing data between applications; it focuses on sharing both business data and business process. Middleware Analysts attending to EAI involves looking at the system of systems

System of systems

System of systems is a collection of task-oriented or dedicated systems that pool their resources and capabilities together to create a new, more complex system which offers more functionality and performance than simply the sum of the constituent systems...

, which involves large scale inter-disciplinary problems with multiple, heterogeneous, distributed systems that are embedded in networks at multiple levels.

Purposes

EAI can be used for different purposes:

Data integration
Data integration
Data integration involves combining data residing in different sources and providing users with a unified view of these data.This process becomes significant in a variety of situations, which include both commercial and scientific domains...

: Ensures that information in multiple systems is kept consistent. This is also known as Enterprise Information Integration
Enterprise Information Integration
Enterprise Information Integration , is a process of information integration, using data abstraction to provide a unified interface for viewing all the data within an organization, and a single set of structures and naming conventions to represent this data; the goal of EII is to get a large set of...

(EII).
Vendor independence: Extracts business policies or rules from applications and implements them in the EAI system, so that even if one of the business applications is replaced with a different vendor's application, the business rules do not have to be re-implemented.
Common facade: An EAI system can front-end a cluster of applications, providing a single consistent access interface to these applications and shielding users from having to learn to use different software packages.

Integration patterns

There are two patterns that EAI systems implement:

Mediation (intra-communication):Here, the EAI system acts as the go-between or broker between multiple applications. Whenever an interesting event occurs in an application (for instance, new information is created or a new transaction completed) an integration module in the EAI system is notified. The module then propagates the changes to other relevant applications.
Federation

Federation (information technology)

A Federation is multiple computing and/or network providers agreeing upon standards of operation in a collective fashion. The term may be used when describing the inter-operation of two distinct, formally disconnected, telecommunications networks that may have different internal structures...

(inter-communication):In this case, the EAI system acts as the overarching facade across multiple applications. All event calls from the 'outside world' to any of the applications are front-ended by the EAI system. The EAI system is configured to expose only the relevant information and interfaces of the underlying applications to the outside world, and performs all interactions with the underlying applications on behalf of the requester.

Both patterns are often used concurrently. The same EAI system could be keeping multiple applications in sync (mediation), while servicing requests from external users against these applications (federation).

Access patterns

EAI supports both asynchronous and synchronous access patterns, the former being typical in the mediation case and the latter in the federation case.

Lifetime patterns

An integration operation could be short-lived (e.g. keeping data in sync across two applications could be completed within a second) or long-lived (e.g. one of the steps could involve the EAI system interacting with a human work flow application for approval of a loan that takes hours or days to complete).

Topologies

There are two major topologies: hub-and-spoke, and bus

Enterprise service bus

An enterprise service bus is a software architecture model used for designing and implementing the interaction and communication between mutually interacting software applications in Service Oriented Architecture...

. Each has its own advantages and disadvantages. In the hub-and-spoke model, the EAI system is at the center (the hub), and interacts with the applications via the spokes. In the bus model, the EAI system is the bus (or is implemented as a resident module in an already existing message bus or message-oriented middleware

Message-oriented middleware

Message-oriented middleware is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple...

Technologies

Multiple technologies are used in implementing each of the components of the EAI system:

Bus/hub: This is usually implemented by enhancing standard middleware

Middleware

products (application server

Application server

An application server is a software framework that provides an environment in which applications can run, no matter what the applications are or what they do...

, message bus) or implemented as a stand-alone program (i. e., does not use any middleware), acting as its own middleware.
Application connectivity: The bus/hub connects to applications through a set of adapters (also referred to as connectors). These are programs that know how to interact with an underlying business application. The adapter performs two-way communication, performing requests from the hub against the application, and notifying the hub when an event of interest occurs in the application (a new record inserted, a transaction completed, etc.). Adapters can be specific to an application (e. g., built against the application vendor's client libraries) or specific to a class of applications (e. g., can interact with any application through a standard communication protocol, such as SOAP

SOAP

SOAP, originally defined as Simple Object Access Protocol, is a protocol specification for exchanging structured information in the implementation of Web Services in computer networks...

, SMTP or Action Message Format

Action Message Format

Action Message Format is a binary format used to serialize objects graphs such ActionScript objects and XML, or send messages between an Adobe Flash client and a remote service, usually a Flash Media Server or third party alternatives....

(AMF)). The adapter could reside in the same process space as the bus/hub or execute in a remote location and interact with the hub/bus through industry standard protocols such as message queues, web services, or even use a proprietary protocol. In the Java world, standards such as JCA allow adapters to be created in a vendor-neutral manner.
Data format

File format

A file format is a particular way that information is encoded for storage in a computer file.Since a disk drive, or indeed any computer storage, can store only bits, the computer must have some way of converting information to 0s and 1s and vice-versa. There are different kinds of formats for...

and transformation

Data transformation

In metadata and data warehouse, a data transformation converts data from a source data format into destination data.Data transformation can be divided into two steps:...

: To avoid every adapter having to convert data to/from every other applications' formats, EAI systems usually stipulate an application-independent (or common) data format. The EAI system usually provides a data transformation service as well to help convert between application-specific and common formats. This is done in two steps: the adapter converts information from the application's format to the bus's common format. Then, semantic transformations are applied on this (converting zip codes to city names, splitting/merging objects from one application into objects in the other applications, and so on).
Integration modules: An EAI system could be participating in multiple concurrent integration operations at any given time, each type of integration being processed by a different integration module. Integration modules subscribe to events of specific types and process notifications that they receive when these events occur. These modules could be implemented in different ways: on Java

Java (programming language)

Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities...

-based EAI systems, these could be web application

Web application

A web application is an application that is accessed over a network such as the Internet or an intranet. The term may also mean a computer software application that is coded in a browser-supported language and reliant on a common web browser to render the application executable.Web applications are...

s or EJBs

Enterprise JavaBean

Enterprise JavaBeans is a managed, server-side component architecture for modular construction of enterprise applications.The EJB specification is one of several Java APIs in the Java EE specification. EJB is a server-side model that encapsulates the business logic of an application...

or even POJO

Pojo

Pojo may refer to:* Pohja, the Swedish name for the Finnish municipality* POJO, abbreviation of Plain Old Java Object in computer programming...

s that conform to the EAI system's specifications.
Support for transaction

Database transaction

A transaction comprises a unit of work performed within a database management system against a database, and treated in a coherent and reliable way independent of other transactions...

s: When used for process integration, the EAI system also provides transactional consistency across applications by executing all integration operations across all applications in a single overarching distributed transaction (using two-phase commit protocols or compensating transaction

Compensating transaction

The execution of a business process consists of one or more transactions. Each transaction may consist of several individual operations yet, as a whole, it moves the system between consistent states....

s).

Communication architectures

Currently, there are many variations of thought on what constitutes the best infrastructure, component model, and standards structure for Enterprise Application Integration. There seems to be consensus that four components are essential for a modern enterprise application integration architecture:

A centralized broker that handles security, access, and communication. This can be accomplished through integration servers (like the School Interoperability Framework (SIF)
Schools Interoperability Framework
The Schools Interoperability Framework, Systems Interoperability Framework , or SIF, is a data sharing open specification for academic institutions from kindergarten through twelfth grade...

Zone Integration Servers) or through similar software like the enterprise service bus
Enterprise service bus
An enterprise service bus is a software architecture model used for designing and implementing the interaction and communication between mutually interacting software applications in Service Oriented Architecture...

(ESB) model that acts as a SOAP-oriented services manager.
An independent data model based on a standard data structure, also known as a canonical data model
Canonical
Canonical is an adjective derived from canon. Canon comes from the greek word κανών kanon, "rule" or "measuring stick" , and is used in various meanings....

. It appears that XML and the use of XML style sheets has become the de facto
De facto
De facto is a Latin expression that means "concerning fact." In law, it often means "in practice but not necessarily ordained by law" or "in practice or actuality, but not officially established." It is commonly used in contrast to de jure when referring to matters of law, governance, or...

and in some cases de jure
De jure
De jure is an expression that means "concerning law", as contrasted with de facto, which means "concerning fact".De jure = 'Legally', De facto = 'In fact'....

standard for this uniform business language.
A connector, or agent model where each vendor, application, or interface can build a single component that can speak natively to that application and communicate with the centralized broker.
A system model that defines the APIs, data flow and rules of engagement to the system such that components can be built to interface with it in a standardized way.

Although other approaches like connecting at the database or user-interface level have been explored, they have not been found to scale or be able to adjust. Individual applications can publish messages to the centralized broker and subscribe to receive certain messages from that broker. Each application only requires one connection to the broker. This central control approach can be extremely scalable and highly evolvable

Event-driven SOA

Event-driven SOA is a form of service-oriented architecture , combining the intelligence and proactiveness of event-driven architecture with the organizational capabilities found in service offerings...

.

Enterprise Application Integration is related to middleware

Middleware

technologies such as message-oriented middleware (MOM

Message-oriented middleware

), and data representation technologies such as XML

XML

Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....

. Other EAI technologies involve using web service

Web service

A Web service is a method of communication between two electronic devices over the web.The W3C defines a "Web service" as "a software system designed to support interoperable machine-to-machine interaction over a network". It has an interface described in a machine-processable format...

s as part of service-oriented architecture

Service-oriented architecture

In software engineering, a Service-Oriented Architecture is a set of principles and methodologies for designing and developing software in the form of interoperable services. These services are well-defined business functionalities that are built as software components that can be reused for...

as a means of integration. Enterprise Application Integration tends to be data centric. In the near future, it will come to include content integration

Enterprise Content Integration

Enterprise content integration is a middleware software technology, often used within large organizations, that connects together various types of computer systems that manage documents and digital content. ECI systems often work in tandem with other technologies such as enterprise content...

and business process

Business process

A business process or business method is a collection of related, structured activities or tasks that produce a specific service or product for a particular customer or customers...

es.

Implementation pitfalls

In 2003 it was reported that 70% of all EAI projects fail. Most of these failures are not due to the software itself or technical difficulties, but due to management issues. Integration Consortium European Chairman Steve Craggs has outlined the seven main pitfalls undertaken by companies using EAI systems and explains solutions to these problems.

Constant change: The very nature of EAI is dynamic and requires dynamic project managers to manage their implementation.
Shortage of EAI expert
Middleware analyst
Middleware analysts are computer software engineers with a specialization in products that connect two different computer systems together. These products can be open-source or proprietary...

s: EAI requires knowledge of many issues and technical aspects.
Competing standards: Within the EAI field, the paradox is that EAI standards themselves are not universal.
EAI is a tool paradigm: EAI is not a tool, but rather a system and should be implemented as such.
Building interfaces is an art: Engineering the solution is not sufficient. Solutions need to be negotiated with user departments to reach a common consensus on the final outcome. A lack of consensus on interface designs leads to excessive effort to map between various systems data requirements.
Loss of detail: Information that seemed unimportant at an earlier stage may become crucial later.
Accountability: Since so many departments have many conflicting requirements, there should be clear accountability for the system's final structure.

Other potential problems may arise in these areas:

Emerging Requirements: EAI implementations should be extensible and modular to allow for future changes.
Protectionism: The applications whose data is being integrated often belong to different departments that have technical, cultural, and political reasons for not wanting to share their data with other departments

Advantages and disadvantages

Advantages

Streamlines business processes and helps raise organizational efficiency
Ease of development and maintenance

Disadvantages

High initial development costs, especially for small and mid-sized businesses (SMBs)
Require a fair amount of up front business design, which many managers are not able to envision or not willing to invest in.

Most EAI projects usually start off as point-to-point efforts, quickly becoming unmanageable as the number of applications increase.

Future

EAI technologies are still being developed and there still is no consensus on the ideal approach or the correct group of technologies a company should use. There are many ongoing projects that provide support to design EAI solutions. They may range from proprietary projects like Microsoft BizTalk Server

Microsoft BizTalk Server

Microsoft BizTalk Server, often referred to as simply "BizTalk", is an Enterprise Service Bus. Through the use of "adapters" which are tailored to communicate with different software systems used in an enterprise, it enables companies to automate business processes...

, IBM WebSphere Message Broker

IBM WebSphere Message Broker

WebSphere Message Broker is IBM's integration broker from the WebSphere product family that allows business information to flow between disparate applications across multiple hardware and software platforms. Rules can be applied to the data flowing through the message broker to route and transform...

or open projects like Apache Camel

Apache Camel

Apache Camel is a rule-based routing and mediation engine which provides a Java object-based implementation of the Enterprise Integration Patterns using an API to configure routing and mediation rules...

and Spring Integration

Spring Integration

Spring Integration is an open source framework for Enterprise application integration. It is a lightweight framework that builds upon the core Spring framework. It is designed to enable the development of integration solutions typical of event-driven architectures and messaging-centric...

to academic projects like Guaraná DSL

Guaraná DSL

Guaraná DSL is a Domain-Specific Language to design enterprise application integration solutions at a high-level of abstraction. The resulting models are platform-independent, so engineers do not need to have skills on a low-level integration technology when designing their solutions...

. The future is to provide technologies that allow the design of EAI solutions at a high level of abstraction and use MDA to automatically transform the design into an executable solution. A common pitfall is to use other proprietary technologies that claim to be open and extensible but create vendor lock-in

Vendor lock-in

In economics, vendor lock-in, also known as proprietary lock-in or customer lock-in, makes a customer dependent on a vendor for products and services, unable to use another vendor without substantial switching costs...

Overview

Improving connectivity

Purposes

Integration patterns

Access patterns

Lifetime patterns

Topologies

Technologies

Communication architectures

Implementation pitfalls

Advantages and disadvantages

Future

See also

Initiatives and organizations

Commercial products

Open-source projects