DLL hell
Encyclopedia
In computing
Computing
Computing is usually defined as the activity of using and improving computer hardware and software. It is the computer-specific part of information technology...

, DLL Hell is a term for the complications that arise when working with dynamic link libraries (DLLs) used with Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...

 operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...

s, particularly legacy 16-bit editions which all run in a single memory space.
While the phrase is Windows-specific and a derivation of the general cross-platform phrase "dependency hell
Dependency hell
Dependency hell is a colloquial term for the frustration of some software users who have installed software packages which have dependencies on specific versions of other software packages. This was mainly attributable to old Linux package managers...

", the rhyme "DLL hell" makes its use popular for discussing a general Windows-related dependency hell case.

DLL Hell often shows up in a Windows alert pop-up that reports something similar to "A Required DLL File, Z.DLL, was not found" or "The procedure entry point Y couldn't be located in X.DLL" when users try to run an application, or during startup. This can also manifest itself more quietly as applications not working properly.
It takes a number of forms, as detailed below.

Problems

There are a number of problems commonly encountered with DLLs – especially after numerous applications have been installed and uninstalled on a system.
The difficulties include conflicts between DLL versions, difficulty in obtaining required DLLs, and having many unnecessary DLL copies.

Incompatible versions

A particular version of a library can be compatible with some (and incompatible with other) programs that use it. Windows has been particularly vulnerable to this because of its emphasis on dynamic linking of C++ libraries and Object Linking and Embedding
Object Linking and Embedding
Object Linking and Embedding is a technology developed by Microsoft that allows embedding and linking to documents and other objects. For developers, it brought OLE Control eXtension , a way to develop and use custom user interface elements...

 (OLE) objects. C++ classes export many methods, and a single change to the class (such as a new virtual method) can make it incompatible with programs that were built against an earlier version. Object Linking and Embedding has some very strict rules to prevent this—interfaces are required to be stable and memory managers are not shared. But this is not enough, for the semantics of a class can change. A "bug fix" for one application may be the removal of a "feature" from another. Before Windows 2000
Windows 2000
Windows 2000 is a line of operating systems produced by Microsoft for use on personal computers, business desktops, laptops, and servers. Windows 2000 was released to manufacturing on 15 December 1999 and launched to retail on 17 February 2000. It is the successor to Windows NT 4.0, and is the...

, Windows was vulnerable to this because the COM
Component Object Model
Component Object Model is a binary-interface standard for software componentry introduced by Microsoft in 1993. It is used to enable interprocess communication and dynamic object creation in a large range of programming languages...

 class table was shared across all users and processes. Only one COM object, in one DLL/EXE could be declared as having a specific COM ClassID. If any program needed to create an instance of that class, it got whatever was the current centrally registered implementation. As a result, an installation of a program that installs a new version of a common object may inadvertently break other programs that were previously installed.

DLL stomping

A common and troublesome problem occurs when a newly-installed program overwrites a working system DLL with an earlier, incompatible version. A famous example of this was the ctl3d.dll and ctl3dv2.dll libraries for Windows 3.1, which had numerous updated versions appearing in the wild. DLL Stomping occurs because:
  • Microsoft in the past distributed runtime DLLs as shared system components, (originally C:\WINDOWS and C:\WINDOWS\SYSTEM), as a way of efficiently sharing code in a shared-memory OS with limited RAM and disk space. Consequently, third-party developers also distributed these in such a manner.
  • Application installers are typically executed in a privileged security context that has access to install DLLs into the system directories, and to edit the system Registry to register new DLLs as COM
    Component Object Model
    Component Object Model is a binary-interface standard for software componentry introduced by Microsoft in 1993. It is used to enable interprocess communication and dynamic object creation in a large range of programming languages...

     objects. A poorly written or misconfigured installer can therefore downgrade a system library on legacy versions of Windows on which Windows File Protection
    Windows File Protection
    Windows File Protection , a sub-system included in Microsoft Windows operating systems of the Windows 2000 and Windows XP era, aims to prevent programs from replacing critical Windows system files. Protecting core system files mitigates problems such as DLL hell with programs and the operating system...

     or Windows Resource Protection
    Windows Resource Protection
    Windows Resource Protection is a feature in Windows Vista that replaces Windows File Protection. It protects registry keys and folders in addition to critical system files. The way it protects resources differs entirely from the method used by Windows File Protection.- Overview :Windows File...

     does not roll back the change. On Windows Vista and later, only the "trusted installer" account can make changes to core operating system libraries.
  • Windows applications are permitted to include OS updates in their own installation programs. That is, many Microsoft DLLs are redistributable, meaning the applications get to include them if they need the services of the particular libraries.
  • Before Windows Installer
    Windows Installer
    The Windows Installer is a software component used for the installation, maintenance, and removal of software on modern Microsoft Windows systems...

    , Windows installers historically were commercial products; many people attempted to write their own installers, overlooking or mishandling versioning problems in the process.
  • Some development environments did not automatically add a version resource in their compiled libraries, so many developers overlooked this aspect. Checking file dates, overwriting existing files or skipping the copy operation if the DLL was already installed were the only options available in lieu of correct versioning.
  • Sometimes the OS itself removes or replaces DLLs with older or obsolete versions. For example, Windows 2000 would install black and white printer DLLs on top of color-aware DLLs, if a black and white printer was installed after the color printer.

Incorrect COM registration

In COM
Component Object Model
Component Object Model is a binary-interface standard for software componentry introduced by Microsoft in 1993. It is used to enable interprocess communication and dynamic object creation in a large range of programming languages...

 and other parts of Windows, prior to the introduction of side-by-side Registry-free assemblies, the Registry
Windows registry
The Windows Registry is a hierarchical database that stores configuration settings and options on Microsoft Windows operating systems. It contains settings for low-level operating system components as well as the applications running on the platform: the kernel, device drivers, services, SAM, user...

 was used for determining which underlying DLL to use. If a different version of a module was registered, this DLL would be loaded instead of the expected one. This scenario could be caused by conflicting installations that register different versions of the same libraries, in which case the last installation would prevail.

Shared in-memory modules

16-bit versions of Windows load only one instance of any given DLL; all applications reference the same in-memory copy, until no applications are using it and it is unloaded from memory. (For 32-bit and 64-bit versions of Windows, inter-process sharing occurs only where different executables load a module from exactly the same directory; the code but not the stack
Stack (data structure)
In computer science, a stack is a last in, first out abstract data type and linear data structure. A stack can have any abstract data type as an element, but is characterized by only three fundamental operations: push, pop and stack top. The push operation adds a new item to the top of the stack,...

 is shared between processes through a process called "memory mapping".) Thus, even when the desired DLL is located in a directory where it can be expected to be found, such as in the system directory or the application directory, neither of these instances will be used if another application has started with an incompatible version from a third directory. This issue can manifest itself as a 16-bit application error that occurs only when another 16-bit application is running.

Lack of serviceability

In direct conflict with the DLL stomping problem: If updates to a DLL do not affect all applications which use it, then it becomes much harder to 'service' the DLL - that is, to eliminate problems which exist in the current versions of the DLL. (Security fixes are a particularly compelling, and painful, case.) Instead of fixing just the latest version of the DLL, the implementor must ideally make their fixes, and test them for compatibility, on every released version of the DLL.

Causes

DLL incompatibility has been caused by:
  • Memory constraints, combined with lack of separation of process memory space in 16-bit versions of Windows
  • Lack of enforced standard versioning, naming, and file system location schemata for DLLs;
  • Lack of an enforced standard method for software installation and removing (package management);
  • Centralized authoritative support for DLL application binary interface
    Application binary interface
    In computer software, an application binary interface describes the low-level interface between an application program and the operating system or another application.- Description :...

     management and safeguards, allowing incompatible DLLs with the same file name and internal version numbers to be released;
  • Overly simplified management tools, preventing the identification of changed or problematic DLLs by users and administrators.
  • Developers breaking backward-compatibility of functions in shared modules;
  • Microsoft releasing out-of-band updates to operating system runtime components;
  • Inability of earlier versions of Windows to run side-by-side conflicting versions of the same library;
  • Reliance on the current directory or %PATH% environment variable, both of which vary over time and from system to system, to find dependent DLLs (instead of loading them from an explicitly configured directory.)
  • Developers re-using the ClassIDs from sample applications for the COM interfaces of their applications, rather than generating their own new GUIDs
    Globally Unique Identifier
    A globally unique identifier is a unique reference number used as an identifier in computer software. The term GUID also is used for Microsoft's implementation of the Universally unique identifier standard....

    .


DLL Hell was a very common phenomenon on pre-Windows NT versions of Microsoft operating systems, the primary cause being that the 16-bit operating systems did not restrict processes to their own memory space, thereby not allowing them to load their own version of a shared module that they were compatible with. Application installers were expected to be good citizens and verify DLL version information before overwriting the existing system DLLs. Standard tools to simplify application deployment (which always involves shipping the dependent operating system DLLs) were provided by Microsoft and other 3rd party tools vendors. Microsoft even required application vendors to use a standard installer and have their installation program certified to work correctly, before being granted use of the Microsoft logo. The good citizen installer approach did not mitigate the problem, as the rise in popularity of the Internet provided more opportunities to obtain non-conforming applications.

Use by malware

The ambiguity with which DLLs that are not fully qualified can be loaded in the Windows operating system has been exploited by malware in recent years, opening a new class of vulnerability that affects applications from many different software vendors, as well as Windows itself.

Static linking

One of the simplest solutions to DLL Hell in an application is to statically link
Static library
In computer science, a static library or statically-linked library is a set of routines, external functions and variables which are resolved in a caller at compile-time and copied into a target application by a compiler, linker, or binder, producing an object file and a stand-alone executable...

 against all the libraries. This is common in C/C++ applications, where, instead of having to worry about which version of MFC42.DLL is installed, the application is compiled to be statically linked against the same libraries. This eliminates the DLLs entirely, and is viable for standalone applications which only use libraries which offer a static option, like Microsoft Foundation Class Library
Microsoft Foundation Class Library
The Microsoft Foundation Class Library is a library that wraps portions of the Windows API in C++ classes, including functionality that enables them to use a default application framework...

 does. The main purpose of DLLs (runtime library sharing between programs to reduce memory overhead) is sacrificed though giving more software bloat
Software bloat
Software bloat is a process whereby successive versions of a computer program include an increasing proportion of unnecessary features that are not used by end users, or generally use more system resources than necessary, while offering little or no benefit to its users.-Causes:Software developers...

 and can prevent proper propagation of security fixes.

Windows File Protection

The DLL overwriting problem (referred to as DLL Stomping inside Microsoft) was somewhat reduced with Windows File Protection
Windows File Protection
Windows File Protection , a sub-system included in Microsoft Windows operating systems of the Windows 2000 and Windows XP era, aims to prevent programs from replacing critical Windows system files. Protecting core system files mitigates problems such as DLL hell with programs and the operating system...

 (WFP) which was introduced in Windows 2000
Windows 2000
Windows 2000 is a line of operating systems produced by Microsoft for use on personal computers, business desktops, laptops, and servers. Windows 2000 was released to manufacturing on 15 December 1999 and launched to retail on 17 February 2000. It is the successor to Windows NT 4.0, and is the...

. This prevents unauthorized applications from overwriting system DLLs, unless they use the specific Windows API
Windows API
The Windows API, informally WinAPI, is Microsoft's core set of application programming interfaces available in the Microsoft Windows operating systems. It was formerly called the Win32 API; however, the name "Windows API" more accurately reflects its roots in 16-bit Windows and its support on...

s that permit this. There is still a risk that updates from Microsoft (such as Internet Explorer 7
Internet Explorer 7
Windows Internet Explorer 7 is a web browser released by Microsoft in October 2006. Internet Explorer 7 is part of a long line of versions of Internet Explorer and was the first major update to the browser in more than 5 years...

) are incompatible with existing applications.

Third-party applications cannot stomp on OS files unless they bundle legitimate Windows updates with their installer, or if they disable the Windows File Protection
Windows File Protection
Windows File Protection , a sub-system included in Microsoft Windows operating systems of the Windows 2000 and Windows XP era, aims to prevent programs from replacing critical Windows system files. Protecting core system files mitigates problems such as DLL hell with programs and the operating system...

 service during installation, and on Windows Vista or later also take ownership of system files and grant themselves access. The SFC
System File Checker
System File Checker is a utility in Microsoft Windows that allows users to scan for and restore corruptions in Windows system files. This utility is available on Windows 98, Windows 2000, Windows XP, and Windows Server 2003...

 utility could revert these changes at any time.

Running conflicting DLLs simultaneously

The solutions here consist of having different copies of the same DLLs for each application, both on-disk and in memory.

An easy manual solution to conflicts was placing the different versions of the problem DLL into the applications' folders, rather than a common system-wide folder. This works in general as long as the application is 32-bit or 64-bit, and that the DLL does not use shared memory. In the case of 16-bit applications, the two applications cannot be executed simultaneously on a 16-bit platform, or in the same 16-bit virtual machine under a 32-bit operating system. OLE
Object Linking and Embedding
Object Linking and Embedding is a technology developed by Microsoft that allows embedding and linking to documents and other objects. For developers, it brought OLE Control eXtension , a way to develop and use custom user interface elements...

 prevented this before Windows XP, because earlier versions of Windows had a single Registry of COM objects for all applications.

Windows XP introduced a solution called Side-by-side assembly
Side-by-side assembly
Side-by-side technology is a standard for executable files in Microsoft Windows XP and later versions that attempts to reduce DLL hell. Side-by-side technology is also known as WinSxS or SxS. Executables that include an SxS manifest are designated SxS assemblies.DLL hell designates a group of...


(MSDN page), which loads separate copies of DLLs for each application that requires them (and thus allows applications that require conflicting DLLs to run simultaneously). This approach eliminates conflicts by allowing applications to load unique versions of a module into their address space, while preserving the primary benefit of sharing DLLs between applications (i.e. reducing memory use) by using memory mapping techniques to share common code between different processes that do still use the same module. Yet DLLs using shared data between multiple processes cannot take this approach. One negative side effect is orphaned instances of DLLs may not be updated during automated processes.

Portable Applications

Portable Applications (also known as "portable apps") are an effective way to reduce DLL problems, since every program contains their required DLLs bundled-in (also sometimes called "private libraries"). Mechanism is the Windows shared library policy which favours locally (application directory) available DLLs before the one in the system library path. Sometimes applications run in a "bubble", using Application virtualization
Application Virtualization
Application virtualization is an umbrella term that describes software technologies that improve portability, manageability and compatibility of applications by encapsulating them from the underlying operating system on which they are executed. A fully virtualized application is not installed in...

, which avoids installing DLL files into the operating system.

Other countermeasures

There are other countermeasures to avoid DLL Hell, some of which may have to be used simultaneously:
Some other features that help to mitigate the problem are
  • Installation tools are now bundled into Microsoft Visual Studio
    Microsoft Visual Studio
    Microsoft Visual Studio is an integrated development environment from Microsoft. It is used to develop console and graphical user interface applications along with Windows Forms applications, web sites, web applications, and web services in both native code together with managed code for all...

    , one of the main environments for Windows development. These tools perform version checking before DLL installation, and can include predefined installation packages in a .MSI installation. This allows third party applications to integrate OS component updates without having to write their own installers for these components.
  • System Restore
    System Restore
    System Restore is a component of Microsoft's Windows Me, Windows XP, Windows Vista and Windows 7, but not Windows 2000, operating systems that allows for the rolling back of system files, registry keys, installed programs, etc., to a previous state in the event of system malfunction or failure.The...

     can recover a system from a bad installation, including registry damage. While this does not prevent the problem, it makes it easier to recover from.
  • WinSxS (Windows Side-by-Side) folder, which allows multiple versions of the same libraries to co-exist.
  • Run 16-bit applications in separate memory space under a 32-bit version of Windows, to allow two applications to use conflicting versions of the same DLL at the same time.
  • Use a version of Windows that includes Windows File Protection
    Windows File Protection
    Windows File Protection , a sub-system included in Microsoft Windows operating systems of the Windows 2000 and Windows XP era, aims to prevent programs from replacing critical Windows system files. Protecting core system files mitigates problems such as DLL hell with programs and the operating system...

    . Windows Me
    Windows Me
    Windows Millennium Edition, or Windows Me , is a graphical operating system released on September 14, 2000 by Microsoft, and was the last operating system released in the Windows 9x series. Support for Windows Me ended on July 11, 2006....

     and Windows 2000
    Windows 2000
    Windows 2000 is a line of operating systems produced by Microsoft for use on personal computers, business desktops, laptops, and servers. Windows 2000 was released to manufacturing on 15 December 1999 and launched to retail on 17 February 2000. It is the successor to Windows NT 4.0, and is the...

    , both released in 2000, support this form of system file protection, as do Windows XP
    Windows XP
    Windows XP is an operating system produced by Microsoft for use on personal computers, including home and business desktops, laptops and media centers. First released to computer manufacturers on August 24, 2001, it is the second most popular version of Windows, based on installed user base...

     and Windows Server 2003
    Windows Server 2003
    Windows Server 2003 is a server operating system produced by Microsoft, introduced on 24 April 2003. An updated version, Windows Server 2003 R2, was released to manufacturing on 6 December 2005...

    . Its replacement, Windows Resource Protection
    Windows Resource Protection
    Windows Resource Protection is a feature in Windows Vista that replaces Windows File Protection. It protects registry keys and folders in addition to critical system files. The way it protects resources differs entirely from the method used by Windows File Protection.- Overview :Windows File...

    , was introduced in Windows Vista and Windows Server 2008, and uses a different method of protecting system files from being changed.
  • Registration-free COM: Windows XP
    Windows XP
    Windows XP is an operating system produced by Microsoft for use on personal computers, including home and business desktops, laptops and media centers. First released to computer manufacturers on August 24, 2001, it is the second most popular version of Windows, based on installed user base...

     introduced a new mode of COM object registration called "Registration-free COM" that was not well-publicized due to the simultaneous release of information related to .NET. This feature makes it possible for applications that need to install COM objects to store all the required COM registry information in the application's directory, instead of in the global registry, where strictly speaking if only a single application will ever use it is all that is needed. Thus, it provides a mechanism for multiple versions of the same DLL to be present at the same time as needed to cater for multiple applications (Microsoft calls this "Side-by-Side Assembly
    Side-by-side assembly
    Side-by-side technology is a standard for executable files in Microsoft Windows XP and later versions that attempts to reduce DLL hell. Side-by-side technology is also known as WinSxS or SxS. Executables that include an SxS manifest are designated SxS assemblies.DLL hell designates a group of...

    "). DLL hell can be substantially avoided using Registration-free COM, the only limitation being it requires at least Windows XP
    Windows XP
    Windows XP is an operating system produced by Microsoft for use on personal computers, including home and business desktops, laptops and media centers. First released to computer manufacturers on August 24, 2001, it is the second most popular version of Windows, based on installed user base...

     or later Windows versions and that it must not be used for EXE COM servers or system-wide components such as MDAC
    Microsoft Data Access Components
    Microsoft Data Access Components is a framework of interrelated Microsoft technologies that allows programmers a uniform and comprehensive way of developing applications that can access almost any data store. Its components include: ActiveX Data Objects , OLE DB, and Open Database Connectivity...

    , MSXML
    MSXML
    Microsoft XML Core Services is a set of services that allow applications written in JScript, VBScript, and Microsoft development tools to build Windows-native XML-based applications...

    , DirectX
    DirectX
    Microsoft DirectX is a collection of application programming interfaces for handling tasks related to multimedia, especially game programming and video, on Microsoft platforms. Originally, the names of these APIs all began with Direct, such as Direct3D, DirectDraw, DirectMusic, DirectPlay,...

     or Internet Explorer
    Internet Explorer
    Windows Internet Explorer is a series of graphical web browsers developed by Microsoft and included as part of the Microsoft Windows line of operating systems, starting in 1995. It was first released as part of the add-on package Plus! for Windows 95 that year...

    .
  • Shipping the operating system with a capable package management system
    Package management system
    In software, a package management system, also called package manager, is a collection of software tools to automate the process of installing, upgrading, configuring, and removing software packages for a computer's operating system in a consistent manner...

     that is able to track the DLL dependencies, encouraging the use of the package manager and discouraging manual installation of DLLs. Windows Installer
    Windows Installer
    The Windows Installer is a software component used for the installation, maintenance, and removal of software on modern Microsoft Windows systems...

    , included with Windows Me
    Windows Me
    Windows Millennium Edition, or Windows Me , is a graphical operating system released on September 14, 2000 by Microsoft, and was the last operating system released in the Windows 9x series. Support for Windows Me ended on July 11, 2006....

    , Windows 2000
    Windows 2000
    Windows 2000 is a line of operating systems produced by Microsoft for use on personal computers, business desktops, laptops, and servers. Windows 2000 was released to manufacturing on 15 December 1999 and launched to retail on 17 February 2000. It is the successor to Windows NT 4.0, and is the...

     and all later versions provides this functionality.
  • Having a central database or authority for DLL conflict resolution and software distribution. Changes to a library can be submitted to this authority; thus, it can make sure compatibility is preserved in the developed branches. If some older software is incompatible with the current library, the authority can provide a compatibility interface for it, or bundle the old version as a distinct package.
  • If software developers need to customize a library, and if the main library release is unlikely to incorporate the changes that they need, they can ship the customized DLL for the program's private use (commonly by placing it in the program's private directory) or statically link the program against the customized library.
  • While DLLs are best for modularizing applications and the system's components and as third-party libraries, their usage is not imperative in all cases. For example, if an application needs a library that will not be used anywhere else, it can be linked statically, with no space penalty and with a speed gain.
  • Windows Vista and later use a special TrustedInstaller service to install operating system files. Other user accounts, including the SYSTEM, have no access to overwrite core system binaries. Windows 7 expands this functionality to some critical parts of the Registry.
  • Web-based applications avoid many side-by-side problems by running the bulk of the code on a server and using a browser interface on the client.

See also

  • Dependency hell
    Dependency hell
    Dependency hell is a colloquial term for the frustration of some software users who have installed software packages which have dependencies on specific versions of other software packages. This was mainly attributable to old Linux package managers...

  • Portable application
    Portable application
    A portable application , sometimes also called standalone, is a computer software program designed to run independently from an operating system...

  • Portable application creators
    Portable application creators
    Portable application creators allow the creation of portable applications . They usually use application virtualization.- Creators of independent portable applications :...

  • JAR hell

External links

The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
 
x
OK