Overlay (programming)
Encyclopedia
In a general computing sense, overlaying means "replacement of a block of stored instructions or data with another" Overlaying is a programming
method that allows programs to be larger than the computer's main memory. An embedded system
would normally use overlays because of the limitation of physical memory, which is internal memory for a system-on-chip and the lack of virtual memory
facilities.
blocks called overlays. The size of an overlay is limited according to memory constraints. The place in memory where an overlay is loaded is called an overlay region or destination region. Although the idea is to reuse the same block of main memory, multiple region systems could be defined. The regions can be different sizes. An overlay manager, possibly part of the operating system
, will load the required overlay from external memory into its destination region in order to be used. Some linkers provide support for overlays.
Overlay programming requires the program designer to be very aware of the size of each part of the program. This means using programming languages or assemblers that allow the designer or architect control over the size of the program and more importantly, the size of the overlay. This constraint adds many design difficulties that do not exist with virtual memory.
, most business applications are intended to run on platforms with virtual memory
. A developer on such a platform can design a program as if the memory constraint does not exist unless the program's working set
exceeds the available physical memory. Most importantly, the architect can focus on the problem being solved without the added design difficulty of forcing the processing into steps constrained by the overlay size. Thus, the designer can use higher-level programming languages that do not allow the programmer much control over size (e.g. Java
, C++
, Smalltalk
).
Still, overlays remain popular in embedded systems because many cannot use virtual memory. Many embedded systems are real-time systems. When a program enters a part that has been paged out, the virtual memory may not respond in time, causing a real-time computing
system to miss deadlines.
The delay of loading an overlay is more predictable. In addition, paging hardware is not without cost, and in many embedded systems sold in millions of units, the reduction in manufacturing costs from simpler hardware outweighs the extra cost of programmer time.
Even on platforms with virtual memory, software components such as codec
s may be decoupled to the point where they can be loaded in and out as needed.
model 5150, for example, and the base model PC didn't even have a floppy disk drive and relied on cassette storage and BASIC in ROM. (Few of these diskless model 5150 PCs were actually sold.)
Early PCs also had very little RAM by current standards; while a double sided double density 5¼" floppy disk held 360K, the RAM in PCs as they came from the factory between 1981 and 1985 was rarely significantly more than that; most of those PCs shipped with 64K, 128K, 256K, or perhaps 384K of system RAM. The original PC had between 16K and 64K depending on configuration. If one chose to add RAM to one's system, conventional RAM maxed out at 640K. Adding more memory beyond the 640K of "conventional memory" required resorting to extension standards like EMS
(expanded memory) and XMS
(extended memory). (XMS was impossible before the introduction of the 80286 CPU, ca. 1983.) The memory added using these methods was installed on ISA expansion cards and was slower to access than conventional RAM, because it had to be copied into and out of a "page frame" in the lower 1 MB address space in order for its contents to be used under DOS (or used at all with a pre-286 CPU). It was in these times of very limited RAM that the overlay originated. For DOS, the binary
files containing memory overlays had a defacto standard extension, .OVL. This file type was used among others by WordStar
and dBase
.
The GFA BASIC
compiler was able to produce .OVL files.
Computer programming
Computer programming is the process of designing, writing, testing, debugging, and maintaining the source code of computer programs. This source code is written in one or more programming languages. The purpose of programming is to create a program that performs specific operations or exhibits a...
method that allows programs to be larger than the computer's main memory. An embedded system
Embedded system
An embedded system is a computer system designed for specific control functions within a larger system. often with real-time computing constraints. It is embedded as part of a complete device often including hardware and mechanical parts. By contrast, a general-purpose computer, such as a personal...
would normally use overlays because of the limitation of physical memory, which is internal memory for a system-on-chip and the lack of virtual memory
Virtual memory
In computing, virtual memory is a memory management technique developed for multitasking kernels. This technique virtualizes a computer architecture's various forms of computer data storage , allowing a program to be designed as though there is only one kind of memory, "virtual" memory, which...
facilities.
Usage
The method assumes dividing a program into self-contained object codeObject code
Object code, or sometimes object module, is what a computer compiler produces. In a general sense object code is a sequence of statements in a computer language, usually a machine code language....
blocks called overlays. The size of an overlay is limited according to memory constraints. The place in memory where an overlay is loaded is called an overlay region or destination region. Although the idea is to reuse the same block of main memory, multiple region systems could be defined. The regions can be different sizes. An overlay manager, possibly part of the operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
, will load the required overlay from external memory into its destination region in order to be used. Some linkers provide support for overlays.
Overlay programming requires the program designer to be very aware of the size of each part of the program. This means using programming languages or assemblers that allow the designer or architect control over the size of the program and more importantly, the size of the overlay. This constraint adds many design difficulties that do not exist with virtual memory.
, most business applications are intended to run on platforms with virtual memory
Virtual memory
In computing, virtual memory is a memory management technique developed for multitasking kernels. This technique virtualizes a computer architecture's various forms of computer data storage , allowing a program to be designed as though there is only one kind of memory, "virtual" memory, which...
. A developer on such a platform can design a program as if the memory constraint does not exist unless the program's working set
Working set
Peter Denning defines “the working set of information W of a process at time t to be the collection of information referenced by the process during the process time interval ”. Typically the units of information in question are considered to be memory pages...
exceeds the available physical memory. Most importantly, the architect can focus on the problem being solved without the added design difficulty of forcing the processing into steps constrained by the overlay size. Thus, the designer can use higher-level programming languages that do not allow the programmer much control over size (e.g. Java
Java (programming language)
Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities...
, C++
C++
C++ is a statically typed, free-form, multi-paradigm, compiled, general-purpose programming language. It is regarded as an intermediate-level language, as it comprises a combination of both high-level and low-level language features. It was developed by Bjarne Stroustrup starting in 1979 at Bell...
, Smalltalk
Smalltalk
Smalltalk is an object-oriented, dynamically typed, reflective programming language. Smalltalk was created as the language to underpin the "new world" of computing exemplified by "human–computer symbiosis." It was designed and created in part for educational use, more so for constructionist...
).
Still, overlays remain popular in embedded systems because many cannot use virtual memory. Many embedded systems are real-time systems. When a program enters a part that has been paged out, the virtual memory may not respond in time, causing a real-time computing
Real-time computing
In computer science, real-time computing , or reactive computing, is the study of hardware and software systems that are subject to a "real-time constraint"— e.g. operational deadlines from event to system response. Real-time programs must guarantee response within strict time constraints...
system to miss deadlines.
The delay of loading an overlay is more predictable. In addition, paging hardware is not without cost, and in many embedded systems sold in millions of units, the reduction in manufacturing costs from simpler hardware outweighs the extra cost of programmer time.
Even on platforms with virtual memory, software components such as codec
Codec
A codec is a device or computer program capable of encoding or decoding a digital data stream or signal. The word codec is a portmanteau of "compressor-decompressor" or, more commonly, "coder-decoder"...
s may be decoupled to the point where they can be loaded in and out as needed.
PC/MS DOS
In the DOS era, overlays were popular because the operating system and many of the computer systems it ran on lacked virtual memory. Originally, PCs even lacked hard drives, so the only storage available for paging out virtual memory would have been 5-1/4" double-density floppy disk. IBM never sold a hard disk for the original IBM PCIBM PC
The IBM Personal Computer, commonly known as the IBM PC, is the original version and progenitor of the IBM PC compatible hardware platform. It is IBM model number 5150, and was introduced on August 12, 1981...
model 5150, for example, and the base model PC didn't even have a floppy disk drive and relied on cassette storage and BASIC in ROM. (Few of these diskless model 5150 PCs were actually sold.)
Early PCs also had very little RAM by current standards; while a double sided double density 5¼" floppy disk held 360K, the RAM in PCs as they came from the factory between 1981 and 1985 was rarely significantly more than that; most of those PCs shipped with 64K, 128K, 256K, or perhaps 384K of system RAM. The original PC had between 16K and 64K depending on configuration. If one chose to add RAM to one's system, conventional RAM maxed out at 640K. Adding more memory beyond the 640K of "conventional memory" required resorting to extension standards like EMS
Expanded memory
In DOS memory management, expanded memory is a system of bank switching introduced April 24, 1985 that provided additional memory to DOS programs beyond the limit of conventional memory. Expanded memory uses parts of the address space normally dedicated to communication with peripherals for program...
(expanded memory) and XMS
Extended memory
In DOS memory management, extended memory refers to memory above the first megabyte of address space in an IBM PC or compatible with an 80286 or later processor. The term is mainly used under the DOS and Windows operating systems...
(extended memory). (XMS was impossible before the introduction of the 80286 CPU, ca. 1983.) The memory added using these methods was installed on ISA expansion cards and was slower to access than conventional RAM, because it had to be copied into and out of a "page frame" in the lower 1 MB address space in order for its contents to be used under DOS (or used at all with a pre-286 CPU). It was in these times of very limited RAM that the overlay originated. For DOS, the binary
Binary file
A binary file is a computer file which may contain any type of data, encoded in binary form for computer storage and processing purposes; for example, computer document files containing formatted text...
files containing memory overlays had a defacto standard extension, .OVL. This file type was used among others by WordStar
WordStar
WordStar is a word processor application, published by MicroPro International, originally written for the CP/M operating system but later ported to DOS, that enjoyed a dominant market share during the early to mid-1980s. Although Seymour I...
and dBase
DBASE
dBase II was the first widely used database management system for microcomputers. It was originally published by Ashton-Tate for CP/M, and later on ported to the Apple II and IBM PC under DOS...
.
The GFA BASIC
GFA BASIC
GFA BASIC is a dialect of the BASIC programming language, by Frank Ostrowski. The first version was finished in 1986. In the mid and late 80s, it became very popular for the Atari ST homecomputer range . Later, ports for the Commodore Amiga, DOS and Windows were marketed...
compiler was able to produce .OVL files.