Reverse engineering
Encyclopedia
Reverse engineering is the process
of discovering the technological principles of a device, object, or system through analysis of its structure, function
, and operation. It often involves taking something (e.g., a mechanical device
, electronic component
, software program, or biological, chemical, or organic matter) apart and analyzing its workings in detail to be used in maintenance, or to try to make a new device or program that does the same thing without using or simply duplicating (without understanding) the original.
Reverse engineering has its origins in the analysis of hardware for commercial or military advantage. The purpose is to deduce design decisions from end products with little or no additional knowledge about the procedures involved in the original production. The same techniques are subsequently being researched for application to legacy software systems, not for industrial or defence ends, but rather to replace incorrect, incomplete, or otherwise unavailable documentation.
(CAD) has become more popular, reverse engineering has become a viable method to create a 3D virtual model of an existing physical part for use in 3D CAD, CAM
, CAE
or other software. The reverse-engineering process involves measuring an object and then reconstructing it as a 3D model. The physical object can be measured using 3D scanning
technologies like CMMs
, laser scanners, structured light digitizers, or Industrial CT Scanning
(computed tomography). The measured data alone, usually represented as a point cloud
, lacks topological information and is therefore often processed and modeled into a more usable format such as a triangular-faced mesh, a set of NURBS
surfaces, or a CAD model.
Reverse engineering is also used by businesses to bring existing physical geometry into digital product development environments, to make a digital 3D record of their own products, or to assess competitors' products. It is used to analyse, for instance, how a product works, what it does, and what components it consists of, estimate costs, and identify potential patent
infringement, etc.
Value engineering
is a related activity also used by businesses. It involves de-constructing and analysing products, but the objective is to find opportunities for cost cutting.
. From their paper, they state, "Reverse engineering is the process of analyzing a subject system to create representations of the system at a higher level of abstraction." It can also be seen as "going backwards through the development cycle". In this model, the output of the implementation phase (in source code form) is reverse-engineered back to the analysis phase, in an inversion of the traditional waterfall model
. Reverse engineering is a process of examination only: the software system under consideration is not modified (which would make it re-engineering
). Software anti-tamper technology is used to deter both reverse engineering and re-engineering of proprietary software and software-powered systems. In practice, two main types of reverse engineering emerge. In the first case, source code is already available for the software, but higher-level aspects of the program, perhaps poorly documented or documented but no longer valid, are discovered. In the second case, there is no source code available for the software, and any efforts towards discovering one possible source code for the software are regarded as reverse engineering. This second usage of the term is the one most people are familiar with. Reverse engineering of software can make use of the clean room design
technique to avoid copyright infringement.
On a related note, black box testing
in software engineering
has a lot in common with reverse engineering. The tester usually has the API
, but their goals are to find bugs and undocumented features by bashing the product from outside.
Other purposes of reverse engineering include security auditing, removal of copy protection ("cracking
"), circumvention of access restrictions often present in consumer electronics
, customization of embedded systems (such as engine management systems), in-house repairs or retrofits, enabling of additional features on low-cost "crippled" hardware (such as some graphics card chip-sets), or even mere satisfaction of curiosity.
BIOS
which launched the historic IBM PC compatible
industry that has been the overwhelmingly dominant computer hardware platform for many years. An example of a group that reverse-engineers software for enjoyment (and to distribute registration cracks) is CORE which stands for "Challenge Of Reverse Engineering". Reverse engineering of software is protected in the U.S. by the fair use
exception in copyright law
. The Samba software, which allows systems that are not running Microsoft Windows
systems to share files with systems that are, is a classic example of software reverse engineering, since the Samba project had to reverse-engineer unpublished information about how Windows file sharing worked, so that non-Windows computers could emulate it. The Wine
project does the same thing for the Windows API
, and OpenOffice.org
is one party doing this for the Microsoft Office
file formats. The ReactOS
project is even more ambitious in its goals, as it strives to provide binary (ABI and API) compatibility with the current Windows OSes of the NT branch, allowing software and drivers written for Windows to run on a clean-room reverse-engineered GPL free software or open-source counterpart.
The three main groups of software reverse engineering are
tools refer to the process of importing and analysing source code to generate UML diagrams as "reverse engineering". See List of UML tools.
are sets of rules that describe message formats and how messages are exchanged (i.e., the protocol state-machine). Accordingly, the problem of protocol reverse-engineering can be partitioned into two subproblems; message format and state-machine reverse-engineering.
The message formats have traditionally been reverse-engineered through a tedious manual process, which involved analysis of how protocol implementations process messages, but recent research proposed a number of automatic solutions. Typically, these automatic approaches either group observed messages into clusters using various clustering analyses, or emulate the protocol implementation tracing the message processing.
There has been less work on reverse-engineering of state-machines of protocols. In general, the protocol state-machines can be learned either through a process of offline learning
, which passively observes communication and attempts to build the most general state-machine accepting all observed sequences of messages, and online learning, which allows interactive generation of probing sequences of messages and listening to responses to those probing sequences. In general, offline learning of small state-machines is known to be NP-complete
, while online learning can be done in polynomial time. An automatic offline approach has been demonstrated by Comparetti et al. and an online approach very recently by Cho et al.
Other components of typical protocols, like encryption and hash functions, can be reverse-engineered automatically as well. Typically, the automatic approaches trace the execution of protocol implementations and try to detect buffers in memory holding unencrypted packets.
. With this technique, it is possible to reveal the complete hardware and software part of the smart card. The major problem for the attacker is to bring everything into the right order to find out how everything works. Engineers try to hide keys and operations by mixing up memory positions, for example, bus scrambling.
In some cases, it is even possible to attach a probe to measure voltages while the smart card is still operational. Engineers employ sensors to detect and prevent this attack. This attack is not very common because it requires a large investment in effort and special equipment that is generally only available to large chip manufacturers. Furthermore, the payoff from this attack is low since other security techniques are often employed such as shadow accounts.
operations. It was often used during the Second World War and the Cold War
. Well-known examples from WWII and later include:
s, reverse-engineering the artifact or process is often lawful as long as it is obtained legitimately. Patent
s, on the other hand, need a public disclosure of an invention
, and therefore, patented items do not necessarily have to be reverse-engineered to be studied. (However, an item produced under one or more patents could also include other technology that is not patented and not disclosed.) One common motivation of reverse engineers is to determine whether a competitor's product contains patent infringement
s or copyright infringement
s.
The reverse engineering of software in the US is generally a breach of contract as most EULAs specifically prohibit it, and courts have found such contractual prohibitions to override the copyright law; see Bowers v. Baystate Technologies
.
Sec. 103(f) of the DMCA
(17 U.S.C. § 1201 (f)) says that if you legally obtain a program that is protected, you are allowed to reverse-engineer and circumvent the protection to achieve interoperability between computer programs (i.e., the ability to exchange and make use of information). The section states:
In 2009, the EU Computer Program Directive was superseded and the directive now states:
Process (engineering)
In engineering a process is a set of interrelated tasks that, together, transform inputs into outputs. These tasks may be carried out by people, nature, or machines using resources; so an engineering process must be considered in the context of the agents carrying out the tasks, and the resource...
of discovering the technological principles of a device, object, or system through analysis of its structure, function
Function (engineering)
In engineering, a function is interpreted as a specific process, action or task that a system is able to perform .-In engineering design:In the lifecycle of engineering projects, there are usually distinguished subsequently: Requirements and Functional specification documents. The Requirements...
, and operation. It often involves taking something (e.g., a mechanical device
Machine
A machine manages power to accomplish a task, examples include, a mechanical system, a computing system, an electronic system, and a molecular machine. In common usage, the meaning is that of a device having parts that perform or assist in performing any type of work...
, electronic component
Electronic component
An electronic component is a basic electronic element and may be available in a discrete form having two or more electrical terminals . These are intended to be connected together, usually by soldering to a printed circuit board, in order to create an electronic circuit with a particular function...
, software program, or biological, chemical, or organic matter) apart and analyzing its workings in detail to be used in maintenance, or to try to make a new device or program that does the same thing without using or simply duplicating (without understanding) the original.
Reverse engineering has its origins in the analysis of hardware for commercial or military advantage. The purpose is to deduce design decisions from end products with little or no additional knowledge about the procedures involved in the original production. The same techniques are subsequently being researched for application to legacy software systems, not for industrial or defence ends, but rather to replace incorrect, incomplete, or otherwise unavailable documentation.
Motivation
Reasons for reverse engineering:- InteroperabilityInteroperabilityInteroperability is a property referring to the ability of diverse systems and organizations to work together . The term is often used in a technical systems engineering sense, or alternatively in a broad sense, taking into account social, political, and organizational factors that impact system to...
. - Lost documentation: Reverse engineering often is done because the documentation of a particular device has been lost (or was never written), and the person who built it is no longer available. Integrated circuitIntegrated circuitAn integrated circuit or monolithic integrated circuit is an electronic circuit manufactured by the patterned diffusion of trace elements into the surface of a thin substrate of semiconductor material...
s often seem to have been designed on obsolete, proprietary systems, which means that the only way to incorporate the functionality into new technology is to reverse-engineer the existing chip and then re-design it. - Product analysis. To examine how a product works, what components it consists of, estimate costs, and identify potential patent infringementPatent infringementPatent infringement is the commission of a prohibited act with respect to a patented invention without permission from the patent holder. Permission may typically be granted in the form of a license. The definition of patent infringement may vary by jurisdiction, but it typically includes using or...
. - Digital update/correction. To update the digital version (e.g. CAD model) of an object to match an "as-built" condition.
- Security auditing.
- Acquiring sensitive data by disassembling and analysing the design of a system component.
- Military or commercial espionageEspionageEspionage or spying involves an individual obtaining information that is considered secret or confidential without the permission of the holder of the information. Espionage is inherently clandestine, lest the legitimate holder of the information change plans or take other countermeasures once it...
. Learning about an enemy's or competitor's latest research by stealing or capturing a prototype and dismantling it. - Removal of copy protectionCopy protectionCopy protection, also known as content protection, copy obstruction, copy prevention and copy restriction, refer to techniques used for preventing the reproduction of software, films, music, and other media, usually for copyright reasons.- Terminology :Media corporations have always used the term...
, circumvention of access restrictions. - Creation of unlicensed/unapproved duplicates.
- Materials harvesting, sorting, or scrapping.
- Academic/learning purposes.
- Curiosity.
- Competitive technical intelligence (understand what your competitor is actually doing, versus what they say they are doing).
- Learning: learn from others' mistakes. Do not make the same mistakes that others have already made and subsequently corrected.
Reverse engineering of machines
As computer-aided designComputer-aided design
Computer-aided design , also known as computer-aided design and drafting , is the use of computer technology for the process of design and design-documentation. Computer Aided Drafting describes the process of drafting with a computer...
(CAD) has become more popular, reverse engineering has become a viable method to create a 3D virtual model of an existing physical part for use in 3D CAD, CAM
Computer-aided manufacturing
Computer-aided manufacturing is the use of computer software to control machine tools and related machinery in the manufacturing of workpieces. This is not the only definition for CAM, but it is the most common; CAM may also refer to the use of a computer to assist in all operations of a...
, CAE
Computer-aided engineering
Computer-aided engineering is the broad usage of computer software to aid in engineering tasks. It includes computer-aided design , computer-aided analysis , computer-integrated manufacturing , computer-aided manufacturing , material requirements planning , and computer-aided planning .- Overview...
or other software. The reverse-engineering process involves measuring an object and then reconstructing it as a 3D model. The physical object can be measured using 3D scanning
3D scanner
A 3D scanner is a device that analyzes a real-world object or environment to collect data on its shape and possibly its appearance . The collected data can then be used to construct digital, three dimensional models....
technologies like CMMs
Coordinate-measuring machine
A coordinate measuring machine is a device for measuring the physical geometrical characteristics of an object. This machine may be manually controlled by an operator or it may be computer controlled. Measurements are defined by a probe attached to the third moving axis of this machine...
, laser scanners, structured light digitizers, or Industrial CT Scanning
Industrial CT Scanning
Industrial CT scanning is a process which uses X-ray equipment to produce three-dimensional representations of components both externally and internally. Industrial CT scanning has been used in many areas of industry for internal inspection of components...
(computed tomography). The measured data alone, usually represented as a point cloud
Point cloud
A point cloud is a set of vertices in a three-dimensional coordinate system. These vertices are usually defined by X, Y, and Z coordinates, and typically are intended to be representative of the external surface of an object....
, lacks topological information and is therefore often processed and modeled into a more usable format such as a triangular-faced mesh, a set of NURBS
Nonuniform rational B-spline
Non-uniform rational basis spline is a mathematical model commonly used in computer graphics for generating and representing curves and surfaces which offers great flexibility and precision for handling both analytic and freeform shapes.- History :Development of NURBS began in the 1950s by...
surfaces, or a CAD model.
Reverse engineering is also used by businesses to bring existing physical geometry into digital product development environments, to make a digital 3D record of their own products, or to assess competitors' products. It is used to analyse, for instance, how a product works, what it does, and what components it consists of, estimate costs, and identify potential patent
Patent
A patent is a form of intellectual property. It consists of a set of exclusive rights granted by a sovereign state to an inventor or their assignee for a limited period of time in exchange for the public disclosure of an invention....
infringement, etc.
Value engineering
Value engineering
Value engineering is a systematic method to improve the "value" of goods or products and services by using an examination of function. Value, as defined, is the ratio of function to cost. Value can therefore be increased by either improving the function or reducing the cost...
is a related activity also used by businesses. It involves de-constructing and analysing products, but the objective is to find opportunities for cost cutting.
Reverse engineering of software
The term reverse engineering as applied to software means different things to different people, prompting Chikofsky and Cross to write a paper researching the various uses and defining a taxonomyTaxonomy
Taxonomy is the science of identifying and naming species, and arranging them into a classification. The field of taxonomy, sometimes referred to as "biological taxonomy", revolves around the description and use of taxonomic units, known as taxa...
. From their paper, they state, "Reverse engineering is the process of analyzing a subject system to create representations of the system at a higher level of abstraction." It can also be seen as "going backwards through the development cycle". In this model, the output of the implementation phase (in source code form) is reverse-engineered back to the analysis phase, in an inversion of the traditional waterfall model
Waterfall model
The waterfall model is a sequential design process, often used in software development processes, in which progress is seen as flowing steadily downwards through the phases of Conception, Initiation, Analysis, Design, Construction, Testing, Production/Implementation and Maintenance.The waterfall...
. Reverse engineering is a process of examination only: the software system under consideration is not modified (which would make it re-engineering
Reengineering (software)
The reengineering of software was described by Chikofsky and Cross in their 1990 paper, as "The examination and alteration of a system to reconstitute it in a new form"...
). Software anti-tamper technology is used to deter both reverse engineering and re-engineering of proprietary software and software-powered systems. In practice, two main types of reverse engineering emerge. In the first case, source code is already available for the software, but higher-level aspects of the program, perhaps poorly documented or documented but no longer valid, are discovered. In the second case, there is no source code available for the software, and any efforts towards discovering one possible source code for the software are regarded as reverse engineering. This second usage of the term is the one most people are familiar with. Reverse engineering of software can make use of the clean room design
Clean room design
Clean room design is the method of copying a design by reverse engineering and then recreating it without infringing any of the copyrights and trade secrets associated with the original design. Clean room design is useful as a defense against copyright and trade secret infringement because it...
technique to avoid copyright infringement.
On a related note, black box testing
Black box testing
Black-box testing is a method of software testing that tests the functionality of an application as opposed to its internal structures or workings . Specific knowledge of the application's code/internal structure and programming knowledge in general is not required...
in software engineering
Software engineering
Software Engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software...
has a lot in common with reverse engineering. The tester usually has the API
Application programming interface
An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
, but their goals are to find bugs and undocumented features by bashing the product from outside.
Other purposes of reverse engineering include security auditing, removal of copy protection ("cracking
Software cracking
Software cracking is the modification of software to remove or disable features which are considered undesirable by the person cracking the software, usually related to protection methods: copy protection, trial/demo version, serial number, hardware key, date checks, CD check or software annoyances...
"), circumvention of access restrictions often present in consumer electronics
Consumer electronics
Consumer electronics are electronic equipment intended for everyday use, most often in entertainment, communications and office productivity. Radio broadcasting in the early 20th century brought the first major consumer product, the broadcast receiver...
, customization of embedded systems (such as engine management systems), in-house repairs or retrofits, enabling of additional features on low-cost "crippled" hardware (such as some graphics card chip-sets), or even mere satisfaction of curiosity.
Binary software
This process is sometimes termed Reverse Code Engineering, or RCE. As an example, decompilation of binaries for the Java platform can be accomplished using Jad. One famous case of reverse engineering was the first non-IBM implementation of the PCIBM PC
The IBM Personal Computer, commonly known as the IBM PC, is the original version and progenitor of the IBM PC compatible hardware platform. It is IBM model number 5150, and was introduced on August 12, 1981...
BIOS
BIOS
In IBM PC compatible computers, the basic input/output system , also known as the System BIOS or ROM BIOS , is a de facto standard defining a firmware interface....
which launched the historic IBM PC compatible
IBM PC compatible
IBM PC compatible computers are those generally similar to the original IBM PC, XT, and AT. Such computers used to be referred to as PC clones, or IBM clones since they almost exactly duplicated all the significant features of the PC architecture, facilitated by various manufacturers' ability to...
industry that has been the overwhelmingly dominant computer hardware platform for many years. An example of a group that reverse-engineers software for enjoyment (and to distribute registration cracks) is CORE which stands for "Challenge Of Reverse Engineering". Reverse engineering of software is protected in the U.S. by the fair use
Fair use
Fair use is a limitation and exception to the exclusive right granted by copyright law to the author of a creative work. In United States copyright law, fair use is a doctrine that permits limited use of copyrighted material without acquiring permission from the rights holders...
exception in copyright law
Copyright
Copyright is a legal concept, enacted by most governments, giving the creator of an original work exclusive rights to it, usually for a limited time...
. The Samba software, which allows systems that are not running Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
systems to share files with systems that are, is a classic example of software reverse engineering, since the Samba project had to reverse-engineer unpublished information about how Windows file sharing worked, so that non-Windows computers could emulate it. The Wine
Wine (software)
Wine is a free software application that aims to allow computer programs written for Microsoft Windows to run on Unix-like operating systems. Wine also provides a software library, known as Winelib, against which developers can compile Windows applications to help port them to Unix-like...
project does the same thing for the Windows API
Windows API
The Windows API, informally WinAPI, is Microsoft's core set of application programming interfaces available in the Microsoft Windows operating systems. It was formerly called the Win32 API; however, the name "Windows API" more accurately reflects its roots in 16-bit Windows and its support on...
, and OpenOffice.org
OpenOffice.org
OpenOffice.org, commonly known as OOo or OpenOffice, is an open-source application suite whose main components are for word processing, spreadsheets, presentations, graphics, and databases. OpenOffice is available for a number of different computer operating systems, is distributed as free software...
is one party doing this for the Microsoft Office
Microsoft Office
Microsoft Office is a non-free commercial office suite of inter-related desktop applications, servers and services for the Microsoft Windows and Mac OS X operating systems, introduced by Microsoft in August 1, 1989. Initially a marketing term for a bundled set of applications, the first version of...
file formats. The ReactOS
ReactOS
ReactOS is an open source computer operating system intended to be binary compatible with application software and device drivers made for Microsoft Windows NT versions 5.x and up...
project is even more ambitious in its goals, as it strives to provide binary (ABI and API) compatibility with the current Windows OSes of the NT branch, allowing software and drivers written for Windows to run on a clean-room reverse-engineered GPL free software or open-source counterpart.
Binary software techniques
Reverse engineering of software can be accomplished by various methods.The three main groups of software reverse engineering are
- Analysis through observation of information exchange, most prevalent in protocol reverse engineering, which involves using bus analyzerBus analyzerA bus analyzer is a computer bus analysis tool, often a combination of hardware and software, used during development of hardware or device drivers for a specific bus, for diagnosing bus or device failures, or reverse engineering....
s and packet snifferPacket snifferA packet analyzer is a computer program or a piece of computer hardware that can intercept and log traffic passing over a digital network or part of a network...
s, for example, for accessing a computer busComputer busIn computer architecture, a bus is a subsystem that transfers data between components inside a computer, or between computers.Early computer buses were literally parallel electrical wires with multiple connections, but the term is now used for any physical arrangement that provides the same...
or computer networkComputer networkA computer network, often simply referred to as a network, is a collection of hardware components and computers interconnected by communication channels that allow sharing of resources and information....
connection and revealing the traffic data thereon. Bus or network behavior can then be analyzed to produce a stand-alone implementation that mimics that behavior. This is especially useful for reverse engineering device driverDevice driverIn computing, a device driver or software driver is a computer program allowing higher-level computer programs to interact with a hardware device....
s. Sometimes, reverse engineering on embedded systemEmbedded systemAn embedded system is a computer system designed for specific control functions within a larger system. often with real-time computing constraints. It is embedded as part of a complete device often including hardware and mechanical parts. By contrast, a general-purpose computer, such as a personal...
s is greatly assisted by tools deliberately introduced by the manufacturer, such as JTAGJTAGJoint Test Action Group is the common name for what was later standardized as the IEEE 1149.1 Standard Test Access Port and Boundary-Scan Architecture. It was initially devised for testing printed circuit boards using boundary scan and is still widely used for this application.Today JTAG is also...
ports or other debugging means. In Microsoft WindowsMicrosoft WindowsMicrosoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, low-level debuggers such as SoftICESoftICESoftICE is a kernel mode debugger for Microsoft Windows. Crucially, it is designed to run underneath Windows such that the operating system is unaware of its presence. Unlike an application debugger, SoftICE is capable of suspending all operations in Windows when instructed...
are popular. - Disassembly using a disassemblerDisassemblerA disassembler is a computer program that translates machine language into assembly language—the inverse operation to that of an assembler. A disassembler differs from a decompiler, which targets a high-level language rather than an assembly language...
, meaning the raw machine language of the program is read and understood in its own terms, only with the aid of machine-language mnemonicMnemonicA mnemonic , or mnemonic device, is any learning technique that aids memory. To improve long term memory, mnemonic systems are used to make memorization easier. Commonly encountered mnemonics are often verbal, such as a very short poem or a special word used to help a person remember something,...
s. This works on any computer program but can take quite some time, especially for someone not used to machine code. The Interactive DisassemblerInteractive DisassemblerThe Interactive Disassembler, more commonly known as simply IDA, is a disassembler for computer software which generates assembly language source code from machine-executable code. It supports a variety of executable formats for different processors and operating systems. It also can be used as a...
is a particularly popular tool. - Decompilation using a decompilerDecompilerA decompiler is the name given to a computer program that performs, as far as possible, the reverse operation to that of a compiler. That is, it translates a file containing information at a relatively low level of abstraction into a form having a higher level of abstraction...
, a process that tries, with varying results, to recreate the source code in some high-level language for a program only available in machine code or bytecodeBytecodeBytecode, also known as p-code , is a term which has been used to denote various forms of instruction sets designed for efficient execution by a software interpreter as well as being suitable for further compilation into machine code...
.
Source code
A number of UMLUnified Modeling Language
Unified Modeling Language is a standardized general-purpose modeling language in the field of object-oriented software engineering. The standard is managed, and was created, by the Object Management Group...
tools refer to the process of importing and analysing source code to generate UML diagrams as "reverse engineering". See List of UML tools.
Reverse engineering of protocols
ProtocolsCommunications protocol
A communications protocol is a system of digital message formats and rules for exchanging those messages in or between computing systems and in telecommunications...
are sets of rules that describe message formats and how messages are exchanged (i.e., the protocol state-machine). Accordingly, the problem of protocol reverse-engineering can be partitioned into two subproblems; message format and state-machine reverse-engineering.
The message formats have traditionally been reverse-engineered through a tedious manual process, which involved analysis of how protocol implementations process messages, but recent research proposed a number of automatic solutions. Typically, these automatic approaches either group observed messages into clusters using various clustering analyses, or emulate the protocol implementation tracing the message processing.
There has been less work on reverse-engineering of state-machines of protocols. In general, the protocol state-machines can be learned either through a process of offline learning
Offline learning
In machine learning, systems which employ offline learning do not change their approximation of the target function once the initial training phase has been absolved. These systems are also typically examples of eager learning.-See also:...
, which passively observes communication and attempts to build the most general state-machine accepting all observed sequences of messages, and online learning, which allows interactive generation of probing sequences of messages and listening to responses to those probing sequences. In general, offline learning of small state-machines is known to be NP-complete
NP-complete
In computational complexity theory, the complexity class NP-complete is a class of decision problems. A decision problem L is NP-complete if it is in the set of NP problems so that any given solution to the decision problem can be verified in polynomial time, and also in the set of NP-hard...
, while online learning can be done in polynomial time. An automatic offline approach has been demonstrated by Comparetti et al. and an online approach very recently by Cho et al.
Other components of typical protocols, like encryption and hash functions, can be reverse-engineered automatically as well. Typically, the automatic approaches trace the execution of protocol implementations and try to detect buffers in memory holding unencrypted packets.
Reverse engineering of integrated circuits/smart cards
Reverse engineering is an invasive and destructive form of analyzing a smart card. The attacker grinds away layer by layer of the smart card and takes pictures with an electron microscopeElectron microscope
An electron microscope is a type of microscope that uses a beam of electrons to illuminate the specimen and produce a magnified image. Electron microscopes have a greater resolving power than a light-powered optical microscope, because electrons have wavelengths about 100,000 times shorter than...
. With this technique, it is possible to reveal the complete hardware and software part of the smart card. The major problem for the attacker is to bring everything into the right order to find out how everything works. Engineers try to hide keys and operations by mixing up memory positions, for example, bus scrambling.
In some cases, it is even possible to attach a probe to measure voltages while the smart card is still operational. Engineers employ sensors to detect and prevent this attack. This attack is not very common because it requires a large investment in effort and special equipment that is generally only available to large chip manufacturers. Furthermore, the payoff from this attack is low since other security techniques are often employed such as shadow accounts.
Reverse engineering for military applications
Reverse engineering is often used by militaries in order to copy other nations' technologies, devices, or information that have been obtained by regular troops in the fields or by intelligenceMilitary intelligence
Military intelligence is a military discipline that exploits a number of information collection and analysis approaches to provide guidance and direction to commanders in support of their decisions....
operations. It was often used during the Second World War and the Cold War
Cold War
The Cold War was the continuing state from roughly 1946 to 1991 of political conflict, military tension, proxy wars, and economic competition between the Communist World—primarily the Soviet Union and its satellite states and allies—and the powers of the Western world, primarily the United States...
. Well-known examples from WWII and later include:
- Jerry can: British and American forces noticed that the Germans had gasoline cans with an excellent design. They reverse-engineered copies of those cans. The cans were popularly known as "Jerry cans".
- Tupolev Tu-4Tupolev Tu-4The Tupolev Tu-4 was a piston-engined Soviet strategic bomber that served the Soviet Air Force from the late 1940s to mid 1960s...
: Three American B-29B-29 SuperfortressThe B-29 Superfortress is a four-engine propeller-driven heavy bomber designed by Boeing that was flown primarily by the United States Air Forces in late-World War II and through the Korean War. The B-29 was one of the largest aircraft to see service during World War II...
bombers on missions over JapanJapanJapan is an island nation in East Asia. Located in the Pacific Ocean, it lies to the east of the Sea of Japan, China, North Korea, South Korea and Russia, stretching from the Sea of Okhotsk in the north to the East China Sea and Taiwan in the south...
were forced to land in the USSRSoviet UnionThe Soviet Union , officially the Union of Soviet Socialist Republics , was a constitutionally socialist state that existed in Eurasia between 1922 and 1991....
. The Soviets, who did not have a similar strategic bomber, decided to copy the B-29. Within a few years, they had developed the Tu-4, a near-perfect copy. - V2 Rocket: Technical documents for the V2 and related technologies were captured by the Western Allies at the end of the war. Soviet and captured German engineers had to reproduce technical documents and plans, working from captured hardware, in order to make their clone of the rocket, the R-1R-1 (missile)The R-1 rocket was a copy of the German V-2 rocket manufactured by the Soviet Union. Even though it was a copy, it was manufactured using Soviet industrial plants and gave the Soviets valuable experience which later enabled the USSR to construct its own much more capable rockets.In 1945 the...
, which began the postwar Soviet rocket program that led to the R-7R-7 SemyorkaThe R-7 was a Soviet missile developed during the Cold War, and the world's first intercontinental ballistic missile. The R-7 made 28 launches between 1957 and 1961, but was never deployed operationally. A derivative, the R-7A, was deployed from 1960 to 1968...
and the beginning of the space raceSpace RaceThe Space Race was a mid-to-late 20th century competition between the Soviet Union and the United States for supremacy in space exploration. Between 1957 and 1975, Cold War rivalry between the two nations focused on attaining firsts in space exploration, which were seen as necessary for national...
. - K-13/R-3SVympel K-13The K-13 is an short-range, infrared homing air-to-air missile developed by the Soviet Union. It is similar in appearance and function to the American AIM-9 Sidewinder from which it was reverse-engineered...
missile (NATO reporting nameNATO reporting nameNATO reporting names are classified code names for military equipment of the Eastern Bloc...
AA-2 Atoll), a Soviet reverse-engineered copy of the AIM-9 SidewinderAIM-9 SidewinderThe AIM-9 Sidewinder is a heat-seeking, short-range, air-to-air missile carried mostly by fighter aircraft and recently, certain gunship helicopters. The missile entered service with United States Air Force in the early 1950s, and variants and upgrades remain in active service with many air forces...
, was made possible after a Taiwanese AIM-9B hit a Chinese MiG-17 without exploding. The missile became lodged within the airframe, and the pilot returned to base with what Russian scientists would describe as a university course in missile development. - BGM-71 TOWBGM-71 TOWThe BGM-71 TOW is an anti-tank missile. "BGM" is a weapon classification that stands for "Multiple Environment , Surface-Attack , Missile ". "TOW" is an acronym that stands for "Tube-launched, Optically-tracked, Wire command data link, guided missile"...
Missile: In May 1975, negotiations between Iran and Hughes Missile Systems on co-production of the TOW and Maverick missiles stalled over disagreements in the pricing structure, the subsequent 1979 revolution ending all plans for such co-production. Iran was later successful in reverse-engineering the missile and are currently producing their own copy: the ToophanToophanToophan is an Iranian anti-tank missile that is a reverse-engineered copy of the US military TOW missile. The Toophan's payload is a 3.6 kg high-explosive anti-tank warhead that can penetrate up to 550mm of steel armor. The range is 3,850m, the top speed 310m/s...
. - China has reversed engineered many examples of Western and Russian hardware, from fighter aircraft to missiles and HMMWV cars.
- During the Second World War, British military intelligence at the Bletchley Park centre studied captured German "Enigma" message encryption machines. Their operation was then simulated on electro-mechanical devices called "Bombes" that tried all the possible scrambler settings of the "Enigma" machines to help break the coded messages sent by the Germans.
United States
In the United States even if an artifact or process is protected by trade secretTrade secret
A trade secret is a formula, practice, process, design, instrument, pattern, or compilation of information which is not generally known or reasonably ascertainable, by which a business can obtain an economic advantage over competitors or customers...
s, reverse-engineering the artifact or process is often lawful as long as it is obtained legitimately. Patent
Patent
A patent is a form of intellectual property. It consists of a set of exclusive rights granted by a sovereign state to an inventor or their assignee for a limited period of time in exchange for the public disclosure of an invention....
s, on the other hand, need a public disclosure of an invention
Invention
An invention is a novel composition, device, or process. An invention may be derived from a pre-existing model or idea, or it could be independently conceived, in which case it may be a radical breakthrough. In addition, there is cultural invention, which is an innovative set of useful social...
, and therefore, patented items do not necessarily have to be reverse-engineered to be studied. (However, an item produced under one or more patents could also include other technology that is not patented and not disclosed.) One common motivation of reverse engineers is to determine whether a competitor's product contains patent infringement
Patent infringement
Patent infringement is the commission of a prohibited act with respect to a patented invention without permission from the patent holder. Permission may typically be granted in the form of a license. The definition of patent infringement may vary by jurisdiction, but it typically includes using or...
s or copyright infringement
Copyright infringement
Copyright infringement is the unauthorized or prohibited use of works under copyright, infringing the copyright holder's exclusive rights, such as the right to reproduce or perform the copyrighted work, or to make derivative works.- "Piracy" :...
s.
The reverse engineering of software in the US is generally a breach of contract as most EULAs specifically prohibit it, and courts have found such contractual prohibitions to override the copyright law; see Bowers v. Baystate Technologies
Bowers v. Baystate Technologies
Bowers v. Baystate Technologies was a U.S. Court of Appeals Federal Circuit case involving Harold L. Bowers and Baystate Technologies over patent infringement, copyright infringement, and breach of contract...
.
Sec. 103(f) of the DMCA
Digital Millennium Copyright Act
The Digital Millennium Copyright Act is a United States copyright law that implements two 1996 treaties of the World Intellectual Property Organization . It criminalizes production and dissemination of technology, devices, or services intended to circumvent measures that control access to...
(17 U.S.C. § 1201 (f)) says that if you legally obtain a program that is protected, you are allowed to reverse-engineer and circumvent the protection to achieve interoperability between computer programs (i.e., the ability to exchange and make use of information). The section states:
(f) Reverse Engineering.—
(1) Notwithstanding the provisions of subsection (a)(1)(A), a person who has lawfully obtained the right to use a copy of a computer program may circumvent a technological measure that effectively controls access to a particular portion of that program for the sole purpose of identifying and analyzing those elements of the program that are necessary to achieve interoperability of an independently created computer program with other programs, and that have not previously been readily available to the person engaging in the circumvention, to the extent any such acts of identification and analysis do not constitute infringement under this title.
(2) Notwithstanding the provisions of subsections (a)(2) and (b), a person may develop and employ technological means to circumvent a technological measure, or to circumvent protection afforded by a technological measure, in order to enable the identification and analysis under paragraph (1), or for the purpose of enabling interoperability of an independently created computer program with other programs, if such means are necessary to achieve such interoperability, to the extent that doing so does not constitute infringement under this title.
(3) The information acquired through the acts permitted under paragraph (1), and the means permitted under paragraph (2), may be made available to others if the person referred to in paragraph (1) or (2), as the case may be, provides such information or means solely for the purpose of enabling interoperability of an independently created computer program with other programs, and to the extent that doing so does not constitute infringement under this title or violate applicable law other than this section.
(4) For purposes of this subsection, the term 「interoperability」 means the ability of computer programs to exchange information, and of such programs mutually to use the information which has been exchanged.
European Union
Article 6 of the 1991 EU Computer Programs Directive allows reverse engineering for the purposes of interoperability, but prohibits it for the purposes of creating a competing product, and also prohibits the public release of information obtained through reverse engineering of software.In 2009, the EU Computer Program Directive was superseded and the directive now states:
(15) The unauthorised reproduction, translation, adaptation or transformation of the form of the code in which a copy of a computer program has been made available constitutes an infringement of the exclusive rights of the author. Nevertheless, circumstances may exist when such a reproduction of the code and translation of its form are indispensable to obtain the necessary information to achieve the interoperability of an independently created program with other programs. It has therefore to be considered that, in these limited circumstances only, performance of the acts of reproduction and translation by or on behalf of a person having a right to use a copy of the program is legitimate and compatible with fair practice and must therefore be deemed not to require the authorisation of the rightholder. An objective of this exception is to make it possible to connect all components of a computer system, including those of different manufacturers, so that they can work together. Such an exception to the author's exclusive rights may not be used in a way which prejudices the legitimate interests of the rightholder or which conflicts with a normal exploitation of the program.
See also
- Antikythera mechanismAntikythera mechanismThe Antikythera mechanism is an ancient mechanical computer designed to calculate astronomical positions. It was recovered in 1900–1901 from the Antikythera wreck. Its significance and complexity were not understood until decades later. Its time of construction is now estimated between 150 and 100...
- BenchmarkingBenchmarkingBenchmarking is the process of comparing one's business processes and performance metrics to industry bests and/or best practices from other industries. Dimensions typically measured are quality, time and cost...
- Bus analyzerBus analyzerA bus analyzer is a computer bus analysis tool, often a combination of hardware and software, used during development of hardware or device drivers for a specific bus, for diagnosing bus or device failures, or reverse engineering....
- ChondaChondaChonda is a generic term describing a small engine produced in China that is reverse engineered from a Honda small engine. It is also called a Honda Clone. The name is a portmanteau of "Chinese" and "Honda", and is informal and not an actual trademark...
- Clean room designClean room designClean room design is the method of copying a design by reverse engineering and then recreating it without infringing any of the copyrights and trade secrets associated with the original design. Clean room design is useful as a defense against copyright and trade secret infringement because it...
- CMMCoordinate-measuring machineA coordinate measuring machine is a device for measuring the physical geometrical characteristics of an object. This machine may be manually controlled by an operator or it may be computer controlled. Measurements are defined by a probe attached to the third moving axis of this machine...
- Code morphingCode morphingCode morphing is one of the approaches to protect software applications from reverse engineering, analysis, modifications, and cracking used in obfuscating software. This technology protects intermediate level code such as compiled from Java and .NET languages rather than binary object code...
- Connectix Virtual Game StationConnectix Virtual Game StationThe Virtual Game Station is an emulator by Connectix that allows Sony PlayStation games to be played on a desktop computer. It was first released for the Macintosh, in 1999. VGS was created by Aaron Giles...
- CryptanalysisCryptanalysisCryptanalysis is the study of methods for obtaining the meaning of encrypted information, without access to the secret information that is normally required to do so. Typically, this involves knowing how the system works and finding a secret key...
- DecompilerDecompilerA decompiler is the name given to a computer program that performs, as far as possible, the reverse operation to that of a compiler. That is, it translates a file containing information at a relatively low level of abstraction into a form having a higher level of abstraction...
- Digital Millennium Copyright ActDigital Millennium Copyright ActThe Digital Millennium Copyright Act is a United States copyright law that implements two 1996 treaties of the World Intellectual Property Organization . It criminalizes production and dissemination of technology, devices, or services intended to circumvent measures that control access to...
(DMCA) - Forensic engineeringForensic engineeringForensic engineering is the investigation of materials, products, structures or components that fail or do not operate or function as intended, causing personal injury or damage to property. The consequences of failure are dealt with by the law of product liability. The field also deals with...
- Industrial CT scanningIndustrial CT ScanningIndustrial CT scanning is a process which uses X-ray equipment to produce three-dimensional representations of components both externally and internally. Industrial CT scanning has been used in many areas of industry for internal inspection of components...
- Interactive DisassemblerInteractive DisassemblerThe Interactive Disassembler, more commonly known as simply IDA, is a disassembler for computer software which generates assembly language source code from machine-executable code. It supports a variety of executable formats for different processors and operating systems. It also can be used as a...
- Knowledge Discovery MetamodelKnowledge Discovery MetamodelKnowledge Discovery Metamodel is publicly available specification from the Object Management Group . KDM is a common intermediate representation for existing software systems and their operating environments, that defines common metadata required for deep semantic integration of Application...
- Laser scanner
- List of production topics
- Logic analyzerLogic analyzerA logic analyzer is an electronic instrument which displays signals in a digital circuit. A logic analyzer may convert the captured data into timing diagrams, protocol decodes, state machine traces, assembly language, or correlate assembly with source-level software.Presently, there are three...
- Paycheck (film)Paycheck (film)Paycheck is a 2003 film adaptation of the short story of the same name by science fiction writer Philip K. Dick. The film was directed by John Woo and stars Ben Affleck, Uma Thurman and Aaron Eckhart...
- Software archaeologySoftware archaeologySoftware archaeology or software archeology is the study of poorly documented or undocumented legacy software implementations, as part of software maintenance...
- Structured light digitizer
- Value engineeringValue engineeringValue engineering is a systematic method to improve the "value" of goods or products and services by using an examination of function. Value, as defined, is the ratio of function to cost. Value can therefore be increased by either improving the function or reducing the cost...
External links
- What Is Reverse Engineering
- Java Call Trace to UML Sequence Diagram A reverse engineering tool for Java. This tool helps you to reverse engineer UML Sequence Diagram for your java program at runtime. It works well with both complex java programs (that have multiple threads) and J2EE applications deployed on Application Servers.
- CASE Tools for Reverse Code Engineering
- Additional tutorials, bibliographies and related sites for reverse engineering of available source code.