Ernst Dickmanns
Encyclopedia
Ernst Dieter Dickmanns is a former professor at Bundeswehr University Munich (1975–2001), and a pioneer of dynamic computer vision and of driverless car
s. Dickmanns has been visiting professor to CalTech, Pasadena, and to MIT, Boston teaching courses on 'dynamic vision'.
and aeronautics
at RWTH Aachen
(1956–1961), and control engineering at Princeton University
(1964/65); from 1961 to 1975 he was associated with the German Aero-Space Research Establichment (now DLR
) Oberpfaffenhofen
, working in the fields of flight dynamics and trajectory optimization
.In 1971/72 he spent a Post-Doc Research Associateship with the NASA
-Marshall Space Flight Center
, Huntsville
(orbiter re-entry). From 1975 to 2001 he was with UniBw Munich, where he initiated the 'Institut fuer Flugmechanik und Systemdynamik' (IFS), the Institut fuer die 'Technik Autonomer Systeme' (TAS), and the research activities in machine vision for vehicle guidance.
van with cameras and other sensors. The 5-ton van
was re-engineered such that it was possible to control steering wheel
, throttle
, and brakes through computer
commands based on real-time evaluation of image sequences. Software was written that translated the sensory data into appropriate driving commands. For safety reasons, initial experiments in Bavaria
took place on streets without traffic
. Since 1986 the Robot Car "VaMoRs" managed to drive all by itself, since 1987 at speeds up to 96 km/h, or roughly 60 mph.
One of the greatest challenges in high-speed autonomous driving arises through the rapidly changing visual street scenes. Back then, computers were much slower than they are today (~1% of 1%); therefore, sophisticated computer vision
strategies were necessary to react in real time. The team of Dickmanns solved the problem through an innovative approach to dynamic vision
. Spatiotemporal models were used right from the beginning, dubbed '4-D approach', which did not need storing previous images but non-the-less was able to yield estimates of all 3-D velocity components. Attention control including artificial saccadic movements of the platform carrying the cameras allowed the system to focus its attention on the most relevant details of the visual input. Kalman filter
s have been extended to perspective imaging and were used to achieve robust autonomous driving even in presence of noise
and uncertainty
. Feedback of prediction errors allowed bypassing the (ill-conditioned) inversion of perspective projection by least-squares parameter fits.
When in 1986/87 the EUREKA
-project 'PROgraMme for a European Traffic of Highest Efficiency and Unprecedented Safety' (PROMETHEUS
) was initiated by the European car manufacturing industry (funding in the range of several hundred million Euros), the initially planned autonomous lateral guidance by buried cables was dropped and substituted by the much more flexible machine vision approach proposed by Dickmanns, and partially encouraged by his successes. Most of the major car companies participated; so did Dickmanns and his team in cooperation with the Daimler-Benz AG. Substantial progress was made in the following 7 years. In particular, Dickmanns' robot cars learned to drive in traffic under various conditions. An accompanying human driver with a "red button" made sure the robot vehicle could not get out of control and become a danger to the public. Since 1992, driving in public traffic was standard as final step in real-world testing. Several dozen Transputers, a special breed of parallel computers, were used to deal with the (by 1990s standards) enormous computational demands.
Two culmination points were achieved in 1994/95, when Dickmanns´ re-engineered autonomous S-Class Mercedes-Benz
performed international demonstrations. The first was the final presentation of the PROMETHEUS project in October 1994 on Autoroute 1 near the airport Charles-de-Gaulle in Paris. With guests onboard, the twin vehicles of Daimler-Benz (VITA-2) and UniBwM (VaMP
) drove more than one thousand kilometers on the three-lane highway in standard heavy traffic at speeds up to 130 km/h. Driving in free lanes, convoy driving with distance keeping depending on speed, and lane changes left and right with autonomous passing have been demonstrated; the latter required interpreting the road scene also in the rear hemisphere. Four cameras with two different focal lengths for each hemisphere have been used in parallel for this purpose.
The second culmination point was a 1758 km trip in the fall of 1995 from Munich
in Bavaria
to Odense
in Denmark
to a project meeting and back. Both longitudinal and lateral guidance were performed autonomously by vision. On highways, the robot achieved speeds exceeding 175 km/h (roughly 110 mph; there is no general speed limit on the German
Autobahn). Publications from Dickmann's research group indicate a mean autonomously driven distance without resets of ~9 km; the longest autonomously driven stretch reached 158 km. More than half of the resets required were achieved autonomously (no human intervention). This is particularly impressive considering that the system used black-and-white video-cameras and did not model situations like road construction sites with yellow lane markings, lane-changes at over 140 km/h, and other traffic with more than 40 km/h relative speed. In total, 95% autonomous driving (by distance) was achieved.
In the years 1994 to 2004 the elder 5-ton van 'VaMoRs' was used to develop the capabilities needed for driving on networks of minor (also unsealed) roads and for cross-country driving including avoidance of negative obstacles like ditches. Turning off onto crossroads of unknown width and intersection angles required a big effort, but has been achieved with "Expectation-based, Multi-focal, Saccadic vision" (EMS-vision). This vertebrate-type vision uses animation capabilities based on knowledge about subject classes (including the autonomous vehicle itself) and their potential behaviour in certain situations. This rich background is used for control of gaze and attention as well as for locomotion.
Beside ground vehicle guidance, also applications of the 4-D approach to dynamic vision for unmanned air vehicles (conventional aircraft and helicopters) have been investigated. Autonomous visual landing approaches and landings have been demonstrated in hardware-in-the-loop simulations with visual/inertial data fusion.
Another success of this machine vision technology was the first ever visually controlled grasping experiment of a free-floating object in weightlessness onboard the Space Shuttle Columbia D2-mission in 1993 as part of the 'Rotex'-experiment of DLR.
Driverless car
An autonomous car, also known as robotic or informally as driverless, is an autonomous vehicle capable of fulfilling the human transportation capabilities of a traditional car. As an autonomous vehicle, it is capable of sensing its environment and navigating on its own...
s. Dickmanns has been visiting professor to CalTech, Pasadena, and to MIT, Boston teaching courses on 'dynamic vision'.
Biography
Dickmanns was born in 1936. He studied aerospaceAerospace
Aerospace comprises the atmosphere of Earth and surrounding space. Typically the term is used to refer to the industry that researches, designs, manufactures, operates, and maintains vehicles moving through air and space...
and aeronautics
Aeronautics
Aeronautics is the science involved with the study, design, and manufacturing of airflight-capable machines, or the techniques of operating aircraft and rocketry within the atmosphere...
at RWTH Aachen
RWTH Aachen
RWTH Aachen University is a research university located in Aachen, North Rhine-Westphalia, Germany with roughly 33,000 students enrolled in 101 study programs....
(1956–1961), and control engineering at Princeton University
Princeton University
Princeton University is a private research university located in Princeton, New Jersey, United States. The school is one of the eight universities of the Ivy League, and is one of the nine Colonial Colleges founded before the American Revolution....
(1964/65); from 1961 to 1975 he was associated with the German Aero-Space Research Establichment (now DLR
German Aerospace Center
The German Aerospace Center is the national centre for aerospace, energy and transportation research of the Federal Republic of Germany. It has multiple locations throughout Germany. Its headquarters are located in Cologne. It is engaged in a wide range of research and development projects in...
) Oberpfaffenhofen
Oberpfaffenhofen
Oberpfaffenhofen is a village which is part of the municipality of Weßling in the district of Starnberg, Bavaria, Germany. It is 20 kilometers from the city center of Munich....
, working in the fields of flight dynamics and trajectory optimization
Trajectory optimization
Trajectory optimization is the process of designing a trajectory that minimizes or maximizes some measure of performance within prescribed constraint boundaries...
.In 1971/72 he spent a Post-Doc Research Associateship with the NASA
NASA
The National Aeronautics and Space Administration is the agency of the United States government that is responsible for the nation's civilian space program and for aeronautics and aerospace research...
-Marshall Space Flight Center
Marshall Space Flight Center
The George C. Marshall Space Flight Center is the U.S. government's civilian rocketry and spacecraft propulsion research center. The largest center of NASA, MSFC's first mission was developing the Saturn launch vehicles for the Apollo moon program...
, Huntsville
Huntsville, Alabama
Huntsville is a city located primarily in Madison County in the central part of the far northern region of the U.S. state of Alabama. Huntsville is the county seat of Madison County. The city extends west into neighboring Limestone County. Huntsville's population was 180,105 as of the 2010 Census....
(orbiter re-entry). From 1975 to 2001 he was with UniBw Munich, where he initiated the 'Institut fuer Flugmechanik und Systemdynamik' (IFS), the Institut fuer die 'Technik Autonomer Systeme' (TAS), and the research activities in machine vision for vehicle guidance.
Pioneering work in autonomous driving
In the beginning of the 1980s his team equipped a Mercedes-BenzMercedes-Benz
Mercedes-Benz is a German manufacturer of automobiles, buses, coaches, and trucks. Mercedes-Benz is a division of its parent company, Daimler AG...
van with cameras and other sensors. The 5-ton van
Van
A van is a kind of vehicle used for transporting goods or groups of people.In British English usage, it can be either specially designed or based on a saloon or sedan car, the latter type often including derivatives with open backs...
was re-engineered such that it was possible to control steering wheel
Steering wheel
A steering wheel is a type of steering control in vehicles and vessels ....
, throttle
Throttle
A throttle is the mechanism by which the flow of a fluid is managed by constriction or obstruction. An engine's power can be increased or decreased by the restriction of inlet gases , but usually decreased. The term throttle has come to refer, informally and incorrectly, to any mechanism by which...
, and brakes through computer
Computer
A computer is a programmable machine designed to sequentially and automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem...
commands based on real-time evaluation of image sequences. Software was written that translated the sensory data into appropriate driving commands. For safety reasons, initial experiments in Bavaria
Bavaria
Bavaria, formally the Free State of Bavaria is a state of Germany, located in the southeast of Germany. With an area of , it is the largest state by area, forming almost 20% of the total land area of Germany...
took place on streets without traffic
Traffic
Traffic on roads may consist of pedestrians, ridden or herded animals, vehicles, streetcars and other conveyances, either singly or together, while using the public way for purposes of travel...
. Since 1986 the Robot Car "VaMoRs" managed to drive all by itself, since 1987 at speeds up to 96 km/h, or roughly 60 mph.
One of the greatest challenges in high-speed autonomous driving arises through the rapidly changing visual street scenes. Back then, computers were much slower than they are today (~1% of 1%); therefore, sophisticated computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
strategies were necessary to react in real time. The team of Dickmanns solved the problem through an innovative approach to dynamic vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
. Spatiotemporal models were used right from the beginning, dubbed '4-D approach', which did not need storing previous images but non-the-less was able to yield estimates of all 3-D velocity components. Attention control including artificial saccadic movements of the platform carrying the cameras allowed the system to focus its attention on the most relevant details of the visual input. Kalman filter
Kalman filter
In statistics, the Kalman filter is a mathematical method named after Rudolf E. Kálmán. Its purpose is to use measurements observed over time, containing noise and other inaccuracies, and produce values that tend to be closer to the true values of the measurements and their associated calculated...
s have been extended to perspective imaging and were used to achieve robust autonomous driving even in presence of noise
Noise
In common use, the word noise means any unwanted sound. In both analog and digital electronics, noise is random unwanted perturbation to a wanted signal; it is called noise as a generalisation of the acoustic noise heard when listening to a weak radio transmission with significant electrical noise...
and uncertainty
Uncertainty
Uncertainty is a term used in subtly different ways in a number of fields, including physics, philosophy, statistics, economics, finance, insurance, psychology, sociology, engineering, and information science...
. Feedback of prediction errors allowed bypassing the (ill-conditioned) inversion of perspective projection by least-squares parameter fits.
When in 1986/87 the EUREKA
EUREKA
EUREKA, often abbreviated as "E!" or "Σ!", is a pan-European research and development funding and coordination organization. EUREKA aims to coordinate efforts of governments, research institutes and commercial companies concerning innovation...
-project 'PROgraMme for a European Traffic of Highest Efficiency and Unprecedented Safety' (PROMETHEUS
EUREKA Prometheus Project
The EUREKA Prometheus Project was the largest R&D project ever in the field of driverless cars. In today's money it received more than 1 billion dollars of funding from the European Commission, and defined the state of the art of autonomous vehicles...
) was initiated by the European car manufacturing industry (funding in the range of several hundred million Euros), the initially planned autonomous lateral guidance by buried cables was dropped and substituted by the much more flexible machine vision approach proposed by Dickmanns, and partially encouraged by his successes. Most of the major car companies participated; so did Dickmanns and his team in cooperation with the Daimler-Benz AG. Substantial progress was made in the following 7 years. In particular, Dickmanns' robot cars learned to drive in traffic under various conditions. An accompanying human driver with a "red button" made sure the robot vehicle could not get out of control and become a danger to the public. Since 1992, driving in public traffic was standard as final step in real-world testing. Several dozen Transputers, a special breed of parallel computers, were used to deal with the (by 1990s standards) enormous computational demands.
Two culmination points were achieved in 1994/95, when Dickmanns´ re-engineered autonomous S-Class Mercedes-Benz
Mercedes-Benz
Mercedes-Benz is a German manufacturer of automobiles, buses, coaches, and trucks. Mercedes-Benz is a division of its parent company, Daimler AG...
performed international demonstrations. The first was the final presentation of the PROMETHEUS project in October 1994 on Autoroute 1 near the airport Charles-de-Gaulle in Paris. With guests onboard, the twin vehicles of Daimler-Benz (VITA-2) and UniBwM (VaMP
VaMP
The VaMP driverless car was one of the first truly autonomous cars along with its twin vehicle, the VITA-2. They were able to drive in heavy traffic for long distances without human intervention, using computer vision to recognize rapidly moving obstacles such as other cars, and automatically...
) drove more than one thousand kilometers on the three-lane highway in standard heavy traffic at speeds up to 130 km/h. Driving in free lanes, convoy driving with distance keeping depending on speed, and lane changes left and right with autonomous passing have been demonstrated; the latter required interpreting the road scene also in the rear hemisphere. Four cameras with two different focal lengths for each hemisphere have been used in parallel for this purpose.
The second culmination point was a 1758 km trip in the fall of 1995 from Munich
Munich
Munich The city's motto is "" . Before 2006, it was "Weltstadt mit Herz" . Its native name, , is derived from the Old High German Munichen, meaning "by the monks' place". The city's name derives from the monks of the Benedictine order who founded the city; hence the monk depicted on the city's coat...
in Bavaria
Bavaria
Bavaria, formally the Free State of Bavaria is a state of Germany, located in the southeast of Germany. With an area of , it is the largest state by area, forming almost 20% of the total land area of Germany...
to Odense
Odense
The city of Odense is the third largest city in Denmark.Odense City has a population of 167,615 and is the main city of the island of Funen...
in Denmark
Denmark
Denmark is a Scandinavian country in Northern Europe. The countries of Denmark and Greenland, as well as the Faroe Islands, constitute the Kingdom of Denmark . It is the southernmost of the Nordic countries, southwest of Sweden and south of Norway, and bordered to the south by Germany. Denmark...
to a project meeting and back. Both longitudinal and lateral guidance were performed autonomously by vision. On highways, the robot achieved speeds exceeding 175 km/h (roughly 110 mph; there is no general speed limit on the German
Germany
Germany , officially the Federal Republic of Germany , is a federal parliamentary republic in Europe. The country consists of 16 states while the capital and largest city is Berlin. Germany covers an area of 357,021 km2 and has a largely temperate seasonal climate...
Autobahn). Publications from Dickmann's research group indicate a mean autonomously driven distance without resets of ~9 km; the longest autonomously driven stretch reached 158 km. More than half of the resets required were achieved autonomously (no human intervention). This is particularly impressive considering that the system used black-and-white video-cameras and did not model situations like road construction sites with yellow lane markings, lane-changes at over 140 km/h, and other traffic with more than 40 km/h relative speed. In total, 95% autonomous driving (by distance) was achieved.
In the years 1994 to 2004 the elder 5-ton van 'VaMoRs' was used to develop the capabilities needed for driving on networks of minor (also unsealed) roads and for cross-country driving including avoidance of negative obstacles like ditches. Turning off onto crossroads of unknown width and intersection angles required a big effort, but has been achieved with "Expectation-based, Multi-focal, Saccadic vision" (EMS-vision). This vertebrate-type vision uses animation capabilities based on knowledge about subject classes (including the autonomous vehicle itself) and their potential behaviour in certain situations. This rich background is used for control of gaze and attention as well as for locomotion.
Beside ground vehicle guidance, also applications of the 4-D approach to dynamic vision for unmanned air vehicles (conventional aircraft and helicopters) have been investigated. Autonomous visual landing approaches and landings have been demonstrated in hardware-in-the-loop simulations with visual/inertial data fusion.
Another success of this machine vision technology was the first ever visually controlled grasping experiment of a free-floating object in weightlessness onboard the Space Shuttle Columbia D2-mission in 1993 as part of the 'Rotex'-experiment of DLR.