Wayback Machine
Encyclopedia
The Wayback Machine is a digital time capsule created by the Internet Archive
non-profit organization, based in San Francisco, California
. It is maintained with content from Alexa Internet
. The service enables users to see archived versions of web page
s across time, which the Archive calls a "three dimensional index". Internet Archive
bought the domain waybackmachine.org for their own site. It is currently in its beta test.
The name Wayback Machine is a reference to a segment from The Rocky and Bullwinkle Show
in which Mr. Peabody and Sherman use a time machine
called the "WABAC machine" to witness, participate in, and, more often than not, alter famous events in history.
, with Bruce Gilliat
, developed software to crawl and download all publicly accessible World Wide Web pages, the Gopher hierarchy, the Netnews bulletin board system, and downloadable software. The information collected by these "crawlers" does not collect all the information available on the Internet since much of the data is restricted by the publisher or stored in databases that are not accessible. These "crawlers" also respect the robots exclusion standard
for websites wishing to opt-out of appearing in search results or being cached. To overcome inconsistencies in partially cached websites, Archive-It.org was developed in 2005 by the Internet Archive as a means of allowing institutions and content creators to voluntarily harvest and preserve collections of digital content, and create digital archives.
The digital library grew and grew and grew. But a lot of people knew about it. Information was kept on digital tape for five years, with Kahle occasionally allowing researchers and scientists to tap into the clunky database. When the archive reached its five-year anniversary, it was unveiled and opened to the public in a ceremony at the University of California-Berkeley.
Snapshots usually become available more than 6 months after they are archived, or in some cases, even later, 24 months or longer. The frequency of snapshots is variable, so not all tracked web site updates are recorded. Intervals of several weeks or years sometimes occur.
After August 2008 sites had to be listed on the Open Directory
in order to be included. According to Jeff Kaplan of the Internet Archive in November 2010, other sites were still being archived, but more recent captures would only become visible after the next major indexing, an infrequent operation.
the Wayback Machine contained approximately three petabyte
s of data and was growing at a rate of 100 terabyte
s each month; the growth rate reported in 2003 was 12 terabytes/month. The data is stored on PetaBox
rack systems manufactured by Capricorn Technologies
.
In 2009 the Internet Archive migrated its customized storage architecture to Sun Open Storage
, and hosts a new datacenter in a Sun Modular Datacenter
on Sun Microsystems
' California campus.
In 2011 a new, improved version of the Wayback Machine, with an updated interface and fresher index of archived content, was made available for public testing.
In March 2011 it was said on the Wayback Machine forum that "The Beta of the new Wayback Machine has a more complete and up-to-date index of all crawled materials into 2010, and will continue to be updated regularly. The index driving the classic Wayback Machine only has a little bit of material past 2008, and no further index updates are planned, as it will be phased out this year."
Netbula objected to the motion on the ground that defendants were asking to alter Netbula's web site and that they should have subpoenaed Internet Archive for the pages directly. However, an employee of Internet Archive filed a sworn statement supporting Chordiant's motion, stating that it could not produce the web pages by any other means "without considerable burden, expense and disruption to its operations."
Magistrate Judge Howard Lloyd
in the Northern District of California, San Jose Division, rejected Netbula's arguments and ordered them to temporarily disable the robots.txt blockage in order to allow Chordiant to retrieve the archived pages that they sought.
v. Echostar Satellite", a litigant attempted to use the Wayback Machine archives as a source of admissible evidence, perhaps for the first time. Telewizja Polska is the provider of TVP Polonia
and EchoStar operates the Dish Network
. Prior to the trial proceedings, EchoStar indicated that it intended to offer Wayback Machine snapshots as proof of the past content of Telewizja Polska’s website. Telewizja Polska brought a motion in limine
to suppress the snapshots on the grounds of hearsay
and unauthenticated source, but Magistrate Judge Arlander Keys rejected Telewizja Polska’s assertion of hearsay and denied TVP's motion in limine to exclude the evidence at trial. However, at the actual trial, district Court Judge Ronald Guzman, the trial judge, overruled Magistrate Keys' findings, and held that neither the affidavit of the Internet Archive employee nor the underlying pages (i.e., the Telewizja Polska website) were admissible as evidence. Judge Guzman reasoned that the employee's affidavit contained both hearsay and inconclusive supporting statements, and the purported webpage printouts themselves were not self-authenticating.
will accept date stamps from the Internet Archive as evidence of when a given Web page was accessible to the public. These dates are used to determine if a Web page is available as prior art
for instance in examining a patent application.
A number of cases have been brought against the Internet Archive for its Wayback Machine archiving efforts. See Internet Archive controversies and legal disputes.
began to provide links to previous versions of pages archived on the Wayback Machine.
Internet Archive
The Internet Archive is a non-profit digital library with the stated mission of "universal access to all knowledge". It offers permanent storage and access to collections of digitized materials, including websites, music, moving images, and nearly 3 million public domain books. The Internet Archive...
non-profit organization, based in San Francisco, California
San Francisco, California
San Francisco , officially the City and County of San Francisco, is the financial, cultural, and transportation center of the San Francisco Bay Area, a region of 7.15 million people which includes San Jose and Oakland...
. It is maintained with content from Alexa Internet
Alexa Internet
Alexa Internet, Inc. is a California-based subsidiary company of Amazon.com that is known for its toolbar and Web site. Once installed, the toolbar collects data on browsing behavior which is transmitted to the Web site where it is stored and analyzed and is the basis for the company's Web traffic...
. The service enables users to see archived versions of web page
Web page
A web page or webpage is a document or information resource that is suitable for the World Wide Web and can be accessed through a web browser and displayed on a monitor or mobile device. This information is usually in HTML or XHTML format, and may provide navigation to other web pages via hypertext...
s across time, which the Archive calls a "three dimensional index". Internet Archive
Internet Archive
The Internet Archive is a non-profit digital library with the stated mission of "universal access to all knowledge". It offers permanent storage and access to collections of digitized materials, including websites, music, moving images, and nearly 3 million public domain books. The Internet Archive...
bought the domain waybackmachine.org for their own site. It is currently in its beta test.
The name Wayback Machine is a reference to a segment from The Rocky and Bullwinkle Show
The Rocky and Bullwinkle Show
The Rocky & Bullwinkle Show is an American animated television series that originally aired from November 19, 1959 to June 28, 1964 on the ABC and NBC television networks...
in which Mr. Peabody and Sherman use a time machine
Time travel
Time travel is the concept of moving between different points in time in a manner analogous to moving between different points in space. Time travel could hypothetically involve moving backward in time to a moment earlier than the starting point, or forward to the future of that point without the...
called the "WABAC machine" to witness, participate in, and, more often than not, alter famous events in history.
Origins, growth and storage
In 1996, Brewster KahleBrewster Kahle
Brewster Kahle is a computer engineer, internet entrepreneur, activist, and digital librarian.- Biography :Kahle graduated from the Massachusetts Institute of Technology in 1982 with a Bachelor of Science in computer science and engineering, where he was a member of the Chi Phi Fraternity. The...
, with Bruce Gilliat
Bruce Gilliat
-References:...
, developed software to crawl and download all publicly accessible World Wide Web pages, the Gopher hierarchy, the Netnews bulletin board system, and downloadable software. The information collected by these "crawlers" does not collect all the information available on the Internet since much of the data is restricted by the publisher or stored in databases that are not accessible. These "crawlers" also respect the robots exclusion standard
Robots Exclusion Standard
The Robot Exclusion Standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web crawlers and other web robots from accessing all or part of a website which is otherwise publicly viewable. Robots are often used by search engines to...
for websites wishing to opt-out of appearing in search results or being cached. To overcome inconsistencies in partially cached websites, Archive-It.org was developed in 2005 by the Internet Archive as a means of allowing institutions and content creators to voluntarily harvest and preserve collections of digital content, and create digital archives.
The digital library grew and grew and grew. But a lot of people knew about it. Information was kept on digital tape for five years, with Kahle occasionally allowing researchers and scientists to tap into the clunky database. When the archive reached its five-year anniversary, it was unveiled and opened to the public in a ceremony at the University of California-Berkeley.
Snapshots usually become available more than 6 months after they are archived, or in some cases, even later, 24 months or longer. The frequency of snapshots is variable, so not all tracked web site updates are recorded. Intervals of several weeks or years sometimes occur.
After August 2008 sites had to be listed on the Open Directory
Open Directory Project
The Open Directory Project , also known as Dmoz , is a multilingual open content directory of World Wide Web links. It is owned by Netscape but it is constructed and maintained by a community of volunteer editors.ODP uses a hierarchical ontology scheme for organizing site listings...
in order to be included. According to Jeff Kaplan of the Internet Archive in November 2010, other sites were still being archived, but more recent captures would only become visible after the next major indexing, an infrequent operation.
the Wayback Machine contained approximately three petabyte
Petabyte
A petabyte is a unit of information equal to one quadrillion bytes, or 1000 terabytes. The unit symbol for the petabyte is PB...
s of data and was growing at a rate of 100 terabyte
Terabyte
The terabyte is a multiple of the unit byte for digital information. The prefix tera means 1012 in the International System of Units , and therefore 1 terabyte is , or 1 trillion bytes, or 1000 gigabytes. 1 terabyte in binary prefixes is 0.9095 tebibytes, or 931.32 gibibytes...
s each month; the growth rate reported in 2003 was 12 terabytes/month. The data is stored on PetaBox
PetaBox
PetaBox is a storage unit from Capricorn Technologies. It was designed by the staff of the Internet Archive and C. R. Saikley to store and process one petabyte of information.-Goals:...
rack systems manufactured by Capricorn Technologies
Capricorn Technologies
Capricorn Technologies is a low-cost, high-density, energy efficient data storage solutions provider based in San Francisco, California. The founder and CEO is C.R. Saikley....
.
In 2009 the Internet Archive migrated its customized storage architecture to Sun Open Storage
Sun Open Storage
Sun Open Storage is an open source computer data storage platform developed by Sun Microsystems. Based on industry-standard hardware and open source technologies from Sun Microsystems, Sun Open Storage offers an open storage architecture without vendor lock-in....
, and hosts a new datacenter in a Sun Modular Datacenter
Sun Modular Datacenter
Sun Modular Datacenter is a portable data center built into a standard 20-foot intermodal container manufactured and marketed by Sun Microsystems . An external chiller and power are required for the operation of a Sun MD...
on Sun Microsystems
Sun Microsystems
Sun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
' California campus.
In 2011 a new, improved version of the Wayback Machine, with an updated interface and fresher index of archived content, was made available for public testing.
In March 2011 it was said on the Wayback Machine forum that "The Beta of the new Wayback Machine has a more complete and up-to-date index of all crawled materials into 2010, and will continue to be updated regularly. The index driving the classic Wayback Machine only has a little bit of material past 2008, and no further index updates are planned, as it will be phased out this year."
Netbula LLC v. Chordiant Software Inc.
In a 2009 case Netbula, LLC v. Chordiant Software Inc., defendant Chordiant filed a motion to compel Netbula to disable the robots.txt file on its web site that was causing the Wayback Machine to retroactively remove access to previous versions of pages it had archived from Nebula's site, pages which Chordiant believed would support its case.Netbula objected to the motion on the ground that defendants were asking to alter Netbula's web site and that they should have subpoenaed Internet Archive for the pages directly. However, an employee of Internet Archive filed a sworn statement supporting Chordiant's motion, stating that it could not produce the web pages by any other means "without considerable burden, expense and disruption to its operations."
Magistrate Judge Howard Lloyd
Howard Lloyd
Howard Lloyd is a hip hop musician who has been active since the mid-1990s. He has alternatively used both Musa Allah and Musa the Penultimate for production work...
in the Northern District of California, San Jose Division, rejected Netbula's arguments and ordered them to temporarily disable the robots.txt blockage in order to allow Chordiant to retrieve the archived pages that they sought.
Telewizja Polska
In an October 2004 case called "Telewizja Polska SATelewizja Polska
Telewizja Polska Spółka Akcyjna is Poland's public broadcasting corporation...
v. Echostar Satellite", a litigant attempted to use the Wayback Machine archives as a source of admissible evidence, perhaps for the first time. Telewizja Polska is the provider of TVP Polonia
TVP Polonia
TVP Polonia is the international channel of the Telewizja Polska . The channel is co-funded by the TVP and the Polish Ministry for Foreign Affairs and broadcasts from the TVP headquarters in Warsaw...
and EchoStar operates the Dish Network
Dish Network
Dish Network Corporation is the second largest pay TV provider in the United States, providing direct broadcast satellite service—including satellite television, audio programming, and interactive television services—to 14.337 million commercial and residential customers in the United States. Dish...
. Prior to the trial proceedings, EchoStar indicated that it intended to offer Wayback Machine snapshots as proof of the past content of Telewizja Polska’s website. Telewizja Polska brought a motion in limine
In limine
Motion in limine is a legal written "request" or motion to a judge which can be used for civil or criminal proceedings and at the State or Federal level. A frequent use is at a pre-trial hearing or during an actual trial requesting that the judge rule that certain testimony regarding evidence or...
to suppress the snapshots on the grounds of hearsay
Hearsay
Hearsay is information gathered by one person from another person concerning some event, condition, or thing of which the first person had no direct experience. When submitted as evidence, such statements are called hearsay evidence. As a legal term, "hearsay" can also have the narrower meaning of...
and unauthenticated source, but Magistrate Judge Arlander Keys rejected Telewizja Polska’s assertion of hearsay and denied TVP's motion in limine to exclude the evidence at trial. However, at the actual trial, district Court Judge Ronald Guzman, the trial judge, overruled Magistrate Keys' findings, and held that neither the affidavit of the Internet Archive employee nor the underlying pages (i.e., the Telewizja Polska website) were admissible as evidence. Judge Guzman reasoned that the employee's affidavit contained both hearsay and inconclusive supporting statements, and the purported webpage printouts themselves were not self-authenticating.
Patent law
The United States patent office and, provided some additional requirements are met (e.g. providing an authoritative statement of the archivist), the European Patent OfficeEuropean Patent Office
The European Patent Office is one of the two organs of the European Patent Organisation , the other being the Administrative Council. The EPO acts as executive body for the Organisation while the Administrative Council acts as its supervisory body as well as, to a limited extent, its legislative...
will accept date stamps from the Internet Archive as evidence of when a given Web page was accessible to the public. These dates are used to determine if a Web page is available as prior art
Prior art
Prior art , in most systems of patent law, constitutes all information that has been made available to the public in any form before a given date that might be relevant to a patent's claims of originality...
for instance in examining a patent application.
Limitations of utility
There are technical limitations to archiving a website, and as a consequence, it is possible for opposing parties in litigation to misuse the results provided by website archives. This problem can be exacerbated by the practice of submitting screen shots of web pages in complaints, answers or expert witness reports, when the underlying links are not exposed and therefore can contain errors. For example, archives like the Wayback Machine do not fill out forms and therefore do not include the contents of e-commerce databases in their archives.Legal status
In Europe the Wayback Machine could be interpreted to violate copyright laws. Only the content creator can decide where his content is published or duplicated, so the Archive would have to delete pages from its system upon request of the creator. The exclusion policies for the Wayback Machine can be found in the FAQ section of the site. The Wayback Machine also retroactively respects robots.txt files.A number of cases have been brought against the Internet Archive for its Wayback Machine archiving efforts. See Internet Archive controversies and legal disputes.
Search engine links
In 2005, Yahoo! SearchYahoo! Search
Yahoo! Search is a web search engine, owned by Yahoo! Inc. and was , the 2nd largest search engine on the web by query volume, at 6.42%, after its competitor Google at 85.35% and before Baidu at 3.67%, according to Net Applications....
began to provide links to previous versions of pages archived on the Wayback Machine.
External links
- Internet Archive: Wayback Machine
- Internet Archive Wayback Machine beta interface (has some post-2008 content not available through the standard interface)
- Official mirror of the Wayback Machine at the Bibliotheca AlexandrinaBibliotheca AlexandrinaThe Bibliotheca Alexandrina or Maktabat al-Iskandarīyah is a major library and cultural center located on the shore of the Mediterranean Sea in the Egyptian city of Alexandria...