Web Archiving Article Index for
Web 		Website Links For
Web 		 

Information About

Web Archiving
  CATEGORIES ABOUT WEB ARCHIVING
   
archives
   
digital libraries
   
library of congress
   
digital library project
   
web archives
   
world wide web
   
APPAREL
BABY
BEAUTY
BOOKS
CAR TOYS
CELL PHONES
DVD'S
ELECTRONICS
GOURMET FOOD
GROCERIES
HEALTH & PERSONAL
HOME & GARDEN
JEWELRY
MUSIC
MUSIC INSTRUMENTS
OFFICE PRODUCTS
SOFTWARE
SPORTING GOODS
TOOLS & HARDWARE
TOYS
VIDEO GAMES
SHOPPING HOME
MORE SHOPPING...




COLLECTING THE WEB


Web archivists generally archive all types of web content including HTML web pages, Style Sheet s, JavaScript , Images , and Video . They also archive Metadata about the collected resources such as access time, MIME Type , and content length. This metadata is useful in establishing Authenticity and Provenance of the archived collection.


METHODS OF COLLECTION


Remote harvesting

The most common web archiving technique uses Web Crawler s to automate the process of collecting Web Page s. Web crawlers typically view web pages in the same manner that users with a browser see the Web, and therefore provide a comparatively simple method of remotely harvesting web content. Examples of web crawlers frequently used for web archiving include:



On-demand

There are numerous services that may be used to archive web resources "on-demand", using web crawling techniques:
  • WebCite , a service specifically for scholarly authors, journal editors and publishers to permanently archive and retrieve cited Internet references (Eysenbach and Trudel, 2005).

  • Archive-It , a subscription service, allows institutions to build, manage and search their own web archive

  • hanzo:web is a personal web archiving service created by Hanzo Archives that can archive a single web resource, a cluster of web resources, or an entire website, as a one-off collection, scheduled/repeated collection, an RSS/Atom feed collection or collect on-demand via Hanzo's open API.

  • Spurl.net is a free on-line bookmarking service and search engine that allows users to save important web resources.



Database archiving

Database archiving refers to methods for archiving the underlying content of database-driven websites. It typically requires the extraction of the and the National Library Of Australia respectively. DeepArc enables the structure of a Relational Database to be mapped to an XML Schema , and the content exported into an XML document. Xinq then allows that content to be delivered online. Although the original layout and behavior of the website cannot be preserved exactly, Xinq does allow the basic querying and retrieval functionality to be replicated.


Transactional archiving

Transactional archiving is an event-driven approach, which collects the actual transactions which take place between a Web Server and a Web Browser . It is primarily used as a means of preserving evidence of the content which was actually viewed on a particular Website , on a given date. This may be particularly important for organizations which need to comply with legal or regulatory requirements for disclosing and retaining information.

A transactional archiving system typically operates by intercepting every HTTP request to, and response from, the web server, filtering each response to eliminate duplicate content, and permanently storing the responses as bitstreams. A transactional archiving system requires the installation of software on the web server, and cannot therefore be used to collect content from a remote website.

Examples of commercial transactional archiving software include:



DIFFICULTIES AND LIMITATIONS


Crawlers

Web archives which rely on web crawling as their primary means of collecting the Web are influenced by the difficulties of web crawling:
  • The Robots Exclusion Protocol may request crawlers not access portions of a website. Some web archivists may ignore the request and crawl those portions anyway.

  • Large portions of a web site may be hidden in the Deep Web . For example, the results page behind a web form lies in the deep web because a crawler cannot follow a link to the results page.

  • Some web servers may return a different page for a web crawler than it would for a regular browser request. This is typically done to fool search engines into sending more traffic to a website.

  • Crawler Trap s (e.g., calendars) may cause a crawler to download an infinite number of pages, so crawlers are usually configured to limit the number of dynamic pages they crawl.


The Web is so large that crawling a significant portion of it takes a large amount of technical resources. The Web is changing so fast that portions of a website may change before a crawler has even finished crawling it.


General limitations


Not only must web archivists deal with the technical challenges of web archiving, they must also contend with intellectual property laws. Peter Lyman (2002) states that "although the Web is popularly regarded as a Public Domain resource, it is Copyright ed; thus, archivists have no legal right to copy the Web."
Some web archives that are made publicly accessible like won.


REFERENCES





SEE ALSO




EXTERNAL LINKS