Scholarly electronic publishing and libraries

20.11.1995

Juha Hakala

Systems analyst

Automation Unit of Finnish Research Libraries

Helsinki University Library

Juha.Hakala@Helsinki.FI

Introduction

This paper contains reflections on the likely future of electronic publishing, taking into account mainly the technological changes that are likely to happen in the near future. Another main theme will be the means libraries can use to integrate electronic documents to their existing services. Long-term storage and availability of electronic documents will also be discussed.

The wider issue of digital libraries will not be discussed; a good starting point for those readers who want to know more about that theme is the digital library special issue of the Communications of the ACM journal [vol. 38 no. 4 (April 1995)].

Electronic publishing in general

Electronic publishing may be defined as the process of making information available in electronic form through a variety of publication channels. The magnetic diskette, CD-ROM, and file transfer are among the most typical distribution media (Pieskä, p. 16).

The input of the electronic publishing process may be printed text or text files, images, graphics or structured information (i.e. a database). The end result may be a supplement or a replacement of the previous printed product, or a completely new product (Pieskä, p. 17). Electronic publishing business is very new, and commercial publishers are proceeding with caution. It has been estimated that electronic publishing may account for 7 -15 % of all commercial publishing by the year 2000. This estimate may prove to be over-optimistic, due to problems that will be discussed below.

Scholarly electronic publishing is defined here as publishing by universities and university researchers. Several universities are important commercial publishers, but often university publications like dissertations and research reports are gray literature. It is this material that is more and more often published in electronic form. For instance in Finland there are plans for electronic publishing of doctoral dissertations. An electronic copy of the dissertation - instead of the printed one - will probably be delivered to the university library which will also be given the copyright to the document.

Electronic publishing has been an important theme in many articles and reports outlining the future of the libraries. A good example is the well-known Follett report which, when discussing the future developments of information technology and libraries, handles the following related themes: Electronic Document and Article Delivery, Electronic Storage of Books and Journals, Electronic Journals, On- Demand Publishing and the Electronic Book. The other themes are Databases and Datasets, Retrospective Conversion of Catalogues, and Awareness and Training (Joint, p. 63-68).

Electronic documents and libraries

Electronic publishing is a relatively new phenomenon. Therefore it is not suprising, that libraries are not yet too well equipped to handle this material. But who would be? From the technological point of view this field of human enterprise is under extremely rapid development, and it seems unlikely that techniques and standards used in the domain of electronic publishing will stabilize in the near future. These changes are at the very heart of the problems that libraries are set to solve. I will in the next few chapters outline the difficulties ahead of us, starting from the "bottom" and proceeding towards more generic problems.

1. Longevity

Paper and it's predecessors like parchment, papyrus, clay tables or rhinestones, are well known for their longevity. Digital documents are not - in fact, the very reverse is true. For instance, the first electronic mail message there ever was is utterly lost. We only know that it travelled in 1964 from one university to another in the US. There are numerous other examples which prove that the longevity of electronic documents is far worse than that of printed media. Although digital information is theoretically eternal, the physical media on which it is stored are far from that. And even before the digital contents of some media has disappeared, it may have become unreadable because the format with which it has been created is not supported any longer (Rothenberg, p. 24). For instance in the Scandinavia, there is not a single service center which could read punch cards any more; these cards have to be sent to UK for conversion.

To make things worse, we need contextual information to interprete a bit stream correctly. We must know which application was used to create a certain file. Without a Word Perfect text editor (and a correct version of it) it is difficult and time consuming to read a WP document.

Archivists use two different strategies for preserving digital documents. The first is to translate these documents into standard forms that are independent of any computer system. The second approach is to extend the longevity of computer systems and their original software to keep documents readable. Both strategies have serious shortcomings (Rothenberg, p. 27).

Translating a document into successive short-term standards is not a good solution. Each translation brings with it new losses of content, in a similar way successive translations to different languages will alter the content of the original text. On the other hand, digital documents are evolving a lot faster than living languages. Therefore, old documents can not always be converted into new forms in meaningful ways, and it is often impossible to convert a current file back to old form (Rothenberg, p. 28).

Anyone who has ever converted documents from one text editing system to another either directly or via an intermediate format - for instance, Microsoft Rich Text Format, RFT - is likely to agree with Rothenberg that conversion is not a good alternative. So libraries intending to store digital documents for the future generations, face the task of storing not only the documents themselves, but also programs that were used to generate these documents. Finally, the library must store the system software (and hardware) required to run those programs.

Obviously, the libraries intending to store digital documents for future generations are facing a monumental task. The greatest burden will fall on National Bibliography departments in National Libraries, as they are responsible of storing free deposit copies in principle for ever. Many countries (including Finland) are rewriting the legislature controlling free deposit activities in such a way that digital documents are included. There is obviously a great need for general principles on how to archive digital documents. Luckily enough this kind of information is already available.

In December 1994, the Commission on Preservation and Access and the Research Libraries Group created the Task Force on Digital Archiving. The purpose of the Task Force is to investigate the means of ensuring "continued access indefinitely into the future of records stored in digital electronic form."

The Commission and RLG are making the draft report, entitled "Preserving Digital Information," widely available in paper and electronic form. Electronic versions are available from the RLG FTP server and Web site. HTML version is available at http://www-rlg.stanford.edu/ArchTF/. See list of references for information about versions available via FTP.

The current version of the report contains the following statement (Preserving, p. 2) which I find relevant to my article as well:

"Given the analysis in this report, its findings and recommendations, we

expect that the best use of the work of the Task Force will ultimately be

to heighten awareness of the seriousness of the digital preservation

problem, its scope and complexity -- and its manageability. There are

numerous challenges before us, but also enormous opportunities to

contribute to the development of a national infrastructure that positively

supports the long-term preservation of digital information."

2. Character sets

The first universally accepted character encoding method was the 6-bit Binary Coded Decimal (BCD). This system with 64 characters in all (A-Z, 0-9, 28 special symbols) was widely used in the late 1950s and early 1960s. To overcome the limitations of this encoding system, a 7-bit system called ASCII (American Standard Code for Information Interchange) was developed in 1965 and certified by ANSI (American National Standard Institute) in 1977 (McClure, p. 20).

There are only two languages in the world that can manage with the 7-bit ASCII encoding, namely english and suahili. No wonder various 8-bit encoding methods have been developed; foremost among these the ISO 8859 standards (for instance, the ISO 8859-1 is better known as ISO Latin-1. But in the long run, even 256 characters is woefully lacking. The British Library OPAC, for instance, contains more than 1100 different characters, many of which have been encoded into the database with escape sequences (combinations of several characters which represent a character outside the basic set).

In the long run the current 8-bit character sets will be replaced by either Unicode or ISO 10646. Unicode is a 16-bit encoding set; i.e. it accommodates 65 536 characters, including all major living languages including ideographs used in Japan and China, non-Roman alphabets such as Cyrillic, Hebrew, Arabic, Greek, and an expanded set of math and technical symbols, subscripts, superscripts, accent marks, control codes and special codes that indicate the direction of the text (left to right or vice versa) (McClure, p. 21).

A Unicode-based document or database will need two times the disk space of a ISO 8859-based document. To minimize conversion costs, ASCII and ISO Latin-1 are subsets of Unicode 1.0. Unicode, on the other hand, is a subset of 32-bit international standard ISO 10646, which allows for encoding of more than 4 billion characters.

It seems likely that Unicode will become widely used in the future, both in digital documents and in libraries' bibliographic databases. As regards the readability of electronic documents, this is good news since Unicode is compatible with the current encoding rules.

3. Markup of digital documents

The text formats used by computer manufacturers and software vendors are often incompatible for both competitive and functional reasons. Information about the way documents are to be structured and presented on computer screen and on paper should be transferred with the text. Unfortunately the information describing the structure of the document is often coded in a machine-specific manner. In a case like this, nothing more than an ASCII text can be transferred to another computer system (Pieskä, p. 1/1).

Scholarly documents have high requirements in regards to the presentation. Delivering for instance a mathematical thesis in ASCII would not do, since mathematical formulae often require sub- and superscripts and other "specialities" not included in ASCII. Conversion to flat ASCII would therefore make the whole text unreadable.

Various markup systems like Adobe's Postscript and PDF (Portable Document Format) have been developed to allow for presentation of complex documents. In "real life" it's also very common to use Word Perfect or Microsoft Word formats or "lowest common denominator" like Microsoft Rich Text Format for document exchange. However, all these markup systems are closed in the sense that development of these markup systems is not an open process. Neither are these systems truly machine- or application independent. It's also not clear if any one of these systems is versatile enough for the requirements of scholarly publishing. Luckily an international standard has been developed to overcome these problems.

SGML (Standard Generalized Markup Language, ISO 8879) allows a document's structure to be coded in a machine-independent way. SGML was designed to take account of all publishing requirements, whether on paper or on screen. Briefly, SGML stands for (Pieskä, p. 1/2):

Standard

International standard (ISO 8879)

Generalized

Stores documents in neutral format

Uses generic identifiers for elements instead of application-specific identifiers

Independent of systems, devices, languages and applications

Markup

Adds markup to a document to indicate it's structure

Gives rules on how to mark up a document

The markup itself is not standardized

Language

Metalanguage which consists of syntax but not semantics

An important SGML concept is DTD (Document Type Definition). DTD creates a class of documents by defining allowable document structures. It comprises all the rules that apply to a type of document, including the names of the various elements, any attributes they may have, and the relationships between the elements. A DTD can be reltatively simple like the HTML (Hypertext Markup Language) used in the WWW, or very complex like TEI (Text Encoding Initiative) designed for markup of documents in humanities. (Pieskä, p. 1/7).

As the popularity of WWW has increased in enormous speed during the last two years, the importance of HTML as a markup method of scholarly (and other) documents has also grown very rapidly. Some authors have regarded HTML as "DOS of WWW"; i.e. they think that the very simplicity of HTML is slowing down the progress of WWW. It's true that the now obsolete HTML version1.0 does not contain elements necessary for markup of scholarly texts - for instance, it does not contains elements for definition of tables or mathematics. Neither does the current version 2.0 contain these element. Luckily they are included in the version 3.0 currently under development.

As regards the layout of the documents, HTML 3.0 specifies style sheets, which are a powerful means of defining what the document should look like. With style sheets it's possible to define fonts and many other many other features which have an influence on what a document looks like. As the style sheet is not part of the document itself, the same style sheet can be applied to any number of HTML documents.

A problem related to current HTML browsers is that some vendors (notably Netscape Communications) have added non-HTML elements to their browser's capabilities. For instance, Netscape understands HTML tags like <BLINK> or <CENTER>, whereas other browsers do not necessarily know what to do when they find these tags in an HTML document.

It seems that HTML 3.0 (and later HTML versions) will be the most widely used markup method of scholarly documents in the near future. It is versatile enough for virtually any scholarly document and it is based on international standard. HTML is, unlike any other markup system, totally independent of application and hardware environment in which it is used. There is a wide variety of HTML editors (see a ranking list at http://www.netppl.fi/consummate/html.html) and browsers (see a ranking list at http://www.netppl.fi/consummate/www.html) available in the public domain. There are also free HTML converters available for e.g Microsoft Word and Word Perfect 6.1.

There is a lot of information about HTML 3 (and the previous versions) available, both in printed form and in the Internet. A decent introduction is included in Peter Flynn's The World Wide Web Handbook (Flynn, p. 297-312). The latest version of the HTML3 DTD is available at ftp://hplose.hpl.hp.com/pub/WWW/html3.dtd

There is also a descriptive text available at ftp://hplose.hpl.hp.com/pub/WWW/html3.txt.

W3 consortium's WWW server (http://www.w3.org) is a good source for all kinds of WWW information as well.

It's difficult to say how long WWW will be the "killer application" of the Internet. Three years ago Gopher was the best choice available; three years from now we may already have a new system which will replace WWW, though that seems highly unlikely at the moment. However, even if WWW will probably be surpassed with something more advanced in the future, HTML documents will still be readable in the new system, as the markup of these documents is based on SGML.

In the long run, it's very difficult to tell how digital documents and their markup systems will evolve. SGML has already been enhanced with HyTime (ISO10744), or Hypermedia/Time-based Structuring Language, which is a set of extensions to SGML. The aim of the HyTime is to preserve information about scheduling and interconnection of related components of a hypermedia document (e.g, audio, music score and libretto in a CD-ROM version of an opera) (Carr, p. 163).

HTML pages are currently static things. This will soon change: Sun Microsystems has developed a programming language Java, with which it is possible to create applications that can be linked to HTML documents. This way HTML pages van be made interactive. For instance, a physics textbook in HTML form may contain interactive simulations as examples of things that are discussed. In addition to Java - which is available for free - Sun has also put it's HotJava browser into public domain. In the near future, the "standard" browsers Netscape and Mosaic will also be Java compliant; Netscape in the version 2.0 to be released early in 1996. For more information on Java, see http://java.sun.com/.

In addition to HTML, there will be other SGML DTDs used in the WWW environment. The first one is VRML, or Virtual Reality Modeling Language. VRML allows creation of 3-dimensional "virtual worlds" and hyperlinking of them via the WWW to each other or to "traditional" Web resources like documents and images. For more information about VRML, see http://www.w3.org/pub/WWW/MarkUp/VRML/. Version 1.0 of the VRML specification can be accessed at http://www.virtpark.com/theme/vrml/. Finally, a list of VRML browsers is available at http://www.netppl.fi/consummate/vrml.html. Even though the VRML is as of this writing very new, there were in November 1995 already six VRML browsers.

Widespread usage of HyTime, VRML and Java will make the WWW of the future very different from what we see now. To the library the emergence of new kinds of digital documents means new challenges and constantly changing requirements. How can we make a description of for instance a VRML document which actually is a virtual world which every user experiences in a different way, and which may even be changed by every visitor?

4. Images - many file formats and compression methods

4.1 File formats

Scholarly electronic publishing does not comprise of texts only. Images (whether still or moving) will have an important role. In many cases old manuscripts and monographs will be converted to electronic form not via OCR (Optical Character Recognition) but by photographing the pages. If markup is a very important theme in handling of texts, file formats and compression are crucial questions.

Graphics files can be divided in three groups: bitmap and vector files and metafiles, which can contain both bitmap and vector data. A short introduction to these file types is available at James Murray's and William vanRyper's highly recommendable book Encyclopedia of Graphics File Formats (Murray, p. 57- 91).

There are several competing file formats for graphics files. Murray and vanRyper describe no less than 91 of them. Many formats have gone through several versions; for instance, there has been six TIFF versions in six years. There is no international standard for graphics file format; all we have is proprietary systems of varying quality and openness. Murray and vanRyper tell that there were cases in which they could not find even the organization which had made the format specification, and in two cases the organization responsible of the format could not find the specification it has made (Murray, p. xxiii). I hope there are no libraries that have based their digital archives on these file formats...

Any graphics file consists of header and (bitmap or image) data. Header consists of 1-n fixed fields, which tell to the application reading the document which file format has been used and how the bitmap data has been stored. The header structure varies considerably from format to format, but generally a header starts with unique identification value called a file ID, which defines the format itself (Murray, p. 59-74).

It is impossible to recommend a single file format to all uses as many formats have been tailored to special purpose. Of all the general formats, it seems that TIFF (Tag Image File Format) is one of the most flexible and widely used file formats. It was originally released in 1976 by Aldus Corp., and it has since then undergone several revisions. TIFF 6.0 was introduced in 1992. TIFF is widely used as data interchange format. Alas, TIFF has proved to be somewhat difficult to implement - it happens all too often that one TIFF implementation can not read data produced by another (Murray, p. 663-666).

TIFF supports perhaps more types of data compression than any other image file format. Among the supported compression method are CCITT Group 3 and 4, LZW and JPEG (see below). It should be noted that JPEG support (which was added in TIFF 6.0) has been designed poorly. This may further increase the probability of different TIFF implementations of not being interoperable (Murray, p. 684- 687).

When a library chooses a file format for it's image archive it should be aware of different alternatives and their limitations. For instance, Kodak Photo CD is quite widely used, but very closed in the sense that Kodak will not make not available information on the format that would enable developers to directly access the image data (Murray, p. 384). Images stored on Photo CD can only be accessed with Kodak software. GIF (Graphics Interchange Format) is also very popular, but is still unfit for serious use as it allows at best only 256 colours.

4.2 Compression methods

Bitmap graphic images require a lot of disk space if each pixel is stored. Therefore various compression methods have been developed. These include run-length encoding (RLE), Lempel-Ziv-Welch (LZW) compression, CCITT (Huffman) encoding and JPEG compression (for more information, see Murray, p. 125-172). JPEG (Joint Photographic Experts Group) can be recommended because it's an international standard. Moreover, there is a lot of public domain JPEG applications available. It's also easy to find out more about JPEG; the JPEG FAQ is probably one of the best starting points. It is available at

ftp://ftp.sunet.se/pub/usenet/news.answers/jpeg-faq/ .

Another good starting point is the compression FAQ, available in three parts at

ftp://ftp.sunet.se/pub/usenet/news.answers/compression-faq/.

Joint Photographic Experts Group is the original name of the committee that wrote the ISO JPEG standard. JPEG is a lossy compression mechanism; the original image can not be restored with no loss of data. For archival purposes it's therefore necessary to store the original image in addition to the compressed copy. JPEG lets the compression rate to be chosen freely. Generally a JPEG compressed copy of the original image is more or less identical to the original up till 20-fold compression rate (20 MB -> 1 MB).

It's quite unlikely that JPEG will be the final compression method. Fractal compressions may yield a lot better compression rates, though there has been a lot of hype around this method. Another method lurking in the horizon is wavelet compression, which also looks very promising. The bottom line of all this is that libraries will in the long run store bitmap graphics images using different compression methods. It's not always feasible to convert from one compression method to another as the conversion may give relatively poor results. So, the need for storing uncompressed original images is further underlined.

The compression methods discussed above are meant for compression of still images. Motion pictures compression standard similar to JPEG are MPEG and MPEG-2. The latter allows for higher quality of compressed images, but uses the same basic technology than MPEG.

Digital documents and library - some viewpoints

Digital documents are being made available in many different varieties. In the future, some of these may be thoroughly innovative; for instance, the Follett report states that "the Review Group has used common working descriptions for new developments such as the electronic book, electronic document delivery and electronic journal. It recognizes however that these are imprecise and overlapping concepts, which may change out of all recognition if and when electronic media come to predominate over the printed word." (Joint, p. 63).

Multitude of markup methods, file formats and compression methods available today may serve researcher's varying needs, but the current situation is a burden to the libraries. Which document interchange formats and compression algorithms should be chosen? What file transfer mechanism to use?

In order to avoid an electronic Babel, a number of library organisations from around the world came together and in October 1990 formed the "Group on Electronic Document Interchange - GEDI". The group members include BLDSC, INIST, PICA, OCLC and RLG. Since all these organizations have mutual in the electronic exchange of the full texts of documents, GEDI has been seeking to define a framework which will provide de facto standards for their electronic formats and protocols (Electronic, p. 2).

In October 1991, the first version of GEDI agreements was agreed by the GEDI members. From the experience that has been built since then, changes have been proposed to the agreements. The present version of GEDI agreements is version 2.1 dated July 1995.

The current GEDI document recommends usage of FTAM and/or FTP for file transfer (Electronic, p. 35-38). FTP is the file transfer protocol used in the Internet, FTAM is the OSI File Transfer, Access and Management protocol used nowhere. In addition to file transfer protocols GEDI will in the future also support document transfer via email (SMTP & MIME, X.400).

The document exhanged will consist of two parts: cover information and document image. Images will be transmitted in the TIFF version 6.0 format, using no compression at all, or either CCITT Group 3 or Group 4 compression (Electronic, p. 34). In addition to TIFF, future versions of GEDI will also support SGML and PDF document types in addition to images (Electronic, p. 3).

For each file transmission method and document type the GEDI document contains, there is a precise definition of how it should be used. This document is therefore a very important starting point for system engineers when new document delivery services are being planned and built. If GEDI recommendations are not used, the likelihood of interoperability between systems is significantly worse even if the same file formats and file transfer methods are used by all software vendors.

Electronic documents may make possible a general shift from holding strategy ("just in case") to access strategy ("just in time") in library activities. Electronic delivery of articles and other digital documents would enable materials stored in one library to be readily transmitted elsewhere (Joint, p. 63). Of course, this kind of service is only possible if a) digital documents are searchable either via OPAC, or by some other means, and b) the methods used in production and transfer of the digital document are "understood" by the applications used in the receiving library.

In addition to these problems, the Follett report lists two additional ones. First, within the academic world, a major difficulty is the current lack of acceptability and status of publishing in electronic form for the purposes of peer review processes associated with research funding and individual academic career progression. Second, existing (commercial) publishers fear that electronic journals will undermine the present market without offering a reasonable prospect that it will be replaced. Publishers are also afraid of losing control over the contents of journals, which would threaten their own copyright and that of authors (Joint, p. 65).

The present Internet is very poorly equipped as regards the security measures like encryption. On the other hand, electronic money is still under development, and there are many contestants, any one of which may win. Reluctancy of commercial publishers is really quite understandable, especially when the end users like university researches generally believe that electronic journals will be cheaper than the printed ones (Follett report did not find any reason to believe this). Of course, if a researcher publishes by himself, then there are no costs involved except time used to complete the document.

There are a few projects in the area of electronic documents delivery like Tulip and ELSA on which commercial publishers are actively involved. It will be interesting to see if these projects will be as successfull as library-initiated projects. For commercial and copyright reasons publishers have been very cautious in making their documents available in electronic form, and in some projects this has led into somewhat awkward arrangements.

In any case, electronic document delivery seems to be very promising area of development. Libraries do not need to obtain books and serials in printed form any more; instead, a library can consist of "virtual shelves", filled with digital documents which have been catalogued in library's OPAC, alongside with "real books". But if the end users are able to first locate and then retrieve by themselves digital documents, without even visiting the library in person, what will happen to traditional ILL activities?

Cataloguing of digital documents

The Internet consists of huge number of digital resources. It's definitely beyond the resources of any library to catalogue even a significant portion of these resources; neither are many of them worth of time- consuming manual cataloguing.

Libraries can utilize different strategies when providing metainformation about the Internet resources and thus making them more easily accessible to their users. These strategies are called here traditional cataloguing, automatic indexing and usage of"intelligent documents".

Traditional cataloguing

Proper cataloguing of Internet resources requires a few basic ingredients. First, there must be ISBD(CF). or International Standard Bibliographic Description for computer files (computer file in this context means all kind of computer resources like databases, documents, etc.). Enhancement of the 1st edition of ISBD(CF) has been under way for a while, and the final draft has been sent to people involved with the development in September 1995. There has been some formidable problems in the development work, one of the most severe having been the version control: how do we differentiate between various versions of a single resource? A single document may exist in various forms, more or less identical (for instance, there may be a Postscript, WP, ASCII and HTML versions of the same text). On the other hand, it is very easy to change an electronic resource; some items like databases tend to change all the time. How should this non-stableness be taken into account?

ISBD(CF) is not yet complete, but development of MARC formats for computer files is well under way in many countries. For instance, work for FINMARC(CF) has already started and will be completed next spring. Once the national MARC format for computer files is stable, software vendors will begin development of enhancements needed for computer resource cataloguing. In LINNEA network, cataloguing of digital resources with correct format will start roughly one year from now. For testing purposes some cataloguing has already been done; for instance, the Helsinki University Library has started in September 1995 an imaging project, in which a few hundred images will be digitized and then catalogued in a VTLS database.

In the USA there are already a few spectacular projects. For instance, OCLC has widely publicized it's co-operative Intercat project, in which a public access database (available at http://www.oclc.org:6990/) of Internet resources will be built. Large number of libraries is producing records into this database which has a simple WWW-based user interface. Unfortunately the number of records in the database is still quite small (less than 2000). OCLC is still seeking for more volunteers; see the project's call for participation at http://www.oclc.org/oclc/man/catproj/catcall.htm

A basic requirement in Internet resource cataloguing is that the "electronic shelf location" of the resource must be supplied. In practice, the MARC record must contain information on where and how a user can fetch the resource, or how he/she can use it. In the Internet, this information is provided by the URL code.

URL (Uniform Resource Locator) is an Internet standard RFC1738 (Berners-Lee). Like the HTML markup, it's a result of the WWW project started at CERN in the end of 1980's. In short, URL contains the syntax and semantics of formalized information for location and access of resources via the Internet. There have been several URL codes already in this article; an average Internet user has become so used to them that they are provided without any further explanations in many different contexts, including articles in newspapers. Internet users are also accustomed to the annoying fact that the lifetime of an URL tends to be short; an URL which worked a few weeks ago does not necessarily do it any more.

A more stable mechanism called URN (Uniform Resource Name) is under development. While URL replaces signum in virtual library, URN should act like ISBN; it ought to uniquely identify a given Internet resource, no matter what it's format is. The problem is that it has been painfully difficult to agree on URN syntax. On the other hand, it's difficult to decide who should possess the right to give URL's to publishers of digital documents. As basically anyone can "publish" in the Internet, it is difficult to follow this process. Anyway, it is to be hoped that URN code will be completed too in the near future, since then it will be possible to build URN->URL directories. This directory would allow users to search copies of a document from the Internet with it's URN, and locate all appearances of the resource. Of course, a normal library database might act like this. At the moment you can search OPAC with ISBN in order to find signums for a book you want to read. Using URN in order to find URLs is basically the very same thing in a virtual library.

In order to make possible cataloguing of digital resources, a new field called Electronic location and access (tag 856) has been defined into USMARC format. The contents of this tag has been described in an electronic document available at gopher://marvel.loc.gov:70/00/.listarch/usmarc/93-4.doc

In it's original form, 856 did already contain all kinds of useful information. However, there was no room for the URL as it was still under development back in 1993. In 1994, a subfield u was added for URL. See more information at gopher://marvel.loc.gov:70/00/.listarch/usmarc/94-3.doc

Most, if not all, MARC formats will follow USMARC's example in defining 856 field. In Finland, Automation Unit of Finnish Research Libraries has authored provisíonal guidelines on how to use this field. A small number of records containing 856-field are available in the article index database ARTO.

Many library systems allow already usage of 856 tag in cataloguing. There are also several systems (including VTLS and Geac) which have developed 856-capable user interfaces. This means that once the library system's client detects an URL code in 856 subfield u, it asks the user if he/she wants to see the resource. If the user is willing, then the OPAC client asks Mosaic or Netscape to retrieve and present the document. In the WWW-based OPAC interfaces like the ones used in the OCLC internet resource database and the ALIS DTV database (available at http://jupiter.dtb.dk/ALEPH/eng/dtb/dtb/) retrieval of linked electronic resources is even simpler.

Automatic indexing

Various tools have been developed in order to make possible browsing and indexing of Internet information resources. For instance, Archie indexes FTP servers and Veronica Gopher resources. There are several tools like Veronica for World Wide Web, including Lycos and World Wide Web Worm. For more information of these tools, see the comprehensive WWW page compiled by the Lund University Electronic Library, available at http://www.ub2.lu.se/nav_menu.html.

Some libraries - for instance, the Lund university library (UB2) - have been very active in this area. However, most indexing tools have been developed by the computer scientists, which may in part explain dismal quality of some of these services. There is a great need to library involvement in this area, so as to enhance the quality of this kind of databases. The EC project Desire (see http://www.ub2.lu.se/desire/) has as one of it's aims improvement of this kind of indexing services.

Another relevant aspect is that a service maintained by libraries is more likely to remain non-commercial than services maintained by firms, which may make either indexing of pages or searches from the database a commercial activity.

Intelligent documents

Full-text indexing of WWW documents is fairly frustrating. As there is no control whatsoever of terminology, searches tend to have very high recall and extremely low precision. For instance, a search with term "sweden" from Web index of all the Finnish Web pages maintained by the Finnish Telecom available at http://www.inet.tele.fi/trampoliini/) creates an enormous amount of non-relevant hits. On the other hand, search by subject "science fiction", using proximity operator gives fairly good results.

One means of making Web searching easier is usage (and indexing) of "intelligent documents". These are documents which contain metainformation about themselves. In practice, these can be done by defining a set of SGML header tags with which the author may tell about himself and the document. Once an indexing robot finds this kind of document, it may index only the header information instead of the full text.

In theory it is possible to encode information in MARC record into this kind of SGML header, or vice versa, convert a SGML header into some kind of MARC record. In practice, SGML headers tend to be a very much simplified version of MARC, with a limited set of fields with no subfield structure so familiar to library cataloguers. The purpose of this simplification is to make coding so easy that even a non- librarian author can provide descriptive information needed for decent indexing of the document.

One of these header schemes is the Dublin Core specified by the OCLC

(see http://www.oclc.org:5047/oclc/research/conferences/metadata/dublin_core_report.html).

Dublin core consists of 13 fields:

Element Description

Subject: The topic addressed by the work

Title: The name of the object

Author: The person(s) primarily responsible for the intellectual content of the object

Publisher: The agent or agency responsible for making the object available

OtherAgent: The person(s), such as editors and transcribers, who have made other

significant intellectual contributions to the work

Date: The date of publication

ObjectType: The genre of the object, such as novel, poem, or dictionary

Form: The data representation of the object, such as Postscript file or Windows

executable file

Identifier: String or number used to uniquely identify the object

Relation: Relationship to other objects

Source: Objects, either print or electronic, from which this object is derived, if applicable

Language: Language of the intellectual content

Coverage: The spatial locations and temporal durations characteristic of the object

A Dublin core record might look like this:

Subject: IETF, URI, Uniform Resource Identifiers

Title: A Unifying Syntax for the Expression of Names

and Addresses of Objects on the Network as used

in the World-Wide Web.

Author: Berners-Lee, T.

Publisher: CERN

Date: 1994

Object-Type: Internet RFC

Form (scheme=IMT): text/plain

Identifier(scheme=URL): gopher://gopher.es.net:70/0R0-57601-/pub/rfcs/rfc1630.txt

It is not clear whether authors or publishers of digital documents will be willing to provide this kind of information. Libraries, on the other hand, should discuss whether traditional MARC cataloguing should be replaced or supplemented by this kind of provision of metainformation.

Conclusion

The number of digital documents available in the Internet will grow very rapidly in the future. These documents will be made available in a multitude of different markup systems and file formats, and major share of them will belong to the category of gray literature. Library OPACs, and especially union catalogues like the ones in use in Scandinavian countries, will become important sources of digital materials as well as traditional library materials, as libraries start routine cataloguing of digital documents. In addition to OPACs, libraries may provide databases built by automatic indexing of Internet resources, which hopefully will in the future contain some information of themselves in one form or another.

Document delivery services will - at least to some extent - be made directly available to end users. Technical problems related to this kind of activity are mainly related to differing formats and delivery methods; careful planning is needed to minimize these hindrances. However, compared with copyright problems, technical problems will be of minor importance.

References

Berners-Lee, Tim et al. (eds): Uniform Resource Locators (URL). RFC 1738. December 1994. Electronic document, available at ftp://nic.nordu.net/rfc/rfc.1700-1799/rfc1738.txt

Carr, L. et al.: Why use HyTime? Electronic publishing vol. 7 (1994) no. 3 p. 163-178.

Electronic document delivery : towards further standardization of internation interchange. Agreements of the Group on Electronic Document Interchange (GEDI). Version 2.1 July 1995. Electronic document.

Flynn, Peter: The WorldWideWeb handbook: an HTML guide for Users, Authors and Publishers. London: Thomson, 1995.

Joint Funding Councils' Libraries Review Group: Report. [s.l], 1993. Electronic version of this report is available at http://www.ukoln.bath.ac.uk/follett/.

McClure, Wanda & Hannah, Stan: Communicating Globally : the Advent of Unicode. Computers in Libraries vol. 15 no. 5 (1995) p. 19-24.

Murray, James D. & vanRyper, William: Encyclopedia of Graphics File Formats. Sebastopol: O'Reilly, 1994.

Pieskä, Kari & Alkula, Riitta & Heimburger, Anneli: Publishing on CD-ROM. Espoo: Technical Research Centre of Finland, 1995.

Preserving digital information : Draft report of the Task Force on Archiving Digital Information commissioned by the Commission on preservation and access and the Research Libraries Group. Version 1.0 August 23, 1995. Electronic document; available in ASCII form at

ftp://ftp.lyra.edu /pub/ArchTF/Draft-Report.txt

Rothenberg, Jeff: Ensuring the Longevity of Digital Documents. Scientific American vol. 273 no. 1 (January 1995) p. 24-29.

Weibel, Stuart et al.: OCLC/NCSA metadata workshop report. Digital document available at http://www.oclc.org:5047/oclc/research/conferences/metadata/dublin_core_report.html

An index of Standards for Open Information Interchange is available at http://www.echo.lu/impact/oii/. This list contains useful links to many of the standards mentioned in this presentation, and to several relevant protocols (like MPEG) that has not been included. There is also a news section; see for instance http://www.echo.lu/impact/oii/oiimay95.html for May 1995.