Intelligent Multimedia Computer Systems: Emerging Information Resources in the Network Environment

By Charles W Bailey, Jr.

Print version: Bailey, Charles W., Jr. "Intelligent Multimedia Computer Systems: Emerging Information Resources in the Network Environment." Library Hi Tech 8, no. 1 (1990): 29-41.

      A multimedia computer system is one that can create, import, integrate, store, retrieve, edit, and delete two or more types of media materials in digital form, such as audio, image, full-motion video, and text information. This paper surveys four possible types of multimedia computer systems: hypermedia, multimedia database, multimedia message, and virtual reality systems. The primary focus is on advanced multimedia systems development projects and theoretical efforts that suggest long-term trends in this increasingly important area.

Introduction

A multimedia computer system is a computer system that can create, import, integrate, store, retrieve, edit, and delete two or more types of media materials in digital form, such as audio, image, full-motion video, and text information. Multimedia computer systems also may have the ability to analyze media materials (e.g., counting the number of occurrences of a word in a text file). A multimedia computer system can be a single- or multiple-user system. Networked multimedia computer systems can transmit and receive digital multimedia materials over a single computer network or over any number of interconnected computer networks. As multimedia computer systems evolve, they may become intelligent systems by utilizing expert system technology to assist users in selecting, retrieving, and manipulating multimedia information.

This paper surveys four possible types of multimedia computer systems: hypermedia, multimedia database, multimedia message, and virtual reality systems. It examines the potential benefits and problems associated with the use of multimedia computer systems as public-access computer systems, which can be employed directly by library patrons.1 Without question, multimedia computer systems will have a profound impact on library systems that are used for internal purposes; however, this area is beyond the scope of the present paper. This paper also does not attempt to survey the wide array of supporting hardware (e.g., CD-I, CD-ROM XA, and DVI) and software products (e.g., BASISplus and HyperCard) that will be used to build multimedia computer systems. Several recent papers introduce the reader to these topics.2-4 Rather, it primarily focuses on advanced multimedia system development projects and theoretical efforts that suggest long-term trends in this increasingly important area.

We are in a period of rapid technological change where new computing technologies will quickly evolve and converge, creating hybrid computing systems from the cross-fertilization of previously discrete products and research areas. The four categories of systems suggested in this paper are intended to provide the reader with a conceptual framework for thinking about the emerging area of multimedia computing. Whether these categories develop into significant applications and/or remain distinct technologies remains to be seen.

The Evolving Computing Environment

Multimedia computer systems are developing in the broader context of a highly dynamic computing environment. Van Houweling has surveyed numerous important computing trends, some of which are summarized and elaborated upon here.5,6 Computer and data communications technologies are continuing to develop at a rapid rate, providing higher performance while reducing both the cost and size of system components. As prices decline, users will increasingly employ powerful computer workstations, which will be equipped with high-resolution displays and high-capacity magnetic and optical storage units. Information in many media formats will be stored in digital form, and improved, cost-effective methods of converting older materials in non-digital formats will be developed. As the necessary network infrastructure is put in place and the problems associated with interconnecting heterogeneous computer networks are solved, library workstations will become linked to a growing number of institutional and external networks. In this networked environment, the user will employ the services of specialized, more powerful computers, known as servers, in conjunction with workstation-based processing of data. Computer messaging and conferencing systems will be increasingly important sources of information in the networked environment.

Hypermedia Systems

What is hypermedia? Hypermedia represents an evolution of the concept of hypertext.

According to Yankelovich et al., hypertext "denotes nonsequential writing and reading. Both an author's tool and a reader's medium, a hypertext document system allows authors or groups of authors to link information together, create paths through a corpus of related material, annotate existing texts, and create notes that point readers to either bibliographic data or the body of the referenced text."

"By extension," they explain later, "we use the word hypermedia to denote the functionality of hypertext but with additional components such as two- and three-dimensional structured graphics, paint graphics, spreadsheets, video, sound, and animation."7

Hypertext and, consequently, hypermedia systems can have several key characteristics, depending on the features system designers have chosen to include in particular systems.8 Information is stored in nodes, which are modular units of information. Nodes may be categorized by type to aid retrieval (e.g., citation node), and they may be organized in a hierarchical fashion. Nodes are connected together by links to form a network. The user traverses the network by activating the node links, which may be highlighted words, icons, locations on a graphic image, or other types of markers. This process is called "navigation." Links may be one-way or two-way. They may be associated with executable procedures that are activated when the link is traversed. Users may be able to identify locations to which they want to return by placing "bookmarks" at those nodes. An historical record of node traversal may be maintained, allowing users to select a prior node and quickly return to it. Authors may define paths through the network, called "tours," that automatically present a sequence of nodes. Textual search keys may be used to retrieve nodes or data within nodes. Selective views of the network may be accomplished using filters, which display a subset of nodes based on user-defined criteria. Graphic browsers, which provide a map of nodes and links, may be available to assist the user in selecting appropriate nodes.9 Landmark views, which present major nodes using a spatial metaphor and allow users to move to nodes at different distances from the current node, may assist users in navigating as well.10

Hypermedia systems typically allow users to create and edit component multimedia nodes and links as well as to import multimedia information from external sources. They may be able to create read-only distribution versions of hypermedia documents.

Based on experience with the NoteCards system, Halasz offers numerous suggestions for improving the next generation of hypermedia systems.11 It is a common perception that navigation of hypermedia systems becomes increasingly difficult as these systems grow in size. Two types of enhanced searching capabilities could ease this problem: 1) more powerful techniques for searching the content of nodes and links; and 2) graphic "structure" searching of interconnection patterns in the hypermedia network. Another current problem of hypermedia systems is the inability to deal with a group of highly interrelated nodes as a unit. This problem could be solved by permitting the definition and manipulation of group structures of nodes. In current hypermedia systems, users must manually link related nodes together, which inhibits structure change in the network. Providing automatic definition of virtual links and virtual group structures of nodes, based on user-defined criteria, could ameliorate this problem. Other potential enhancements deal with use of expert systems to infer new information, improved control over different historical versions of both hypermedia components (e.g., nodes) and entire networks, better controls in locking mechanisms in multiuser hypermedia systems that prevent users from simultaneously changing the same material, and provision of high-level tools to permit users to customize hypermedia systems to meet their needs.

Suggested enhancements to hypertext systems, which generalize to hypermedia systems, include automatically generating hypertext materials from linear files12 and automatically constructing linear documents from hypertext materials.13

Although hypermedia systems offer a powerful new way organizing information, it is not clear at this point whether current success with modestly sized systems will translate into success with much larger systems. Lynch has suggested that the implementation of large-scale hypertext systems may involve significant technical challenges that are not evident in smaller systems, with the result being that hypertext systems become "a valuable supplement--not a replacement--for existing information retrieval technology in a large database environment."14

Examples of Hypermedia Systems

For an in-depth survey of hypertext systems, the reader should consult Conklin.15 Nielsen and Olszewski provide recent annotated bibliographies on hypertext and hypermedia that are good supplements to Conklin's survey.16 One hypertext system that is of particular interest is the Hypertext Abstract Machine, a generic multiuser hypertext server that can support different hypertext implementations (e.g., Guide or NoteCards).17

The power of hypermedia can be seen in Stanford University professor Larry Friedlander's Shakespeare Project.18, 19 The hypermedia system developed by the project utilizes the Macintosh HyperCard software. With this system, the user can review, on a video monitor, filmed versions of major scenes from Hamlet, King Lear, and Macbeth, which are stored on videodisk. Synchronized with the performance, the user can display, on the screen of the Macintosh computer, the text of the play and stage blocking information. The user can save and reuse portions of these performances. While reviewing a filmed sequence, the user can use other tools--such as dictionaries and historical notes--to aid his or her understanding of the performance. For comparison purposes, the user can easily move back and forth between different filmed versions of the same scene. The user also can produce animated versions of a play, utilizing a database of characters, props, and stages. Interactive tutorials provide instruction on basic theatrical topics. While employing the system's other capabilities, the user can take notes.

Other representative projects that are developing hypermedia materials include: 1) the Electric Cadaver20 project at Stanford University, which deals with human anatomy; 2) the HyperTeam Project21 at Dartmouth College, which works in a variety of subject areas such as art, history, molecular biology, and pathology; 3) the Intermedia project 22, 23 at Brown University, which works in several areas such as cell biology and English literature; 4) the multi-institutional Perseus project24 based at Harvard University, which deals with classical Greek civilization; and 5) PROJECT EMPEROR-I25 at Simmons College, which focuses on China during the reign of the First Emperor of China.

Multimedia Database Systems

Multimedia database systems are analogous to contemporary database systems for textual and numeric data; however, multimedia database systems have been tailored to meet the special requirements of dealing with different types of media materials. Multimedia database systems can create, import, integrate, store, retrieve, edit, and delete multimedia information. They may incorporate some hypermedia capabilities. Multiuser multimedia database systems are likely to perform these functions in a manner that reduces redundant data storage, permits different views of data by users, and provides secure access to data. Multimedia database systems are in an early stage of development; however, both theoretical and experimental work have been done.

Based on their work with the prototype MINOS system at the University of Waterloo, Christodoulakis and Faloutsos describe how networked multimedia servers, utilizing high-density optical storage devices, may be used to deliver information to sophisticated user workstations, such as SUN workstations.26 The multimedia database would utilize what is known as an "object-oriented architecture." Object orientation is a powerful and increasingly popular concept, albeit a somewhat complex one. In brief, objects are organized into hierarchical classes based on their common characteristics.27 Lower-lower objects "inherit" the characteristics of their antecedents. Objects may be composed of many different components. The specific details of how objects are implemented are hidden from the system as a whole; however, objects will respond to specific structured messages and perform appropriate actions. Objects have unique "identities" that transcend their temporary characteristics.

The multimedia database system proposed by Christodoulakis and Faloutsos, which would contain text, image, and voice data, would operate as follows.28 Based on textual or verbal search keys, the system would retrieve objects and present the user with image or voice representations of these objects. The user would then select the object of interest. The system would permit the user to browse within the multimedia object using several techniques: 1) by "page" (a page could be text, combined text and image, or audio); 2) by marked subunits of the object, such as section or chapter; and 3) by pattern matching. When an object was retrieved, one media presentation mode (i.e., visual or audio) would be dominant; however, information in another mode could be attached to it. Visual pages, for instance, could have audio annotations. Objects could, in hypermedia fashion, also be linked to external objects. In visual mode, images could be viewed in reduced form and portions of them could be selected for close-up inspection. Images also could be presented superimposed over each other like overhead transparencies. A pre-defined sequence of visual pages could be shown automatically, permitting the author of the multimedia object to imitate a slide-tape presentation or to create basic animation effects. Finally, executable programs, which could be embedded in multimedia objects, could accept user input and perform certain pre-defined functions.

Christodoulakis and Graham outline a method for browsing "time-driven" multimedia objects, such as a video sequence.29 Major points in a multimedia sequence, which would be viewed on the left-hand side of the screen, would be described by a vertical row of icons in the right-hand side of the screen. Shaded icons would provide hypermedia links to different multimedia sequences. In addition to symbols, still images from a multimedia sequence or words could be employed as icons. These icons could be browsed to move to different points in the multimedia sequence. Horizontal lines between icons, much like the markings on a ruler, would indicate the time intervals between icons. The marked points between icons also could be directly accessed. Above the vertical row of icons would be a "context icon" and an "elapsed time indicator." The context icon would identify the broader unit of information that the user is browsing; these icons also could be browsed. The indicator would show the number of elapsed seconds from the beginning of the unit of information represented by the context icon, and the user could enter a different elapsed time to move to that point in the presentation. The elapsed time indicator also could be used to freeze or re-start a presentation. It would be necessary to stop the presentation in order to perform the various browsing functions of the system.

Ghafoor et al. outline the architecture of a proposed Heterogeneous Multimedia Database (HMD) system, which is capable of providing distributed access to textual, audio, image, and full-motion video information.30 In the object-oriented HMD system, a network controller would analyze the user's database command, identify the server or servers that housed needed multimedia information, designate one server as the "master" server where the majority of processing would occur, decide how to process information from multiple servers, and perform general network management functions. The master multimedia server, using the services of its local controller, would, if required, integrate the multimedia information from all participating servers for delivery to the user workstation. Each multimedia server, which would be a multiprocessor system with substantial memory, could house multiple database management systems, each oriented towards a particular type of data (e.g., image). A broadband optical fiber network, operating at speeds as high as 2-5 gigabits per second, would provide data transmission services for the HMD system. The need for a high-speed network is shown by the projected transmission speeds for two types of media: 1) still image--50 kilobits per second to 48 megabits per second, contingent on the resolution and color characteristics of the image; and 2) full-motion video in the High-Definition Television format--1.2 gigabits per second without compression and 200-300 megabits per second with compression.

In Germany, the BERKOM (Berliner Kommunikationssystem) project is developing the Multi-Media Document Model, a standard for providing access to multimedia documents via Broadband Integrated Services Digital Network (B-ISDN) systems.31 By focusing on B-ISDN technology, the BERKOM project is bypassing contemporary Integrated Services Digital Network (ISDN) technology in order to achieve the higher speeds required to transmit a full range of digital multimedia materials. While ISDN systems provide users with 64 kilobits-per-second data channels, the evolving B-ISDN standard is likely to support 135.168 megabits-per-second data channels.32

The BERKOM project's Multi-Media Document Model, which is based on the Open Systems Interconnection (OSI) Reference Model, has two components: the Data Model and the Communication Model.33 The Data Model describes different types of information: text, graphic, audio/speech, raster image, video/movie, modelling data, special forms (e.g., mathematical and chemical formulas), and transparent (i.e., additional data that is not apparent to the user). The Communication Model describes the telecommunications services required to deliver multimedia documents.

Examples of Other Multimedia Database Systems

Two prototype systems are harbingers of future multimedia database management systems.

The Athena Muse system at the Massachusetts Institute of Technology allows users to create complex multimedia "packages" that contain audio, graphic, textual, and video information.34 It provides users with an object-oriented multimedia environment that includes features such as hypermedia links and state-transition networks, advanced multidimensional (e.g., temporal and spatial dimensions) information management capabilities, and multimedia editing tools. Users employ DEC MicroVAX II or IBM RT workstations equipped with a special board that permits the display of full-motion video.

The Multimedia Office Server (MULTOS) system at the Istituto di Elaborazione della Informazione in Italy allows users, who employ networked SUN workstations, to retrieve multimedia documents based primarily on the textual and numeric components of those documents.35

Multimedia Message Systems

Multimedia message systems are extensions of contemporary electronic mail and conference systems which include multimedia data handling capabilities. Multimedia message systems can create, transmit, receive, reply to, forward, save, retrieve, and delete multimedia messages. As part of the message creation and editing processes, multimedia message systems can import different media materials and integrate them. Since multimedia message systems can incorporate sophisticated data handling capabilities, the distinction between this type of system and multimedia database systems can sometimes appear hazy; however, the primary purpose of these two kinds of systems is quite different. Multimedia database systems are optimized for database functions, while multimedia message systems are optimized for communication functions.

Examples of Multimedia Message Systems

BBN Software Products Corp. is marketing BBN/Slate, a multimedia electronic mail and conferencing system.36 The system runs on SUN workstations. BBN/Slate messages can include five types of information: bit-map images, geometric graphics, speech annotations, spreadsheets with associated charts, and text. An integrated media editor, which has specialized editing functions for each type of information, permits the user to easily edit messages. The system allows users to create, store, retrieve, send, receive, sort, reply to, forward, and delete messages. If the user sends a message to a colleague whose system does not have multimedia capability, BBN/Slate will automatically send the message in text-only form. Utilizing the conferencing capability of BBN/Slate, geographically dispersed users can jointly edit documents. The system can be tailored to meet the user's needs with a built-in programming language called the Slate Extension Language.

Several prototype multimedia systems intended for electronic mail or real-time conferencing have been developed, including: 1) the Command and Control Workstation (CCWS),37 an electronic mail system at SRI International; 2) the Multi-Media Bridge System,38 a real-time conference system at Bell Communications Research; 3) the Multimedia Mail Handler, an electronic mail system at the University of Southern California Information Sciences Institute,39 and 4) Pygmalion, an electronic mail and database system at the Massachusetts Institute of Technology.40

Virtual Reality Systems

The preceding types of multimedia computer systems enrich the computing environment with a wider variety of visual and auditory data. Virtual reality systems transform the computing environment by immersing the user in a simulated world, which also can include movement and tactile control. When this is accomplished, the user enters a "virtual reality."41 Virtual reality systems will permit users to interact with computer systems in a manner that more closely mimics how humans naturally operate in the real world.

Examples of Virtual Reality Systems

This evolutionary process is still in its very early stages and many barriers must be overcome for it to come to fruition; however, several experimental projects are forerunners of virtual reality systems.

Three projects at the Massachusetts Institute of Technology have explored virtual reality concepts: 1) the Aspen Movie Map,42 which simulates automobile travel in Aspen, Colorado and allows the user to explore the interiors of selected buildings in the city; 2) the Athena Language Learning Project,43 which has created two foreign-language systems that involve the user in interactive stories or documentaries based on simulated travel and conversations; and 3) The Navigation Learning Environment,44 which simulates sailing a boat in the coastal waters of Maine.

The most advanced research efforts are currently focusing on providing users with auditory, tactile, and visual experiences in simulated bodies that can move through virtual reality spaces and manipulate objects within them. VPL Research has developed various computer-based components that allow a user to experience virtual realities. By putting on specially-designed goggles with earphones, gloves, and a body suit, the user can enter a color, three-dimensional virtual reality that has approximately as much detail as a cartoon.45 Sounds in the virtual reality are made to have a three-dimensional quality as well. The user can move about in the virtual reality, and handle the objects within it. If multiple users are in the virtual reality, they can see each other's virtual bodies, including facial expressions. Autodesk, Inc., the producer of the AutoCAD program, is also developing a virtual reality system, called Cyberspace, that will simulate movement through a color, three-dimensional architectural design and allow users to manipulate virtual objects.46 NASA has established the Visualization for Planetary Exploration Project, which is developing a virtual reality system that will allow users to explore the planets of our solar system.47

Intelligent Agent Systems

Currently, our interactions with computers are primarily directive. We issue a command to the system, and if the command is properly structured the computer executes it. System interfaces may be designed to include command prompting (e.g., pull-down menus) and context-sensitive help displays, but basically they are passive entities that await our specific instructions.

What is emerging is a different model. Intelligent systems will act as our assistants and play a variety of roles in this capacity.48-49 These systems are unlikely to become truly sentient in the foreseeable future; however, utilizing artificial intelligence techniques, they will exhibit behavior that mimics intelligence within limited realms of activity.50 Natural language interactions with these systems are likely to be possible within the narrow areas of their expertise, but unrestricted natural language dialogues are still beyond our technological capabilities.51 As intelligent systems come into use, it may be appropriate consider Weyer's conceptualization of the roles of the system and the user:

      Besides helping to mine information from library-like repositories, an intelligent system must help refine and create knowledge--it should have many of the qualities of coach, tutor, and colleague, encouraging the learner to question, conjecture, create, and experiment. Although an educational information system still must help us find "the answer" more efficiently, the emphasis must be on creating questions, proposing solutions, and contributing to understanding. Talking about "learners" instead of "users" should help make these information needs primary in our minds over information sources.52

Specialized intelligent systems, called "intelligent agents,"53 will be developed that have limited, well-defined responsibilities, such as screening electronic mail. Although intelligent agent systems may initially perform generic services for all kinds of users, their real potential will be realized when they provide personalized services that are tailored to meet the user's specific needs.54

In the context of multimedia computer systems, intelligent agents could perform several functions. They could: 1) monitor multimedia databases and capture relevant new information; 2) filter incoming multimedia messages; 3) assist the user in identifying and searching appropriate multimedia databases as well as downloading data from these databases; 4) help the user manage and access personal databases; 5) guide the user in analyzing retrieved information using statistical, textual, or other analysis tools; and 6) help the user create new intellectual works from retrieved and original information.

The latter four functions could be performed in at least three possible modes: 1) advise mode, where the system monitors the user interaction with the target systems, identifies user errors, and suggests courses of action; 2) tutor mode, where the system instructs the user in the mechanics of the target system, using either simulated or real interactions with that system; and 3) transparent mode, where the system performs needed interactions with the target system for the user based on the user's high-level instructions.

Examples of Intelligent Agent Systems

Although intelligent agent systems are in their infancy, several prototype systems prefigure intelligent agent systems of the future. The Carleton Office Knowledge Engineering System (COKES) at Carleton University employs user-specific agents that can communicate with other users' agents to perform collaborative tasks, such as compiling a monthly report.55 The Composite Document Expert/Extended/Effective Retrieval (CODER) system at the Virginia Polytechnic Institute and State University provides knowledge-based searching of archived electronic mail messages about artificial intelligence from the electronic journal AIList Digest.56 The Conversational Desktop system at the Massachusetts Institute of Technology, a voice-recognition system, emulates several of the functions of a human secretary, including giving the user reminders, scheduling meetings, taking phone messages, and checking plane reservations.57 The NewsPeek system at MIT monitors computerized information services, such as NEXIS, and, based on a profile of user interests and other criteria, constructs a personalized electronic newspaper.58 The Object Lens system at MIT permits users to construct agents that automatically perform a variety of user-defined actions, such as sorting and filing incoming electronic mail, in an environment that combines electronic mail, hypertext, and object-oriented database capabilities.59

Implications for Libraries

Multimedia computer systems have the potential to improve dramatically the information transfer process in libraries. In spite of a growing diversity of media types in this century, the collections of most libraries remain predominately print based. Print is a powerful medium that has had a major impact on the development of our highly technological civilization. However, like all types of media, it has both strengths and weaknesses. Primarily, the weakness of print are its: 1) use of a single sensory channel (vision); 2) reliance on a fixed, linear sequence of presentation; 3) lack of interactivity; 4) absence of built-in editing tools to create new intellectual works; and 5) restriction to single-user mode only. DeFanti et al. suggest that print may no longer be an adequate tool to convey scientific information:

      Much of modern science can no longer be communicated in print. DNA sequences, molecular models, medical imaging scans, brain maps, simulated flights through a terrain, simulations of fluid flow, and so on, all need to be expressed and taught visually over time. To understand, discover, or communicate phenomena, scientists want to compute the phenomena over time, create a series of images that illustrate the interrelationships of various parameters at specific time periods, download these images to local workstations for analysis, and record and play back one or more seconds of the animation.60

It should be apparent that these weaknesses of print are the strengths of multimedia computer systems. Depending on its underlying technological infrastructure, a multimedia computer system can provide the user with a multisensory, nonlinear, highly interactive, edit-oriented, multiuser environment. To some degree, the reluctance of libraries to embrace traditional media materials reflects the fact that these materials come in a number of different formats, each requiring a different kind of equipment. As a "metamedium,"61 multimedia computer systems can provide unified access to diverse types of media information via a single delivery mechanism. Multimedia computer systems also can significantly increase the communication potential of media materials by relating them to each other in totally new ways.

While I am hesitant to proclaim yet another technological "revolution," it is not inconceivable that multimedia computer systems will be viewed by future generations as a major milestone in the development of information technology.

Nonetheless, the power and flexibility of multimedia computer systems might cause us to long for the relative simplicity of printed materials. By relying on a linear presentation of text and image, printed materials provide the user with a highly structured, easily comprehended information format. Once the average printed item is in hand, a literate person generally requires no significant guidance in its use. With multimedia systems, the very richness of the information environment may disorient users and overwhelm them with choices. As the functionality of these systems increases, their operational complexity will also likely increase. System designers will attempt to make user interfaces intuitive. However, user interface design is still an art, not a science. The potential solution to this problem lies in the development of increasingly powerful intelligent agents, which can shield the user from the complexities of the system. However, this area may develop at a slower pace than basic multimedia capabilities.

Until significant standardization efforts occur, the information content in multimedia computer systems will be inextricable from the hardware and software used to deliver that information, resulting in the potential need for libraries to provide a multiplicity of heterogeneous systems, each with a unique user interface. Users of heterogeneous textual/numeric database systems with graphical user interfaces are already experiencing difficulties adjusting to the differences between these systems,62 and the interfaces of these systems are less complex than those that are employed in advanced multimedia computer systems. Standardized formats for various types of source multimedia materials in digital form still need to be established, and this will limit libraries' ability to import diverse multimedia files from different information producers into local systems. Lack of these and other standards will exacerbate problems related to financing and supporting multimedia computer systems as well as slow down the diffusion of this innovation in the library community.

Many multimedia system pioneers envision a computing environment where users can easily create their own multimedia documents from the contents of hypermedia or multimedia databases. In hypermedia systems, this process could involve having the user create new personal paths through existing databases, potentially supplementing existing material with new information. Another option, shared with multimedia database systems, is for users to create entirely new works, combining parts of existing materials with new information. From a technological point of view, these activities could occur on a library system or on a user workstation, utilizing downloaded data. These scenarios raise several issues.

Historically, libraries have contained collections of fixed, immutable works. The emphasis of library systems and services have been on providing the user with access to this information, not on giving the user tools to manipulate it to create new works. Libraries would need to shift their collective service orientation in order to provide this capability. Multiuser multimedia computer systems that would allow users to generate new information would require more powerful and capacious computer hardware and a security system oriented to individual end-user accounts. Maintaining the integrity of the library's multimedia information would be a paramount concern. If information in source files maintained by the library is altered by end-users, the library may be liable should the use of this changed information by other end-users have unfortunate consequences. These factors would increase the cost of providing multimedia computer systems.

In addition to giving users the ability to create materials, libraries also could make materials created by end-users available to all users of the system. The notion of allowing unrestricted contributions to library multimedia databases is contrary to the concept of controlled collection development; however, the selective acceptance of contributions by users with special expertise could be accommodated by a relatively minor adjustment to existing collection development models. The proposed BIBLIO system, which is primarily oriented to citation data and related notes, provides an interesting conceptual model for incorporating end-user produced information in university information systems.63

The creation of multimedia materials by end-users raises definite intellectual property rights concerns. In the case of end-users creating new paths in existing databases, a new information entity (i.e., the hypermedia path) is created without physically replicating the original data. If access to this new entity were confined to the user who created the entity, it would likely be considered legal. However, if the new path was available to all system users, access might be viewed as violating intellectual property rights laws. Creation of new materials derived from captured data is likely to be legal for personal use; however, making such works available on library multimedia systems or distributing them through other means without getting appropriate clearances from the database producer could be illegal, depending on the extent to which captured information is utilized. End-users may significantly alter library-provided multimedia materials in the process of creating derivative materials from those works, potentially leading to further possible legal complications. Certainly, these scenarios raise interesting legal problems in the context of the current body of intellectual property law, which has not evolved adequately to meet the demands of a dynamic computing environment.

Problems related to system incompatibilities and standardization will initially inhibit downloading of multimedia information for use on end-user workstations; however, once these issues have been addressed, the photocopy machine will look like a Neolithic tool for the theft of intellectual property compared with multimedia computer systems. Initially, libraries are likely to have the option of offering read-only versions of multimedia materials on their systems; however, as advanced workstations and multimedia software become affordable, users are likely to obtain sophisticated multimedia tools that will allow them to easily edit downloaded information. In the long run, the likelihood is that users who have the ability to download multimedia information will utilize it as they choose, and libraries will need to both intensify their efforts to educate users about intellectual property rights laws and to influence the ongoing development of this body of law in order to ensure that end-users have reasonable and equitable access to information in the electronic environment.

Given the restrictions that contemporary license agreements impose on locally-mounted databases, remote access to multimedia library systems may be inherently problematic. Already, libraries face the problem of obligating themselves contractually to limit the population of system users to a well-defined group, while at the same time trying to make the system easily available to remote users with a minimum of record keeping. In an environment where "easy access" will increasing mean access via interconnected computer networks, unauthorized users could be on the other side of the globe as well as in the local area. For libraries whose large user populations preclude use of password-based security systems for end-users, this issue could be difficult to resolve satisfactorily.

Another issue deals with appropriate mechanisms for describing the content of computerized multimedia materials. Multimedia computer systems will provide various retrieval mechanisms; however, the effectiveness of these mechanisms will be contingent on developing appropriate intellectual access tools to deal with materials that may have little or no textual content. Computerized multimedia materials could have a high level of granularity--information may be packaged and accessed in small, modular units, such as a single image or a short video sequence. In relation to the indexing visual materials, Brooks identifies two problems:

      First, because a subject authority for visual images, such as works of art, does not exist as it does for print works, the characteristic of "aboutness" can be much more difficult to determine for visual media, especially for film segments and other components that form a unified whole. The second problem is the shortage of standardization by which subject indexing of images may be guided.64

Barring unforeseen advances in retrieval software, controlled vocabularies are likely to be important tools for retrieving multimedia information. Controlled vocabularies for multimedia materials are likely to be textual; however, it is possible that information units may be described by iconic or other symbolic representation methods. The development of these controlled vocabularies and their application at an appropriate level of detail will not be a trivial task.

Perhaps the most significant barrier to library's use of multimedia systems is financial. It is likely that commercial multimedia database producers, who will have invested significant time and resources in database development, will try to maximize their profits in the marketplace, resulting in potentially expensive products. If multimedia databases are licensed, libraries will pay annual fees, and they will not have permanent ownership rights to these materials.

The technological infrastructure needed to support these systems also may be costly. Single-user systems are likely to be affordable in small numbers, much like CD-ROM workstations are today; however, as the number of workstations increases, total hardware costs and ongoing maintenance costs will rise. New staff may be required to provide end-user and technical support. Multiuser multimedia systems may require libraries to invest significant resources in acquiring and maintaining new computer systems and software. (However, adding multiuser multimedia capability to existing library systems may reduce initial costs, assuming that these systems will not require major hardware upgrades to support the new functions and add-on software license fees are moderate.) As indicated earlier, multiuser multimedia systems may require high-speed networks to support transmission of certain kinds of materials, such as full-motion video. Libraries may need to consider more expensive data communication options than they have become accustomed to with textual systems, and their parent institutions may need to upgrade institutional networks as well.

Conclusion

For the full potential of multimedia computer systems to be realized by libraries there are many fiscal, legal, organizational, standardization, and technological challenges to be met. Step-by-step, these barriers are likely to be overcome; however, the length of this process is not predictable. The Information Workstation Group and Desktop Presentations have written a report that predicts very rapid growth for all sectors of the multimedia marketplace, with total expenditures increasing from $0.4 billion in 1989 to $16.6 billion in 1994 (educational expenditures alone would rise from $0.1 billion to $2.5 billion during this period).65 On the other hand, Gantz sees several factors that might slow the growth of multimedia computer systems, including too many competing standards and vendor systems, the inherent complexity of multimedia systems, and the initial reluctance of users to embrace this new technology.66 Although the Information Workstation Group and Desktop Presentations forecast may be overly optimistic, it suggests that multimedia computer systems represent important emerging technologies that should be taken seriously.

Microcomputer-based hypermedia systems are likely to be the first multimedia computer systems that most libraries will make available to their patrons. Libraries have already developed a variety of hypermedia systems to assist their users, and commercial hypermedia materials are becoming available as well. Multimedia database and message systems are moving from the research lab to the commercial marketplace, and technological advances in this area will be driven by the increasingly sophisticated office automation needs of business and government. As the underlying technological infrastructure to support these systems matures, hardware and software costs drop, needed standards are developed, and the base of available computerized multimedia materials increases, libraries are likely to gradually provide end-users with access to multimedia database systems and to utilize multimedia message systems for information delivery purposes. Virtual reality systems are likely to be the last multimedia computer system that libraries will provide to end-users. As the use of artificial intelligence technology becomes more prevalent, intelligent agents are likely to be interwoven with all types of multimedia computer systems. Initially, these systems are likely to be rather simple; however, they could become very sophisticated and powerful, evolving into an important tool for simplifying our interactions with an increasingly complicated information environment.

In the long-term, multimedia computer systems could have a major impact on libraries. The four types of multimedia computer systems outlined in this paper are in the early stages of their development and any projections about their future must be purely speculative. However, it is not too soon for librarians to begin tracking the development of multimedia computer systems and to consider the implications of using these systems in their libraries.

Notes

1. Charles W. Bailey, Jr., "Public-Access Computer Systems: The Next Generation of Library Automation Systems," Information Technology and Libraries 8, no. 2 (June 1989): 178-85.

2. Ching-chih Chen, "Why Current Technology is Ready for Multimedia and Hypermedia Applications," Microcomputers for Information Management 6, no. 2 (June 1989): 135-145.

3. Carl Franklin, "Hypertext Defined and Applied," Online 13, no. 3 (May 1989): 37-49.

4. Karen A. Frenkel, "The Next Generation of Interactive Technologies," Communications of the ACM 32, no. 7 (July 1989): 872-881.

5. Douglas E. Van Houweling, "Information Technology: A Survey from the Perspective of Higher Education," Library Hi Tech 4, no. 4 (Winter 1986): 23-31.

6. Douglas E. Van Houweling, "The Information Network: Its Structure and Role in Higher Education," Library Hi Tech 5, no. 2 (Summer 1987): 7-17.

7. Nicole Yankelovich, Norman Meyrowitz, and Andries van Dam, "Reading and Writing the Electronic Book," Computer 18, no. 10 (October 1985): 18-19.

8. Ben Shneiderman and Greg Kearsley, Hypertext Hands-On! An Introduction to a New Way of Organizing and Accessing Information (Reading, MA: Addison-Wesley, 1989), 2-16.

9. Chia-Jer Tsai, "Hypertext: Technology, Applications, and Research Issues," Journal of Educational Technology Systems 17, no. 1 (1988-89): 6.

10. Kamran Parsaye, Mark Chignell, Setrag Khoshafian, and Harry Wong, Intelligent Databases: Object-Oriented, Deductive Hypermedia Technologies (New York: John Wiley & Sons, 1989), 275-79.

11. Frank G. Halasz, "Reflections on NoteCards: Seven Issues for the Next Generation of Hypermedia Systems," Communications of the ACM 31, no. 7 (July 1988): 836-52.

12. Udo Hahn and Ulrich Reimer, "Automatic Generation of Hypertext Knowledge Bases," SIGOIS Bulletin 9, nos. 2 and 3 (April and July 1988): 182-188.

13. Roy Rada, "Writing and Reading Hypertext: An Overview," Journal of the American Society for Information Science 40, no. 3 (May 1989): 165-66.

14. Clifford A. Lynch, "Hypertext, Large Databases, and Relational Database Management Systems," in National Online Meeting Proceedings--1989, comp. by Carol Nixon and Lauree Padgett (Medford, NJ: Learned Information, Inc., 1989), 270.

15. Jeff Conklin, "Hypertext: An Introduction and Survey," Computer 20, no. 9 (September 1987): 17-41.

16. Jakob Nielsen, "Hypertext Bibliography," Hypermedia 1, no. 1 (Spring 1989): 74-91. Also Larry Olszewski, "HyperLibrary Applications," Library Hi Tech Bibliography, volume 5 (Ann Arbor, MI: Pierian Press, 1990), forthcoming.

17. Brad Campbell and Joseph M. Goodman, "HAM: A General Purpose Hypertext Abstract Machine," Communications of the ACM 31, no. 7 (July 1988): 856-61.

18. Larry Friedlander, "Moving Images Into the Classroom: Multimedia in Higher Education," The Laserdisk Professional 2, no. 4 (July 1989): 35-36.

19. Michael Miley, "The Inter-Play's the Thing," MacUser 5, no. 3 (March 1989): 111-13.

20. Martin Zagari, "The Electric Cadaver," Journal of the American Medical Association 261, no. 5 (3 February 1989): 781,783.

21. J. Robert Beck and Donald Z. Spicer, "Hypermedia in Academia," Academic Computing 2, no. 5 (February 1988): 24-25, 49-50.

22. Nicole Yankelovich, Bernard J. Haan, Norman K. Meyrowitz, and Steven M. Drucker, "Intermedia: The Concept and the Construction of a Seamless Information Environment," Computer 21, no. 1 (January 1988): 81-96.

23. Nicole Yankelovich, Karen E. Smith, L. Nancy Garrett, and Norman Meyrowitz, "Issues in Designing a Hypermedia Document System: The Intermedia Case Study," in Interactive Multimedia: Visions of Multimedia for Developers, Educators, and Information Providers, ed. by Sueann Ambron and Kristina Hooper (Redmond, WA: Microsoft Press, 1988), 33-85.

24. Gregory Crane and Elli Mylonas, "The Perseus Project: An Interactive Curriculum on Classical Greek Civilization," Educational Technology 28, no. 11 (November 1988): 25-31.

25. Ching-chih Chen, "As We Think: Thriving in the HyperWeb Environment," Microcomputers for Information Management 6, no. 2 (June 1989): 83-97.

26. Stavros Christodoulakis and Christos Faloutsos, "Design and Performance Considerations for an Optical Disk-Based, Multimedia Object Server," Computer 19, no. 12 (December 1986): 45-56.

27. Kamran Parsaye, Mark Chignell, Setrag Khoshafian, and Harry Wong, Intelligent Databases: Object-Oriented, Deductive Hypermedia Technologies (New York: John Wiley & Sons, 1989), 97-159.

28. Stavros Christodoulakis and Christos Faloutsos, "Design and Performance Considerations for an Optical Disk-Based, Multimedia Object Server," Computer 19, no. 12 (December 1986): 46-52.

29. Stavros Christodoulakis and Stephen Graham, "Browsing Within Time-Driven Multimedia Documents," SIGOIS Bulletin 9, nos. 2 and 3 (April and July 1988): 219-27.

30. Arif Ghafoor, C. Y. Roger Chen, and P. Bruce Berra, "A Distributed Multimedia Database System," in Proceedings: Workshop on the Future Trends of Distributed Computing Systems in the 1990s (Washington, DC: Computer Society Press of the IEEE, 1988), 461-67.

31. Peter Egloff and Angela Scheller, "Multi-Media Documents in Broadband ISDN: Development of a Generalized Telecommunication Model in BERKOM," in Proceedings: Workshop on the Future Trends of Distributed Computing Systems in the 1990s (Washington, DC: Computer Society Press of the IEEE, 1988), 103-10.

32. Larry L. Learn, "ISDN: Fantasy or Future?" Library Hi Tech News no. 60 (May 1989): 3.

33. Peter Egloff and Angela Scheller, "Multi-Media Documents in Broadband ISDN: Development of a Generalized Telecommunication Model in BERKOM," in Proceedings: Workshop on the Future Trends of Distributed Computing Systems in the 1990s (Washington, DC: Computer Society Press of the IEEE, 1988), 103-10.

34. Ben Davis, Russell Sasnett, and Matthew Hodges, "Educational Multimedia at MIT," Advanced Imaging 4, no. 7 (July 1989): 32-35.

35. Elisa Bertino, Fausto Rabitti, and Simon Gibbs, "Query Processing in a Multimedia Document System," ACM Transactions on Office Information Systems 6, no. 1 (January 1988): 1-41.

36. Herb Lison and Terrence Crowley, "Sight and Sound," Unix Review 7, no. 10 (October 1989): 76-86.

37. A. Poggio, J. J. Garcia Luna Aceves, E. J. Craighill, D. Moran, L. Aguilar, D. Worthington, and J. Hight, "CCWS: A Computer-Based, Multimedia Information System," Computer 18, no. 10 (October 1985): 92-103.

38. E. J. Addeo, A. B. Dayao, A. D. Gelman, and V. F. Massa, "An Experimental Multi-Media Bridging System," SIGOIS Bulletin 9, nos. 2 and 3 (April and July 1988): 236-42.

39. Jonathan B. Postel, Gregory G. Finn, Alan R. Katz, and Joyce K. Reynolds, "An Experimental Multimedia Mail System," ACM Transactions on Office Information Systems 6, no. 1 (January 1988): 63-81.

40. Wendy E. Mackay and Glorianna Davenport, "Virtual Video Editing in Interactive Multimedia Applications," Communications of the ACM 32, no. 7 (July 1989): 808.

41. Stewart Brand, The Media Lab: Inventing the Future at MIT (New York: Viking Penguin, 1987), 110-17.

42. Ibid, 49.

43. Matthew E. Hodges, Russell M. Sasnett, and Mark S. Ackerman, "A Construction Set for Multimedia Applications," IEEE Software 6, no. 1 (January 1989): 38-39.

44. Wendy E. Mackay and Glorianna Davenport, "Virtual Video Editing in Interactive Multimedia Applications," Communications of the ACM 32, no. 7 (July 1989): 805-6.

45. Jaron Lanier, "Virtual Reality: An Interview with Jaron Lanier," interview by Adam Heilbrun and Barbara Stacks (additional material by Kevin Kelly), Whole Earth Review, no. 64 (Fall 1989): 108-10.

46. Steve Ditlea, "Another World: Inside Artificial Reality," PC Computing 2, no. 11 (November 1989): 98-99, 102.

47. Ibid, 92-93.

48. Harold Abelson, Michael Eisenberg, Matthew Halfant, Jacob Katzenelson, Elisha Sacks, Gerald J. Sussman, Jack Wisdom, and Kenneth Yip, "Intelligence in Scientific Computing," Communications of the ACM 32, no. 5 (May 1989): 546-62.

49. Beverly Woolf, "Intelligent Tutoring Systems: A Survey," in Exploring Artificial Intelligence: Survey Talks from the National Conferences on Artificial Intelligence, ed. by Howard E. Shrobe and the American Association for Artificial Intelligence (San Mateo, CA.: Morgan Kaufmann, 1988), 1-43.

50. Robert G. Bowerman and David E. Glover, Putting Expert Systems into Practice (New York: Van Nostrand Reinhold, 1988), 39-61.

51. Tamas E. Doszkocs and Bella Hass Weinberg, "Natural Language Interfaces for Information Retrieval," in Gateway Software and Natural Language Interfaces: Options for Online Searching, ed. James A. Benson and Bella Hass Weinberg (Ann Arbor, MI: Pierian Press, 1988), 127-128.

52. Stephen A. Weyer, "As We May Learn," in Interactive Multimedia: Visions of Multimedia for Developers, Educators, and Information Providers, ed. Sueann Ambron and Kristina Hooper (Redmond, WA: Microsoft Press, 1988), 91.

53. John Sculley, "The Relationship Between Business and Higher Education: A Perspective on the 21st Century," Communications of the ACM 32, no. 9 (September 1989): 1060.

54. Raymond DeBuse, "So That's a Book . . . Advancing Technology and the Library," Information Technology and Libraries 7, no. 1 (March 1988): 14-15.

55. A. Roger Kaye and Gerald M. Karam, "Cooperating Knowledge-Based Assistants for the Office," ACM Transactions on Office Information Systems 5, no. 4 (October 1987): 297-326.

56. Edward A. Fox, "Development of the CODER System: A Testbed for Artificial Intelligence Methods in Information Retrieval," Information Processing & Management 23, no. 4 (1987): 341-66.

57. Stewart Brand, The Media Lab: Inventing the Future at MIT (New York: Viking Penguin, 1987), 51-54.

58. D'Ellen Bardes, "Update: MIT's Electronic Publishing Group," Optical Information Systems 7, no. 2 (March-April 1987): 115-16.

59. Kum-Yew Lai, Thomas W. Malone, and Keh-Chiang Yu, "Object Lens: A 'Spreadsheet' for Cooperative Work," ACM Transactions on Office Information Systems 6, no. 4 (October 1988): 332-53.

60. Thomas A. DeFanti, Maxine D. Brown, and Bruce H. McCormick, "Visualization: Expanding Scientific and Engineering Research Opportunities," Computer 22, no. 8 (August 1989): 12-13.

61. Alan Kay and Adele Goldberg, "Personal Dynamic Media," Computer 10, no. 3 (March 1977): 31.

62. Charles W. Bailey, Jr., "The Need to Develop a Graphic User Interface Command Standard for Information Retrieval," LITA Newsletter 10, no. 3 (Summer 1989): 13-14.

63. Richard K. Belew and Maurita Peterson Holland, "BIBLIO: A Computer System Designed to Support the Near-Library User Model of Information Retrieval," Microcomputers for Information Management 5, no. 3 (September 1988): 147-67.

64. Diane Brooks, "System-System Interaction in Computerized Indexing of Visual Materials: A Selected Review," Information Technology and Libraries 7, no. 2 (June 1988): 111.

65. "Information Workstation Group and Desktop Presentations Project $17 Billion Multimedia Market in 1994," Library Hi Tech News no. 64 (October 1989): 12.

66. John Gantz, "Multimedia's Murky Market: Standards and Applications Favor and Foil the Growth of Multimedia," Computer Graphics World 12, no. 8 (August 1989): 33.

Copyright (C) 1990 by Charles W. Bailey, Jr. All Rights Reserved.