Content area
The Librarians Association of the University of California created New Horizons in Scholarly Communication as a Web site directory of online resources targeting academic information technology. This article covers the initial site development and the move to dynamic site management using a database. The article discusses the successful application of the Lasso Web site management software for multiple library information service purposes, and a later move to the ZOPE and SQL platforms.
Keywords
Databases, Internet, Information services, Computer programming, Information resources management
Abstract
The Librarians Association of the University of California (LAUC) created New Horizons in Scholarly Communication as a Web site directory of online resources targeting academic information technology. The article covers the initial site development, and the move to dynamic site management using a database. Discusses the successful application of the Web-compatible Filemaker Pro database and the Lasso Web site management software for multiple library information service purposes, and a later move to the ZOPE and SQL platforms.
In 1995, the Librarians Association of the University of California (LAUC) launched New Horizons in Scholarly Communication[1]. This service, a directory of online resources, highlights emerging information technologies, with a focus on issues affecting their adoption in the academic world. The editorial board track projects, discussion papers, reports, standards, organizations, conferences, workshops, and periodicals relevant to these issues.
From its modest beginnings of a few dozen items listed in a single page, New Horizons grew rapidly with the availability of relevant material on the Web. Within only a few months of its launch, the site was reorganized into a multi-- paged listing broken into subject categories. Even this turned out to be a short-lived solution as the number of items grew beyond a couple of hundred. The longer and longer lists on each page made items increasingly inaccessible to users. In addition, the site managers were having more trouble knowing whether and where anything was listed. By late 1995 another reorganization was imminent.
Fortunately, the New Horizons board included some of the leaders in Web development in the library world: they argued persuasively for moving the site to a "dynamic" platform, one that would generate pages "on the fly" from content residing in a database. (The term "dynamic" denotes pages assembled from real-time data at the point they are requested, as opposed to "static" pages, which are the same every time they are viewed.)
At that time, tools for developing even simple Web sites were few and far between. Leadingedge site development, such as dynamic pages, usually required homemade applications often written in the Perl scripting language or more advanced programming languages such as C++. Though New Horizons did not have the resources for a "from scratch" solution, we were again fortunate to have the services of a talented systems administrator who put together a package including Filemaker Pro (R)[2], MacHTTP(R) [3], and a series of Common Gateway Interface (CGI) utilities culminating with the use of Lasso (R)[4].
CGIs are utilities for translating data to and from the Web. Lasso similarly passes search requests from Web pages to Filemaker Pro and formats the results as Web pages. Therefore, Lasso has two basic functions; search and display.
For searching, Lasso provides a query language capable of performing anything from simple retrievals to complex queries involving combinations of fields and operators. Lasso searches can be performed as part of a standard Web form or even be embedded in a link, as part of a URL. Using a form allows users to enter or select their own search criteria. Searches embedded in URLs allow for predetermined or "canned" searches, which appear to users as a simple link to a new page. The New Horizons site takes advantages from both approaches.
Lasso also provides a formatting function to display results of searches. This is managed by a template or "format file," an HTML document using a combination of special tags and standard markup to indicate where and how the data from retrieved records is displayed. Often, an initial format displays all the retrieved items in a brief listing of records, which then links to a second display containing the full record for selected items.
One useful aspect of format files is the ability to create different displays and, in effect, different services from the same source database. In other words, a second (or third ...) set of Lasso screens could be created to retrieve data from the same Filemaker Pro file and present it in a completely different way. Thus a team of developers could collaboratively enter and maintain a single database, yet each could create as many different Web pages from subsets of that central source as needed.
Lasso commands can be delivered as part of a URL (as in the example below) or via a Web form. Either way, a basic Lasso command includes the name of the Filemaker Pro database ([database]=) to be searched, the criteria to be met (subj=), and instructions for displaying the results ([format]=).
../lasso.acgi? [database] =scomm&subj= diglib&
[format] =list.html&[search]
Here, the Lasso utility (lasso.acgi) is told to look in the Filemaker Pro database scomm, retrieve all records with the string diglib in the subj field, and send the data from the matching items to a format file called list.html.
Making Lasso work
It was surprisingly quick work to assemble the necessary pieces of software and data with a used Macintosh as our test server. Using a word processing program's search-and-replace function, we reformatted the existing subject lists from the New Horizons site and imported the data into a Filemaker Pro database. The original New Horizons was made up of a simple series of subject lists providing the title, URL and a brief abstract for each site included:
New Horizons in Scholarly Communication
http.//Iibweb.ucsc.ed/scomm/index.html
collection of online resources concerning emerging technologies in colleges and universities.
The Filemaker Pro database was designed to hold these records, provided fields for title, URL, abstract, and added a subject category.
title: New Horizons in Scholarly Communication
url: http://libweb.ucsc.edu/scomm/ index.html
abstract: collection of online resources...
subject: diglib
Next it was necessary to create the Lasso screens that would retrieve and display the appropriate information from the Filemaker Pro database. The first iteration of the upgraded New Horizons simply featured a series of links with embedded Lasso searches. For instance, the heading DIGITAL LIBRARIES is a link containing a set of commands requesting Lasso to perform a search of the specified database retrieving those records with the term diglib in the subject: field:
<a href=../lasso.acgi? [database] = scomm&subj = diglib&
[format] =list.html&[search] >Digital Libraries</a>.
The data from the retrieved records is sent to a list format file (list.html) for display. This file, an otherwise normal HTML document, contains special Lasso formatting instructions which can be combined with standard markup features. For each record retrieved by the above search string, the Lasso tags below will display the contents of the title, url, and abstract fields in a list.
[RECORD] [field:title] [field: URL] [field:abstract] [/RECORD] You have the option of using HTML formatting capabilities to enhance the display of this information. For instance, the following example puts the results in a descriptive list format <DL>, displays the title in the descriptive title position <DT> and in a bold font <STRONG>, indents the URL and abstract fields <DD>, makes a link from the URL field <A HREF= >, and italicizes the abstract text.
<DL>
[RECORD]
<DT> <STRONG> [field:title] </STRONG>
< DD ><A HREF = [field: URL] >[field: URL] </A> <DD> <em> [field:abstract]
</em>
[/RECORD]
</DL>
In other words, you can use almost any combination of Lasso, FilemakerPro and HTML functions to achieve an amazing range of display results.
It took only two or three days to install all the components, upload the original lists into a Filemaker Pro database, and create the necessary coded pages, replicating all the features and even the look and feel of the earlier New Horizons pages. One of the strengths of this technology is how quickly and easily it can mimic existing services. After testing the new site for several more days, New Horizons in Scholarly Communication moved to its new home on the Macintosh server supporting the Lasso/Filemaker platform.
New horizons for New Horizons
One of the key advantages of the new configuration was the marriage of a database's inherent strengths to Web site management. The ability to search and sort records by any number or combination of criteria and make global changes was a very powerful tool when overseeing large amounts of information. We can search for duplicate records or retrieve items added before a specified date in order to be able to check accuracy and currency of older items. For example, when the International Federation of Library Associations (IFLA) moved its Web pages from the National Library of Canada site to its own, new domain, finding and updating the myriad IFLA entries within New Horizons was not only possible, but simple, because of the underlying database management tools. The predictions of the visionaries who recommended this change bore fruit almost immediately.
The new system also opened the way for development in several directions. First, we began to add features to the New Horizons site based upon the capabilities of the new platform. One relatively modest, but telling, change was the ability to place items in more than one category. Under the old scheme, items were manually placed on one list or another but rarely in multiple lists. However, reality is such that many sites fit more than one category. University of California's "Guidelines on the Reproduction of Copyrighted Materials for Teaching and Research" could be filed under "Teaching," "Research," or "Copyright." With the new scheme, it became possible to code it to appear under all three headings.
The FilemakerPro/Lasso platform made it relatively easy to add, refine and reorganize categories because it holds the site's contends in a database. When the number of items in the TEACHING category grew too large to be useful to browse in a single listing, one member of the editorial committee reviewed the entries and broke them into five subheadings. ACCESS was similarly broken into two new main headings with numerous subheadings by recoding items in the database to reflect a finer division.
It was also relatively easy to add entirely new categories as the need arose. Another ten categories, covering everything from new subject areas to formats such as periodicals and conferences, have been added over the life of New Horizons. In fact, in some subject categories, it was possible to cross subjects and formats to find, for instance, meetings addressing teaching with new technologies or periodicals concerning building digital collections.
The Lasso platform has also made it possible to add entirely new functions to the site. One particularly useful enhancement is a search function. Instead of accessing only the fixed retrieval lists provided by embedded subject links, addition of a Web form allows users to search for specific items or topics outside of the general arrangement of the site. Search terms can be entered for the Title:, URL: and/or Keyword fields, in any combination, and results are returned in a list sorted by Title: or the date added to the site, according to the users' preference.
Another added feature allowed users to recommend new entries for New Horizons. This was done with another Web form able to add entries directly into the database, holding them in a temporary queue until they can be reviewed by the site managers. This allows approval and quality control before entries are actually made public.
In a similar fashion, another set of pages allowed board members to review and approve items in the queue, as well as screen older entries, search, edit, add, and delete entries. This allowed for more collaborative site management, particularly among members of a geographically dispersed team. Access to these internal functions was password protected. This feature was initially developed for some of the other users, described below, to allow for realtime changes to the database without direct access to Filemaker Pro. It has turned out to be a very valuable and widely applicable feature.
As of this writing, New Horizons in Scholarly Communication includes 13,000 entries and has received more than 26,000 visits since December 1995. There is no question that the FilemakerPro/Lasso configuration is partly responsible for its success. It has provided a capable platform for managing and maintaining a site of such size. Improved site management, more flexibility in organization, added user functions, and collaborative maintenance are clear benefits of the new configuration.
Beyond New Horizons
Having such a capable resource in hand, it was natural to want to share it or, in some cases, have users beat a path to our door wanting to take advantage of its features. In the intervening years, more than a dozen standing services or shorter term projects have resided on our FilemakerPro/Lasso server:
*Film & Video Catalog[5]. This service, a searchable listing of UCSC's Media Center holdings, began life as an in-house inventory and accounting system using Filemaker Pro. It became a public access service, initially running on a server featuring Tango(R), a product competing with Lasso. However, the Tango server was shared with other services which caused occasional conflicts and outages and the Media Center sought a new home for their catalog. They designed and created the pages and scripts required for migrating to Lasso in very short order, demonstrating the flexibility and applicability of this platform for a wider range of services.
*Lick Observatory Archives Master Card File[6]. Mary Lea Shane Archives of the Lick Observatory holds many important documents regarding the history of astronomy. Archivist Dorothy Schaumberg is in the process of converting the card file catalog of the archives' extensive collection of historical records to a Filemaker Pro database as part of a grant-funded project. Availability of these records in a database form made for an easy conceptual leap to Web access via Lasso. A few changes to file structure (for instance, to allow for searching across multiple subject fields) were required, but otherwise the design and creation of the main features required only half a day. This is truly a world class resource and the ability to make it available worldwide so easily is one of the strongest testaments to the power of the Lasso configuration.
*Multicultural Children's Literature Curriculum Collection. [http://libweb.ucsc. edu/mcc/] [7].
*Senior Theses: Community Studies and Environmental Studies [8].
*City on the Hill Press Index[9]. These three services were originally developed for and resided on the UCSC Library's Gopher server as WAIS databases. As we focused more on Web-based development, there was less support for maintaining the Gopher system as it began to fade away. It turned out to be easier to import the existing files into Filemaker Pro and build the Lasso interfaces than to fix the Gopher system. Added features, such as improved search capabilities and direct editing of the databases, offered by the new system won it ready acceptance.
*Hihn-Younger Archive[10]. The papers of the Hihn and Younger families held by the UCSC Library provide useful insight into the history of Santa Cruz County. Emeritus Librarian Stanley Stevens is in the process of preparing volumes of their correspondence for publication. He has been entering his work into Filemaker Pro as a management tool before exporting the text for final formatting. It was relatively easy to reformat the available database and create the Lasso screens to make this accessible via the Web. As other sections have been indexed, we have redesigned the site to accommodate new components. There are now two separate databases underlying this site, which also features PDF versions of the printed indexes. A new segment is expected soon.
*Subject Guide to the Web[11]. This service (closed since March 2000) began as an early attempt to highlight Web resources useful for research and teaching at the university level. Following the same path as the New Horizons site, it grew too large to be managed as a set of HTML files and eventually migrated to the Filemaker Pro / Lasso platform.
The wide subject approach and ability to manage the contents collaboratively suggested that this should be a shared project across the Library. In fact, format files allow multiple displays to be designed for a single database. It is possible to create multiple, unique views from the same data. In this way, different site managers can share the one central database and yet develop their own pages. This is a concept now gaining wider popularity but at the time the project foundered under lack of interest.
*Santa Cruz Guide[12].
*Pottery Resources on the Web[13]. Though the main Subject Guide has closed, other services - spin-offs - were developed using the same database. As mentioned earlier, one of the strengths of the Lasso/Filemaker Pro platform was the ability to create different views with subsets from the same data. Two of these have continued after the main service closed.
In addition to the above ongoing services, the Lasso/Filemaker platform has supported a series of shorter term projects. Some useful examples include:
*Conference registration system - Registrants completed a Web-based form, entering their information into a Filemaker Pro database, and getting a printable form to include with their payment. Organizers could monitor registrations and the resulting database was used to order meals, arrange transportation, and print name tags.
*User survey - New equipment allocations required that we collect user data to measure success of the funding. A Library homepage link invited users to contribute to the survey. A brief, Lasso-based form collected demographic information into a Filemaker Pro database, which was then used to generate statistics for our report.
*Funding requests - The UCSC Library's annual in-house "one-time" funding process has migrated over the past several years from paper forms, to Filemaker Pro databases distributed and collected on diskettes, to Web forms using Lasso to enter requests directly into a central file. The database can be used to review and sort requests prior to funding decisions and then supports tracking purchases afterwards.
*Work-flow tracking - A Library unit needed to keep detailed records for materials they were processing. We built them a Filemaker Pro database with a Lasso front-end which allowed each member of the team to annotate when they completed each stage of the processing for each piece of material.
To some extent, Lasso has become the hammer leading us to see every problem as a nail. The range of uses which have been found for this system, to say the least, is broad. Once you can see a need in terms of information which can be organized as a database, the Lasso utility makes it possible to throw together a simple implementation in an hour, or a half-day for a more complex system. It may not be the only tool in our kit, but it is a tool that is hard to ignore.
A bigger hammer
As the Lasso/Filemaker Pro system has become more central and widespread in UCSC Library operations, we have made improvements to ensure its reliability and performance. Therefore, one of the other directions for development has been a series of moves to more stable and robust platforms.
The original "experimental" configuration was quickly replaced. The castoff Macintosh was upgraded to a new PowerMac 6400 tower and then by a PowerMac Server. Memory and a larger hard drive have also been added. Where the hardware once supported other services, it is now dedicated solely to Lasso/Filemaker functions. The MacHTTP Web service utility was replaced with WebStar[14]. A sequence of three free CGI utilities was eventually superseded by the full, paid version of Lasso. We are now in the process of upgrading to a newer version of Lasso.
However, despite the successful history and remaining potential in the Lasso/Filemaker platform, we have begun to move away from this scheme for future development. While there were very few systems available when we adopted Lasso in 1995, there are a quite a few choices available now. Some of these will actually be easier to integrate into and support within our core systems.
One promising technology we have begun using is ZOPE[15] - Z Object Programming Environment. This is a Web site publishing and management platform which runs on the Library's main Linux server. In addition to a range of site management tools, ZOPE offers the same capabilities we found using Lasso and Filemaker Pro, using more robust SQL (pronounced sequel) databases. Running on a larger, more well-supported server integrated with other Web tools, allows ZOPE-based services to take advantage of security, backup and other features already in place.
During 2000, two existing Library services - a database of Electronic Resources[16] and another listing Full-text Online Journals[17] -- were ported from Lasso/Filemaker Pro to ZOPE. The move provided more reliability, improved performance, and extended capabilities than available previously. More recently, a new catalog for the Women's Studies Library[18] was developed firsthand using ZOPE. We plan to move other UCSC Library services now based on Lasso to ZOPE during the coming year and most new development of any consequence will likely be undertaken using ZOPE.
However, Lasso remains a viable working platform from several perspectives. For some existing services, "if it ain't broke, don't fix it" applies. These are active and even growing resources with no pressing need for change. Lasso is also ideal for smaller and shorter term projects, where quick development time (often only two to four hours) makes it the more cost-- effective tool. Thus, Lasso will remain an important part of the UCSC Library's Web repertoire for some time to come.
Lasso generally remains a viable development platform. Blueworld continues to introduce new versions and enhancements, allowing it to work in a wider choice of environments with a larger range of resources. While it is ideal for smaller projects or those with limited technical resources, Lasso has a good track record of use for very large enterprise Web sites. If nothing else, our success in being able to create and support such a range of services is a ready recommendation of their software technology.
Ultimately, however, the point here is not about any one specific product. We might have been just as successful, if not more so, had we chosen a competing package. The distinctions between one tool and another were mostly academic. The important decision was the realization that we could not continue managing our Web pages the same way we had started: we had to find a better way. The real breakthrough was the decision to make a change.
Notes
1 New Horizions in Scholarly Communication <http:ll libweb.ucsc.edu/scomm/>
2 Filemaker Pro is a well-established database program available for both Macintosh and Window systems. FilemakerPro has gone through several incarnations since 1996, and now includes its own Web server and data sharing capabilities <http://www.filemaker. com/>
3 MacHTTP is a freeware Web server application for the Macintosh. MacHTTP evolved into the popular WebStar server application and there are now other numerous Web server choices available.
4 Lasso is a CGI utility (also available as a Web server plug-in module) from BlueWorld which allows sharing data in Filemaker Pro files over the Web. Before adopting Lasso, we experimented with ROFM, WebFM and Lasso Lite, free packages with similar capabilities <http://www.blueworld.com/>.
5 UCSC Film & Video Catalog <http:/Iibweb.ucsc.edu/ film/>.
6 UCSC Lick Archives Master Card File <http:/libweb. ucsc.edu/lick/Search.html>.
7 Multicultural Children's Literature Curriculum Collection <http:/Iibweb.ucsc.edu/mcc/>.
8 Senior Theses: Community Studies & Environmental Studies <http:Iibweb.ucsc.edu/thesis/Search.html>.
9 City on the Hill Press Index <http://libweb.ucsc.edu/ chp/>.
10 Hihn-Younger Archive <http://libweb.ucsc.edu/hihn/>.
11 Subject Guide to the Web <http://1ibweb.ucsc.edu/ help/subject.html>.
12 Santa Cruz Guide <http://libweb.ucsc.edu/cruz/>.
13 Pottery Resources on the Web <http:libweb.ucsc. edu/pottery/>.
14 WebStar <http://www.webstar.com/>.
15 ZOPE - Z Object Programming Environment <http:ll www.zope.org/>.
16 UCSC Electronic Resources <http:/library.ucsc.edu/ eresources/index.html>
17 UCSC Full-text Online Journals <http:library.ucsc. edu/Zope/science/ejournals/>.
18 UCSC Women's Studies Library <http:ll library.ucsc.edu/Zope/ws/>.
Further reading
Dwight Walker's Web Construction, Tools, Hints <http:// www.zip.com.au/-dwight/webconst.htm>.
El.pub: Electronic publishing product information <http:// inf2.pira.co.uk/base09.htm>.
The authors
Lee Jaffe is with Microcomputer & Network Services, McHenry Library, University of California, Santa Cruz, California, USA.
Copyright MCB UP Limited (MCB) 2000