by Greg R. Notess from Online 24(1): 57- , January 2000.
Usgovsearch (http://www.usgovsearch.com/) is the result of a partnership between the Web search engine Northern Light and the commercial government organization of NTIS (National Technical Information Service). Under the Commerce Department, the NTIS is the government's central source for the sale of scientific, technical, engineering, and business information "produced by or for the U.S. Government."
What brought these somewhat disparate entities together was the idea of creating an online service combining the databases and expertise of the two organizations. NTIS brought their own bibliographic database of technical reports to the table. Northern Light had their Special Collection database, a general Web database, and some unique search technology. Unfortunately, the concept of access to government information on the Internet, for a price, did not immediately gain universal acceptance. The May rollout announcement was greeted with several protests from both the public and government sectors and the service was forced to re-evaluate its pricing structure. Meanwhile, rumors flew and the press had a field day.
The complaints centered around usgovsearch's fees. The commercial database world ran head first into the Web public's assumption that information on the Net is free. Thousands of government sites have been putting up publications, press releases, and other content for free on the Web. And the public is keenly aware that government produced information is funded by taxpayers. The fact that government information on the Web had been fragmented and difficult to find and navigate was irrelevant. Paying to search for information on the Web, especially government information, was a difficult pill to swallow.
On the other hand, the full NTIS database had not previously been available in any format for free searching. The original pricing proposed for usgovsearch offered one of the least expensive means to access that database. But what raised the ire was not the NTIS database, or even the Northern Light Special Collection database, but the database of government Web pages.
Northern Light did not help its case in the initial press release when it claimed that it "unveiled the first custom search engine that allows anyone to efficiently search the voluminous amount of information created by and for the U.S. Government." Sorry folks, but that sounds like a free search product. Even a later statement describing usgovsearch as a "one-stop shopping service" implied to a Web-molded audience that searching the service would be free when in fact searchers must pay for access.
Northern Light and NTIS responded by extending the free-trial period and eventually lowering the cost of the day pass access option. Yet usgovsearch remains a fee service. Is it worth the price? How does it differ from the freely accessible Northern Light and from other government search engines?
Usgovsearch is a combination of databases: the NTIS bibliographic database of NTIS technical reports, Northern Light's Special Collection database of thousands of full-text publications, and a special U.S. federal government subset of the Northern Light database of Web pages. In addition to the databases, usgovsearch provides specialized search services and features for using the databases.
The NTIS database is a familiar old standby dressed up in new Web clothes. The usgovsearch version is the full NTIS database, with some of the oldest records going back to 1901, but the main coverage starts in 1964. The NTIS database is updated at least weekly.
Most of the fields are available on the usgovsearch version of the NTIS database. NTIS numbers, subject headings, authors, document type, and date fields are all available on the initial display. Requesting the full record, at the cost of $1 per record, displays additional fields: the abstract, page length, media type, availability, and supplementary notes.
So what fields are missing? The price code is not directly displayed. Instead, usgovsearch offers direct links to the NTIS Order Now program for online NTIS document ordering. Choose the "Buy from NTIS" option, and the title and order number are inserted into the form for you, and the current prices are listed in a drop-down menu. The DTIC Limitation Code (related to access limitations on a specific report) is not displayed, but a generic message is displayed based on the value of the code, when the code appeared in the original NTIS record.
For the advanced searcher, the lack of any Report, Grant, Project, or Task numbers is a bigger negative. The language field only displays when it is a non-English language report. In addition, the corporate source codes, journal announcement (as in which issue of Government Reports and Announcement Index the citation originally appeared), and the descriptor, identifier, and section header and code fields are not displayed. Also, as noted in Mary Ellen Bates' review in the August 1999 issue of EContent (formerly DATABASE), the NTIS number is not searchable.
The lack of NTIS subject fields is unfortunate for the searcher familiar with the NTIS vocabulary. Subject terms are included in the record. However, they're not the same as the descriptors, identifiers, section headers, and section codes of the original record. Instead, the NTIS terms are mapped to a Northern Light thesaurus of subject terms that is used for both the NTIS database and the Web page database. These Northern Light subject headings are searchable and are used for sorting results into the Custom Search Folders.
The two other databases available via usgovsearch come from Northern Light: their Special Collection and a database of U.S. federal government Web pages. The usgovsearch Special Collection is the same as the Special Collection available on the free Northern Light service. It indexes the full text of thousands of publications and provides the full text of each article for a fee. While it does include some government related titles, it also includes the full collection of business and popular titles as well.
The Web database of government-related Web sites is a subset of the full Northern Light Web database. Currently, its main focus is on pages from U.S. federal government Web sites. While the majority have the top level domains of .gov or .mil, usgovsearch goes beyond that limitation to include .com, .net, .org, .edu, and .us domains when they are used by government sites such as the Smithsonian or the Federal Reserve Board. With more than four million pages in this database, the majority are hosted on .gov domains. Yet it still includes tens of thousands of pages from .edu and .org domains. Many other search engines restrict government searches to only .gov and .mil, so usgovsearch's effort in conjunction with NTIS to identify other government Web sites is an especially valuable component of their service.
What combination of databases can be searched? The Special Collection choice on both the usgovsearch Simple Search and Power Search also includes the NTIS database. There is no separate choice for searching just the Special Collection without NTIS, although adding a simple "not ntis" statement to a Special Collection search achieves the same goal. However, the NTIS database can be searched separately as can the Web pages database. And all three databases can be searched together. These options are available on both the Simple Search and Power Search. However, note that the Agency Web Search only searches the Web database and not the NTIS records or the Special Collection.
In a sense, all the content of the three databases is available from other sources. It is the search features and services that set usgovsearch apart. On the NTIS database, rather than just simply providing phone number and addresses for ordering full NTIS reports, usgovsearch links to the online shopping cart service of NTIS' Order Now. From a citation, Order Now automatically fills in the NTIS number, title, and price. The Order Now feature is available both at the citation level (before paying the $1 to view the abstract) and at the full record level.
The combination of the NTIS database with the Special Collection and one of the largest databases of government Web pages provides a unique approach to searching for government information. In addition, they share a combined subject thesaurus that is not available from other services.
A direct comparison between the NTIS descriptors and identifiers and the Northern Light subject headings that replace them shows less specificity on the Northern Light headings. For example, in one record, NTIS has 25 descriptors and identifiers such Autocorrelation, Interferometers, Femtosecond pulses, and Acoustooptics. These terms end up mapped to three Northern Light headings: Optics and Lasers; Optics (Physics); and Lasers (Photonics).
For anyone familiar with the detailed NTIS terms, the usgovsearch terms are far too broad. On the other hand, for searchers unfamiliar with the greater specificity of NTIS indexing, the broader Northern Light headings may well be effective. In addition, the Northern Light subject headings are also used in usgovsearch Web databases. Usgovesearch certainly will not replace a full-fledged NTIS database because of the different subject terms, but it works well for basic title, author, and keyword searches.
Of far more interest to many users is the government Web database. Northern Light provides two specialized search forms that provide especially useful search options. The Power Search can search all the databases and the Agency Search is used just for the Web database.
The Power Search is similar to the one available at Northern Light. It provides options for choosing which of the usgovsearch databases to search, field search boxes for titles, publication names, and URLs. It has a date limit and the option to sort results by date (rather than by the default relevance ranking).
Then comes the main difference: the agency and subject limits. Figure 1 shows a portion of this. The agency limits are not as extensive on the Power Search as on the Agency Search, but if one of the general agency boxes is selected, then the results will only come from the Web database.
The subject limits come from the special government-related vocabulary developed for usgovsearch. It only lists the more general upper level hierarchical terms, but ones such as Electrotechnology, Ordnance, and Government Inventions for Licensing are likely to be of interest to government searchers.
The Agency Search screen will only search the Web database. At first glance, it seems like a Power Search with less options. It does have the title word and date searches. But then it just lists the top level agencies for an agency limit (see Figure 2). These were all available on the Power Search. However, note the Expand List button.
Clicking that button results in a much more expanded list of the federal U.S. government hierarchy (see Figure 3). It includes the major executive branch departments, the more frequently requested judicial and congressional agencies, over thirty independent agencies, and more. The EPA, Social Security, Sentencing Commission, GAO, the Federal Reserve System, and many others are available for limits here. More than one can be selected. Not every government agency is listed, but Northern Light tried to include the most commonly requested ones.
On the expanded list, selecting the broad categories at the top will search all the agencies listed below it as well as any that fall in that section of the hierarchy, but are not listed separately. The one exception to this is the All Quasi-Official Agencies which only searches the agencies specified.
How does Northern Light classify all these Web pages to the proper agency? NTIS reviews and classifies the Web sites, so the proper hierarchy should be represented appropriately. But as with all Internet services, expect some inaccuracies.
According to Northern Light, the Agency Search form is quite popular among their users, especially government users. A particular favorite feature is the ability to search several selected agencies at the same time.
While no language limit is available on the usgovsearch Power Search (unlike regular Northern Light), just add a phrase such as "language french" to an NTIS search to limit to a non-English language. However, since there is no language field information displayed for English language documents, there is no easy way to limit a search to English only.
Another difference between usgovsearch and Northern Light is that in the Custom Search Folders, even when there are Special Collection documents in the hit list, the Special Collection folder is not displayed on usgovsearch. The emphasis on the NTIS collection and the government Web sites is probably behind this.
The foremost difference between usgovsearch and regular Northern Light is the cost. Search access is configured differently, but all the records on usgovsearch are also available in regular Northern Light. That's right, the NTIS database, the Special Collection, and all of the government Web pages are all there! As of July, the NTIS records were incorporated into the regular Special Collection. Of course, the NTIS abstracts still cost $1 per record and all of the rest of the Special Collection documents have a per document charge for the full text.
Why even charge for usgovsearch? The special search forms and the focused Web database of government Web pages make it a useful service for searchers who frequently focus on the government sector. Eliminating the non-government-related Web pages from the database, providing more detailed agency search capabilities, and using government-related subject terms for searching and the custom folders provide the added value to this government-targeted service.
As for cost, the access fees for usgovsearch are:
$5.00/day, document purchases extra
$30.00/month, document purchases extra, or
$250/year, document purchases extra
The per document pricing is the same for each access option, with NTIS abstracts at $1 a piece and other Special Collection items typically from $1 to $4 each.
Public libraries, public K-12 school libraries, and U.S. federal depositories can get free access for one computer; however, per document fees still apply.
All of the government sites and pages in usgovsearch's Web database are in the larger Web database on Northern Light. However, what is not available are the specialized search forms, the agency search, the government-related subjects, and the division between the government sites and all the other Web sites.
Usgovsearch came about as a targeted service for the professional government information researcher, especially from within government agencies and businesses heavily involved with the government. The controversy in the press helped attract even more attention to the service, and Northern Light reported a higher level of interest from consumers and academic users than they expected.
So what are the main advantages offered by usgovsearch? First and foremost is the inexpensive access to the NTIS database. While lacking the full indexing of other versions, the cheaper access rate at usgovsearch and free access at Northern Light to the NTIS numbers and titles from the complete database are a welcome step toward seeing more open access to the NTIS records.
Usgovsearch is well worth a look from anyone who frequently searches for government information. Looking just at the government Web pages database, its scope, size, and search limits set it apart from any other current search service. Compared to other government search services like GovBot, gov-search.com, Google's Government Search, the Pathway Indexer, and even AltaVista domain: gov searches, usgovsearch has a larger database and is one of the few that looks for government sites beyond the .gov and .mil domains.
For the casual users of government information or NTIS records, Northern Light's free access service will generally suffice. Fortunately, the work on usgovsearch has enabled Northern Light to offer limited free access to the entire NTIS database. It can only benefit all of us to see more similar efforts that provide value-added services for a fee, but that also bring about improvements and increased access for the free services.
Greg R. Notess (greg@notess.com; http://www.notess.com) is a reference librarian at Montana State University and author of Government Information on the Internet.
On August 12, Department of Commerce Secretary William Daley announced that he would work with Congress to close down NTIS. As with any such plan, there is no telling the eventual outcome, and when, if ever, NTIS will actually cease to exist. However, Northern Light announced that it plans to continue running its usgovsearch service regardless of the fate of NTIS.
What might this mean for usgovsearch? The NTIS bibliographic database and/or the availability of the NTIS archive of technical reports might cease to be maintained. That' would affect more than just usgovsearch and could create huge problems for anyone trying to track down old NTIS documents.
Since NTIS, helped with the subject terms and the identification of government sites, the loss of the agency could have some impact, but according to Northern Light, that help came mostly in the development stage. It certainly seems that the database of government Web sites and the Special Collection should be able continue as they exist now. So, other than the worst case scenario of a dead NTIS database and no availability of NTIS documents, usgovsearch should be able to continue quite well on its own, even if NTIS folds.
Usgovsearch
Northern Light Technology LLC
Suite 1320;
222 Third Street
Cambridge, MA 02142
617/577-4900
Fax: 617/621-3549
cs@nlsearch.com
Price
$5.00/day, document purchases extra
$30.00/month, document purchases extra
$250/year, document purchases extra
Pros
Government Web pages databases is a unique resource.
Looks for government sites beyond the .gov and .mil domains.
Detailed agency search capabilities, government-related subject terms, and the custom folders provide added value.
Cons
The NTIS database lacks Report, Grant, Project, or Task numbers and doesn't have NTIS subject headings.
No easy way to limit a search to English only.
-------------------------------------------------------------------------------
COPYRIGHT 2000 Online, Inc.
-------------------------------------------------------------------------------