The Internet Gold Rush of the late twentith century created
an environment of experiemental business models, some of which
were pretty wacky. The flood of venture capital often allowed
the underlying weaknesses of those business models to be hidden
from the management of those dubious enterprises. As CIO of
O'Reilly C.J. Rayhill says, when the swamp is full, the tree
stumps are covered. In today's VC-poor atmosphere, the water
is low and the Internet swamp is littered with stumps.
Yahoo Seriously Troubled
Of all the Internet businesses to emerge from those salad VC days,
perhaps the most useful one for the entire Internet community
is the search engine. Where would the average Internet surfer
be without the humble Yahoo,
Google or Lycos?
Without search engines, the Internet revolution would have
been merely a footnote in Computer Science courses.
Almost all of the large search engines allowed anyone to add his
own link to their database. Of course, each search engine had
software robots called spiders to extract the hyperlinks from a
given web page and follow them. Thus, for each human submitted page
an exponential number of new pages might be discovered.
Not only could one submit links for free, but users could also
search the link database for free. Search engines usually depended
on banner ads for revenue. Although this seemed to work for a time,
banner ads are clearly not going to be the source for unbriddled
revenue growth. How can the large search engines survive?
What Businesses Need
When I worked for a small Needham, Massachusetts company called
CareerSearch, I spent most of my time mapping databases
from book publishers of industry contact information into CareerSearch's
own database system. CareerSearch is a subscription-based web application
that targets the very small market of out-placement firms and university
career centers. CareerSearch's charter is to service the needs of other
businesses. They do not collect their own data, but they do present a robust
interface into an amalgamated database of over a million companies. Most
importantly, they are a profitable company and they have never used banner
ads. Instead, customers need to buy rather expensive subscriptions to access
the CareerSearch application. CareerSearch exists because its data suppliers,
those book publishers of contact information, couldn't or wouldn't make
their data available in a format that other businesses wanted.
What does this have to do with Internet search engines?
The problem with the business model of search engines is that it
depends on casual viewers for revenue. This is a huge mistake. Search engines
need the users to supply URLs for their database. In fact, search engines
should consider paying users to add verifiably correct links to
their systems, because here's the secret that these companies don't seem
to understand: the value of a search engine is its link database.
Search engines seemed to have conceived of only one use and only one kind of
customer for their data. The question search engine companies should be
asking is, how can we sell our data to businesses? There is a wealth of
information locked away in the link databases of Yahoo and Google.
Remember, each URL in their system is linked to a series of keywords
(at a minimium). There are so many uses for this data that companies have
to build web spiders of their own just to get at the information. Clearly,
there is a need for a richer interface into these systems.
Making Link Databases Available
Once a search engine decides to service business clients, the next question
what sort of interface is most apropriate? For example, should Google
build a subscription-only web site with a radically different layout that
allows for more robust searching options? O'Reilly Senior Software Engineer
Tim Allwine and I think this is the wrong way to go. Businesses know what they
want and they are likely to have programmers who can build applications
that manipulate the link data in a relevant way for them. Search engines
need to build subscription-based Web Service interfaces that provide rich
querying capabilities and easy data retrieval. Remember, Web Services (like
SOAP and XML-RPC) are designed for programs, not for people. Each business
client will need to build an application that makes the search engine Web
Service fit its particular business need.
For example, XYZ Corp wants to measure how effective its press releases are.
One way to do this is to see how often the content from that document
gets quoted in online media. XYZ Corp would love an interface that allows
them to type in a unique sentence from the press release and get the number
of sites that use that phrase. XYZ Corp can expand on this simple idea by
creating an internal tool that automatically looked for press releases and
charted the citations over time. If search engines provided Web Services
interfaces, this type of application would be very easy to create. By
supporting IT infrastructure, search engines would have a business model
that makes sense.
Web Services can be very easy to build. The kind that search engines would build are particularly easy because they would be read-only (i.e. clients aren't modifying the link database) and non-transactional. By helping businesses help themselves with Web Services interfaces, large search engines get a license to print money, which is exactly what places Yahoo, Lycos and Google are looking for most right now.
