Search Engine Poem

The Meta Search Engines: A Web Searcher's Best Friends

I know what you're thinking: Google gives you such accurate results that you don't need any other search tool. Well, let's see about that.

You might - or might not - know that no major search engine indexes ALL the existing Web pages. OpenFind states that it indexes 3.5 billion Web pages, Google claims 2.4 billion, AlltheWeb - 2.1 billion, Inktomi - a little more than 2 billion, WiseNut - 1.5 billion and AltaVista - 1 billion Web pages.

The truth is, nobody knows how wide the Web is. Some say 5 billion pages, some 8 billion, some even more. Anyway, what's definite is that the major search engines (SEs) index only a fraction of the "publicly indexable Web". Moreover, every SE indexes different Web pages, which means if you use only one SE you will miss relevant results that can be found in other search engines.

One way to more effectively search the Web is to use a meta search engine.

What Is A Meta Search Engine?

A meta search engine (also know as multi-threaded engine) is a search tool that sends your query simultaneously to several search engines (SEs), Web directories (WDs) and sometimes to the so-called Invisible (Deep) Web, a collection of online information not indexed by traditional search engines.

After collecting the results, the meta search engine (MSE) will remove the duplicate links and, according to its algorithm, combine/rank the results into a single merged list.

An important note:

Unlike the individual search engines and directories, the meta search engines

1. Do not have their own databases and
2. o not accept URL submissions.

Pros and Cons of Meta Search Engines

Pros: MSEs save searchers a considerable amount of time by sparing them the trouble of running a query in each search engine. The results - most of the time - are extremely relevant. MSEs can be used by Webmasters to find their site's presence, rankings and link popularity in the major SEs.

Cons: Because some SEs or WDs do not support advanced searching techniques such as quotation marks to enclose phrases or Boolean operators, no (or irrelevant) results from those SEs will appear in the MSEs results list when those techniques are used.

MSEs Come In Four Flavors:

1. "Real" MSEs which aggregate/rank the results in one page
2. "Pseudo" MSEs type I which exclusively group the results by search engine
3. "Pseudo" MSEs type II which open a separate browser window for each search engine used and
4. Search Utilities, software search tools.

The following provides detailed information on each of the four MSE types, along with my ranking:

1. "Real" MSEs

These real MSEs simultaneously search the major search engines, aggregate the results, eliminate the duplicates and return the most relevant matches, according to the engine's algorithm.

Following is a list of a few meta search engines that you might find useful. It's by no means complete, but it might help you find what you need.

(The criteria I used to determine the best MSEs were:
the amount and the relevance of the results,
the capability to handle advanced searches,
the ability to enable users to customize searches,
the speed of their searches and others.)

ez2www [ http://ez2www.com ]
Searches the best SEs - AlltheWeb, Google, AltaVista, Teoma, Wisenut - and directories - Yahoo! and Open Directory. Through its "Advanced Search" function it also searches a small part of the Invisible (Deep) Web. It also searches news, newsgroups, MP3, images and many, many more. Provides excellent results in a very neat interface. Created in September 2000 by the French search engine developer Holomedia. THE best!

Vivisimo [ http://vivisimo.com ]
Uses the clustering technology, meaning matches are organized in folders. Don't like the frames? Just modify the size of both the upper and the left frames. This MSE was created by researchers at Carnegie-Mellon University. Advanced searching options available: exact phrase, Boolean operators, fields searching (domain, host, title, URL, etc.) and more. A jewel for the serious searcher.

Query Server [ http://www.queryserver.com/web.htm ]
Searches an impressive list of 11 SEs - everything important except Google. But don't worry: Query Server searches Yahoo!, Netscape and AOL, all partially powered by Google. This is another example of the clustering technology. Highly customizable metasearch tool. You can modify the appearance of the results page, selecting the search engines, the amount of results, their timeout, etc. It supports quotation marks to enclose phrases, the Boolean syntax and parentheses. Very professional.

Infonetware [ http://www.infonetware.com ]
Searches the Web and provides relevant results, organized in topics, in a very clean interface. This MSE is based in Edinburgh, Scotland and is a very good tool.

Metaseek [http://www.pcdigest.net/metasearch/en/main.shtml]
Excellent MSE from Ukraine. Searches major international and local search engines. Besides the Web you can search images, MP3, FTP files, news and more. You can use "Phrase" (""), "natural language" processing, Boolean logic and field searching (by URL, title, site/domain or link). Very nice.

IBoogie [ http://iboogie.com ]
Uses a minimalist design. This MSE "performs intelligent clustering of results". It searches the Web, the Invisible (Deep) Web, images, video and audio files.

Vinden.NL [ http://www.vinden.nl ]
Searches "the best," providing very good results in a clean interface. This MSE comes from the Netherlands.

Search Online [ http://www.searchonline.info ]
Uses an excellent selection of search engines and directories. This MSE provides relevant results in a relatively crowded interface. For each result you can see the search engine where the hit was found, and its ranking.

Meta Bear [ www.metabear.com ]
Provides relevant results from both international and Russian sites. Be sure you type the query in the "Search The World" box.

Web Scout [ http://www.webscout.com ]
Searches the Web, news, newsgroups, auctions, MP3 files and jobs. This Australian MSE utilizes the major SEs - except Google - and provides relevant matches in a clean results list.

argosa:de [ http://www.argosa.de ]
Searches 17 international and local SEs. I suggest you avoid checking the boxes of Acoon, Abacho, GoClick and ah_ha.com search engines, because they give many irrelevant hits. This is Germany's first MSE, and provides excellent results organized by relevance, source (quelle) or title.

Experts Avenue [ http://www.expertsavenue.com ]
Searches different search engines simultaneously for Web pages, auctions, jobs and forums and provides very relevant results in a neat interface. Enables online language translation of Web pages. Click on "Translate" and you will be brought to AltaVista's Babelfish translation service, powered by SYSTRAN.

InfoGrid [ http://www.infogrid.com ]
Provides excellent results in an easy to read layout, despite a very confusing Home Page crowded and with frames. This MSE searches the "big ones" including Google, AlltheWeb, Yahoo! and Open Directory. It also searches newswires, auctions, discussion forums, MP3, FTP files and more. To avoid the frames in the results list select the option "Open in the Current Window".

Suchspider.de [http://www.suchspider.de/meta-suchmaschinen]
Searches a whopping 100 (!) international SEs and WDs. Google, AlltheWeb, Open Directory, you name it. You can sort the results by relevance, source or - much better - grouped by domain name. This "Meta-Suchmachine" is based in Germany.

EmailPinoy [ http://www.emailpinoy.com ]
Sends your query to 15 search engines. Don't use Kanoodle, ah_ha.com and GoClick pay-per-click search engines, because you'll get irrelevant results. For better results enclose phrases in quotation marks. This MSE is from the Philippines.

1 SECOND [ http://1second.com ]
Searches a good selection of 14 major SEs and WDs, throws out the duplicates and summarizes the results in a neat listings page. Use the Advanced Search if you want to customize the search, especially the timeout of the search engines.

My Prowler [ http://myprowler.com ]
Searches over a dozen search engines, news, images, audio/MP3, music videos, auctions and various other sites. Compiles the results, weeds out irrelevant matches and provides a summarized report. It accepts "natural language" query.

Gimenei [ http://www.gimenei.com ]
Use of the "Advanced Search" option is strongly recommended. You can customize the results page, including my favorite option, "All Results" in one page. This is a comprehensive and fast MSE based in Manhattan, New York.

Dug Dugi [ http://www.dugdugi.com ]
Queries the major search engines, collates the results, eliminates information and aggregates the results in an ultra-clean layout. For each match you'll get the search engine and the ranking of the page.

Search 66 [ http://search66.com ]
Groups together pages from the same domain. Beautiful. To avoid SEs timeouts, select the "Speed": "Comprehensive". Obviously, you'll get more results from this excellent Australian MSE.

Besides the very good MSEs listed above, there are also some others that are worth a try:

Internav [ http://internav.com ],

NetXplorer [ http://www.netxplorer.de ] (Germany),

Aaise [ http://www.aaise.com/meta ],

Metengine [ http://www.metengine.com ] (Antigua),

One2Seek [ http://www.one2seek.com ],

Ithaki [ http://www.ithaki.net ],

Fossick [ http://www.fossick.com/Search.htm ] (Australia),

Pandia [ http://www.pandia.com/powersearch/index.html ] (Norway),

meta EUREKA [ http://www.metaeureka.com ] (Netherlands),

Widow [ http://www.widow.com ],

VROOSH! [ http://www.vroosh.com ] (Canada),

Meta 360 [ http://meta360.com ],

7 Meta Search [ http://7metasearch.com ],

Metor [ http://www.metor.com ] (Germany) and

Ixquick [ http://www.ixquick.com ].

The following is a list of some unimpressive meta search engines. These MSEs do not provide the breadth of coverage offered by the sites recommended above. Each of these has its own flawed characteristics, but generally they are old and have not kept up with the latest capabilities or they suffer from too many functional problems.

Metacrawler [ http://www.metacrawler.com ],

Dogpile [ http://www.dogpile.com ],

C4 (formerly Cyber 411) [ http://www.c4.com ],

Mamma [ http://www.mamma.com ] (Canada),

Pro Fusion [ http://www.profusion.com ],

moonmist [ http://www.moonmist.info ] (UK),

Bytedog [ http://www.bytedog.com ] (Canada),

il motore [ http://www.ilmotore.com ] (Italy),

METASEEK.NL [ http://www.metaseek.nl ] (Netherlands) and

ApocalX [ http://search.apocalx.com ] (France).

2. "Pseudo" MSEs Type I

The type I "Pseudo" MSE sends the query to the search engines, and then presents the results grouped by search engine in one long, easy to read scrollable list. Be careful. Based on how many SEs you select, the waiting time can be very long. Some people might find these MSEs useful, however.

The best MSEs in this category are:

Mall Agent [ http://www.mallagent.com/web.html ], which provides results from 38 SEs and WDs,

qb Search [ http://www.qbsearch.com/ ] (from 17),

Better Brain [ http://www.betterbrain.com/ ] (12),

My Net Crawler [ http://www.mynetcrawler.com/ ] (12),

NBCi [ http://nbci.msnbc.com ] (11),

Planet Search (Sherlock Hound) [ http://www.planetsearch.com/ ] (10),

Rede Search [ http://www.redesearch.com/ ] (8),

1 BLINK [ http://www.1blink.com/ ] (7),

Search Wiz [ http://www.searchwiz.com ] (6) and

Search Fido [ http://www.searchfido.com ] from 4 SEs and WDs.

3. "Pseudo" MSEs Type II

There are two types of Type II "Pseudo" MSEs:

a) You type your query one time and then select the search engines. One browser window will open for each SE selected. The best are:

Multi-Search-Engine.com [http://www.multi-search-engine.com] which opens 36 windows,

GoGettem [ http://www.gogettem.com ] (30),

Search Bridge [ http://www.searchbridge.com ] (24),

The Info [ http://www.theinfo.com ] (15) and

Net Depot [ http://www.netdepot.org ] (15).

b) You choose the SE, type the query in the SE' form and a new window will open. Every search engine has its own query form. Many users will find these window-opening MSEs annoying. The best of this type are:

Alpha Seek [ http://www.alfaseek.com ],

Westlaser [ http://www.westlaser.com/ ],

Dan's No Overhead Search Thingy [ http://www.danielc.com/thingy.html ],

Express Find [ http://www.expressfind.com ] and

Freeality [ http://www.freeality.com/meta.htm ] .

4. Search Utilities (also called Desktop Search Applications)

These are downloadable meta search tools that search multiple search engines. Results are collated and ranked for relevancy with redundancies removed. They are not free but most of them have a free trial version available. The price? A few dozen dollars. The most popular are:

BullsEye [ http://www.intelliseek.com/ ],

Copernic [ http://www.copernic.com/ ],

LexiBot (formerly know as Mata Hari) [ http://lexibot.com ],

WebFerret [ http://www.zdnet.com/ferret/index.html ] and

WolfBot [ http://www.wolfbot.com/ ].

Conclusions:

Now, you might ask yourself: If MSEs are so good, do we still need the search engines?

Well, it depends. I use a search engine - yup, Google - when I search for general information. I use a meta search engine when I'm looking for a unique or obscure search term or if I want to make an in-depth analysis of what's out there on a specific subject.

My suggestion is to find some time and give a test drive to the MSEs; you might fall in love with these lesser-known search tools. And next time you use your search engine of choice, remember that there are search tools that can provide you many more relevant results.

Good luck with your searches!

Author, Daniel Bazac, is Search Engine Marketer for Web Design in New York, a site design, Search Engine Optimization and promotion company http://www.web-design-in-new-york.com. He's been online since 1995 and he's also a seasoned Internet Information Researcher.