What Is the Difference Between MetaCrawler & a Search Engine?

A search engine indexes information in Web pages on the Internet to present you with a list of relevant Web pages when you run a search. MetaCrawler, a specific search engine owned by InfoSpace, was one of the first metasearch engines. Instead of crawling the Web, metasearch engines search multiple search engines and aggregate the results. For general search purposes, initiatives such as the Google Knowledge Graph accomplish many of the same goals as a metasearch engine. However, in specific industries such as the travel industry, metasearch engines are gaining in popularity with consumers because they aggregate results across many websites.

Search Engine

A search engine, such as Google or Bing, sends out Web crawlers or spiders that visit websites and follow the links in each Web page, similar to the way you might browse pages on the Internet. The Web crawlers read the contents of each Web page to determine the subject and context of the information. They also collect other data, such as the number of links to a page and the reputation of the websites that link to it, to make a determination about the quality of the information. The Web crawler updates an index that enables the search engine to quickly present you with a list of Web pages when you perform a search.

Search Algorithms

Each search engine has its own set of algorithms it uses to decide which Web pages it presents to you when you perform a search and the order of the pages. If you type the same search term into Google and Bing, for example, you'll likely get two different sets of results. While many of the same Web pages might appear in both search engine results, the order in which they appear is usually different, and each search engine's results might contain one or more pages that aren't included in the other search engine's list of relevant Web pages.

Metasearch Engine

Instead of scouring the Internet and creating its own index, a metasearch engine submits a query to multiple search engines, Internet directory sites, government sites, news sites and specialty search engines. It then consolidates the responses into a single view. This can result in a more comprehensive list of Web pages, because it includes pages that you might not see if you relied on a single search engine's results. Popular metasearch engines include Dogpile and Excite.

Industry Nice Metasearch

Metasearch engines are gaining in popularity in certain industry niches. For example, a decision about which hotel room to book for a trip is usually not based on price alone. In fact, the average consumer visits five or six websites before booking a room. Metasearch engines consolidate results from multiple sites and categorize results so consumers can view results based on characteristics other than price. Some general-purpose metasearch engines also use linguistic analysis to categorize and theme search results to add value and to make it easier to drill down deeper into subcategories of the search results.