How Do Major Search Engines Work?
2.1 How Google Works?
Google has a comprehensive and highly developed technology, a straightforward interface and a wide-ranging array of search tools which enable the users to easily access a variety of information online. Google users can browse the web and find information in various languages, retrieve maps, stock quotes and read news, search for a long lost friend using the phonebook listings available on Google for all of US cities and basically surf the 3 billion odd web pages on the internet! Google boasts of having world’s largest archive of Usenet messages, dating all the way back to 1981. Google’s technology can be accessed from any conventional desktop PC as well as from various wireless platforms such as WAP and i-mode phones, handheld devices and other such Internet equipped gadgets.
The web search technology offered by Google is often the technology of choice of the world’s leading portals and websites. It has also benefited the advertisers with its unique advertising program that does not hamper the web surfing experience of its users but still brings revenues to the advertisers.
Google’s Web Search Technology When you search for a particular keyword or a phrase, most of the search engines return a list of page in order of the number of times the keyword or phrase appears on the website. Google web search technology involves the use of its indigenously designed PageRank Technology and hypertext-matching analysis which makes several instantaneous calculations undertaken without any human intervention. Google’s structural design also expands simultaneously as the internet expands.
PageRank technology
PageRank technology involves the use of an equation which comprises of millions of variables and terms and determines a factual measurement of the significance of web pages and is calculated by solving an equation of 500 million variables and more than 3 billion terms. Unlike some other search engines, Google does not calculate links but utilizes the extensive link structure of the web as an organizational tool. When the link to a Page, lets say Page B is clicked from a Page A, then that click is attributed as a vote towards Page B on behalf of Page A.
Quintessentially, Google calculates the importance of a page by the number of such ‘votes’ it receives. Not only that, Google also assesses the importance of the pages that are involved in the voting process. Consequently, pages that are themselves ahead in ranking and are important in that way also help to make other pages important. One thing to note here is that Google’s technology does not involve human intervention in anyway and uses the inherent intelligence of the internet and its resources to determine the ranking and importance of any page.
Hypertext-Matching Analysis: Unlike its conventional counterparts, Google is a search engine which is hypertext-based. This means that it analyzes all the content on each web page and factors in fonts, subdivisions, and the exact positions of all terms on the page. Not only that, Google also evaluates the content of its nearest web pages. This policy of not disregarding any subject matter pays off in the end and enables Google to return results that are closest to user queries.
Query-handling – The Google Way
Google has a very simple 3-step procedure in handling a query submitted in its search box.
1. When the query is submitted and the enter key is pressed, the web server sends the query to the index servers. Index server is exactly what its name suggests; it consists of an index much like the index of a book which displays where is the particular page containing the queried term is located in the entire book.
2. After this, the query proceeds to the doc servers, and these servers actually retrieve the stored documents. Page descriptions or “snippets” are then generated to suitably describe each search result.
3. These results are then returned to the user in less than a second!
The Google Dance
Approximately once a month, Google update their index by recalculating the Pageranks of each of the web pages that they have crawled. The period during the update is known as the Google dance.
Because of the nature of PageRank, the calculations need to be performed about 40 times and, because the index is so large, the calculations take several days to complete. During this period, the search results fluctuate; sometimes minute-by minute. It is because of these fluctuations that the term, Google Dance, was coined. The dance usually takes place sometime during the last third of each month.
Google has two other servers that can be used for searching. The search results on them also change during the monthly update and they are part of the Google dance. For the rest of the month, fluctuations sometimes occur in the search results, but they should not be confused with the actual dance. They are due to Google's fresh crawl and to what is known "Everflux".
Google has two other searchable servers apart from www.google.com. They are www2.google.com and www3.google.com. Most of the time, the results on all 3 servers are the same, but during the dance, they are different.
For most of the dance, the rankings that can be seen on www2 and www3 are the new rankings that will transfer to www when the dance is over. Even though the calculations are done about 40 times, the final rankings can be seen from very early on. This is because, during the first few iterations, the calculated figures merge to being close to their final figures.
You can see this with the Pagerank Calculator by checking the Data box and performing some calculations. After the first few iterations the search results on www2 and www3 may still change, but only slightly.
During the dance, the results from www2 and www3 will sometimes show on the www server, but only briefly. Also, new results on www2 and www3 can disappear for short periods. At the end of the dance, the results on www will match those on www2 and www3.
This Google Dance Tool allows you to check your rankings on www, www2 and www3 and on all 9 datacenters simultaneously.
How to maximize Google’s Search Features?
Find It ASAP
The Google Web Directory works in combination of the Google Search Technology and the Netscape Open Directory Project which makes it possible to search the Internet organized by topic. Google displays the pages in order of the rank given to it using the PageRank Technology. It not only searches the titles and descriptions of the websites, but searches the entire content of sites within a category, which ultimately delivers a comprehensive search to the users. Google also has a fully functional web directory which categorizes all the searches in order.
In case you are feeling lucky
The I'm Feeling Lucky™ search button is recommended when searching for a highest ranked web page for a particular search. This saves time in searching for a webpage.
The multi-faceted Google Toolbar
The Google Toolbar™ can seamlessly integrate with a user's web browser and be of quick assistance.
Pandora’s Box
• Google enables its users to search for U.S. street maps immediately by just typing the street name in the query box.
• Latest stock quotes are just a click away. Just type in the company ticker symbol or the name of one of the stock indices stock and mutual fund information and Google will return the relevant information in association with high-profile financial and trading concerns.
Cache me if you can
Google takes a snapshot of each page it examined as it crawls the web and stores or caches them as a back-up in case the original page is unavailable. This cached link always displays the page in the same manner as it was indexed and this is used by Google to match the relevancy of the page to the query submitted by the user. The "Cached" link will be missing for sites that have not been indexed, as well as for sites whose owners have requested Google not to cache their content.
Google Web Search Features
Google offers a variety of special features which helps users to find exactly what they are looking which is all in addition to providing easy access to more than 3 billion web pages. The following is an overview of its key features:
• Calculator. Google has a built-in calculator function which can be used to calculate mathematical expressions involving basic arithmetic, more complicated math, units of measure and conversions, physical constants and even hexadecimal and binary numbering systems. You can simply enter the expression you'd like evaluated in the search box and hit the Enter key or click the Google Search button.
• Dictionary Definitions. When searching about any particular term, if the Google database has a definition or meaning for the term, then it will be highlighted with an underline on the results page. This definition is derived in association with a reliable dictionary source.
• File Types. In addition to HTML files, Google search also supports 12 other formats such as PDF, Microsoft Office, PostScript, Corel WordPerfect, Lotus 1-2-3, and others. Additionally, Google also offers the user the ability to "View as HTML", which allows users to view these files in case the corresponding software is not installed on the user’s PC. It also eliminates the hazards of opening a virus-infected document.
• News Headlines. While searching for a particular term, if that term is also in any of the current news, it is displayed as a separate news link on the results page. This is derived from various news providers who work in association with Google and who let Google monitor them.
A new offering – Advanced News Search
A new offering from Google - Advanced News Search, allows visitors to scour headlines by date, location, exact phrases or publication. People can use it retrieve articles from more than 4,500 news outlets publishing on the Web. Advanced News Search lets visitors search for headlines using several parameters. Among other features, people can locate stories that contain an exact phrase, within the Unites States or abroad, or written by a specific publisher.
• Similar Pages. The results page also displays a link for ‘similar pages’ which uses the GoogleScout technology to explore the web for similar pages. This is particularly helpful if you have hit upon a page which has relevant content, but you want something similar but more.
• Web Page Translation. This feature is particularly helpful if your search has non-English results. Google offers a facility to automatically translate a page for you in English. Currently, Google supports Italian, French, Spanish, German, and Portuguese languages.
• SafeSearch Filtering
Google provides a SafeSearch option to filter pornographic contents from its results page. This is especially useful for shared computers which need to be protected for children surfing the Internet. Google’s technology tries to check keywords and phrases, URLs and Open Directory categories and eliminate these from the search results.
Submitting your URL to Google
Google is primarily a fully-automatic search engine with no human-intervention involved in the search process. It utilizes robots known as ‘spiders’ to crawl the web on a regular basis for new updates and new websites to be included in the Google Index. This robot software follows hyperlinks from site to site. Google does not require that you should submit your URL to its database for inclusion in the index, as it is anyway done
automatically by the ‘spiders’. However, manual submission of URL can be done by going to the Google website and clicking the related link. One important thing here is that Google does not accept payment of any sort for site submission or improving page rank of your website. Also, submitting your site through the Google website does not guarantee listing in the index.
Cloaking
Sometimes, a webmaster might program the server in such a way that it returns different content to Google than it returns to regular users, which is often done to misrepresent search engine rankings. This process is referred to as cloaking as it conceals the actual website and returns distorted webpages to search engines crawling the site. This can mislead users about what they'll find when they click on a search result. Google highly disapproves of any such practice and might place a ban on the website which is found guilty of cloaking.
Google Guidelines
Here are some of the important tips and tricks that can be employed while dealing with Google.
Do’s
• A website should have crystal clear hierarchy and links and should preferably be easy to navigate.
• A site map is required to help the users go around your site and in case the site map has more than 100 links, then it is advisable to break it into several pages to avoid clutter.
• Come up with essential and precise keywords and make sure that your website features relevant and informative content.
• The Google crawler will not recognize text hidden in the images, so when describing important names, keywords or links; stick with plain text.
• The TITLE and ALT tags should be descriptive and accurate and the website should have no broken links or incorrect HTML.
• Dynamic pages (the URL consisting of a ‘?’ character) should be kept to a minimum as not every search engine spider is able to crawl them.
• The robots.txt file on your web server should be current and should not block the Googlebot crawler. This file tells crawlers which directories can or cannot be crawled.
Don’ts
• When making a site, do not cheat your users, i.e. those people who will surf your website. Do not provide them with irrelevant content or present them with any fraudulent schemes.
• Avoid tricks or link schemes designed to increase your site's ranking.
• Do not employ hidden texts or hidden links.
• Google frowns upon websites using cloaking technique. Hence, it is advisable to avoid that.
• Automated queries should not be sent to Google.
• Avoid stuffing pages with irrelevant words and content. Also don't create multiple pages, subdomains, or domains with significantly duplicate content.
• Avoid "doorway" pages created just for search engines or other "cookie cutter" approaches such as affiliate programs with hardly any original content.
Google Services
The following are some of the popular and innovative services offered by Google and there are always improvisations in it.
Google Answers
Google Answers is an interesting cross between ‘online marketplace’ and probably a ‘virtual classroom’. Those who wish to participate must register with Google Answers.
Here, the researchers who have considerable expertise in online researching provide answers to the queries posted by other users for a fee. When a user posts a question, he or she also needs to mention the price the user is willing to pay in case the question is answered. When the question is answered by any user, then the payment is made accordingly to the user answering the question. Moreover, the questions and the discussion that ensues will be publicly viewable and other registered users can also share their opinions and insights.
There is a non-refundable listing fee of $0.50 per question plus an additional 'price' you set for your question that reflects how much you're willing to pay for an answer. Three- quarters of your question price goes directly to the Researcher who answers your question; the remaining 25 percent goes to Google to support the service.
Google Groups
Google Groups is an online discussion forum and it contains the entire archive of Usenet discussion groups dating back to 1981. These discussions cover the full range of human dissertation and present a fascinating look at evolving viewpoints, debate and advice on every subject from politics to technology. Users can access all of this information all in a database that contains more than 800 million posts by using the search feature of Google.
Google’s Image Search
Google offers a wide collection of images from around the web; its comprehensive database consists of more than 425 million images. All a user has to do is to enter a query in the image search box, then click on the "Search" button. On the results page, by clicking the thumbnail a larger version of the image can be seen, as well as the web page on which the image is located. By default, Google's Image Search uses its mature content filter on the initial search by any user. The filter removes many adult images but it cannot guarantee that all such content will be filtered out. It is not possible to ensure with 100% accuracy that all mature content will be removed from image search results using filters.
Google analyzes the text on the page near the image, the image caption and dozens of other factors which enables it to determine the image content. Google also utilizes several sophisticated algorithms which make it possible to remove duplicates and it in turn ensures that the highest quality images are presented first in the results. Google’s Image search supports all the complex search strategies like Boolean operators, etc.
Google’s Catalog Search
Google offers a unique service in the form of its Catalog Search. Google’s Catalog Search has made it easy to find information published in mail-order catalogs that were not previously available online. It includes the full content of hundreds of mail-order
catalogs selling everything from industrial adhesives to clothing and home furnishings. Google’s Catalog Search can help you if you are looking to buy for either yourself or for your business.
The printed copies of catalogs are scanned and the text portion is converted into a format which makes it easy for users to search for the catalog. The same sophisticated algorithm employed by the Google Web Search is then employed to search for catalogs. This makes sure that most recent and relevant catalogs are displayed. Google is not associated with any catalog vendors and is not liable for any misuse of this service on part of the users.
Froogle
The word ‘froogle’ is a combination of the word ‘frugal’ which means ‘penny-wise’ or ‘economical’ and of course ‘Google’. Currently in its beta version, or testing format, Froogle is a recent concept put forth by Google. Google’s spidering software crawls the web looking for information about products for sale online. It does so by focusing entirely on product search and applying the power of Google's search technology to locate stores that sell items you want and consequently pointing you to that specific store.
Just like the Google Web Search, Froogle also ranks store sites based only on their relevance to the search terms entered by the users. Google does not accept payment for placement within their actual search results. Froogle also includes product information submitted electronically by merchants. Its search results are automatically generated by Google’s ranking software.
|
