Wednesday, June 17, 2009

How to effectively use a Search Engine?

Most of the people think that Search Engine is a website where we can search a term or a phrase and the search engine gives back the results. When this is absolutely true, the power of a Search Engine is not limited to just searching a few terms/phrases. Search Engines are a very powerful tool that can help us find virtually everything on the internet.

At present there are three major search engines competing in market, Google, Yahoo and Bing! In this article let’s unleash the power of a search engine and see how effectively we can use it. But before going ahead with the details, let’s first brush-up our basics about a search engine.

What is a Search Engine?
A Web search engine is a tool designed to search for information on the World Wide Web. The search results are usually presented in a list and are commonly called hits. The information may consist of web pages, images, information and other types of files.

Web search engines work by storing information about many web pages, which they retrieve from the WWW itself. These pages are retrieved by a Web crawler (sometimes also known as a spider) — an automated Web browser which follows every link it sees. Exclusions can be made by the use of robots.txt. The contents of each page are then analyzed to determine how it should be indexed (for example, words are extracted from the titles, headings, or special fields called meta tags).

Now that we know what the search engine is, let’s begin by understanding how to effectively use search terms or phrases. Badly given search terms may result in unexpected search results. Hence it is very important to know – What to search for?

Building Effective Search Words/Phrases
When we want to search a particular article, we first think of what is the focused area of that article. For example, if I want to search for “anonymous methods in C#”, then my search phrase should contain the highly occurring words from that article. These words should be spelled correctly and should be short and to-the-point. Let’s see some of the key points for making searches more effective:
  • Spell words correctly
  • Remember to leave a space between each word in your search query
  • Use most effective words – i.e. words that you expect to occur frequently in that web site. The choice of words makes a lot difference to the search engines. Remember that every word matters to the search engine!
  • Use OR and NOT keywords to combine or exclude words. The support for all or some of these keywords differs from search engine to search engine. Also, note that the words OR and NOT should be capitalized
  • Use fewer descriptive words, or try words that have a different but similar meaning. This may result in different search results and the one you are expecting 
  • Search for exact phrases by placing the search words within quotation marks
  • Do not use long search phrases, since search engines limit the number of words that could be searched
  • Click on the category to see category-specific search results such as web, images, people, maps etc.
Here are few key points to be noted about most of the search engines:
  • Search engines aren’t case sensitive
  • Common words such as a, the, an, as etc are ignored by the search engine. These are called as STOP words. If you want these words to be included, then enclose them in double quotation marks
  • If you are searching for a date, then make sure you use standard date formats. Any custom date format that you are using might not be known to the search engine. Also as far as possible, use month names instead of their integer equivalents
  • There is no need to use the word “AND” in your search query. By default all the searches are “AND” searches .i.e. they will use your search words using AND. For example: “green trees” means “green AND trees”
Apart from these you can use binary operators and wildcards to fine tune your search results:
  • Phrase Search (“”): As discussed earlier, use double quotation marks for searching exact phrases.
  • Search within a website (site:[site_name]): The search of a query can be limited to a particular website by including the term “site:[site_name]” for example: C# This search query will search for C# in the website
  • Search for a title (intitle:[search_words]): If you want to search for pages with specific title, then you can use the “intitle” tag for searching the query. For example: intitle:Sandeep, will search all the pages which has title containing word “sandeep”.
  • Search exact term (+): By attaching a + immediately before a word (remember, don't add a space after the +), you can get the search results for the precisely typed word. This will specifically ignore any synonyms.
  • Terms to exclude (-):Attaching a minus sign immediately before a word indicates that you do not want pages that contain this word to appear in your results. This is typically used to exclude a word from your search result. The (-) sign works similar to the NOT keyword we discussed above.
  • Fill in the blanks (*): The * represents a wildcards and indicates that the search engine should consider * as a set of unknown words.
Apart from these generic tips, there are some search engine specific tips that you can find on their respective help pages. By following these common rules you can make an effective search.

Searching for Files
It is often required to search for files on the internet. And at the same time it is very difficult to visit all the websites and check if the required file is present in that website or not. For example if I want to search for PowerPoint presentations on C#. I can search the presentations i.e. a file having extension .ppt or .pptx using the “filetype” tag as shown below:

Search Query: C# filetype:ppt

This will give me search results that directly points to the PowerPoint presentation which are related to C#. This is supported by Google, Yahoo and Bing.

Finding Vulnerabilities Using Search Engine
This might sound weird at first sight, but this is absolutely true. Hackers or Malicious attackers make use of search engines to find web based vulnerabilities. A powerful search engine is actually a helpful tool for hackers for finding various flaws and vulnerabilities. Let’s understand this using an example. TSWEB is a tool used to acquire a Remote Desktop Connection via Internet. Many of the companies expose their computer systems via TSWEB for flexible operation and controlling of the system. You need to expose an URL for letting internet user access you computer system via TSWEB. The TSWEB tool is specifically used by system administrators to control the systems remotely. But when this exposed URL gets indexed by search engine, it actually gets exposed to Hackers. Hackers search for TSWEB enabled system using the search engine and then these systems are attacked. Hence it is a best practice to hide such URL’s from hackers, using robots.txt. Thus it becomes important to make the website Search Engine Optimized for efficient searching and security reasons.

Search Engine As Calculator
Yes! Your search engine is your handy online calculator. Search engines are made intelligent enough to search for mathematical calculations and at the same time performing those calculations and giving out the result. So if you search for 5+2 in search engine, then it will give the result as 7 and will also search for websites containing the expression 5+2. Most of the arithmetic calculations are performed by search engines.

Custom Search Engine For Your Website
Often we require a search engine which will search for keywords ONLY in our site or in a list of site. This is typically required for products site, where a huge list of products are display on the web and the user wants to search for a particular product on your site. This can be achieved by creating a custom search engine. Both Google and Bing provide facility for creating a custom search engine that suits your needs. Bing also provides programmatic interface to the search engine where it sends the search result in the form of XML or JSON. Bing API’s are exposed for achieving this.

Why Not Earn Money From Search Engines?
Yes, search engines provide facility to embed advertisements into the result set of your custom search engine. When users click on these advertisements, you get paid. Well this is not the primary use of search engine, but definitely one of the most used options for earning ;)

Search Engine Optimization
Search engine optimization (SEO) is the process of improving the volume or quality of traffic to a web site from search engines via "natural" ("organic" or "algorithmic") search results. By following the SEO practices you increase the probability of your site appearing at higher positions in the search result. It’s important to note that SEO practices provide the potential for higher content coverage and ranking, but do not guarantee it. SEO discussion is out of scope for this topic. You can read my blog on SEO tips for search engines.

This is only a part of the Search Engine capabilities that we have discussed here. The Search Engine is a huge beast! Kudos to Google, Yahoo and Bing for providing such powerful search engines!

Tuesday, June 16, 2009

SEO and IIS SEO Toolkit

I have created a presentation to help you understand the basics of Search Engine Optimization (SEO). SEO techniques are used to optimize the web pages for search engines. Some key SEO tips and tricks helps us increase the page rank and eventually list our site in the top few search results.
The presentation also has links for the usage of the IIS SEO Toolkit.

SEO and the IIS SEO Toolkit

Tuesday, June 2, 2009

Bing! Microsoft's new search engine is live!

It' Bing!

Bing is a phonic sound to indicate you found the thing you were searching for. I think, by the term "Bing" Microsoft wants to tell that you can find the things that you are searching at Microsoft has evolved the search engines from to Bing has a very powerful and improved search engine. Even the indexing mechanism has been improved. Microsoft claims Bing to be a decision engine rather than just a search engine.

Lets see, how Stefan Weitz, a director on the Microsoft Search team, discusses the development of Bing around users’ needs, focusing on four key areas: speed, relevance, previews and multi-media.

You can find the press release of Bing here.

Monday, June 1, 2009

General Motors (GM) files bankruptcy!

General Motors (GM) has filed for Chapter 11 bankruptcy protection Monday morning, submitting its reorganization papers to a federal clerk in Lower Manhattan. This is in accordance to the Obama administration's plan to shrink the automaker to a sustainable size and give a majority ownership stake to the federal government. This has made the 100 years old giant to ruin in soil. GM was one of the largest auto makers in America and had a strong economic hold. About 20,000 workers are expected to lose their jobs due to this bankruptcy directly and numerous indirectly.

GM's bankruptcy is going to have a huge impact on Indian economy too. Many of the Indian giants such as TCS, Infosys hold a share in GM's growth, which has now stunted. The recession is deepening and there are no signs of getting a rescue from this situation. Let’s hope to improve this situation in next two or three quarters. If this situation continues, it’s going to create more criminals than wise men.

Let’s keep our fingers crossed…and try to improve this situation!