Who the Heck is Frank Bauer anyway?

Spiders You Want To Attract… Web Crawlers

What Are Web Crawlers?

Well, according to Wikipedia, their definition is:

“A web crawler (also known as a web spider or web robot or – especially in the FOAF community – web scutter) is a program or automated script which browses the World Wide Web in a methodical, automated manner. Other less frequently used names for web crawlers are ants, automatic indexers, bots, and worms.”

This is a process that is also known as spidering. This process is used by the search engines in order to provide the most up-to-date information. This information is then used to make the searches for you and me much faster. For instance, you have a website about how to finish a basement, you got the information, therefore, you want the engines to know about your site… so when someone searches this topic, voila, your site comes up in their search.

Web crawlers are also used for maintenance purposes, such as link validation and checking HTML codes. Another use is for gathering specific information from the Web, such as email address harvesting which is mainly used for spamming purposes.

Why Do You Want Web Crawlers Coming To Your Site?

You want these web crawlers coming to your site to improve your rankings. When Google and MSN use Web crawlers, they are actually analyzing an unlimited amount of Web sites, finding which ones are relevant for the most popular keywords. When you are using a good SEO, you will be showing these Web crawlers that your Website is one to be seen. It will determine whether your site is unique enough to provide information and data for a specific search.

Do All Web Crawlers Work the Same Way?

As you can see, even spiders make their style of web… as do the search engines…

No, Google, Yahoo, MSN, all have their own methods and formula for ranking, and they do not share their formulas with anyone, a highly guarded secret. But I can tell you that they will look at several factors concerning each Web site, checking its importance and overall relevance to a particular topic. They will take the keyword or keywords you are using, and use this to determine whether your page is relevant or not.

You do not need to worry so much about how heavy your keyword saturation is… if you put it in too many times, then they will pick up on that and it will hurt you in the long run. You need to put your keywords in as it pertains to your article and Web site, write for your readers, not for the Web crawlers and search engines.

Back-linking and Why It Is Important

Web crawlers will take a look at how many links there are coming to your site (its popularity), what kind of sites are linked, their quality and popularity, and so forth. Getting back-links to your site builds up your relevance in the eyes of the search engines as you can see in the illustration below.

How Do You Get These All Important Links?

Ask for them! The most popular method of the “experts” is to just ask your readers for links. If you go to a blog and leave a link or comment, just ask them to do the same for you… and guess what… they will! If there is something to gain for a blogger to link to your blog, then you can bet that he or she will do it. First, get to know them, write a blog about them, become a friend with them, then once that relationship is built, you have a friend who will provide you some great back-links.

‘Whatever the mind can conceive and believe… it can achieve.’
Napoleon Hill, Think & Grow Rich

To your success,

Frank Bauer

Owner of FrankBauer.name