Preamble:

A while back some coworkers of mine came to me and suggested that, as I was new to SEO, I should follow in the footsteps of so many SEO neophytes and write a blog post going over the difficulties I've been having. I agreed.

You know how this goes, neophyte reads SEOmoz, he reads sugarrae, he reads blue hat SEO, he downloads Ideavirus onto his ipod so he can read it on the bus and look like a total jackass. He googles every combination of SEO, Guide and, futilely, "Not Crap", and giving up on that he returns to link building pet sites. He leaves the office at the end of the day overwrought with the desire to buy a chihuahua and wrap it in a miniature American flag. Then he realizes that he is Canadian, and feels confused.

But the fruit of much effort and avoidance of pet websites is here in the beginning of my SEO learning process. A guide for people like me who need to be directed. The SEO lemmings Guide to Ranking Analysis.

Introduction:

The Issue: How do you handle the realization that a once proud SEO achievement has been over shined by an even better SEO effort? Lets say you have a client trying to rank for the term 'maui homes'. Maybe he was once ranked first, then dropped. "Whatever," you think, "there's always some one out there who can get ahead in a single term". Then you find you've dropped some more. Ouch.

I find the Kübler-Ross model useful in situations like these:

Denial: Has Google made a mistake? Have we? Are all of our pages indexed?

Anger: What have these other sites got that our site hasn't got? What kind of cheating is he doing? I wonder how much it would cost to buy them some site wides...

Bargaining: Maybe we can get his links to link to us instead?

Depression:

Acceptance: ... %&$# it.

Relevance, popularity, and the Mob:

First, and I know this is basic, but bare with me: relevance and popularity serve as the two key aspects of the search engine rankings, they are the components that search engines use to decide if people care about you. Neither of these concepts are measurable by us, because Google won't let us know what metrics they use, but we can make some fairly good guesses.

Search engines are interested in showing you sites that you may find useful. They do this by combining relevance and popularity.

You also need to be relevant in order to rank well. Relevance is decided in part by off page factors, and in part by on page factors. Off page, the text that people use to link to you with is associated with what you do, so if lots of people refer to you as a "professional cleaner" you will likely rank well for that. It is also theorized that contextual terms (the rest of the paragraph around the link to you) also affect relevance.

Page titles also play a key role here, with titles that match their subject being more relevant than those which do not. Further, proper use of keywords in the page title is important. Wait. Why is that important? David?

David: The presence of keywords in your title, headers, and body text help google to determine whether or not that page is relevant to the search query. The contents of the title tags are assumed to describe the topic of the page, and thus play the most important role when determining the relevance of the document. H1,2, etc. all play important roles as well.

So titles designate what people (or search engines) think you do. Gotcha.

Popularity, on the other hand, is almost entirely developed off site. It is a social measurement usually explained with the analogy that a link is like a vote for your site. This, in turn, compares the internet to a democracy... and that's where the analogy falls apart. Maybe we could consider it a dictatorship driven meritocracy? Ugh, too wordy.

So let me use a different analogy altogether: the internet is like organized crime, and the search engines are the heads of the families. One day, some one in the family (that is, some one ranked) suggests you to their 'associates' (by linking to you). The more trusted the person who suggests you, the more trust he passes to you, and the more trusted you will be to the Don, so when some one comes to him for a favor he will recommend those he trusts the most. And as you gain the trust of the Don (page rank) your opinions are considered more important.

Of course, like any good crime syndicate, opportunities are abound to whack your competitors (say, where are my site-wides?) or to offer the odd bribe (how much is it to get into DMOZ's directory these days?), and of course if you suggest the wrong people it can lower how much the Don trusts you. Or worse.

So yeah, this analogy isn't perfect either, but the foundation holds. Search engine popularity functions on a meritocratic principle where by the recommendations of individuals build along popularity lines to give particular sites more influence over who ranks first. But hey, the mob analogy is easier to understand, right?

When talking about inlinks as trust it is important not to confuse the "trust" of a website with a new technique for combating spam developed by Stanford University and Yahoo referred to as "Trust Rank. Googles use of trust rank is theoretical, but given Yahoos investment in the study we can assume that Yahoo does. This rank propagates and calculates the likelihood that a site is trusted based on a combination of pagerank bias , outlinks, and so called 'oracle evaluations'. The result is a filtered version of page rank that gives increased relevance, like a super PageRank, that makes the Google and Yahoo pay much more attention to them.

It is also frequently thought that there are trusted domains, for example .edu, or .gov domains, which are given a higher initial for their page rank, as well as higher initial page rank. The real world analogy might be that one would probably trust a government or university representative over a random stranger or commercial representative.

Age also has an effect on rankings. The assumption is that older sites are more likely to be the sites that surfers want to visit. The real world analogy doesn't work as well here.

What other trust issues do we have, David?

David: Age isn't actually as important as the age of your links. If you've had a good, trusted link for some time it will be worth more than if you just got it. However, a five year old site that has gotten a good link in the last six months is not going to be worth much more than a year old site that's had a trusted link for the same period of time.

Also, the actual value of a .edu or .gov domain is vague. We don't know if having that domain alone is actually going to give you a higher page rank, but it is known that .edu and .gov sites are frequently authority sites, and they are awarded a higher PageRank.

Finally, there are a lot of factors to consider when it comes to ranking. Pagerank and Trustrank are just the easiest to guess. There is likely also combinations of search page clickthrough data, toolbar data, and potentially even data from Google Chrome, being used to calculate ranking positions.

Ok, so maybe I've described this somewhat simply, but the basic idea is accurate. If you want to rank well you need to be relevant to the needs of the people coming to Google for help, and vouched for by other important ranked sites, kapish?

Analysis:

Part 1: Figuring out if your site sucks

Now that the basics are out of the way lets look at why your site has dropped. First off, we can look at your page rank.

Ok, no we can't. We can't actually see a pages page rank, and the common perception is that toolbar page rank (TPR) means squat, so this is less a metric of your sites value to google, and more of a dip stick for low level SEO problems. The value of this measurement is that it forces you to take notice of major changes within your own site and provides a visual cue when there is something wrong with a page (greybar).

If there aren't any internal problems, though we could theoretically use some page rank sculpting to pick up the rank, assuming the clients site doesn't already have a site architecture that favors his key pages. There is debate over whether this is important or not, but Google has confirmed that they aren't going to penalize for using nofollow to perform page sculpting.

Next lets look at the competition. Anywhere you like to start, David?

Dave: I like to start by just googling the search terms and comparing how Google displays the results. You can check the titles of the results for how they use keywords, as well as some basic data about the sites. Right away we can see that the first result is called mauirealestate.net. This appears to be their most targeted search term I wouldn't want to compete with them for that.

Yeah. Beyond the huge relevance it gains just by having its keywords in the URL, one of the things people often forget is that people are usually going to link to a site by its url name, so Mauirealestate.net will get linked to as Maui Real Estate. How cunning. But why are they ranking so high for "maui homes"?

Part 2: Links: the meat of an analysis

Next let's check its popularity, which means we're going to be looking at links. In Google we use 'link:', lets try that.

What the #$%&?

Alright, the truth is Google does not want to encourage this kind of analysis because it allows clever SEO's to figure out Google's ranking systems and how backlinks play into popularity. We do have another service that isn't so paranoid: Yahoo.

So lets try the same thing in yahoo site explorer, with the options "show inlinks except from this domain". Clicking that option takes us from site explorer to... a blank Yahoo search page. Good one Yahoo. Fortunately we can still use the linkdomain:x –site:x query in order to find the results.

3,070 for linkdomain:mauirealestate.net –site:mauirealestate.net

Compare this to whatever site you're using. For instance, if you do that with randykeller.com, the fifth result for 'maui homes', you can see that they have...

3,070 for linkdomain:randykeller.com –site:randykeller.com

64 inlinks? Holy ouch. So mauirealestate.net ranks well because they have lots and lots of inlinks right?

Well not necessarily, because the internet isn't a democracy, remember? The questions isn't how many people are vouching for you, its how trusted they are by our kingpin: Google or Yahoo. So we should look at who is doing the linking.

If you are a realtor then the majority of your inlinks are likely from other realtor sites, thanks to the wonderfully inbred nature of realtor linking tactics. This tactic has lead to such wonderful anomalies as this page having a comparatively decent rank

Right off the bat we can see that Mauirealestate.net is different. Their first link is from a mail center, which just happens to have his link at the bottom of it? The dipstick is telling us it has a TBR of 5, which aint bad these days if TBR is at all meaningful. So there is a high point for him. And what's this? A Maui activity site.

These two both exist well outside the realm of realtor interlinking, but both look suspiciously like they were bought. It could very well be legit, but I'm not going to count these as particularly meaningful.

But hey, what's this?

So he has a webcam up eh? Lets check the next page:

Oh Holy HELL. There are loads of them. Many SERPs after the one pictured above are filled with cam links. Not only does he have a LOT of links, but many of the links are coming from .gov's, very legitimate looking .org sites, and lots of sources outside the standard linking circle of realtors. The dipstick ratings of many of these are really high, and perhaps most importantly the links are pointing to places inside the site, so called 'deep links', which Google looks favorably upon. I think we have a winner, folks. Let this be a lesson in the power of simple linkbait to enhance rankings.

David, what else can we do to analyze the site?

David: It's likely you've found the main source of his ranking success: lots of good links. To put it simply: the more quality links you have and the more relevant you are the better your ranking will be.

However, in a less clear cut situation you do a more detailed analysis of his backlinks; checking pagerank, their backlink quality/quantity, and the age of the sites. It would also help to use a key word cloud to see what other search terms he has been using, and what sites are linking to him with those. In this case they are maui, homes, land, condo. This also gives you a sense of the type of real estate he is selling, which may give you an idea of his target markets.

Conclusion:

So lets look at Kübler-Ross questions again: Has Google made a mistake? No. I've watched this site for weeks and there is no sign of it. Have you? Check your site and see. Are all of our pages indexed? No, but then quite a few of them wont be. Just make sure the ones that count are. You could try to increase rank on those in order to get some site wide link juice flowing around the site, but it would likely be a lot of work for little result.

What have these other sites got that our site hasn't got? Webcams. More links. What kind of cheating is he doing? Linkbait. Ok, that isn't really cheating. I wonder how much it would cost to buy them some site wides...? No, don't do that. Really. It was just a joke.

Maybe we can get his links to link to us instead? That's probably not going to happen, unless you put up a webcam

So what can you do? How do we face this herculean task? There are a number of ways.

Combat fire with fire: Produce your own catchy linkbait. Maybe an authorative blog post, articles on the nature of the Maui real estate market during financial collapse? Build a post apocalyptic realty site? A fake webcam that occasionally has a huge wave wipe out the island? A list of issues that a Hawaii property owner might have when facing Mauis massive homelessness problem (isnt it ironic that despite having the highest realtor/populace ratio in the US, Hawaii also has one of the largest homeless populations?)? Ok, those might be kinda lame, but the possibilities are endless.

The difficulty, which was also the brilliance of his decision to use a webcam is that people will continue to link to it, versus a cool one off which tends to get tired after a few months and is forgotten. Webcams have a long, long, longtail.

Make some alliances: I consider this the one under used SEO method that's going to be HUGE. Build real world alliances, much like marketers and PR people do, which allow one to draw upon high quality links. A lot of the major real estate websites do this, though they tend to over do the interlinking which, in turn, leaves them with a big ugly footprint.

Ask for links from related sites: They might not give them up freely, but you'd be amazed how often doing a nice deed for some one will get you a link.

There are loads more on any list of link building strategies that you can find anywhere will tell you. Here are a couple I like:

• To absolutely no ones surprise SEOmoz's guide to link building is great (and expensive)
• Smashing Magazine (I love these guys) on linkbait
• SEOBooks 71 great ways to build links is exactly what the title says it is
• SearchEngineWatch.com's 131 (Legitimate) Link Building Stategies

Remember, this is not a mechanical process, there isn't a system that works here, it's about understanding your audience and knowing how to make them want to link to you. It's a social science, not a science, and as any Engineer will tell you, being social science is just art with numbers.

So until next time, if there is a next time, do not forget the power of something as simple as a web cam, keep checking those back links, and try to avoid cliffs, you brave SEO lemmings.

Post script: It looks like the guy was in third for "maui homes" during the time of writing this has dropped off the face of Google. Those are the kind of cliffs you should avoid.