Google should learn to love black hat SEO.  In fact – they should embrace it.  They should make it a central pillar of their entire strategy.  Shock horror gasp!  I know.  Let me present my case.  I think it’s a pretty good one.  But first, let me tell you my yarn as to how I got started thinking about this issue.

It all started around March of this year when I decided that it was about time I learnt to code.  To encourage this pursuit I decided to pick web development (django/python style) – and I thought I might as well try to turn a coin at it while I do it.  So I’ve been working at writing an application which adds some value in the affiliate space.  It’s not very high minded, and I certainly don’t think I’ve contributed anything yet that is worthwhile (hence I haven’t turned any coin yet) – but that’s okay.  I’m just trying to learn to code.

But although it’s okay if I don’t actually turn a coin in this endeavour, since my primary aim here is learning, still I was a bit dismayed to learn of some of the forces arrayed against me.  The most upsetting of these are the blackhat seo marketers.   After noticing some funny looking websites appearing in the search results for some of my targeted keywords, I began to peer into their strange, dark world.

I should note at this point that I’m not going to link to any of the sites that I found, nor name that particular black-hat seo marketer that I’ll primarily be talking about. Ultimately my reason for this is that I don’t believe in trial by internet.  This alone would be enough to prevent me from releasing the information about who I was talking about.  But of course, I actually believe these people shouldn’t be vilified – and that it’s in all our interests NOT to vilify them.  But before I explain myself on this point, we need to go deeper into the rabbit hole.

The Black Hat SEO Rabbit Hole

So – I noticed a somewhat odd site ranking higher than my site for a particular set of keywords in my chosen niche.  As you’d expect, it was a spammy looking wordpress blog with about five or so poorly written articles – the sort of articles you can find on ezine, but slightly worse.  They are the sort of articles you could write in ten or fifteen minutes by just googling the topic and paraphrasing the most basic information that you find.  They involve the absolute minimum amount of human intelligence required to satisfy a robot that this isn’t algorithmically produced spam.  (Of course, it is just algorithmically produced spam – it’s just that this algorithm still requires people for its execution.)

There were two things I couldn’t understand about this website.  Firstly, I couldn’t understand how it was ranking so well.  It had only been registered a couple of months and had no backlinks that I could discern in google.  Secondly, I couldn’t understand the point of the website.  It had no advertising on it.  It had no links to other websites – so it wasn’t trying to funnel traffic.  So what was the angle here?

The answer to the first question proved to be straightforward.  Google will place any site on the front page of results for a particular set of keywords if it meets the following criteria:

1) The targeted keywords are not particularly competitive (for the given market/country)’ i.e. there are not many strong sites targeting those keywords.

2) The site follows basic on-page optimisation rules which include:

• Having the keywords in the domain name
• Having the keywords in the meta title tag
• Having the keywords in the meta description tag

Astonishingly, that’s it.  I’ve tested this extensively, and it is undeniably a fact.  It works because a) there is an astonishing amount of ignorance out there about on-page optimisation, b) because most legitimate sites use branding keywords in their title that don’t exactly match the keywords people use to search (e.g. “Yolanda’s Jewellery” – as opposed to “Cheap Jewellery for Sale” ), c) SEO folk devote their time to deploying hundreds of these sites to find those keywords that have high traffic and low competition.  This is a specialised endeavour in which individual vendors can’t afford to engage.

Finding out the purpose of this site took a little more legwork – but not much.  Luckily, it turns out that this particular internet marketer has one particular flaw – vanity.  Within minutes of looking up the owner of the domain, I had just about everything I needed.  I had his facebook page.  I had a list of articles he had written for various crummy seo blogs.  I had a list of the other spammy domains that he owned.  It was now just a matter of sifting through it all.  The vanity comes into it because this guy was making a lot of money and of course had to tell all these sites about it.  It seems to me to be the kind of tragic flaw of the black hat SEO.  What they really want is respect and acclaim, but don’t seem to realise that ultimately what they’ll receive is derision.

Anyway – the paydirt of this long and tedious exercise was an hour long interview that I found between our hero and a pretty low-brow SEO blog.  In it he revealed everything I wanted to know.  The whole genesis and purpose of his site breaks down like this:

1) Throw up a wordpress blog for the chosen keywords

2) Outsource the article writing to cheap labour in Malaysia, India etc… (I believe he quoted a dollar or so per article.

3) Outsource link building (if required) to the same cheap labour.

4) Contact legitimate site owners.  Point out to them that their site is not ranking as well as it could be for the given keyword set.  (Of course it wouldn’t be because they’ve just been bumped down by black hat spam).

5) Offer your SEO services at some ridiculous price rate.  If they accept – you can give them a near instant bump by removing your spammy websites.

6) If they refuse SEO, try selling them your spammy domains.

7) If all of the above fails… then you can whack some adsense on the blog and let the recalcitrant website owners suffer the inflated marketing costs this produces.

8) Rinse and repeat… always repeat.

This blew my mind when I first learned of it.   On the face of it, these people exist only to suck value out of the market and make everyone’s life (but their own) harder.  It becomes harder for consumers to find real websites that provide value.  It becomes harder for real websites of value to rank for keywords for which they would have ordinarily ranked well if not for the black hats.  Effectively, the black hats extort money from legitimate website owners as a payoff for not making life shitty.

There are two main ways I think one can try to fight this phenomenon – the wrong way and the right way.  Let’s look at both.

The Wrong Way – Deny and Vilify

This seems to be the default view of both Google and the community as a whole.  Ultimately the aim of such an approach is to increase the costs for the black hats in their endeavours so that it effectively becomes cheaper for them to produce sites with real value.  The main ways in which such costs are to be exacted involve largely trying to improve the algorithms which can select for quality.

News flash – it’s not working!  It is always going to cost disproportionately more to build an algorithmic measure of quality than it will to circumvent said algorithm.  I’m not going to argue at length for this claim – but current evidence seems to be on my side.

But there is another significant way in which the costs of the black hat seo are increased – through vilification.  The idea is that we ostracise these people from the inner fold of the internet community.  We tell everyone to be on the lookout for spammers so we can de-index them and level large big wads of shame in their general direction.

This approach is probably more effective than the algorithmic one – but it’s still not very effective.  These people generally do want esteem.  But there are enough of them around that they have their own communities and will ultimately be happy with the esteem they can garnish in that small pond if they are locked out from getting what they need from the wider community.

The Right Way – Legitimise Them

This is going to sound radical – but the best way to deal with these people is to legitimise what they do.  Hear me out.

Ultimately – the problem is caused by perverse incentives.    Improve the incentives and you’ll get a better result.   First of all, you need to recognise the value that these guys do bring to the table.    These guys uncover low cost, high value keywords that quality businesses could leverage if they knew about them.  This is real value – and it takes work to uncover.  It deserves to be remunerated  But think about how hard it actually is to leverage payment for this information.  As soon as you reveal it – you’ve lost a large measure of your edge, unless you’re willing to engage in shady practices like those I just described.  And you end up with a search engine where these great keywords are getting mapped to complete shit.

But imagine if you could transform these guys into an army of go-getters that work FOR Google and not against?  So rather than encouraging to map great keywords to shitty domains – encourage them to map good keywords to good domains – and find a way to ensure they get remunerated for this job.

Here’s one way you might do it:

1. First of all, you need to continue to let these guys test for good keywords.  So you need to let them continue to throw up their spammy websites.  They need to be able to see how well these spammy websites will do – how much traffic they get etc… (even see what sort of adsense income they get).
2. Encourage them to register these website on Google webmaster – and encourage them to identify these sites as the spammy sites they are.  (but give it a better name so they don’t feel so horrible about it – call them “Funnel Sites” – because that’s what these sites will be, as I’ll show in a moment).
3. Once registered as a funnel site – google will allow that site a grace period in the index of say 3-6 months… wherein that site will be allowed to gather the data that the SEO guy will need to make his pitch.  After that – the site will be de-indexed.
4. Allow SEO guys to sell a mapping from this domain to another domain.  This mapping can count toward a site’s page rank for those particular keywords.  The data collected in the grace period could be used for a website owner to be able to determine the value in purchasing a mapping from a particular funnel site.
5. Setup a section for this within adwords so as to allow for an efficient marketplace for this sort of exchange.  (But not anonymous bidding – SEOs ultimately must choose who to sell to.  Otherwise you are not leveraging their knowledge of the marketplace for a given set of keywords).  Google can scoop cream off the top.
6. Provide an incentive for SEOs to sell their mappings to high quality websites.  You can do this by making it so that they can charge a higher price for a mapping if and only if they have a reputation for selling mappings to high quality sites.  So if a person with a high quality website (as determined by page rank etc) asks to buy a mapping from you, and you allow them, then this improves the value of your account and it therefore becomes more expensive to purchase from you.
7. SEOs that don’t play by the rules run the risk of reduced revenues, sites delisted – etc…  same as before really – but now ostracisation actually means something.

Now there would have to be lots of little bits and pieces that would have to be added to this to make it airtight.  (e.g. how many funnel sites can there be for a particular keyword, what should the weighting be, how to establish a reputation metric for SEOs… difficult questions – but that’s google’s job).   Nevertheless I think the gist is solid because it gets the incentives right.  At the very least, this is the sort of general direction in which Google needs to be thinking.  Consider some of its advantages:

1. It encourages funnel sites out into the open so google can more easily see them and understand how they are being used.
2. It provides a way in which to encourage SEO spammers to have their own spammy sites de-listed.
3. Google has created it’s own perverse incentive structure insofar as its adwords system incentivises them to have a shitty results page so that people either click on the ads or end up clicking an adsense ad.  This mitigates that problem.  Lost revenue from promoting quality in the index will likely show up by leveraging a commission off the transactions for funnel sites.
4. Users of the search index get to where they actually want to go quicker (without having to pass through multiple layers of spam).
5. Since they’ll be part of the internet family – black hat SEOs will be more likely to share their discoveries with the wider community because they will have reason to think that Google will incorporate future discoveries in a way that will positively benefit them.
6. Black Hat SEOs will be encouraged to invest in the future of search.  If search fails then they fail with it.  And they’ll have to figure out how to do business with Facebook – or some god awful analogue.
7. A good deal of the power in this scenario rests with the owners of quality websites – insofar as they can choose who to buy mappings from.
8. Link building for such domains will become irrelevant (so no more incentives to build even MORE spammy sites just to link to another spammy site).

And probably more I haven’t thought of.  No doubt there are some weaknesses in this idea.  But I’m currently in the honeymoon phase with it and am so enamoured that I can’t see them – please offer your criticisms in the comments.

The core of the idea is one that really works for just about all things in life.  If someone is working against your interests and you try to vilify and ostracise them – you’ll most likely just provide further incentives for them to work harder against you.  Instead, you should try to figure out how to bring them into the fold – and they might just become your most powerful allies.