Competitive Keyword Analysis, how do YOU do it?

[JJMcClure]

Ok, I should be working but I thought I'd get this subject going

This is how I do competitive keyword analysis, I'm interested to see how other people do it, how I can improve it, what I'm doing wrong, what I'm doing right etc. Maybe together we can come up with a guide to add to the FAQs.

Keyword count strength analysis

1. Ok, I've done my keyword analysis and got my seed list of highly relevant keyword phrases from all the various keyword suggestion tools. There might be 200 keyword phrases on it, for example. I have them in a column on a spreadsheet.

2. Wordtracker has the least skewed search counts so I use that as a rough guide to find out how many people are actually searching for my keyword phrases. I use to use Yahoo as well but it's out of date now. I add the search counts, actual and estimated in two columns to my spreadsheet.

3. I get rid of all the keywords below a certain number of searches which will vary depending on how many searches the most searched phrases are getting. If some keyword phrases get low search counts but are very relevant and might convert really well, I keep them for the next phase anyway, I'm not just going for the big numbers, anything that might convert well and is competitive is a contender. I might be down to 60 or 70 phrases at this point.

Keyword competition analysis

4. For the remaining phrases I look at Google (because it accounts for 76% of UK daily searches) and I look to see how many pages return for each and every keyword phrase and enter that figure into the next column on my spreadsheet.

5. Then I do an intitle:"keyword" search for every keyword phrase and make a new column for that figure.

6. Then I do an intitle:"keyword" & inanchor:"keyword" (both together) for each keyword phrase. Any page that has my keyword phrases in both the Title and an on page link is likely to be optimised for that phrase so they're my assumed optimised competition, I'm hoping to see low numbers for this particular search (under 30 is good, under 10 is great, I'm not looking to go head to head with loads of optimised competition).

7. I score each keyword phrase using an algorithm I made up that uses a logarithmic weighting for each of the values (search count, total pages, keyword in title pages, keyword in title & anchor pages) and gives each keyword a score (kind of like KEI, I just improved the original idea a bit)

8. Depending on the scores, I carry through maybe 20 keywords phrases that scored the highest to the backlink analysis. The algorithm is design to score the phrases on their competitiveness so even low search count keywords can do well if there is little optimised competition.

Competing pages BackLink analysis

9. I Google each keyword phrase again but this time I'm looking at the top 10 returning pages and analysing the strength of their incoming links. I take each page and use Yahoo's Site Explorer to look at all their backlinks. What I'm looking for in particular is the use of the actual keyword phrase in the anchor text. I don't look at every single IBL, just enough to get a feel for whether or not they're optimised for that phrase in particular. Sometimes, I'll even look at the backlinking site's own backlinks but I don't usually have time for that depth of analysis, gotta keep the client's costs down....

I also use Google's PR figure to get a very rough idea of what Google thinks of the quality of their IBLs. I add all the link numbers to the spreadsheet and average them out to get an idea of how competitive each phrase is for IBLs, are there a lot for this phrase or is the average quite low? It's just an general indicator of whether I can compete or not though and the important thing is the individual pages and their IBLs. I want to get into the top 5 minimum so those competing pages get the most attention usually.

Choosing the keyword phrases to recommend

I usually end up with a handful, 5-10 keyword phrases that have good search counts and a low number of optimised competing pages. From those I choose 3 or 4 that I think I can get the client a high ranking for and that most importantly, are likely to convert to a sale.

All this usually takes about half a day, another half day to implement on-site changes and build up some relevant backlinks and a further half day to monitor, report and tweak over the next 6 months. I think that's pretty good value for money. I give the clients all the keyword analysis data so that they can see what they paid for and sometimes even to understand how I did it (except for the algorithm which is the only bit they couldn't get for free off any search engine).

And that's it.

[I haven't gone into much detail on why each step is important, I'm assuming a basic knowledge of how search engines work and the importance Google places on back links and relevance, I can't wait to see what people would change or add to this, get posting guys ]

[bgscotty]

Hello.

What would also be interesting would be for a number of keyword research experts (which I'm sadly not) to do their research for the same website and to compare the end result keyword lists.

If the methods are diverse, so be it.

What would really matter (to the website business) is that various methods identify broadly the same keyword lists to go with.

However, if the final keyword lists are greatly different, then where does that leave the website business, their confidence in the quality of the research they've paid for, and the whole keyword research approach?

How likely is it that different keyword researchers would produce broadly similar "best" keyword lists?

Cheers,

BG

[aspen]

I don't do, or buy into, the idea that volume of results returned is an indicator of competition.

The fact is some common words might not be competitive at all, but they're common words and so appear on many pages.

Allintitle and allinachor are better.

Personally, I just look at the top ten or so listings, their backlinks, their pagerank, my own opinion as to their on page optimizations, and then decide if I can beat them. Not everyone could do this of course, you need to understand SEO enough to make good judgement calls, but it works for me.

I don't care so much about how many sites there are on the topic, 99% of them are easily beat, it is the top ones that will either hold me down or allow me to be successful. So those are the ones I worry about.

[JJMcClure]

I don't do, or buy into, the idea that volume of results returned is an indicator of competition.

The fact is some common words might not be competitive at all, but they're common words and so appear on many pages.

Allintitle and allinachor are better.

Agreed.

The general search I do is more just to see how many pages return for that particular phrase and it also gives me an indicator of how competitive that phrase might be. I don't use search operators or select my local version of Google because I don't believe that most searchers will do either of those thing initially, so I want to see what they'll see when they use that phrase. Another thing I look for but don't include in the algorithm is the number of PPC ads that return, that's a strong indicator of competition.

I include the general search figure in the algorithm but I don't give it as much weight as the other factors, it's more of a dampening factor than anything else. Regardless of whether or not those pages are optimised for that phrase, I still have to beat them because they returned for that phrase. I start there and work down to optimised competition.

[jgsketch]

regardless of any other comments, this step by step procedure will help anyone who is not a season pro at SEO. It has given me ideas on how I can better my current methods. Thank you. Great Post.

[JJMcClure]

regardless of any other comments, this step by step procedure will help anyone who is not a season pro at SEO. It has given me ideas on how I can better my current methods. Thank you. Great Post.

You're welcome.

With the exception of the backlink relevance analysis and few minor tweaks like not paying so much attention to on page factors on the competing pages like keyword density, ignoring weightings for less relevant keywords (I just don't include them most of the time), the keyword strength algorithm which is my own version of the famous KEI, and one or two other things, it's pretty much process described in the Sitepoint SEM kit.

[JJMcClure]

Bump. Still hoping to get a few more expert opinions on this very important core issue of SEO. If you get your keywords wrong, everything else you know is a waste of time.

[ExtraArm]

Nice post.

I also use Google's PR figure to get a very rough idea of what Google thinks of the quality of their IBLs

I also look at TBPR, both of the backlink pointing to a competitor and of the linker's home page. In some cases, a home page TBPR 0 is useful in filtering out scrapers and other low quality sites that are linking to a competitor.

Home page TBPR can be deceiving, however, (e.g. in cases where the home page TBPR dropped as part of a paid link penalty), so I also run a site:domain.com/* to approximate domain authority. Though this is also inaccurate (there are some quality sites with only a few indexable pages), in general, the numbers are decent indicators of the strength of a domain. And unlike the toolbar, the numbers are always fresh.

[JJMcClure]

I must say I'm a little surprised at how little attention this thread is getting, particularly from the experts. If you take a search engine friendly page and actually optimise it for a particular keyword phrase you have to identify that phrase in the first place. Everything that happens after that is dependent on how well you chose that phrase so why is there so little comment or debate about such a critically important part of SEO?

Did I nail it so exactly that no one feels the need to comment? I'd like to think so but I doubt it so come on guys, let's hear your opinions....

so I also run a site:domain.com/* to approximate domain authority. Though this is also inaccurate (there are some quality sites with only a few indexable pages), in general, the numbers are decent indicators of the strength of a domain. And unlike the toolbar, the numbers are always fresh.

Interesting, what importance do you place on domain authority? If you felt that you could compete on everything but that would you move onto a different phrase? What importance do you think Google place on it?

[JJMcClure]

JJ, I'll try to get a coherent response to your initial post together shortly.

Coolio, look forward to reading it.

[MrCat]

JJ this is a very informative and useful thread you got going in here. Another one to add to your belt so to speak

What I basically do is use Google analytics to see which keywords refer more traffic, then check on the length of time the visitor stays as well as the pages he visited - this will tell me if I got a targeted visitor out from that keyword. The relevancy of that keyword would give me a go signal to optimize more for that.

I would definitely give your method a try. A bit tedious though but maybe worth the effort

[jlisec01]

very nice job, what kind of keyword suggestion tools do you use?

[heatman07]

Those are nice tips anyway how about researching a Long Tailed Keyword? Is it the same? or you use another tool to get them.?

[JJMcClure]

JJ this is a very informative and useful thread you got going in here. Another one to add to your belt so to speak

Thank you sir I'm hoping that when Stymiee and Extraarm get done with their very interesting but a little off topic exchange, they'll post their thoughts on the subject of competitive keyword analaysis

And yes, it can be really tedious sometimes but it's a kick when you find a great keyword phrase...

What I basically do is use Google analytics to see which keywords refer more traffic, then check on the length of time the visitor stays as well as the pages he visited - this will tell me if I got a targeted visitor out from that keyword. The relevancy of that keyword would give me a go signal to optimize more for that.

I would definitely give your method a try. A bit tedious though but maybe worth the effort

Yes that's one way of identifying keywords but if you have a list 300 long, how do you use them all to see which ones work? A good competitive analysis gets rid of 295 of those keywords and leaves the ones that will generate traffic that converts to a sale.

very nice job, what kind of keyword suggestion tools do you use?

Wordtracker mostly, yahoo's keyword tool when it's working but only for related phrases not actual search counts, Google keywords suggestion tool, WEBCeo's keyword suggestion tool, googling the main phrase to see what comes back... whatever I can think of basically. All I'm trying to do at that point is build up a big old list of terms related to the main niche phrase. The analysis comes after that.

Those are nice tips anyway how about researching a Long Tailed Keyword? Is it the same? or you use another tool to get them.?

My seed list is every phrase I can find that is relevant to the main niche phrase so it includes long tails, see above.

Then I winnow it down using the analysis from the first post until I have about 5 keyword phrases left (gotta set a limit cos the client is only paying me so much) and they get used whether they're 2 word phrases or 5 word phrases. It's all about traffic and competition for those phrases.

[stymiee]

Thread split. See Domain Authority (Split from: Competitive Keyword Analysis, how do YOU do it?).

[JJMcClure]

Thread split. See Domain Authority (Split from: Competitive Keyword Analysis, how do YOU do it?).

Thanks mate

[dtm32236]

JJMcClue - do you do this for every page of the site, or do you do it for the whole site in general, then select certain sets of KWs to use on different pages?

[Dan Schulz]

I'm pretty sure he does it for each page, since Web pages are independent documents and are treated as such by the search engines.

[dtm32236]

Yeah - that's what I figured.... but that's a hell of a lot of work if you have a couple hundred of pages.

I figured that maybe he breaks them into different categories or groups, then divides certain keywords among pages, if that makes sense...

[JJMcClure]

I try to come up with 3-5 good strong keywords and then use them in the appropriate places in a site, one phrase per page usually. I don't do it for every single page on a site, that would be a huge amount of work plus it's hard enough to find a couple of good phrases let alone loads.

Plus I want to use the best phrase in IBLs so only one page gets the benefit of those at first.

[dtm32236]

I try to come up with 3-5 good strong keywords and then use them in the appropriate places in a site, one phrase per page usually. I don't do it for every single page on a site, that would be a huge amount of work plus it's hard enough to find a couple of good phrases let alone loads.

Plus I want to use the best phrase in IBLs so only one page gets the benefit of those at first.

Gotcha.

So, you only focus on one KW per page?

Don't most suggest focusing on 3-4 phrases for each page? That seems to make sense to me.

[JJMcClure]

Gotcha.

So, you only focus on one KW per page?

Don't most suggest focusing on 3-4 phrases for each page? That seems to make sense to me.

Suppose you use 3 keyword phrases on one page. They'd have to differ significantly or they'd look like 3 forms of the same phrase and that would be a strange way to write, plus your keyword research should have identified the best version of your primary keyword phrase. Since a keyword phrase describes the contents of the page you'd need to have 3 different subjects on the one page. Not great usability.

Next you have the problem of where to put your keywords. First place is the page Title, you want to have 3 keyword phrases in the Title? If you do, which order do you put them in? How many characters does Google actually read in the page title? You have the same problem in your description meta tag and then again when you start using your H tags.

Then you have the IBLs. You want to use your keyword phrase in your IBLs o boost relevance, it's key. So now you have to get three different sets of IBLS, all pointing to one page. Instead of having 3 dedicated opportunities to rank, you have one, even if you manage to pull it off, it's one page that ranks for three terms. I would always use 1 or maybe 2 keyword phrases on a page, never 3. I'd rather take 3 keyword phrases and use them to make 3 different pages rank well.

[dtm32236]

Wow - all good points.

...something I'll keep in mind for my next site (or next time I decide to do an extreme SEO makeover of my site)

[JJMcClure]

Personally, I just look at the top ten or so listings, their backlinks, their pagerank, my own opinion as to their on page optimizations, and then decide if I can beat them.

You're only describing the second part of competitive analysis here, how do you get to the point of doing this search? By that I mean what keyword phrases are you using and how did you find them?

The first part of competitive keyword analysis is identifying all the keyword phrases that you then go on to analyse the competition for, how are you doing the identification bit?

[kronikmedia]

Although many different keyword analysis tools are available, we mostly use Google's keyword volume tool along with searching for our competitors on Google to see what keywords they are targeting. We have also found the free version of Search engine optimisation software, WEB CEO to be quite useful as it can automate this process for you.