Competitive Keyword Analysis, how do YOU do it?

[Bayliss Trevor]

Hi JJ,
I have just done a huge amount of work finding out data for the site I'm working on putting all the info into an excel:
1.The site's rank in the google results for the specific keyword or keyword phrase
2.The url(s) for that keyword or keyword phrase
3.The number of total results that google shows for that keyword or keyword phrase
4.The number of total results that google shows for that keyword or keyword phrase
with intitle:""
5.The number of total results that google shows for that keyword or keyword phrase
with intitle:"" & inanchor:""
6.The number of total results that google adsense shows for that keyword or keyword phrase

I am now trying to make an algorithm to make a KEI score. How would you suggest doing this using your logarithmic weighting? Thank you

7. I score each keyword phrase using an algorithm I made up that uses a logarithmic weighting for each of the values (search count, total pages, keyword in title pages, keyword in title & anchor pages) and gives each keyword a score (kind of like KEI, I just improved the original idea a bit)

[JJMcClure]

I am now trying to make an algorithm to make a KEI score. How would you suggest doing this using your logarithmic weighting? Thank you

7. I score each keyword phrase using an algorithm I made up that uses a logarithmic weighting for each of the values (search count, total pages, keyword in title pages, keyword in title & anchor pages) and gives each keyword a score (kind of like KEI, I just improved the original idea a bit)

I have a friend who's a maths whiz, I took my problem to him and basically he gave each factor a weighting, if the number is good for your keyword (e.g. it gets lots of searches) then give it good weighting like squaring it, if the number is bad for your keyword then weight it negatively and divide/add/multiply or whatever to the other numbers in your equation so that it has the correct impact on the score for the keyword.

The formula is intended to be a curve with the score approaching zero the more negative factors there are.

Don't forget what I said earlier though about not relying too much on KEI, it's not a 100% accurate indicator of competition, just a rough guide and a way to winnow down that initial list to a more manageable amount for the final analysis.

[Bayliss Trevor]

JJ do you mean get data for inanchor and intitle or intitle and inurl? It seems KEI is calculated by intitle and inurl?

Cheers

[JJMcClure]

JJ do you mean get data for inanchor and intitle or intitle and inurl? It seems KEI is calculated by intitle and inurl?

Cheers

It is for me, I can't remember what the original formula used. Mine uses the number of searches, total returned, intitle and intitle & inurl.

I've done some testing on this and for the most part, the pages that return for a search using no operators are rarely the same as with operators which is why I don't rely on KEI too much, it's obvious just from doing searches using various combinations of operators that Google places more importance on other signals than page title and url etc, but we knew that anyway.

If it wasn't so time consuimiong to do it I'd try and factor in the percentage of backlinks to a competing page thta use the anchor text. That would make it more accurate.

[Bayliss Trevor]

According to what you told me in the excel you kindly gave me was:

1.Made a value called P^3/C which is made by:
=((Monthly average results for keyword/30)^3)/(total results in google^0.5)

2.Then make a value called title factor which is:
=1/(LOG((keyword in title/3)+1.5))^2

3.Then make a value called T+A factor which is I imagine title and anchor results but appears to be using title and url results??:
=1/(LOG((keyword in title and url/4)+1.5))^4

4.Then the KEI which is called in this case New Score is made by multiplying those three results together.

How would we combine the results for the title and anchor results (if 3 is title and url?) Does that formula work for both? I imagine the answer is no because their must be different weights given to the combinations. What would the new combo be to include both? Thank-you for any ideas and for any maths seo people reading who can give a hand!!

[Roger_Silver]

We run a candidate list of keywords through an online dictionary/thesaurus, linked to a database of keyword phrases ACTUALLY keyed in by search engine users (a sample of top search engines) over the last 60 days (the DEMAND side of analysis). We then run up to 100 of the top count search terms against Google (and or MSN) to get a count of the number of ACTUAL result listings (i.e. competitor listings) satisfying each keyword phrase (the SUPPLY side of analysis).

[ddzc]

JJMcClure,

Great thread. I want to ask though...where can I find accurate "real" stats on how many searches are completed daily, monthly on Google.

I have used the below tools:

https://adwords.google.com/select/KeywordToolExternal

http://freekeywords.wordtracker.com/...suggest=Hit+Me

I know you mentioned you use wordtracker, do you use the above link or did you buy the monthly subscription? Will the monthly subscription give different results? Is it worth the $?

I'm getting extremely frustrated b/c I spent a lot of time on a couple of sites targetting specific keywords but i'm getting nothing out of it.

Theres a couple of keywords i'm targetting...both don't come up on wordtracker but google states that one gets 200 searches a day and the other states 700 searches a day. I'm on google page 1 for both sites/keywords and i'm not receiving a single hit on these two key phrases.

I feel that i'm wasting my time with innacurate data.

Please chime in, thanks

[JJMcClure]

JJMcClure,

Great thread. I want to ask though...where can I find accurate "real" stats on how many searches are completed daily, monthly on Google.

I have used the below tools:

https://adwords.google.com/select/KeywordToolExternal

http://freekeywords.wordtracker.com/...suggest=Hit+Me

I know you mentioned you use wordtracker, do you use the above link or did you buy the monthly subscription? Will the monthly subscription give different results? Is it worth the $?

I'm getting extremely frustrated b/c I spent a lot of time on a couple of sites targetting specific keywords but i'm getting nothing out of it.

Theres a couple of keywords i'm targetting...both don't come up on wordtracker but google states that one gets 200 searches a day and the other states 700 searches a day. I'm on google page 1 for both sites/keywords and i'm not receiving a single hit on these two key phrases.

I feel that i'm wasting my time with innacurate data.

Please chime in, thanks

I used to use Wordtracker because it was the best available option but I don't think that it is anymore so I use the Google keyword tool now, it may be skewed but I'd rather see a skewed version of the real stats than a best guess from Wordtracker which is actually based on extrapolations from searches made on Dogpile and Metacrawler.

But, any keyword tool should just be giving you a feel for how popular a keyword is, don't rely on any stats being dead accurate.

[ddzc]

I used to use Wordtracker because it was the best available option but I don't think that it is anymore so I use the Google keyword tool now, it may be skewed but I'd rather see a skewed version of the real stats than a best guess from Wordtracker which is actually based on extrapolations from searches made on Dogpile and Metacrawler.

But, any keyword tool should just be giving you a feel for how popular a keyword is, don't rely on any stats being dead accurate.

Thanks for the response. Do you use the same tool as the one I linked above or does google have other tools?

I also noticed your ranking of 30 and under results for "keyword" & inanchor:"keyword". What type of searches would keywords get with those results? I have a few keywords which are half decent, couple hundred searches a day but the result for the above command is like 15000 to 25000. Is it very difficult to optimize for these types of keywords for a new site?

Thanks

[JJMcClure]

Thanks for the response. Do you use the same tool as the one I linked above or does google have other tools?

Same tool.

I also noticed your ranking of 30 and under results for "keyword" & inanchor:"keyword". What type of searches would keywords get with those results? I have a few keywords which are half decent, couple hundred searches a day but the result for the above command is like 15000 to 25000. Is it very difficult to optimize for these types of keywords for a new site?

Thanks

Heh, that's why you're doing the analysis, to find those words.

[dtm32236]

6. Then I do an intitle:"keyword" & inanchor:"keyword" (both together) for each keyword phrase. Any page that has my keyword phrases in both the Title and an on page link is likely to be optimised for that phrase so they're my assumed optimised competition, I'm hoping to see low numbers for this particular search (under 30 is good, under 10 is great, I'm not looking to go head to head with loads of optimised competition).

JJ - maybe I'm doing this wrong, but when I Google intitle:"cabinet hardware" inanchor:"cabinet hardware" I get 185,000 results. That sounds wrong, AND just doing intitle:"cabinet hardware" I get 8,650 results.

Clearly I'm not searching this correctly. Do you (or anyone) know why?

Thanks!

[JJMcClure]

JJ - maybe I'm doing this wrong, but when I Google intitle:"cabinet hardware" inanchor:"cabinet hardware" I get 185,000 results. That sounds wrong, AND just doing intitle:"cabinet hardware" I get 8,650 results.

Clearly I'm not searching this correctly. Do you (or anyone) know why?

Thanks!

Hi dtm,

I don't use inanchor: anymore, I was discussing this with Trevor earlier in the thread. Now I use inurl: instead.

I decided that a page with the keyword phrase in a link somewhere on the page was more liekly to be linking to a page about that keyword than it was to be about that keyword, and so would be a competing page. However, a page with the keyword phrase int he URL is veyr likely to be a competing page.

I said in the firrt post that I got most of the theory from the Sitepoint SEM kit and have just been changing things as I developed the technique, that's one of the thing I've changed since making the original post.

[dtm32236]

Oh gotcha - I didn't realize you changed that. I'll see how InUrl works. Thanks a lot.

[rome9t9]

Hi dtm,

I don't use inanchor: anymore, I was discussing this with Trevor earlier in the thread. Now I use inurl: instead.

I decided that a page with the keyword phrase in a link somewhere on the page was more liekly to be linking to a page about that keyword than it was to be about that keyword, and so would be a competing page. However, a page with the keyword phrase int he URL is veyr likely to be a competing page.

I said in the firrt post that I got most of the theory from the Sitepoint SEM kit and have just been changing things as I developed the technique, that's one of the thing I've changed since making the original post.

Okay..so what would be the approx search results that will make that niche profitable???

like you said in your OP that if results for intitle:"kw" inanchor:"kw" is below 10 then its a great niche..

so what will it be for when inanchor is replaced by inurl??

also do you use allinurl function??

[PowerBit]

O.K. for all who want to learn and obviously we have a very willing teacher who is dedicated to help as much as he is able; thank you JJMcClure for obvious participation in this thread created with the intent to help others. Your efforts will be rewarded.

I would like to know a little more about what the above algorithms, post #157, actually are finding:

Let's start with a break down of the above first step:

P^3/C which is made by:
=((Monthly average results for keyword/30)^3)/(total results in google^0.5)

1.) What data is this formula finding for us? The end result is?

2.) Further more when I calculate (Monthly average results for keyword/30)^3) what is this result?

3.) I can see obtaining a daily average when dividing the month by 30. But why multiplying by 3?

4.) (total results in google^0.5) is resulting in what? And why 0.5?

I have more questions to create an even more dynamic thread, but that could only be helped along by our companion JJ. JJ when you have time, I am a sponge for answers.

Thanks everybody for participating. Hope all of your days work today has been productive and that you are satisfied with a days work.

Blessings,
Billy

[PowerBit]

A separate question;

Of course this is open to all who want to partake in this open dialog...

Also it had been mentioned that keywords were selected that were less than 30 for intitle: and inachor: search counts for the algorithm analysis; what if they have 200 page returns for inanchor: and intitle: searches? This question may been asked...

[JJMcClure]

Okay..so what would be the approx search results that will make that niche profitable???

like you said in your OP that if results for intitle:"kw" inanchor:"kw" is below 10 then its a great niche..

so what will it be for when inanchor is replaced by inurl??

also do you use allinurl function??

You can't put a number on it, it's down to the number of searches being performed and the quality of the competing pages. Obviously, the less competing pages the better but each situation is unique. Less than 10 just means that it's going to be easy to get onto page one but keywords like that, that actually get a good number of searches and have little competition, are hard to find.

No I don't use the allinurl: function, that might be better than inurl, I'll have a think about that.

Powerbit, I'm away for a few days, when I get back I'll try to answer your questions.

[michaelblit]

Hello to all!

I am new to this forum and a newbie in creating websites. Through the web I've optimized my website and with three keyphrases I'm on a first page on Google. For more than 5 years but it helped me so little to have more unique visitors. Though I am optimizing from time to time I see I need to do this work more frequently and on more professional level. Please advise me if is is a right algorithm as somebody on the net put. Search on Adwords Google a keyword with at least 500 searches per month and check this word in " " Google how pages use that keyword. It is desirable not more than 200,000. What if more than 500 people type that word what kind of proportion should be for the pages? Also of course those keywords should be in Title, description and keyword mata tegs. Plus in the beginning of the pages of our site. IS it all true? Please update some info regarding searching right keywords for free. I would really apprecaite your inputs, tips, and suggestions.

WIth much love to all,

Michael.

[SEOVancouver]

I mostly use Goggle's Adwords tool. Does anyone have any experience with the reliability of the data compared to Wordtracker?

[ddzc]

Hi JJ,

Do you use the free wordtracker tool (http://freekeywords.wordtracker.com) or do you have a monthly membership? I'm wondering if the membership gives you more/better results for keyword search results in SE's compared to the free tool online?

Thanks

[seoguider1]

I also look at TBPR, both of the backlink pointing to a competitor and of the linker's home page. In some cases, a home page TBPR 0 is useful in filtering out scrapers and other low quality sites that are linking to a competitor.

Home page TBPR can be deceiving, however, (e.g. in cases where the home page TBPR dropped as part of a paid link penalty), so I also run a site:domain.com/* to approximate domain authority. Though this is also inaccurate (there are some quality sites with only a few indexable pages), in general, the numbers are decent indicators of the strength of a domain. And unlike the toolbar, the numbers are always fresh.

[miriamjones26]

I use both intitle:"keyword in search and Yahoo site explorer to measure competitve keyword/ back link analysis.

[michaelblit]

Hello al,

Would be great to hear everyone's opinion about a question asked on 23/2 in this thread by SEOVancouver

I mostly use Goggle's Adwords tool. Does anyone have any experience with the reliability of the data compared to Wordtracker?

Is there am optimum formula between a number of searchers and a number of webpages using the same keyword/phrase?

Thanks all in advance.

Michael.

[tmonit69]

use the google keyword tool and make sure to look at exact match to see how much volume (estimated of course) that a keyword gets

[michaelblit]

Hi tmonit69,

Thanks for your guidance. Please indicate the minimum number of searches per month would be appropriate to optimize that keyword and what is the proportion to that number the number of pages using that keyword in Google?

Thank you in advance to all.

Warmly,

Michael.