ClickEquations Blog

I’ve been at SMX Advanced in London for the last few days, in part talking about quality score. The one recurring question I heard was about the best way to scale a new account in a way that would maximize quality score.

The worst way to do it, which seems to be the default method, is to just drop thousands, tens of thousands, hundreds of thousands, or even millions of keywords into new accounts and just turn them on. This generally produces really poor results, and may create damage to the account from which it might never recover.

The reason why you don’t want to do this will be clear after we discuss the right way to build and scale a new account.

Starting On The Right Foot

The ideal way to build a new account is to add a small number of initial keywords, allow them to earn good or great quality scores and for those scores to stabilize, and to then add another batch of keywords, allow those to earn good or great quality scores and to stabilize, and then keep repeating this process until the entire initial account build and expansion is complete.

The reason this iterative approach is best, is that it allows your account to develop a history – a positive history – across all the CTR measures that quality score depends on. You want to build this history slowly and carefully, building a base strong enough to eventually support your entire account.

For a brand new account, with a domain that has never been advertised on AdWords before, Google has no existing history or reputation on which to determine initial quality scores. In this case they will look at the performance history and experience that other advertisers have had with the keywords you choose (assuming you’re not the first advertiser to ever bid on those terms) but they will be cautious given your lack of account history, ad copy history, display URL history, etc. and so keywords will very often start with extremely low quality scores – 2s, 4s, and 5s, are common.

But actual performance is monitored closely. If these keywords can show over the course of their first few hundred and thousand impressions, that they can earn competitive click-through rates as compared to other advertisers, the quality scores will rise quickly. This solid performance starts building a reputation for the account by way of a lifetime account CTR history, a performance history for ad copy, a performance history for the display URLs those ads are using, as well as demonstrating whatever geographical performance patterns may be typical in your case.

In each case, a fact-based history is better than an unknown. AdWords will use this more certain base, and hopefully track record of success, to base each quality score recalculation – which of course happens every time a keyword enters a search auction.

Given the importance of this first layer of base, it should include only your strongest keywords. Brand keywords are the best bet IF you have a recognizable brand that will earn typical ‘brand-level’ CTRs of 10% or higher. If your brand is an unknown and will gain low search volumes and potentially low click-through rates then you may not want to start with your brand, or at least include only a few core versions of it and instead go with a set of ‘short tail’ keywords that have a better chance of high CTR.

If you’ve run a version of this account before, or worked with similar keywords for another advertiser, leverage that experience to choose a strong set of high CTR keywords to launch your account. Don’t start with broad ‘head’ keywords that include category names and the like, because they tend to not earn good CTRs. Specific keywords where you’ve got some special attractiveness – a feature, a better offer, unique content like reviews or something you can promise in the ad copy to really drive great CTR. Put your best foot forward.

I would probably start with between 20 to 50 keywords, and lean to the low side of that range if possible. This sounds like an impossibly low number of keywords when your goal is to build an account with hundreds of thousands or millions of keywords. But even the largest skyscrapers start with relatively small foundations. We’ll discuss scaling in a moment, but the risk to focus on here is a poor start that will permanently cripple the account forever. If you can’t come up with that number of high CTR keywords and quickly earn quality scores of 7 or higher, maybe AdWords isn’t for you or this business.

Next Steps

If you pick the right starting set of keywords, build the account on a sound organization and write good ad copy, you should see quality scores increase by the day and hit 7 or higher within a week. Impression volume is probably more important than time, and keep 1000 impressions per keyword in your head as the target volume to achieve.

Any keywords that stabilize at quality scores below 7 should be improved. Better ad copy, smaller ad groups, and a long hard look in the mirror where you ask yourself if you really deserve to bid on that keyword are in order. Don’t make changes before you have statistically significant data, but when you do don’t delay. Pause any keywords that can’t achieve at least 7s in this early stage.

Now you’re ready to add more keywords. Add enough keywords to double your impressions-per-week. Normally many of the keywords in the initial batch will be relatively high volume, so often this next set will include more than a doubling of the keyword count. The point is to not overwhem your base – if you’ve shown strong performance on 50,000 impressions-a-week worth of keywords, and earned good histories in all the CTR measures mentioned above, adding 1M impressions-a-week worth of keywords in one new batch will overwhelm the account. The base wouldn’t be strong enough to hold it. Doubling the impression count is reasonable. The numbers don’t have to be exact, just get as close as you can.

Watch your new batch of keywords closely. They may start at lower than optimal quality scores too, but should climb at least as fast as the first set, and probably faster. When the second set has achieved about 1000 impressions for most of the keywords and demonstrated the ability to earn quality scores 7 or higher in nearly or every case, you can move on to batch #3.

Building Your Skyscraper

You can feel comfortable doubling the average number of impressions-per-week with every new set of keywords added to the account. At this pace your history will provide a strong base to help the new keywords leverage your past success and the new set are unlikely to weaken the base even if they contain some poor performers.

Use this time and opportunity however to be a harsh judge of performance and work to improve or pause any isolated keywords that get stuck with low quality scores. It’s better to work on them now while they’ve got your attention and haven’t started dropping your average performance over long periods of time. A zero tolerance policy for any keyword that can’t get above a 6 – unless it has a strong business advantage (like being highly profitable despite the poor quality score) – is recommended.

At some point, you could probably increase the pace of additions if you’re working toward a particularly huge account. After 250,000 keywords are in and have earned good stable quality scores, a new crop with 2x or possibly 3x the impression volume could work. You’ve got to weigh the risk of weakening your patiently built base with the business expediance of getting the full account running.

Creating History and Reputation

This process is about building up a history in the signals that drive quality score. You can’t swagger into town as an unknown and expect the powers-that-be to trust you as you start doing big business in forty locations on your first day. Better to start small, show them you know what you’re doing, and scale on top of the initial reputation you develop.

Metaphors aside, you need an account CTR history that suggests that you know how to buy keywords and write text ads. Note that if AdWords ads have been pointed at your root domain in the past even from another account, there will be some historical residue to overcome. Beyond the account history there is history to build for every visible target URL, for all the geographies where you’ll market, and with each keyword, ad copy, combination thereof, and even search queries. Slow. And. Steady. Wins. The. Race.

Revising History

All of the above is great, if you’ve got a brand new AdWords account. If that horse has already left the barn, then you need to apply these same principles in a different way. That’s not necessarily easy depending on the history the account has, so we’ll cover that topic in a blog post here next week.

Quality Score in High Resolution

This post is based on material from Quality Score in High Resolution, a new book that details the workings and secrets of AdWords quality score. Learn more and order your copy today.

Craig has put together the definative book that will help you to undertstand and improve your quality scores. This is a must read book for anyone concerned about their AdWords’s performance.”
- Brad Geddes, Certified Knowledge

Steve Baker at epiphany put up a very interesting post this week, in which he analyzed some quality score data to try and answer three questions:

1. How high is a high click through rate?
2. What is a decent click through rate for a given position?
3. How do you know if your Quality Score is being dragged down by the Account Quality Score or your adverts?

These are things we’d all like to know, and his results are interesting, but I have some concerns about whether or not they really answer any of these questions in any way we can rely on. To be clear, I’m not sure – so I’m posting my thoughts here to hopefully further the discussion.

If you haven’t please go read his entire post.

There three thing that concern me about the methodology and the conclusions:

1. A mistake concerning the idea that ‘quality score is only calculated on Exact Match’.
2. The assumption that ‘visible quality score’ is quality score.
3. The treatment of the relationship between quality score and bids and position.

Quality Score and Match Type

As discussed at length last week, visible quality score only takes into account the performance of past impressions where search query was identical to keyword, regardless of match type. Using a data set comprised only of Exact Match keywords is certainly a study of its own, but may very well not be representative of how all keywords of all match types perform or behave. Since AdWords already disregards non-identical queries, given the other assumptions this analysis would be equally accurate with all match types included.

Analyzing Visible Quality Score

It’s very hard not to conflate quality score and visible quality score, as Google themselves use the one name ‘quality score’ to refer to both – but they’re very different and I think as search managers we need to begin to really understand that these two things are very different and using them interchangably will lead us to a lot of very inaccurate conclusions. I wrote about the differences in a guest post last week on PPC Hero.

The complexity is that if you’re only analyzing queries which are identical to keywords, as visible quality score does, when in fact all non-identical queries are earning potentially very distinct quality scores for those same keywords, then there is no way to know how valid any conclusions really are. In effect, it’s taking a non-random sampling of the available data (only the identical queries, which represents an unknown % of the data) and ignoring the rest. We might assume that the identical queries have higher CTRs and therefore represent the best quality scores of the bunch – but it is literally impossible to know.

Of course, visible quality score is all we’ve got. Therefore it’s entirely natural to analyze this data and try to understand it and learn from it and draw conclusions. I’m not arguing against it. But I am suggesting that the characteristics of that data have to be acknowledged and considered along with any conclusions.

Quality Score, Bid, and Position

We all know that bid x quality score = ad rank, which determines the position in which any ad appears. In this case, quality score is not visible quality score but a version I’ve taken to calling ‘quality score for ad rank’ that includes a number of factors ignored in visible quality score.

In his post, Steve supplies some very nice charts showing the relationship between position and quality score from his data set. He’s found keywords with visible quality scores of 10 that live in nearly every position from 1 down to 8, for example. Actually his post includes similar charts from many different quality scores.

Chart from Steve Baker @ epiphany

Steve’s draws several conclusions from this data:

• “It appears that Google expect the click through rate in any position to be about 65% of the next position up. So where position 1.0 has an average click through rate of 34%, position 2 has an average click through rate of 22.1%”
• “This appears to be Google’s estimate of what ‘should’ happen to your click through rate every time you drop a position – you lose just over 1/3 of your clicks.”
• “Using this, you can potentially ‘health check’ your account. If you have a click through rate of 4.5% in position 4, you should have a Quality Score of around 7 or so. If you are getting less than the predicted Quality Score across the bulk of your keywords (excluding brand, on Google only, on Exact Match), then it’s a sign that your account has other issues, possibly with the landing page, keyword relevance or the overall account quality.”

Ignoring for a moment the issues about match type and visible quality score, I just can’t quite see how these conclusions are valid. My concern is that the impact of bid on determining the position a keyword earns isn’t considered or reflected – it isn’t just quality score that is driving these positions.

I’ve only had a few moments over the past 24 hours to really think about this, but I’d love to hear from the many smart readers we’ve got what they think of this analysis.

It would be great to have ANY answers to the original questions, and Steve has done a great job of collecting data and presenting it to us with some interesting potential conclusions. I hope he doesn’t mind if we try to crowdsource some additional work on his data.

UPDATE: I realize re-reading this that I didn’t comment directly on the three questions Steve set out to answer. They’re great questions, and there is a lot we know about the answers outside of the data being discussed here. I’ll take these up in a future post.

Quality Score in High Resolution

New 225-pg paperback
by Craig Danuloff

Learn more and order your copy today.

Welcome to week two of my mea culpa tour. Last week I revealed an error from an earlier post on how quality score takes search queries into account. Today I’ll talk about some new facts regarding the most popular post I’ve ever written – The Economics of Quality Score.

An Economist Walks Into A Bar…

The original Economics of Quality Score post describes the impact of quality score on CPC. What was interesting about it, I think, is that it included two tables that purported to quantify the actual economic impact of quality score – how much CPC decreases when a keyword earns a 10 and how much extra you pay if a keyword only gets a 3, for example.

The original calculation was based on visual quality score (see the guest post I recently wrote about visual quality score over at PPC Hero). Doing the math while assuming that quality scores are really whole numbers between 1 and 10 produced the tables included in the original post.

Working with these numbers resulted in dramatic results and a powerful graphic that has been borrowed and republished in many blog posts and used in quality score seminars. The story the numbers told was that earning a quality score 10 gets you a 30% discount on every click, while suffering with a quality score for costs you a 75% CPC premium – to take just two examples.

This calculation made the risks and rewards of quality score very clear. Or so it seamed.

It didn’t take too long after the original post went up, to realize the mistake in these calculations. Quality score isn’t really a whole number between 1 and 10. So these results must be inaccurate. Oops.

A disclaimer was added to the original post.

The disclaimer explained the mistaken assumption, and concluded by saying that while the actual numbers in the chart were undoubtedly wrong, the point remains true – the positive and negative effects of qualty score did apply – and ‘hopefully the numbers are roughly proportional’.

Which brings us to the new information. They’re not proportional.

Quadratic, I Didn’t Even Factor

There are many differences between visible quality score and the quality score number used to calculate CPC. Visible quality score is a whole number between 1 and 10. Quality score for CPC is a real number and the scale is non-linear.

The premise of the calculations I did in the ‘economics’ post was that the distance between the numbers was known and constant, and if you divide any number by 7 and then divide that same number by 10, you will always get a 30% difference in your answer. This was intended to be revealing in terms of quality score.

But since the math that drives your CPC involves numbers that aren’t between 1 and 10, and don’t have a predictible relationship to each other – and are a secret held inside a big blue safe in Building 47 on the Google campus – it turns out we can’t reasonably calculate or estimate the actual impact of quality score in CPC.

We can’t calculate or estimate how much a 10 saves you vs a 7. We can’t calculate or estimate how much extra you pay for keywords with poor quality scores such as 3. Google hasn’t shared enough information for us to know.

Why Did The Quality Score Cross The Road?

There are at least three morals to this story.

The first is that we still don’t know how any increase or decrease in quality score economically impacts your account.

I suppose we could track individual keywords and try to find instances where quality score goes from X to Y while position remains constant and calculate the size of that change, and then after doing this a great many times build a new table based on observation. Of course, there are so many other variables in the system (different search queries, geographies, competitors, etc.) that it would take a huge amount of data to even have a chance at accuracy and in the end we’d never know.

The second is that I should better verify the veracity of the information I post. I’ll work on that.

The third is that Google is really good at hiding their secrets.

The ability to actually know the amount of money a change in quality score was worth seemed like such a big deal because it represented a rare bit of clarity in the sea of uncertainty. We orient well around something as clear and familiar as a 1-10 rating system, but when we stop and think about it:

• We don’t know the CTR’s that achieve any given ranking,
• We don’t know how many auctions we’re being ruled ineligible for because of our score,
• It’s extremely hard to know how queries or geography or ad performance impacts our score, and
• While we know ‘higher is better and lower is worse’ we have no way of knowing how much better or how much worse.

It’s like the perfect carnival game – it seems like getting the coin to land on the plate is easy and the variables are within our control…

So in the end, another mystery not solved.

I promise that the new book does get to the bottom of a few.

Quality Score in High Resolution

New 250-pg paperback
by Craig Danuloff

Pre-Publication
Learn more and order your copy today.

It was exactly two years ago today that I posted the first bit of what I thought was going to be a quick series of posts to dispense with this issue of quality score once and for all.

The idea seemed reasonable enough (they all do when they pop into my head): write a chapter (post) a day for two weeks or so and nail a clear definition and tactical plan for all things quality score.

Two years later and I’m pleased to say I’ve nearly got it. Six chapters are back from the copy editor, four more are in her hands, and the last couple will get there soon.

What took so long?

There were a number of factors:

• It’s a big topic. Quality score is not one simple thing nor does it have one simple impact. It’s complex and pervasive in AdWords. Way moreso than it first appears.
  
• Google hasn’t fully explained it. They’ve said a lot about it, but they had left a lot of big obvious questions unanswered.
  
• It’s subtle and complicated – Therefore it’s a bit difficult to explain clearly. This was probably the biggest time sink, trying to boil down the material to be comprehensive and not confusing or just plain dull.
  
• This isn’t my full time job. Life at ClickEquations and outside of it takes up a little time.
  
• My attention wanders. Bob Dylan has played 188 shows since that first post was written, for example.

Why It Matters

Of course, none of that matters. What matters is that every time someone does a search where one of your keywords might match and one of your ads might be shown, quality score determines if it’s shown, where it’s positioned, and how much you pay. Your success in paid search is literally defined by how effectively you earn good or great qualty scores.

It’s something we all should understand.

The role of quality score in paid search is unique: it is both grading your past (quality score is a measure of the success of any keyword) and at the same time influencing your future (quality score is a prediction of future success which it then plays a role in making come true).

Anything that accurately tells you how well you’ve done and then determines how well you’re going to do should be paid a lot of attention. I argue in the book that quality score should drive which keywords are in your account and which ones get deleted. It should drive the organizational structure of your ad groups. It should drive the copy in your text ads. And it certainly has a lot to do with how you’ll have to bid.

It doesn’t make sense to spending tens or hundreds of thousands of dollars per month advertising via paid search and not deeply understand such a core aspect of the paid search process.

Imagine playing a game every day – for money – and not knowing all the rules?

Yet this is how most people have effectively been forced to play. The basics of quality score are well known, and that’s a big improvement over 2 or 3 years ago when the subject was almost universally ignored. But as soon as you get past the basics, past the generalities, into the land of ‘what exactly should I do with this keyword’, you get to a place where even the very best in the business (and I’m lucky enough to talk to a great many of them with some frequency) just haven’t been sure what to do. The facts, at that level of detail, haven’t been available.

Or they’ve been shattered and scattered into different pieces in different places, with pockets of incorrect info mixed liberally throughout just to make it fun.

Is this any way to conduct business or spend millions of dollars? It really isn’t.

I have some fun analogies in the book about what other businesses would be like used similar terms & conditions and communication strategies. Nobody would accept it and nobody would do business with them. But who can stop advertising on Google just because they’re not getting all of their questions answered?

Google’s Role

At which point we have to stop and praise the truly amazing copywriters at Google. There are dozens of extremely well written posts in the AdWords help system, and blog posts and responses in the help forum, and still they managed to not explain to us precisely how we’re being rated or what we should do to score higher and do better. The broad strokes are extremely clear, but the details are entirely lacking.

To be fair, their job is to provide an overview to millions of advertisers, the vast majority of whom need exactly the level of detail they get. And I genuinely believe their writing is incredibly accurate, clear and concise. But it doesn’t go as far as serious advertisers spending real money need or I think deserve.

Some of the key people at AdWords agreed that ‘advanced users’ desired and required a different kind of information. And they were willing to share – in very large part – the information that was necessary to produce this new resource. They were really open and really helpful.

By all appearances they are happy to have a more complete picture of quality score out there, but the opportunity or the format hadn’t presented itself, or maybe nobody had every found the right way to ask them about it before.

Get Yours

In any case, I believe and hope this book will be useful to all of us who who have been craving more depth and details about how quality score is calculated, how it impacts the account, and how to manage it more effectively.

ClickEquations clients will be getting a free copy.

Learn more and order your copy today.

If the wind holds up, books will be out in time for SMX Advanced in Seattle. Maybe we’ll have a little QS-Geek party.

I was wrong. A couple of times.

The subject was ‘how quality score works’. And in both cases I wrote long detailed posts on this very blog, and I have come to learn that these particular posts were not accurate.

My revised world view was provided courtesy of our friends at Google. They have been kind enough to help me to better understand quality score – the gory details and the dark recesses – over the past six months or so, and I’m going to share some of what I’ve learned.

Actually, I’m going to share all of what I’ve learned, but only some of it will be here on the blog. For the full story, you’ll have to get yourself a copy of my upcoming book ‘Quality Score in High Resolution‘ which will be out in June.

If you’re a ClickEquations client, you’ll be getting a courtesy copy.

Otherwise you can pre-order your own copy for a limited time at a 46% discount off the not-so-tiny retail price.

Now on to my most recent mistake.
.

.

The Actual Truth About Quality Score and Search Queries

A few months ago I wrote a post called ‘Match Types & Quality Score – The Truth At Last‘. It turns out the title probably should have been ‘Match Types & Quality Score – More Confusion and Inaccuracy’. I thought I had it right, but my source was reading between the lines in the detailed study of the official word and various conversations over the past few years.

Now this is embarrasing for many reasons. Chief among them is the fact that I’m not a fan of all the how freely incorrect information and poor advice flows through blogs and tweets and even from the conference podium in this market. I generally work hard to know my song well before I start singing, as some old man once said when he was younger than he’s now. But I blew it on this one (and at least one other which I’ll admit next Monday.)

I am glad that I get to correct the error. I wrote this book to clear up the many mistaken assumptions and recommendations I regularly see passed off as quality score information and ‘tips’. The fact that the research process exposed some of my own errors is a fair price to pay to set the overall record straight.

When I sat down with Google to ask for their help in research and tech-editing the book, I told them that I really didn’t want to do all this work and get it wrong. But I knew there were many specific points that I couldn’t be sure of, because the published material wasn’t detailed enough. I was very pleased and excited when they agreed to help. Over time they answered every question I asked, and only rarely with a ‘no comment’.  This includes responding to the ’11 Hard Questions About Qualty Score‘ I posted a few months ago.

The Details of My Mistake

The crux of the mistake I made concerning the role of search queries was taking the fact that google says ‘quality score is calculated based on keyword performance only when a keyword perfectly matches a search query’ too literally.

One of the things I learned while writing the book and trying to follow all the threads presented in the AdWords help files and official information, is that whenever someone (including Google) says ‘quality score’ you’d better quickly ask ‘which one’ (the book lists eight of them).

In this case I fell down the very easiest hole – the statement above refers to what I call ‘visible quality score’ the number we all see next to our keywords in the AdWords interface. Visible quality score differs from the versions of quality score used to calculate important things like ad rank and CPC in a number of ways.

The statement above is entirely true in terms of visible quality score – the numbers you see are only impacted by queries that equal the keywords – but that does not mean, as I claimed in the earlier blog post, that the quality score from queries identical to a keywords is used to make decisions or calculations about queries that are not identical to the keyword.

To be fair and complete (and slightly mysterious) there is a second cause of my error. This one is based on what I think is an intentional misdirection Google uses when talking broadly about quality score. Google is expert at shaping our perceptions and expectations, and one of the ways they do this is by creating impressions that aren’t literaly true but serve some other purpose – sometimes even for our own good.

Suffice it to say that when that blog post was written I still held some nieve (although almost universally held) beliefs and I am no longer so afflicted.

The third element of my mistake, this is more of a proof or an error-of-oversight, is the fact that the core description of the calculation of qualty score includes ‘the relevance of the keyword and the matched ad to the search query’. So Google had in fact already definitively confirmed that search query was considered. I knew that but overlooked it’s implication when writing that post.

How Search Queries Influence Quality Score

When quality score is being calculated, after a query has been made and before the advertisers and pricing has been decided, AdWords looks at a wide range of factors to assign your keyword a quality score. One of those factors is relationship between the current search query and the current keyword. That relationship can dramaticaly impact the resulting quality score, which means that different search queries matched to one keyword may see significantly different rankings and significantly different CPCs even if they achieve the same ranking.

Search queries are a part of what determines quality score, just not the quality score you see in your account every day.

Doubling Down and Getting It Half Right

I compounded my error by going on to say that the solution to the problem of search queries not impacting quality score, was to create new keywords in order to give each query what amounted to ‘access’ to its own quality score.

The point I was making may have been wrong, but the idea still has merit. By adding a new keyword from what was once just a search query, you do gain the ability to see the quality score for that query – because now it will be identical to the keyword.

Suppose you bid on the broad match keyword ‘dog food’ and it was frequently getting matched to the search queries ‘organic dog food’ and ‘cheap dog food’ among many others. Now further suppose that when AdWords looked at the ‘relationship between these queries and the keyword’ what they saw was, relative to the query ‘dog food’ itself, very positive for the query ‘organic dog food’ and fairly negative for the query ‘cheap dog food’.

In that case, the quality score visible in the account would reflect the performance of the ‘dog food’ search queries. But the queries ‘organic dog food’ and ‘cheap dog food’ would get real-time qualty score calculations, and the resulting impression counts, positions, and costs, based on their own merits. But you would never be able to see those differences.

If on the other hand, you added ‘organic dog food’ and ‘cheap dog food’ as their own keywords (probably in phrase or exact match, but that really doesn’t matter) then the visible quality scores that would appear for these keywords would (ultimately perhaps not immediately) reflect the full detail of their performance and value as AdWords saw it.

By splitting them out you’d be able to make their ‘invisible’ quality scores visible.

A Lot of Shadows In A Short Hallway

I hope this post clarifies the facts about search queries and quality score. I regret and apologize for the original mistake.

This episode highlights a lot about the complexity of quality score – both in terms of how it works and how we as paid search managers get information about it. The complexity of both of thse is one of the main reasons I took up the task of figuring this stuff out and writing this book. This post has turned out long enough, so I’ll say more about that in the near future.

In the meantime, if you’d like to support this project, please consider taking advantage of the pre-sale pricing and offers.

I have a New Year’s Resolution this year. I would like to be done with quality score, at least as a primary PPC obsession. I’ve been too deep into it for what seems like two years, and it’s time to move onto something else.

I’ve even decided what that something else would be – bidding. Seems like the only bigger and deeper mess in the world of PPC. Perfect for me to stew in for a while.

But Quality Score isn’t conquered quite yet. Not for a lack of effort. For about 18 months I’ve been reading, discussing, thinking and writing about it. I’ve even got hundreds of pages of a book on it just about complete.

But questions remain. All the available official help files and conference presentations by Googlers and discussions between the paid search Guru’s I trust still leave a number of open mysteries.

Fortunately, some good folks at Google have agreed to help me by answering some questions. They’ve said that they would like the complete truth out there, and agreed to help me capture it and be allowed to share it. So I’ve begun a series of ‘interviews’ with people at Google who’ve been identified as experts in this area. There are some more conversations ahead.

In preparing for these upcoming discussions, I’ve been refining a list of remaining questions. And I thought that sharing them publicly would be interesting and useful.

This is not a complete list of all the aspects of quality score that I’ve found mysterious, it’s just those that even at this late stage of this process remain confusing or unclear. If you’ve got other questions you’d like answered (or comments/answers to those I post here) please leave them in the comments and I’ll either confirm that I’ve got that topic covered or will try to get an official answer.

NOTE: I’m deliberately leaving questions regarding landing pages off this list. There are still some in that area, but we’ll leave those for another day.

Eleven Huge Questions I Still Have About Quality Score

In no particular order:

Does the ‘relevance’ element of quality score consider any semantic or contextual analysis or comparison between keywords, ad copy, and landing page text?

• The popular and default assumption about relevance might make this question seem strange, but discussions thus far have led me to question these precepts and make this question necessary.

Does relevance have a range, or like landing pages is it simply ‘poor’ or ‘ok’?

• Landing pages can hurt but they can’t help. Relevance is reported in a binary fashion, but it’s not clear if the impact is binary or gradient.

Historic CTR of the account is a listed component of quality score. What is the decay rate of this measure? (Is last week more important than last year? Is there data so old it no longer matters at all?)

• Account CTR history has a HUGE influence on quality score, but ‘history’ is in many ways ambiguous. Does the recent past have more weight? Does last year still matter at all? (There is clear evidence in some cases it does. How does this work?)

How is the history of the target domain (in terms of CTR or other measures) within AdWords a factor in the calculation of quality score?

• Most of the doc only talks about the account, but there are references to the domain history in comments that have been made. What is the role the domain history plays?

If quality score is always calculated based on a combination of keyword + ad copy, how is QS calculated the first times new ad copy is available for display or displayed?

• Testing and changing ad copy is a common and important aspect of account management – but changes to copy should impact quality score given that each new ad has no historical CTR. Do new ads have a minor or major ‘cost’ in terms of initially lower or estimated quality scores? If not, how and why?

In the ‘Economics of Quality Score‘ I used the fact that Ad Rank/QS=CPC to document the impact of different quality scores and QS changes. Given that real QS scores aren’t 1-10, is the post accurate?

• Can’t wait to hear the answer to this one.

Is geographic performance factored at the account or keyword level?

• One of many issues where different official references seem to contradict each other.

How often are visible QS numbers updated, and based on what range of historical performance?

• If we’re working to improve quality score, it matters what the visible QS number represents – even though it’s clearly not the QS the system uses anyway.

Is quality score calculated before or after eligibility for an auction?

• One of several chicken and egg confusions in the public record.

Is there ANY impact of performance or QS at the Ad Group or Campaign levels?

• A rumor that looked like it had been put away but one contradictory help file emerged and the web is full of references.

Google has claimed that QS is only calculated when query = keyword. So how is QS applied to queries that are not identical to keywords?

• There are two possible answers and we don’t know which is true: Do they use the QS of the keyword based on the times queries were identical, or is a different QS calculated for those queries in some way?
• Extra Credit 1:  Are phrase matched queries treated as identical to keyword for QS calculation purposes?
• Extra Credit 2: What about queries when there has never been one for that keyword that is identical to keyword, or when exact keyword is a negative in that ad group?

Sharing The Real Answers

Hopefully I’ll get complete answers to all of these soon. I’m sure there are nooks and crannies of the QS issue that neither I nor anyone will ever get to the bottom of. But early next year I hope to start sharing the sum of what I have learned, both in writing and in special sessions at some upcoming conferences and maybe in a few webinars.

Quality score is big and important and confusing, but it really doesn’t need to be this hard for everyone. I hope I can help create at least a good amount of ‘settled law’ on many of the issues that today are the topics of lots of confusions and occasional endless debate. Stay tuned.

In the comments to the previous post on Modified Broad Match, Helena Papirnikova asked an interesting question regarding the role of match type in quality score.

This is an issue that has been clouded in confusion for some time, and I thought worth more discussion than just a comment reply. So here goes…

Match Type / Quality Scores Rumors

Many say that using more exact match within your campaigns is a way to boost quality scores. Others point out that match type has no impact at all on quality score.

Who’s right?

It turns out that the truth is somewhere in between. Quality score is only calculated when the search query is identical to the keyword. A broad match keyword like ‘dog food’ is matched to many different search queries – sometimes it’s matched to ‘dog food’ and sometimes to ‘cheap dog food’ and other times to ‘puppy chow on sale in Kansas City’.

But quality score is only calculated in those instances where the query is ‘dog food’. For all other queries, the quality score that was calculated when the query was ‘dog food’ is used.

Suppose you have a text ad that promotes low prices and free shipping. When the query ‘cheap dog food’ is matched to your ‘dog food’ broad match keyword, you actually get higher click-through-rates then you do when the query is simply ‘dog food’. But since the query ‘cheap dog food’ isn’t identical to the keyword ‘dog food’, a quality score is not calculated using this higher CTR. Instead, the quality score calculated based on the lower ‘dog food’ queries is applied.

The result is that your ad will appear less frequently, in lower positions, and at a higher CPC when the query is ‘cheap dog food’ and the keyword is ‘dog food’ then it would if you were buying the keyword ‘cheap dog food’ in either broad or phrase match type.

Match Type / Quality Score Facts

• Match type plays a role in determining which queries will be matched to which keywords
• Quality score is only calculated when the query is identical to the keyword, regardless of the keyword match type.
• The match type of a keyword is not considered and has no effect on the calculation of quality score for any keyword.
• When the search query is identical to the keyword, a quality score is calculated and applied. When a search query is not identical to the keyword (but matched anyway) the keyword will use a quality score that was calculated based on the performance of earlier searches where the query was identical to the keyword.
• If the same keyword appears in an account using different match types each should earn a nearly identical quality score. (Note: Minor differences in quality score may occur due to differences in ad copy and target URLs and the geography of searchers.)

So more exact match does not improve quality score – at all. The match type you set for any keyword is irrelevant. What matters is the keywords you choose to include in your account.

The Secret To Better Quality Scores

When you add a keyword to your account and use a broad or phrase match type, you attract queries that are related or similar to your keyword, but quality score is not calculated for these queries. When the CTR of those queries is lower than that of the identical query, you get an undeserved boost. When the CTR of those queries is higher than that of the identical query, you pay a quality score price.

The logic behind the suggestion to use more exact match is probably sound, but it suggests the wrong way to achieve the objective. You don’t need more exact match keywords, you need more keywords that are identical to the search queries that perform well (or occur frequently) in your account.

This shows the importance of intelligent keyword expansion. The match types of your keywords should be set to whatever is appropriate for each keyword – see our ‘Match Type Keyword Trap’ and ‘Modified Broad Match’ posts for more details on effectively applying match types.

Broad match keywords, particularly those which are generic or broad terminology terms, will tend to get lower CTRs on the queries that are identical to them and higher CTRs for the longer and more precise phrases to which they’ll match. Broad match helps you find more searchers but it does so inefficiently from a quality score perspective.

It’s critical to ‘query mine’ the keywords in your account (as discussed in this blog post) to find valuable search queries and turn them into new keywords. THIS is how you improve quality score, and increase both search volume and impression share while increasing ROI. (BTW, the Keyword Zoom tool in ClickEquations is the best way in the world to get this done quickly and easily.)

Negatives Don’t Matter Either

This is probably a good place to correct another common rumor. Adding negative keywords doesn’t directly impact quality score either. The reason is the same. When negatives filter out queries that weren’t identical to the purchased keyword, it has no effect because quality score was never calculated for those queries anyway.

This isn’t to say you shouldn’t add all appropriate negatives – just that doing so won’t improve your quality score on specific keywords.

There is an indirect benefit, however. Adding negatives in theory will improve your CTRs overall (by not showing ads to people who probably shouldn’t be interested in them) and these improved CTRs may be used in the way CTR is considered for account history, display URL, and specific geographies – each of which is applied in the overall quality score calculation.

Why It Works This Way

The relationship between match types, keywords, search queries, and quality score is a little confusing. Why does it work this way? Why doesn’t AdWords just calculate quality score for every query?

I can only speculate. I seems like if we earned quality scores based on the performance of all the crazy search queries that broad and even phrase is sometimes matched to then we’d be less in control of our own accounts than with the current method of only judging our quality when people are searching for exactly what we’re advertising (on a keyword level).

I also believe that broadly quality score is a tool Google uses to get advertisers to do the right thing. Expanding keyword purchases based on queries is the right thing to do – it improves the account in every way and would be a best practice even if quality score didn’t exist. Yet I expect that if fully understood the benefits to quality score will motivate many people who wouldn’t otherwise make the effort frequently enough.

Quality score is a rating of how effective you are at advertising on a specific keyword. By only making that judgment based on the results of people who searched with a query that was identical to your keyword AdWords is able to fully and fairly score your performance.

Lessons Learned

When you look at the quality score of any keyword in your account, remember that this is the quality score earned by the identical queries. For broad match and phrase match keywords, there are likely queries getting this quality score – and the resulting Ad Rank and CPC – that could do better if you turn those queries into their own new keywords. Adding an exact match version of an existing keyword won’t help. Making productive queries into independent keywords can help a lot.

UPDATE: Good discussion in comments below, and I’ve heard from Google that ‘something’ is not correct above – will update here as soon as I know what. Happy to get corrected share the facts!

Doing some research searches today (meaning just searching to see what kind of results appear) I noticed something in the AdWords ads I don’t recall ever seeing before – ads broken down by suggested alternate search queries.

My search was for ‘Amtrak Auto Train’ and the AdWords results showed a few ads for that, and then some ads ‘Related to auto transport’ and others ‘Related to amtrack statsions’ and more ‘Related to amtrak jobs’. There are several significant implications of this to AdWords advertisers.

First, Google is increasing the ad density, putting more ads on the page. While there is no way to know how they’re making these decisions, it seems like in the past they may have only shown three or four resulting ads – only those that achieved a minimum ad rank based on my query and geography – but not rather than leaving the rest of the page blank, they’re showing ads for queries I didn’t enter.

So are they filling what would have been white space, or displacing advertisers who would have otherwise shown in positions 4 through 9?

Second, if my ad is shown in an auto-suggested category, but would not have normally triggered for the actual query, I would expect a much lower CTR. But AdWords only reports the blended CTR of all impressions – not telling me that a bunch were non-targeted suggestions or experimental or whatever.

That could mislead me into rewriting text ads that were actually working.

And does the lower CTR drive down my quality score? It shouldn’t for the keyword, since quality score is only calculated when query = keyword, but what about the impact on my account CTR history, or display URL CTR history?

It’s great that AdWords does these experiments (I”ll assume for now that’s what this is).

It would be great-er if they’d issue a blanket statement saying ‘no advertiser was harmed in the performance of these experiments’.

Anyone else seeing this? What do you think it means?

One of the ways I sometimes describe quality score is as a bozo filter. It’s a mechanism that enables Google to discourage and prevent bad advertisers.

There are two kinds of bad advertisers; unintentionally bad advertisers and intentionally bad advertisers.

Unintentionally bad advertisers just don’t know what they’re doing. They jam too many keywords into ad groups, use broad category terms and phrases, write insipid copy, and send all traffic to the home page.

Quality score discourages (or instructs if you like) these nieve young advertisers with low quality scores.

Intentionally bad advertisers aren’t likely to make any of those same mistakes. They build highly targeted ad groups, use multi-word keywords, tune ad copy assiduously, and create custom landing pages.

Yet quality score whacks them too. How can this be?

Quality Score as Stick

The answer almost universally is found in the way landing pages effect quality score. If you read all the Google help files on landing page quality score – which you should – you’ll quickly discover that it’s essentially a citizenship guide.

They’re telling you everything a page and site needs to do to be good and nice and helpful. It also is good advice for most businesses looking for both conversions and long term positive brand identification and customer satisfaction.

But these tactics and techniques may not be the best way to maximize short term conversions. Hype, deception, and murkiness may actually better accomplish that. And that’s exactly what landing page quality searches for and penalizes. And it’s penalized quite heavily.

In fact, getting a poor landing page quality rating can cause many or all of your keywords to become ineligible for a huge portion of the search query auctions where they would otherwise likey rank quite highly. Or it can drop your quality score so low so fast, that the incremental cost-per-click you have to pay is quite considerable.

The other risk of being a bad guy in landing page land, is that quality score penalties based on landing pages can extend to your entire account – beyond just those keywords that were originally pointed at the poorly rated pages.

Once you get a bad reputation they begin to either decide you’ve got one of those business models they don’t want advertising or are otherwise some type of undesirable advertiser. It can be very tough to dig out of that hole.

Quality Score as Carrot

It’s a lot easier for Google to tell the bad landing pages from the not bad ones, than it is to tell the good ones from the great ones. So for the most part – almost the entire part – quality score slams those who do bad (or try to) but does very little to assist those who make great landing pages and sites.

As long as you don’t make poor landing pages, and especially deceptive or otherwise unfriendly ones, you’re almost always OK from a quality score perspective. Think of it as a pass/fail grading system.

Reading the quality score official writings doesn’t give you this impression. They make it sound like really targeted landing pages with perfecly aligned copy will actually drive quality score up. I don’t think it’s technically true, and have had highly placed people from the Google quality team confirm this.

What I think is happening in this case is Google is in this case telling you what you should do, what they want you to do, and even what is good for you to do, but over-reaching what they can actually quantify and apply.

Over time, it would certainly not be surprising if their ability to distinguish truly great landing pages from those that are just good improves. The calculations and applications of quality score continue to evolve and change. The current advice is good, the only point here is that right now if you’re not bad then you’re probably OK.

Landing Pages are About Conversion

Landing pages are an interesting element to think about in terms of AdWords because they’re the only system element that resides outside the system. Keywords, bids, match types, target URLs, and everything else exists inside their little world.

Landing pages are post-click. They’re instruments of conversion. For most advertisers Google doesn’t know if you’re clicks are converting, and since that’s the goal is really is hard for them to judge your success.

It’s good and reasonable for them to ensure that people who search on Google aren’t led into a dark alley and whacked on the head. I think that’s what landing page quality does today.

Mistaken Identity

It is worth noting that algorithmically sometimes they get this one wrong. The AdWords Help Forums are full of stories of people who claim to be good guys – not something you alway want self-assessed – and yet get poor landing page quality scores. Often it seems their pages do give the scent of badness even if it wasn’t intentional. But other times it seems clear the all knowing GooglePlex has erred. When this happens, it’s not fun, but reaching out to AdWords Support and requesting re-evaluation and perhaps some human intervention has proven helpful. Usually not as quickly as people might like, but it works. FYI.

What Do You Think?

This blog post is part of a series extending and amplifying the ideas in our free ebook ’21 Secret Truths of High-Resolution PPC’.

What they’re saying: “Everything you know about AdWords is the basics Google wanted you to know. Just enough to get you hooked. But what if there was fundamental secrets that they neglected to share? Would you want to know them? Now you can! 21 Secrets Truths is what you must read, no, act on, before your competitors do.”

- Bryan Eisenberg Conversion Expert and New York Times Best-Selling Author ’.

Download Your Copy Today
.

.