ClickEquations Blog

Text ads are trying to answer questions.

Writing text ads is difficult because you have only 95 characters to stand out from a sea of competing messages and persuade the searcher that you’re the ad to click.

There are many strategies and tactics to accomplish this, and both technical skill and creativity are required. The process takes considerable time and effort.

And there is only one way to measure success. Testing.

But testing requires more than simply running a couple of ads simultaneously. It requires the conditions for a fair test, a clear goal, and valid measurement and analysis. Much of what passes for text-ad testing in paid search lacks one or more of these requirements.

Let’s look a little closer at each to better understand how to properly test text ads.

Conditions For Text Ad Testing

Text ads cannot be effectively tested too early in the process of optimizing your account or ad groups.

If you haven’t yet optimized the keywords, match types, ad group organization, and negatives then the search queries coming into the ad group will be too diverse. If people are asking 25 different questions it’s impossible to compose any single answer that will satisfy all of them. If you try to test text ads too early, you won’t be able to trust the results. Maybe you’ve got a lousy text ad, or maybe the ad is just running against a lot of very untargeted or unqualified search queries.

So before even beginning to worry about text ad testing, make sure you’ve read and implemented Secret Truths #1 – #8. When the vast majority of the queries coming into an ad group are similar, you’re read to test text ads.

Of course, you have to write some text ads when you setup an ad group. And you should monitor and review their performance and make changes as necessary. But hard core text-ad testing – statistical comparisons – isn’t reasonable or necessary until the ad group has been properly constructed and intelligently optimized and – in terms of search queries – stabilized.

What You Can and Can’t Control

Another factor to consider in text-ad testing is the reality that paid search is a dynamic environment. Keywords get added, negatives expanded, bids change, competitors impact average positions, and other account modifications take place on a regular basis. There are variations in activity and results based on day of the week, week of the month, month of the year, weather, news events, sales, inventory, competitor promotions, and more.

So in the time it takes your ad group to get a sufficient number of impressions or clicks for a good test, how can you be sure that it is the ad copy that you’re really testing?

The answer is that you really can’t. There are no static environments in PPC. But all the ads in the test are subject to almost all the same environmental conditions, so many would argue these external influences don’t influence relative performance. That may be true, it may not. But you can’t control for many of these variables, so we ultimately have to accept them as a fact of life, a limitation in the system.

Whenever possible however, try to limit those changes you do control during deliberate text-ad tests. Don’t introduce new keywords or negatives or dramatically shift bids. Chance are if you find the need to make radical changes of any of these types you’d be better off making them and then restarting your tests.

Clear Text Ad Testing Goals

The goal of text ad testing is to determine which ad copy delivers the best click-through and/or conversion rates.

• Most ad testing focuses on CTR. That’s clearly the direct goal of the ad, and helps to drive up quality score.
• Conversion rate should be tracked and considered, even if CTR is the primary goal. There are many ways to incite a click, but Google gets paid for clicks while you get paid for conversions.
• The conversion-per-1000-impressions metric (CP1K) is a great way to blend these two goals and find the truly optimal ad copy. (I hope to write more about CP1K in the near future).

Statistically Valid Text Ad Testing Analysis

Assuming you have a clear goal in mind and a stable testing environment, test data becomes the next hurdle. How many impressions and clicks does a set of text ads need for a valid test?

The answer to that relatively straightforward question has eluded most PPC managers for years. I assume this is due to the fact that most of us aren’t trained mathemeticians or statisticians. (I’m certainly not.) And most of the software we use to create and edit text ads does not provided the statistical support we really need.

So we’ve slithered forward based on the conventional wisdom that suggests tracking ‘at least 100 impressions or 10 clicks before there is enough data to declare a winner’. Unfortunately this really isn’t very accruate or useful advice.

Statistically, it turns out that those of us who’ve been reacting to text-ads with anything near 100 impressions or a dozen or so clicks have regularly made essentially random decisions. We’ve paused the better ad many times, letting the loser run. We’ve sabataged our own results. Repeatedly. Over long periods of time.

Consider the example shown at right: Three text ads running in an ad group. About 500 impressions each.

Is there enough data to make a wise decision?

It seems pretty clear. The first ad at 1.98% CTR appears to be our winner. But the statistics tell us that it isn’t that clearcut.

I looked at the statistical significance and confidence intervals for these ads. We can only be 80% confident that the CTR difference between the first and second ad are actually different. Same for the difference between the second and third ad.

80% confidence is not very high.

It’s not considered high enough to be sure something is true in most activities where statistical confidence is considered. For scientific activities a 95% rate is the desired standard.

To understand the potential error in accepting these numbers, look (below) at the range of possible CTRs for each of these ads that we can be sure of with a 95% confidence.

The first ad may actually be as low as 0.82% CTR, or could be as high as 3.14%. That’s a pretty wide range – we just don’t know yet, with a high level of confidence, what the CTR of this ad is going to be. You can see similarly wide ranges for the other two ads, and in comparison see there is plenty of overlap in the potential which means if we really let this test play out, we may get a very different result.

So how many impressions would it take to get 95% confident in the differences?

If we let these ads run until they had around 1000 impressions each they’d achieve a 90% confidence. It takes nearly 1500 impressions per ad to hit 95% confidence.

The actual number needed for any given set of ads depends a lot on the CTRs and their relative difference. But it’s a rare circumstance when anything like 100 impressions or 10 clicks is adequate.

You can check the numbers on your own ads using two great tools:

• Vertster.com offers a simple, free, online utility that lets you enter CTRs for two ads and check the confidence level.
• Teascalc is an Excel sheet that costs $49 but offers a both confidence and interval data.

Making The Grade

Everything we do to create and optimize paid search accounts is done in hopes of showing the right people the right ad at the right price.

Their reaction to our ads is feedback on how well we’ve done at targeting them and organizing our accounts as well as on how aligned our answers are to their questions.

Fortunately for us if we do things right – in setting goals, creating testable conditions, and accurately measuring and analyzing we can get this feedback in clear, powerful, and actionable form.

Text ad testing isn’t just another wise and important step in paid search management. It’s the crucial step that pays off all the others.

This blog post is part of a series extending and amplifying the ideas in our free ebook ’21 Secret Truths of High-Resolution PPC’.

What they’re saying: “Everything you know about AdWords is the basics Google wanted you to know. Just enough to get you hooked. But what if there was fundamental secrets that they neglected to share? Would you want to know them? Now you can! 21 Secrets Truths is what you must read, no, act on, before your competitors do.”

- Bryan Eisenberg Conversion Expert and New York Times Best-Selling Author ’.

Download Your Copy Today
.

.

A lot of people who come to ClickEquations have never used dedicated paid search management platform before. They recognize that the right paid search platform can give you a competitive edge – you can spend more time optimizing your paid search programs and less time dealing with repetitive issues.

But, the buying process can be often be overwhelming. It’s easy to get caught up in long spreadsheets of feature lists. If that sounds like you, you might want to attend PPC Summit’s free webinar, “The Top Three Mistakes Marketers Make When Choosing a Bid Management Tool” this Wednesday, 2 EST / 11 pst.

You’ll get glimpse inside PPC Summit’s 2010 PPC Management and Optimization Report, including:

• The one thing to help you make smarter buying decisions
• The common practice that leads to buyer remorse and what to do about it
• Where paid search is heading and why that matters to your business
• Important lessons you don’t have to learn the hard way

We’re sponsoring the webinar, but the report is not sponsored and is vendor neutral. It’s much more about how to approach the process of choosing a platform more effectively.

Register Now!

Tomorrow June 8th @ SMX Advanced in Seattle I’m digging deep into AdWords Quality Score in the 10AM Session. But I’m not going to have time to cover the issue of what to do with poor performers. This post offers some thoughts on that topic, as an addendum offered in advance. I’ll post some version of the entire presentation online next week.

In the dark ages of AdWords, (before quality score) you couldn’t just bid on any old keyword. There was a minimum CTR requirement. When a new keyword was added to your account, Google gave you about 1000 impressions to prove that you could earn a click-through rate of at least 0.05%. If you didn’t meet or exceed that CTR level the word was paused. Game over.

Yes, they did allow you to try to improve by writing a new text ad, or editing your bid to test a higher position. But after another 1000 impressions or so, if one-half of one-percent of the users didn’t click, the keyword was shut down again.

The Age of Quality Enlightenment

In the AQ era (after quality score) things are more complex. Poor performing keywords are sometimes denied all impressions, but more often they’re pushed down in position and generally shown less frequently but still shown occasionally.

More importantly, you are allowed to compensate for bad quality with high (or extra-high) bids, and still get your ads shown regardless of performance.

Protection From Yourself

There are many ways to look at this change. Advertisers didn’t like being denied the ability and opportunity to run ads in the rather abrupt way of the old .05% CTR threshold. It wasn’t entirely fair – obviously there is not one ‘good’ CTR for the many categories and business – and it didn’t recognize the different goals and success thresholds of different advertisers.

But the willingness and even bravery of Google to deny advertisers the ability to advertise should be considered.

They did it to protect user experience – if you couldn’t satisfy or at least interest that tiny percentage of the people that you’re targeting, it does pretty clearly suggest that your ads are disinteresting to a whole lot of people.

I think they also did it to stop advertisers from wasting good money after bad, and ultimately having a poor experience themselves. If some of your keywords perform and make money, you keep those and wish you could find more. But if they allowed you to aimlessly run poorly performing ads, at some point it’s likely that you (or whomever is writing your checks) decides that this channel really isn’t working and cuts off all funding.

This creation of scarcity – only a limited number of keywords work for you – leaves you willing to bid up those remaining keywords to maximize volume, and builds a desire to work harder to find additional keywords that do perform adquately. But in this world they have to perform or they’ll be turned off.

That was a clear signal, and it seems a lot of advertisers needed it.

The Freedom To Waste Money Endlessly

Today, there is a line below which your ads are ‘not showing’ because your advertising is failing on that keyword. It’s ostensibly based on quality score, but we all know that quality score is just a fancy way of saying click-through-rate. But it’s a more complicated calculation and is highly customized to the keyword – it’s clearly advanced from the old 0.05% and you’re out days.

But the line is far lower down the performance spectrum. We’re talking quality scores of 1, 2, and maybe 3 here. These are hideously low CTRs or keywords with terrible relevance.

The everyday bad performers are allowed to keep running. Keywords where something is very clearly wrong: those with quality scores of 3, 4, 5, (and even long-standing 6′s). Keywords where you are clearly and plainly underperforming other advertisers. Keywords where your ad copy is not compelling, your offer is not relevant to very many searchers, or something is just wrong.

By keeping these keywords running you’re wasting a lot of money. You’re over-paying on a per-click basis for the right to keep these stinkers in the game. And you’re lowering your account CTR history to the detriment of all your good performing keywords.

Google lets you pay up and keep spending. You’ll get less impressions per keyword, but with broad or phrase match they’ll find some crazy queries to match you to. You’ll get some clicks and spend spend spend.

But how many keywords with quality scores below 7 have ROI’s above 100%? Very very few.

So Why Do It?

Wouldn’t it be better to turn those keywords off. You tried. It didn’t work. Cut your losses and move on.

What is it you expect to change or improve over time?

I can think of only three valid reasons to let keywords with quality scores below 7 keep running:

• Profitable. It happens. If you’re making money then more power to you. Let ‘em run.
  .
  
• Rehab. If you’re really working on them, testing new creative, removing any relevance or landing page warnings, refining keywords and negatives and match types to find a winning combination – then by all means keep working while improvement is possible.
  .
• High Cost Low Conversion. As discussed in this earlier post, there are situations, often in B2B primarily, where it makes more sense to focus on conversion rates than CTRs. Managing PPC in this case plays be a different set of rules.

If you really can’t muster the willpower or courage to turn off failing keywords when one of these aren’t true, you really should consider opening a second AdWords account and move them there. At least that way it’s easy to see and measure the cost of this decision, and more importantly the collateral damage of poor lifetime CTR is avoided in your main – and hopefully moneymaking – main AdWords account.

We’ll be in Seattle this coming week in full force.

The ClickEquations booth will be on the show floor, we’re speaking in several sessions, introducing at others, and hosting birds-of-a-feather lunch tables. We might even go out at night and have a few drinks. You just won’t be able to miss us if you’re there.

The sessions where we’re speaking:

• Tuesday, 10AM – Quality Score: I’ve been allowed to bust-out of the traditional 15 minute format, and really go wild to dive deep into the mysteries of Quality Score. We’ll cover the facts, myths, conspiracies, and even maybe have an audience quiz with prizes!
• Tuesday, 3PM – Text Ad Testing: Alex Cohen will entertain and inform on the text ad panel. Alex blew out the doors in London so expect a great presentation.

At the ClickEquations booth, we’ll be offering demos of our platform, and offering a sneak-peak at some truly amazing new features that will be made public later this summer. Stop by and ask to see the secret stuff!

We hope to see you in Seattle!

We had a chance to sit down with James Green of Siftable to chat about ClickEquations, the state of the search market and the search marketing problems with think software is uniquely capable of solving.

James profiled our software in his platform review and our built-in Excel tool, ClickEquations Analyst:

ClickEquations had a separate Excel plug-in, which is included in the cost, called ClickEquations Analyst that is used for reporting. All I can say about this is – “Holy Crap”. I’m not much for urban slang but this piece was off the hook. Essentially what they do is have a set of canned reports (they’ll create custom reports to your business upon request) with some customization which is pretty cool but where this excels (pun intended) is in their integration with Excel.

It’s one of our favorite pieces too

James and I sat down for lengthier talk about search marketing in general and the search tool market:

We, as the advertising community, need to demand from Google transparency and control. They’re taking our money and therefore, we have a say. They’ve spent all the money on the marketplace that we profit from, but they’re also profiting from our participation in that marketplace.

We, as the advertising community, constantly need to be pushing the boundaries with Google and constantly need to be pressuring them to give us the level of transparency and control that we want. I think all the tools in our market, including ClickEquations, help because they push Google to align themselves better with our business goals. If Google was offering all these things we were doing, there wouldn’t be a market for tools like ours, but they’re not because the way we serve the advertising market and community isn’t necessarily what serves Google’s purposes.

Read Siftable’s ClickEquations review and interview with me. Thanks James!

If you’d like to interview anyone at ClickEquations, please email me, acohen @ clickequations.com