ClickEquations Blog

Paid Search Managers spend a lot of time analyzing clicks.

Which keywords got them? Did they convert? How much did they cost?

But how much time is spent thinking about the clicks you didn’t get? How much information do you have about those clicks anyway?

Earlier in this series I’ve discussed the idea that paid search marketers have a tough time getting a full and clear picture of what’s really going on in their accounts with the information currently provided by the engines, analytics programs, and PPC tools.

The last few posts discussed the lack of search query details as one example. Gaining insight into missing clicks is another.

Two Ways To Lose

There are two types of clicks you didn’t get. The first are those reflected in your click-through-rate; clicks that didn’t happen when your ad was shown. The count of these can be easily seen by comparing your impression count with the click count for any keyword.

The second type of missed click are those where the query was relevant (or interesting) to you but your ad wasn’t displayed. As Steve Forbert once said: (although I don’t think he was the first) you cannot win if you do not play.

Tracking Missed Impressions

Google has provided a series of Impression Share Metrics for over a year now, which provide important insights into the click missed because ad weren’t even displayed.

• Impression Share (IS): The percentage of times your ads were shown out of the total available impressions in the market you were targeting. This metric is available at the campaign and account level for search.
• Impression Share Exact Match. Impression Share Exact Match reports the impression share of your campaigns as if your keywords were set to ExactMatch.
• Lost IS. Your impression share + Lost IS (Budget) + Lost IS (Rank) = 100%.
• Lost IS (Rank): The percentage of impressions lost due to low Ad Rank (cost-per-click bid x Quality Score).
• Lost IS (Budget): The percentage of impressions lost due to budget constraints.

These are informative and critical reports. You should always know the IS numbers for your campaigns. There are times you can accept a low Impression Share, and times when you cannot.

It’s too bad it takes a trip into the reporting environment (or setting up an email report) to get them rather than having them ‘in line’ with other reporting metrics.

More importantly, this data is only available at the Campaign level, and we could really use it at the Ad-Group level. When you have a large campaign with many Ad-Groups is very possible that some have great Impression Share and a few have lousy Impression Share (or that the reasons why the number is what it is differ between Ad-Groups) and the Campaign-level roll up is of limited use.

In a future post we’ll dig deeper into the meaning and applications of these numbers.

Tracking Missed Clicks

There is less information, ironically, delivered about the clicks you miss when your ads do appear.

There are many reasons people don’t click (see this post for a good list). Many could not be translated into paid search metrics without qualitative research. But there more information that could be shared about these lost clicks.

For example, average click-through rates and various positions are known, both in absolute and relative terms. Given your position of your ads, how many more or less clicks occurred than should have been expected at that position?

And exactly how many clicks would each higher position garner, or lower position lose? This could be predicted with some degree of accuracy.

Since text-ads have their own click-through-rates, which have a massive effect on the CTR’s of keywords, another option is to look at which text-ads were displayed and calculate the number of clicks a keyword would have received if the best of them (CTR-wise) had run all the time.

So with a little work doing some calculations around the position and text-ad running for a keyword, we could start to know what our potential keyword CTR could be, if we just improved our position performance and text-ad copy.

Not Perfectly Clear

Paid search is the pursuit of clicks. The right clicks at the right price.

A clear picture of a paid search campaign would therefore tell us a lot about the clicks we got, and the clicks we didn’t get. Google’s Impression Share is a great start – it delivers actionable information and with the sub-metrics starts to break the main one apart so we can see how different factors are contributing to the remaining click opportunity.

Impression share needs to go mainstream – into the normal dynamic Adwords reports and the API.

And a comparable level of visibility should be given to the clicks we get and don’t get once our ad has been displayed.

• How much better could our CTR have been?
• How many clicks were missed because we under-performed our position?
• How many more were available at higher positions?
• How many were missed because text-ads were under-performing? (Within the text-ad itself, was it the headline or target URL that dragged us down?).
• Was there a specific competitor who took more share from us than another, over time?

These are just some of the things we should be able to know about our clicks.

I’ve suggested before that you should think of each search as a question.

Paid search ads are run in an attempt to raise your hand and deliver answers.

Today Google noted that they are monitoring over 1 trillion URLs, for use in finding the organic rankings which are delivered on search result pages. Wow.

But I was glad to see how they characterized the reason why they’re doing all this work:

As you can see, our distributed infrastructure allows applications to efficiently traverse a link graph with many trillions of connections, or quickly sort petabytes of data, just to prepare to answer the most important question: your next Google search.

In the earlier posts in this series the point was made that it’s hard to get clear and complete information on the performance of PPC accounts.

This is true, in part, because some important information is either unavailable or plays hard-to-get. Examples mentioned included search queries, information about missing clicks, and results in terms of true profitability.

This post drills down on search queries; the others will be covered in future posts.

Finding Queries
Let’s start with an assertion: It is not possible or reasonable to competently manage paid search campaigns without full access to search query details.

This isn’t a nice-to-have. It’s required.

Managing without query details is like managing a baseball team without being allowed to know what happened at the plate. Suppose you’re told who gets on base and who doesn’t, but nothing else.

How do you rate or make changes to your batting lineup without knowing who strikes out, who hits deep long fly balls miraculously caught on the warning track, or who gets hit by pitches?

The analogy may not be perfect, but the point is that choosing to add, delete, change bids, add negatives, not add negatives, or modify match type without knowing queries is a bit of blind-folded juggling.

And yet, most people do manage paid search without full and detailed query reports – they have to because the data is not available to them. With very limited exceptions, you can’t get it from the engines, in web analytics software, or even in specialized paid search management tools.

Why Search Queries Matter (the short version)

Queries are vital because they can contain insight into the desire or intent of the user. If you sell tennis racquets, for example, and buy the keyword ‘tennis racquet’ (using the standard Broad Match) then your ad might be shown to someone who wants ‘tennis racquet restringing’, or ‘New Prince V14 Tennis Racquet’ or  ‘used cheap tennis racquet’ or even someone looking for ‘tennis racquet art’.

Are each of those people relevant to you? Are the ones that are relevant equally relevant? Can you write a single text ad that speaks directly to each of those people and persuades them to click and take action?

If you know the search queries that people clicked on when triggered by the keywords you’re buying, you can answer these questions and take action to improve the targeting and results of your account.

Without knowing, you’re left without the ability to fine tune your campaign, so you waste money and miss revenue opportunities.

Where The Queries Are(n’t)

When talking about this, I’ve found that people usually have either never thought much about the difference between queries and keywords, or have the impression that they do have access to that information but don’t use it aggressively so they haven’t realized the limitations in the little bit of query data they can access.

Let’s review what search query data is available in some of the most widely used SEM analytics and reporting tools:

• In Google Adwords the Search Queries report lists queries at the ad-group level, but it does not tell you which keywords triggered which queries. And they notoriously hide a massive percentage of them in rows marked ‘Other Keywords’.
• In Google Analytics does not display search queries at all, at least by default. It can be hacked to display queries, but from what I can see in the ones I’ve used you cannot see/link the queries to specific keywords (or even bucket them into adgroups).
• In Omniture SiteCatalyst & SearchCenter offer great query support if you purchase the optional ‘db universal’ VISTA rule (typically $5K). With this enabled you gain fairly complete search query reporting and it’s a metric you can use with the full power of SiteCatalyst reporting, meaning you can use the ‘break down by’ feature to subsort by query relating it to keyword, product sold, or just about anything. You can also access it via their Excel tool in powerful ways.
• In most stand-alone paid search management tools (like Clickable, Acquisio, SearchRev, SearchIgnite, Efficient Frontier, and others), search queries do not exist. They’re completely unavailable. These tools rely on the search engine API’s for data – they don’t have their own page/URL tags – so they just can’t get query data. Which means their customers don’t get it either.

There are many other analytics and paid search tools of course, and I don’t personally know the details of many of them. (I believe Marin Software does have their own tags and can gather query data, but I don’t recall the level of reporting, for example.)

If you know the details of available or unavailable query information and reporting, please leave details in the comments.

Missing Data 1, Good Search Reporting 0

Based on this review of the popular platforms people use for paid search reporting, it seems safe to say that the vast majority – probably at least 90% and maybe as many as 98% of search managers do not have the ability to look at which queries drove clicks (and spent their money) on a keyword by keyword basis.

Imagine if your sales records only told you what categories of items you sold, not which specific items or SKU’s were sold. How would you decide on inventory re-orders or future promotional plans. You couldn’t with any level of accuracy so you’d have to just guess and play the averages.

This is what the search engines want you to do. Your inefficiency is their profit margin.

It’s hard to understand why the web analytics and focused paid search software companies place such a low priority on this vital information. I have some theories, which I’ll share in future posts.

A Fair Shot

If it’s the search query/keyword combination that triggers ads, causes your money to be spent, and dramatically clarifies the ‘why’ of who clicked and converted, why should paid search advertisers have to manage their accounts without this information?

I suggest you ask your search engine account managers, or analytics / PPC tool providers that question.

Came across a great post from Chris Zaharias at Omniture today, bemoaning Google’s Automatic Matching option, which I lampooned in an earlier post.

Chris makes two very interesting additional points (Beyond the great name “laziness tax”.)

Advertisers typically set higher daily or monthly AdWords budgets than they want or expect to spend, simply because setting a budget equal to what they actually *expect* to spend would result in Google throttling back on delivery of their ad as the advertiser approaches the budget limit. So what you have is a scenario where a strong double-digit percentage of Google’s customers currently have higher budgets set than what they expect to spend, and an automatic matching beta that wrong assumes the advertisers actually *want* to spend all that budget.

And

Keep in mind that even should you opt out of “automatic match”, you will likely still feel its effects. If the history of AdWords new feature rollouts is any indication, 30-40% of advertisers go with whatever new features Google suggests. Were 30-40% of Google’s advertisers to use “automatic match”, I’d expect the increase in coverage and competition in Google’s ad space to result in keyword inflation as bad as the rising gasoline and food costs we are now experiencing.

Two great points. The Automatic Matching Option helps Google and nobody else.

Turn it off. And complain to your Google Rep.

Yesterday I noted that paid search managers face three challenges in trying to effectively manage paid search campaigns:

• A lack of clarity (reporting problems)
• Difficulty defining priorities (strategic and planning problems)
• Horrible inefficiencies (mechanical and processes problems)

I believe that these problems need to be solved in order to improve paid search management, both the profession and the results.

First you need to see what’s happening, then you’ll want to decide what needs to be done, and then you can hopefully get it done with a reasonable amount of effort.

That doesn’t sound like too much to ask.

But 4-5-6 years into explosive growth in paid search and we’re hardly out of the starting gate. Today I’ll expand on the issues regarding reporting and clarity, and in future posts dive more deeply into the problems of setting priorities and executing paid search tasks.

What Paid Search Reports Don’t Tell You

Paid search is about answering questions. People type queries and search engines return results, which are lists of possible answers to the questions they believe are being posed. I want to structure my campaigns as tightly as possible around those search queries.

Every search engine tells you how many impressions your ads had, and how many clicks you got. They have to I suppose, since the CPC is what drives your billing. What I really want to know is what did I miss? And why? Then I can set goals and define strategies or tactics (or at least design tests) to do better.

Each conversion hopefully generates more revenue than it cost to cause that conversion, which is reflected in the rather innane ROAS metric. Being impressed with a good ROAS seems akin to believing you’ve saved money by buying something you didn’t want when it was on sale. Goods or services have costs (COGS) and the only metric that matters is ROI taking account (at least) both direct-marketing and goods/services expenses.

When my clicks do generate revenues, I’d like to know which ones. Then I can make wise decisions about future investment and effort around certain keywords and queries.

Unreasonable Demands?

So I’d like to know which search queries generated which results, how many clicks I didn’t get and why, the actual amount of profit made on each transaction (and from each keyword, query, and click).

Do any of these sound unreasonable? Far-fetched? Demanding?

Yet these desires are not generally or specifically fulfilled through the paid search reporting capabilities provided by the search engines, popular web analytics software, or even specialized PPC management tools.

Surprised? The devil is certainly in the details, and some of the information defined is available in some packages/places, but generally with huge compromises and limitations that disqualifies or invalidates them as actual or sufficient information.

Really? Yes to the best of my knowledge, as the next post will review in somewhat excruciating detail. I’m happy to learn new facts or discuss this further in the comments – significant corrections will be appended to that post.

User search queries, accurate revenue & expense allocation and matching, and ROI reporting are just three of the ways that the current generation of PPC reporting generally fail paid search advertisers and managers.

The fact that these problems/limitations are seemingly not well known, frequently discussed, and therefore clammored for as improvements is one of the things that has to change to move the business/market forward.

Managing paid search campaigns is hard.

But why?

I’ve come to the conclusion that there are three primary reasons why it’s so hard to manage paid search campaigns efficiently and effectively:

1. There is a lack of clarity. It is amazingly difficult to get accurate and complete data on campaign performance and results. Much of the data you need to see is scattered across three to five different tools and interfaces. Other data is presented in formats or based on calculations that just aren’t right. (they’re wrong.) Still other information is seemingly unavailable. There is no quick and accurate way to get reports which are satisfying.
2. It’s tough to assign priority to possible actions. This is directly related to the clarity problem in many ways, but given the size of today’s campaigns, and the information provided by both the engines and even with leading analytics and paid search tools, it’s hard to know what is the most important next step to take. There are so many choices and the functional and mathematical basis for clearly making these decisions are just not available.
3. Actual paid search management is horribly inefficient. A huge number of the things one needs to do to manage campaigns are dreadfully difficult to accomplish. Many involve potential campaign reorganizations. Some depend on keyword expansion or match type filtering. Others require bid modifications or target landing page testing. Almost all are about 100x harder to accomplish at the scale they need to be done than I wish they were or anyone has time to complete.

These three issues – clarity, priority, and efficiency – are holding back the paid search industry. Perhaps not in terms of pure industry spend – because fear is still driving a lot of rather uninformed dollars into the game – but certainly success and returns from the advertisers point of view are suffering greatly.

While I don’t think they’ve been identified or considered in quite this way, the overall feeling of being ‘out of control’ or ‘without control’ pervades the comments I’ve received in recent discussions with both practitioners and executives responsible for paid search.

When was the last time a search marketer told you how in-control of their campaigns they felt? How sure they were that both expenses and revenues were where they should be?

Does anyone feel this way?

Over the next few posts I’ll dive deeper into each of these problems, attempting to clarify the issues and try to start identifying solutions.

There’s always a lot of talk about the ‘future of search’ but it usually focuses on the searchers or the search engines. I’d like to try and think about it relative to the future of search managers and search management tools.

Another lawsuit aimed and preventing the use of trademarked keywords was dropped this week. This time it was American Airlines who had filed a lawsuit against Google for allowing other to use their name to trigger the display of competitive ads.

According to Bloomberg:

American claimed Google violated its trademark by allowing competing airlines to bid on keyword searches that generate “sponsored link” ads on search-results Web pages. The ads take advantage of the American brand’s popularity, even if the name isn’t used in the ad, the carrier said.

Google settled similar suits by other U.S. companies before the untested area of trademark law could be addressed by a judge or jury. Foreign lawsuits still pose challenges to the advertising practice, part of Google’s AdWords program.

Courts in France have held Google liable for allowing advertisers to select trademarked terms as keywords, according to U.S. regulatory filings. Google, based in Mountain View, California, said it is handling or recently resolved similar cases in Germany, Israel, Italy, Austria and Australia.

Google had argued that its “invisible” use of trademarks isn’t technically “trademark use” under U.S. law. Google compared the program to practices such as placing generic drugs next to name brands in pharmacies and buying billboard ads next to those of competitors.

This last point is the reason I’ve never understood the merits of these suits. Trademark law is designed, in my very simple understanding, to prevent one company from confusing customers with a name that is similar (or identical) to another company.

Does buying a trademarked term as a keyword provide one company benefit from the name and reputation of another? Certainly. But isn’t that why all the car dealers rent space on the same block? Doesn’t it happen when magazines review all the products in one category together?

Every company in the world wants to steal customers and prospects from their competitors. Their efforts to do so yield better features, better pricing, and loads of other consumer benefits.

Using trademark protection to limit confusion benefits consumers. Using it to try and limit consumers knowledge and awareness of competition harms consumers, and should itself be illegal. Great to see a lawsuit go the right way.

The search engines have an inventory of ad space they want you to buy. They want you to pay the highest price possible, although they obviously have to work within market conditions and create (or even deliver) value.

They operate highly complex, extremely secretive ‘auctions’ based on rules and algorithms they choose not to share – in small part for competitive reasons but mostly because doing so would degrade their revenues and the relationships they have with advertisers.

And when they make a change to the rules or algorithms under which we all choose to advertise, they inform us via blogs and notices within their software, which not surprisingly puts the changes in the best possible light.

The same is true when they simply try to explain how the system works in an effort to educate. This is fair and reasonable, to a point.

As mentioned in the earlier post on this topic, their job is to make money, yours is to be an educated and wary consumer. But there is a point where self interest and ‘marketing bluster’ turns into white-washing and deception.

How Honest Should They Be?

I think the line between marketing hype and accuracy should in part be based on the complexity of the topic, the chance that the advertiser can know or will likely be advised that the positioning is rather one-sided, and the magnitude of the cost of the deception, among other variables.

Against these measures, I think all the engines regularly go way too far in surrounding important and costly issues in friendly sounding and rather inaccurate language.

Take for example the following excerpt from a recent Yahoo blog post about what to do (and even think) when your ads aren’t displaying all the time:

If your ads are not being displayed as often as you like, it may be time to take a look at how your spending limits and bids are set. To help get your ads displayed more often, consider increasing your spending limits. If that’s not possible, there are ways to work within your means and still compete with the deep-pocket competition.

While you may not have the advertising budget to maintain round-the-clock ads in top positions, there are ways to help get your ads in front of searcher eyes more often at lower cost. First, go for specifics. Consider selecting keywords that have lower minimum bids and are targeted for a specific service or product that a searcher may specifically type in. Not only are these “tail” terms likely to be more affordable than general terms, they are less liable to blow your budget.

If you translate that happy sounding stuff into english, you find that Yahoo suggests spending more money and buying more of their unused inventory.

They do this while perpetuating the falicies that one searcher is as good as the next (regardless of the keyword they match let alone the query they type) and that higher positions are inherently better than lower ones. And they get a bonus 10 point for gratuitous use of the pop-culture phrase ‘tail terms’.

They also fail to discuss the many other ways you could likely work on improving the situation – the ones that don’t make them any more money.

Yahoo Fisked

Let’s take it line by line:

If your ads are not being displayed as often as you like, it may be time to take a look at how your spending limits and bids are set.

We start by subtly setting up what is probably a false premise, although they do it cleanly by disclaiming that ‘it may be time’.

If your ads are not displaying as often as you’d like, the first things you’d want to consider are the keyword selection itself and the associated match types, and then the reported (or likely) ‘quality index’ in terms of alignment between the keywords, ad-copy, and landing page terminology and content.

If a bike has a flat tire and rusted chain, pushing harder may work but it isn’t the best initial advice. Paying more is usually just pushing harder on a system where something is out of whack.

To help get your ads displayed more often, consider increasing your spending limits.

Again, the first suggestion is to push harder. There’s no margin in putting air in the tires.

If that’s not possible, there are ways to work within your means and still compete with the deep-pocket competition.

OK, a positive statement. You can find a way to make your business work. We’re here to help. Great.

While you may not have the advertising budget to maintain round-the-clock ads in top positions, there are ways to help get your ads in front of searcher eyes more often at lower cost.

Uh Oh. A quick drop into bad-assumption hell. Four things you may not neccessarily want to do – maintain round-the-clock ads, run them in ‘top-positions’, get your ads in front of more ‘eyes’, and even get a ‘lower cost’. It’s a nice sounding sentence, but based entirely on an over-simplified characature of how PPC works.

What we want, in the real world, is to get conversions at the highest ROI. From there we work backwards and get the best clicks (ie from the most qualified users as defined by the topic of their query and/or the history of similar conversions) at the lowest price. To do this we want the right people to see our ads, however many of them there are. Eyes are generically not valuable, cost is relative to return, and time of day and position are not ‘more is better’ things regardless of the common perception.

First, go for specifics. Consider selecting keywords that have lower minimum bids and are targeted for a specific service or product that a searcher may specifically type in.

The first piece of good advice thus far, ironically, is to ‘go for specifics’. To bad it’s passed off in such a general and ultimately misleading way. We want keywords that attract queries which are both highly relevant to our subject and if possible demonstrate intent towards serious prospects.

While we all want to have lower costs, making lower bids the (or only) criteria is silly. We want the lowest possible bids for keywords which generate conversions. If you really want lower minimum bids, leave Yahoo and go bid on a 3rd-tier engine. That’s not good advice because the traffic is of such substantially lower quality that it more than offsets the lower bid – obviously.

The idea of ‘going for specifics’ is excellent advice if it’s clear that it means increasing the targeting of your keywords (or more specifically narrowing the range of the queries you attract) and doing a better job of valuing specific keywords via the use of negatives, match types, and bids. I’m sure that’s what they meant.

Not only are these “tail” terms likely to be more affordable than general terms, they are less liable to blow your budget.

The tail wags the dog here, again. Buy cheaper words so you can stay in the game longer. Shouldn’t the idea of conversions, revenue, and return get a mention?

Are we playing at the $5 table in Vegas or Atlantic City because we know we’re going home broke but we really want to be able to play all weekend? Sure sounds like it.

Let Me Try It

This review may sounds a bit harsh and over-reactive. Yahoo is just trying to give some friendly advice, and taken with a less critical eye it’s just another generic piece of questionable advice.

But this isn’t advice from just another search blogger or ‘guru’. This is from a company getting thousands or tens of thousands of dollars from people who will take this advice to heart and move forward making bad decisions as a result.

Consider this alternative:

If your ads are not being displayed as often as you like, make sure you’re bidding on keywords directly relevant to your offer, and that your text ads and landing pages are directly aligned in both word usage and clearly focused context for those words. If they are, it may be time to take a look at how your spending limits and bids are set.

Your goal should be to see your ads displayed as often as possible to searchers who are likely to purchase your goods or services. With proper setup and management you should be able to generate a good number of sales within your means – even when competing with the deep-pocket competition.

Make sure that you’ve chosen as many different highly relevant keywords as possible – the more specific the better. Very general terms and category names usually have a lot of competitive bidders and therefore see higher per-click costs. If you want your ads to appear for these keywords it may be necessary to both raise bids and increase budgets – although a better Quality Index (a big part of which is higher click through rates that come from well written creative) can help minimize those.

More focused words and phrases are often overlooked by many bidders, despite the fact that advertisers often see higher conversion rates from more detailed keywords. So you may experience lower costs and higher revenues!

An approach like this gives more balanced information to search advertisers, helping them to become successful and therefore keep advertising. Which produces better long term results for Yahoo as well as their clients.

Sure it takes a little longer to think through and then write with more meat and less marketing fluff. And it both provides and expects more from readers – they may not undertand all the terms and concepts and will therefore have to seek out explanations.

But I think advertisers deserve and would appreciate the additional effort and information. And it would be great to not have to be so suspect of everything the engines say and do.

I needed a room tonight near the Denver airport, so for the first time took a shot on Priceline’s ‘name your price’ service.

The process worked well enough, and I probably saved $50.

But one of the questions in the purchase process caught my attention. Look what they ask you to enter as one of the possible ‘secret words’ in case you forget your password. Wow.