ClickEquations Blog

• Testing ‘Bid Cap’ algorithm. Nice to get warned when you’re paying too much. #
• Forgot to Tweet on Tuesday, but ClickEquations Free Trial Invitation can now be requested at http://www.clickequations.com. Give it a try! #
• Finally – Simple and Informative graphs of Impression Share metrics. The best secret Adwords numbers get even more useful. #
• ClickEquations becomes Silver Sponsor of emetrics show in DC Oct 20. http://www.emetrics.org/ #
• Just finished great interview with Webmaster Radio on Paid Search strategies and how ClickEquations will help search managers drive profits. #
• Webmaster Radio Interview posted in a few days, will tweet the link when available. #

Follow ClickEquations on Twitter. Get updates, tips, wierd thoughts, status updates, pre-release information and more.

What one piece of advice would I give to help improve a paid search campaign?

That was a question asked of our panel as SES in San Jose last week.

My answer: Make sure your brand keywords are fully segregated from all others.

Brand keywords – any keyword with your company name or variations in them – have completely different cost and performance characteristics than category or other other generic or product specific keywords.

These differences completely confuse the reporting for any campaigns and Ad-Groups if they’re co-mingled.

Separating Keywords and Queries
The first step is easy – every keyword you buy, regardless of its Match Type, should be in an Ad-Group if not a Campaign with only other keywords that contain the Brand name too.

Preferably, the brand terms are bucketed, with the ‘Pure’ Brand keywords in one group (those that represent just the name and variations itself), the navigational versions in another (www.brand.com, brand homepage, etc.) and the Brand-Plus keywords (Brand Sweatpants, Brand Coupons, etc.) in yet another, and so on.

In these brand focused Ad-Groups, you have to use Broad and Advanced match very sparingly and carefully, and eventually almost entirely eliminate them. If you leave them, you’ll get too many non-brand queries matching and diluting the intent of these highly focused Ad-Groups.

The other side of this Broad/Advanced Match coin is that you’ll also want to add your brand as a negative in all the remaining non-branded Campaigns and Ad-Groups. Otherwise the engines will match brand-inclusive queries against your non-brand targeted keywords.

This can be and feel dangerous, if you’re not completely sure that your Brand campaigns are complete, bid properly, running the full range of Match-Types (with of course the Match Type Keyword Traps fully configured and loaded.)

It’s probably a good idea to skip this step of adding the brand as negatives in the non-branded campaigns for a few days to ensure that there aren’t certain query formulations that your new Brand targeted Campaigns are missing.

Watch the query reports carefully, and add variations to the brand campaigns, and ultimately more negatives to both the brand the non-brand campaigns.

The Payoff
Immediately upon starting this process, especially if your campaigns had brand terms and lots of broad match scattered throughout, you’ll see radical shifts in your search reports.

• You may be amazed how much revenue is coming from and and how little cost is going into your pure brand campaigns. That’s the good news.
• You may be shocked at how much money and how little revenue is coming from your now-strictly-non-brand ad-groups. That’s the bad news. Or the opportunity, depending on how you look at it.

In any case, you’ll have a new level of clarity about the performance and activity in your PPC campaigns.

Coming Up
I’ll share more thoughts on the execution of full brand segregation, and the implications of the changes it makes to your reported results, in future posts. This is another one that may take 3-4 posts to just scratch the surface of.

In the meantime, questions and comments are encouraged. Are your brand terms separated into ad-groups? Does that help you better understand the way your PPC budgets are spent? What problems have you seen trying to control brand via Match Types? Any other ideas?

Just to prove that I really do know the great Avinash. And that we’re wild and crazy guys. Evidence from Google Dance 2008.

Most of the comments and analysis on the Google Quality Score updates, including my own, had mentioned the fact that the changes as described seemed to deal a death-blow to the old ‘good-ok-great’ Quality Score ranking system, but didn’t mention any replacement.

Brad Geddes apparently has the scoop that there will be ‘more transparency’ in the new system:

More visibility coming to Quality Score. The ‘OK, Great, and Poor’ will be replaced with a much more transparent system. At present, the easiest way to see many changes is to run a keyword report and sort by minimum bid high to low. With the new system, you will eventually be able to run a report and sort by Quality Score so that you can get a much better view of your keywords quality score.

Excellent. Hope they’re available in the API!

One morning in San Francisco last week, the happy-time morning folks on one of the TV networks interviewed the whole original cast of Happy Days. Howard Cunningham, Ralph Malph, Fonzie – all of them who aren’t now as rich as Ron Howard.

One question the penetrating journalist just had to ask was about the phrase ‘jumping the shark’. Fonzie and Gary Marshall were quick to point out that the show was #1 for two years after that episode.

I guess they wanted to make it clear that they don’t even understand what ‘jumping the shark‘ means.

But later that day, after the 2nd day of the XChange analytics conference, where many of the WA Gurus and a lot of very prominent Analytics customers gathered to discuss their marketplace, it hit me:

I think high end of web analytics might have jumped the shark too. The money may flow for a while longer, but there are some real problems which may be irreparable.

What I Heard At XChange

With a unique conference format – all sessions except for a brief opening event are round-table discussions between 10-20 attendees – XChange is the perfect place to find out what’s really happening. Everybody gets their say, not just a few selected presenters.

And what they’re clearly expressing is frustration. The world’s most prominent web analytics thinkers and professionals seem to have five issues:

• Data Collection - Analytics can only work if the right data is collected. Yet site tagging is hugely problematic because IT depts are slow and inflexible. Managing web analytics in this environment is like driving a race car where use of the gas and breaks requires a ‘request submission form’ that someone else will consider and implement, fully or partially, at some time of their choosing. You slam into a lot of walls this way. Of course, if you do get the site tagged, the circus that is cookies pretty much obliterates the data integrity anyway. Saying it’s the trends not the numbers only goes so far.
• Data Integration - Even if website-based tracking was perfect, the world is no longer website based. From social media to multi-channel to Flash, Flex, Ajax, Video, and Mobile, web analytics is a guard dog with a 10 mile territory and a 100-ft chain tied around its’ neck. That’s a lot of ground not covered.
• Core Capabilities - Supposed you had all the data you dream of – then you could analyze it as you wished right? Maybe not. There were no ‘I Love My Vendor’ buttons at this show – in fact the session on ‘When and How to Change Vendors’ confirmed only that the top analytics vendors have a lot in common with the airlines – everybody hates the one they use the most. The most common story was of executives wanting the cool reports and features they understood to be promised in the sales demos, and the analytics professionals having a hard time explaining why that was completely impossible.
• Competitive Environment - The party line at XChange was a professed distain for Google Analytics because it’s ‘limited and inflexible’, but they aren’t pleased with the growing lack of alternatives at the high end. Several still going concerns are assumed to be the walking dead, and the remaining green giant has a surprising lack of goodwill that would lead you to believe Microsoft had already bought them.
• Damn Customers - This is where the real trouble lies. Because of the issues listed above, analytics folks haven’t been able to educate or satisfy their customers – the managers, marketers, partners, and technical staff that need to consume the information and insights web analytics are supposed to produce. The stories clearly reveal users who want things they can’t have, don’t understand the things they get, ask for things they don’t need, don’t use the things they’re given, and remain therefore un-enlightened as to the behaviour and performance of their online assets. This is making it very hard for the analysts to tell them that what they really need is more time, more staff, and more money for new tools.

Is there success and satisfaction out there? Yes.

The most advanced of the practitioners are doing wonderful things. The smartest of them have generated huge wins from the tools they have. There are anticdotes aplenty. It’s not impossible.

But it’s not easy. Even those with clear wins aren’t living on easy street. Those without them seem nearly defeated. The barriors are just too high and too hard. The few wins are not worth the enormous costs.

It seems like high end Web Analytics is the new CRM, where companies used to spend hundreds of thousand, or even millions of dollars, only to find their sales staff secretly using ASK on their laptops.

What I Think It Means

And that’s the thought that got me. The high-end packages can out-perform Google Analytics in just about every way you can think of or discuss, except in the ease with which basic data and analysis is delivered.

Which leads to a paradox; the high end package can out perform Google Analytics only if they can be fully and properly configured, solve some very serious data integration problems, actually do most or all of what they promise, and become accessible to a very diverse set of end clients. But they’re failing at these four tasks which leaves most end-users getting only very simple reporting out of very complicated and expensive packages.

Wouldn’t they be better off just getting these simple reports from a simple and cheap (even free) package?

PostScript

I wish it weren’t true. I want the full promise of the high end. And it takes a lot to convince me that something possible is impractical.

But if the collective status of the smartest and best resourced analytics users is as it appeared at XChange, I think I just saw The Fonz water skiing in a leather jacket.

I always wonder if Frank Luntz invented the name Quality Score for Google.

It just sounds like the man behind ‘climate change’ (which was otherwise known as ‘global warming’) would call something a ‘quality score’ when it actually functions as ‘advertising tax’.

The Quality Score is Google’s way of handicapping your keywords/text-ads, in the sense of both ranking and limiting their appropriateness and therefore likelihood to run.

The idea, as Google portrays it, is that keyword/ad/landing-page combinations which are more appropriate for a given search get a higher score, and those less appropriate get a lower one. A higher score helps ads run more frequently and be positioned higher, while a lower quality score drives them to be run less frequently and positioned lower.

This of course all aligns with the idea of putting user experience of searchers first, as better ads (more relevant and ‘voted’ so by clicks) get higher quality scores.

And oh ya, the lower your quality score the more you have to pay for the chance or priviledge of running your ads at all.

This is where the prime directive gets sold out – ads with lower quality scores (to a point) can run and even rank highly if the advertiser is willing to pay enough.

In some cases quality scores were so low that a ‘Minimum Bid’ was put into place, which is the moral equivalent of saying that we have no available seating for dinner this evening, unless you can find it in your heart to slip the maitre de a Benjamin.

Beyond a certain point, however, keywords have been shut down entirely (and marked ‘inactive’ until the words, ads, landing pages, or bids were modified and re-evaluated.)

Quality score is calculated using yet-another-secret-google-algorithm, but we know it reflects the symmetry of language between the query, keyword, ad, and landing page, click-through-rate performance, load time of the landing page, and other elements.

Quality Score Revised

The way Quality Score is calculated and applied is being changed, which as just announced in a blog post entitled ‘Quality Score improvements’. Luntz would be proud.

Here’s what they say about the changes:

A more accurate Quality Score

Most importantly, we are replacing our static per-keyword Quality Scores with a system that will evaluate an ad’s quality each time it matches a search query. This way, AdWords will use the most accurate, specific, and up-to-date performance information when determining whether an ad should be displayed. Your ads will be more likely to show when they’re relevant and less likely to show when they’re not. This means that Google users are apt to see better ads while you, as an advertiser, should receive leads which are more highly qualified.

Keywords no longer marked ‘inactive for search’

The new per-query evaluation of Quality Score affects you in that keywords will no longer appear as ‘inactive for search’ in your account. Instead, all keywords will have the chance to show ads on Google web search and the search network (unless you’ve paused or deleted them). Keep in mind, however, that keywords previously marked ‘inactive for search’ are not likely to accrue a great deal of traffic following this change. This is because their combined per-query Quality Score and bid probably isn’t high enough to gain competitive placement.

‘First page bid’ will replace ‘minimum bid’

As a result of migrating to per-query Quality Score, we are no longer showing minimum bids in your account. Instead, we’re replacing minimum bids with a new, more meaningful metric: first page bids. First page bids are an estimate of the bid it would take for your ad to reach the first page of search results on Google web search. They’re based on the exact match version of the keyword, the ad’s Quality Score, and current advertiser competition on that keyword. Based on your feedback, we learned that knowing your minimum bid wasn’t always helpful in getting the ad placement you wanted, so we hope that first page bids will give you better guidance on how to achieve your advertising goals.

It’s worth mentioning that the impact of these changes will vary from advertiser to advertiser; some might see no changes to their ad serving, while others may see a noticeable difference. As always, we recommend optimizing ads to prevent them from receiving a low Quality Score.

First Impressions
The core idea of calculating Quality Score on the unique characteristics of each search instead of coming up with a single score per keyword is clearly a step in the right direction.

The dynamic nature of the new Quality Score, however, may make it a lot more challenging to know and manage the implications of your Quality Scores. They don’t say if they’ll still report Quality Score in the Adwords interface, of more importantly if they’ll make any QS rating available via the Adwords API.

By scoring independently in each situation, many keywords may suffer what will in effect be a lower impression share – getting shown far less often than their potential – but it’s not clear that this loss will be reported or visible.

We may see volume drops for certain keywords and not have any clear indication that the reason is a low Quality Score in certain situations. And it’s not clear that there will be any feedback as to which situations – certain queries, certain network sites, certain times of day or whatever – are delivering low QS which therefore will make it quite difficult to take corrective action.

Similarly, while not having keywords marked ‘Inactive for Search’ sounds positive, it may be worse to have words running at extremely low impression counts if there is not a clear indication that this is happening or that it’s due to frequently low Quality Scores in the situations where the keyword is being scored and considered.

The ‘First Page Bid’ metric at least makes the process of bribing the matre de more transparent. There’s nothing worse than either slipping someone a $20 only to have them scoff at you because a $100 was necessary, except of course passing off a $100 when $20 would have done.

Having the price of admission clearly marked will enable advertisers to make their own decisions as to value.

One issue it would be great to have Google clarify is the way Quality Score is calculated, and therefore ‘First Page Bid’ too, over the life and history of a keyword. In the past the ‘Minimum Bid’ was frequently insanely and unjustly high for new keywords added to a campaign, and would decrease rapidly as a click-history was established.

This required paying up to $10 per click for terms without any competitive bids and which would later settle at bid prices as low as $0.10. Hopefully these types of ‘hazing’ fees for new keywords won’t be included in the new system – but of course only time will tell.

The Roll-Out
The new Quality Score changes are being rolled out slowly, so you may not see these in your account immediately. There will be another post at the Adwords blog before final system-wide launch.

Do you have Quality Score concerns? Post a comment!

Update: More info on new Quality Score reporting.

At the Search Engine Strategies Conference in San Jose this week I participated on a panel entitled “Identify, Analyze, Act: SEM by the Numbers”. Below are my presentation slides, and some accompanying thoughts and comments.

Of course, it’s great to see such a high-profile discussion of PPC Analytics. I’d love to dig in and talk about very specific strategies and tactics, metrics and applications, etc.

But with limited time, and with most paid search managers up against some real barriors to applying solid analytics to their campaigns, I decided to use the opportunity to talk about these challenges in the hopes that raising visibility would encourage the PPC community to start requesting the kinds of changes we need from the engines and tools vendors and the ‘best practices’ widely discussed across the paid search community in order to make meaningful analysis easier and more commonplace.

Invisibility
The first challenge to paid search analytics is that a lot of the data you’d want to analyze isn’t readily available.

The prime example, which I’ve covered in detail before on this blog is search query information. Google Adwords and Analytics (or Yahoo or MSN) don’t provide it at a keyword level, and neither to a great many web analytics or even PPC management tools.

Similarly, SKU-level margins which enable SKU-level profit to be calculated to enable ROI to replace ROAS, is not available in most tools. That has been discussed a length too.

But if we’re trying to measure how people who search react to our ads in terms of our profitability, it’s rather amazing that a clear view into their search and our profit isn’t a default condition of analysis.

Deception
Another challenge to PPC managers is data that isn’t what it appears to be – untrustworthy data. Some of it is inaccurate, some of it is simply misleading. None of it helps.

Averages are the backbone of most paid search reports – the average ROI/ROAS for a campaign, the average cost-per-click for an adgroup, the average order value for one engine vs another.

But averages mask as much as they summarize. Averages without awareness of the standard deviation (or dispersion) of their data should be considered of limited value. Yet millions of PPC reports are consumed every day leaving impressions and causing decisions based on the ‘feel’ of these averages.

Similarly, raw performance data is presented in all kinds of PPC reports without regard for the statistical significance or margin-of-error of that data. Snap decisions to turn of text-ads, pause keywords, or inversely let them run can be based on data which would tell a very different story if reviewed slighly later in the process.

Why don’t the tools use conditional formatting or some other method to warn you that ‘there isn’t enough sample sizes to evaluate these numbers yet?’

And lastly I didn’t get the chance to speak on one last but very important point. And there isn’t enough time to go into it in detail here, but just about every revenue or profit number used to report or analyze paid search is ‘wrong’ or at least ‘suspect’ based on huge limitations in how keywords/clicks are matched with revenue over time.

Everyone knows that many people visit sites a number of times before they buy, and the question of which visit and driving method gets ‘credit’ for the sale is frequently discussed. Most search and analytics programs use the last-visit standard, although a few allow a first-visit option and some even enable the choice of a linear allocation. On the SES show floor I even learned of one package that does a ‘reverse time-decay’ allocation IF the last keyword is a brand term.

All of these have issues, but what’s more important is that they skew the performance of different types of keywords, and it’s hard to imagine that influence is fully considered when the eventual keyword and bidding decisions are made. (MUCH more about this later).

Unlimited Power and Resources
A few more challenges to taking advantage of search analytics. One is the shear size of the data we’re reviewing. Huge campaigns with hundreds of thousands of keywords, sometimes hundreds of Campaigns and Ad-Groups, and time-marches-on producing a wide range of time frames and performance trends. Plus of course the engines change, businesses have seasonality, competitors keep moving, etc.

Taking this all in, keeping the factors in mind, and driving to meaningful conclusions is not easy.

Within this world, and due to many of these factors, we’re making a near constant stream of changes to our campaigns. But are we making these changes carefully?

When changes are made, the current crop of tools don’t help us to record the time and date of the change, watch what happens over a significant number of days/clicks/conversions, and then remind us to confirm, extend, or roll-back the change. We’re conducting tests without any test plan, test scoring, or final review.

On a related note, most of the changes we’re making – adding keywords, shifting bids, modifying Match-Types, rewriting text ads – have impacts which ripple through our campaigns and certainly reduce the clarity of our reports. If we look at a two week period when 457 changes were made throughout the account in clumps throughout that time frame, what are we really learning about either what happened or what we should do next?

Lessons To Learn
As the above hopefully suggests, the core issues in being more analytical with paid search campaigns are not simply to produce and review more reports.

We have to get clarity and accuracy from our data first, apply sensible practices to the construction and management of our campaigns, and raise the bar for both the tools we demand and use and the way we understand and use the reports they provide.

We don’t need perfect solutions to any of these problems. Those aren’t coming soon and aren’t really necessary. But we should take steps to both understand and factor in these issues as we work to better learn what’s really going on within our accounts and how we can use that information to make better decisions and drive better results moving forward.

We also have to understand these issues and demand that the engines and tool vendors work towards handling them in more comprehensive and reasonable ways. Both sides have ignored these issues for too long.

The Match Type Keyword Trap
To include at least one practical element, I included a slide summarizing the several posts that live earlier in this blog about how to target queries more specifically using Match Types.

Final Thoughts
The feedback after the session was great, although I probably and characteristically bit off more than the time allowed. I hope this exposition helps those who were there and those who were not. As always, I’d be happy to answer questions or discuss it further in the comments.

The session I participated in at SES on Tuesday got a write-up by Avi Wilensky (who clearly takes notes very quickly).

It’s a good summary of my presentation and those of the other panelists. I’ll add some more thoughts and a copy of my slide deck shortly.

Update: Another from Lisa Barone at Bruce Clay Inc.

2nd Update: A full review of my presentation with the slides.

There are many amazing things about Google. The one I’ve always been the most intrigued by is their ability to manage so many projects so well at such a large scale.

We can hardly imagine the number of things going on there – big diverse programs, developments, acquisitions, global scaling issues, etc. Yet relatively speaking things seem to get done and run amazingly smoothly.

This extends to their ability to throw a party.

I spent last evening walking the famous “Google Dance” event with friend and advisor Avinash Kaushik.

If Martha Stewart threw a tech party, this would be it. There was no detail, no extravagance, no space or idea left incomplete. There were gifts, and caricature artists, and music, and food (of all kinds & everywhere) and light shows, and photo-booths, and volleyball, and on and on. With an industry full of people streaming in by the bus load.

And yet like the Google homepage it was simple, friendly, and casual.

I can’t imagine the effort that went into making this event so complex and so seemingly effortless.

It was a great event, but it inspired even more awe about what these guys are doing and will continue to do during work hours.