Understanding Every Searcher's Intent

Advanced Keyword Management Technique Using The Search Terms Report

The search terms report in AdWords should probably be the first thing you check when you are trying to better understand your account health or looking to make it more efficient. Keyword management is an art of balancing reach and cost efficiency - if you have too many broad keywords, you could be paying for clicks that are driving little or no value to the campaign goal. On the other hand, if you have too few exact keywords, you could be falling short of the target number of conversions. To find the sweet spot, the search terms report should be periodically used to add keywords and block irrelevant searches. 

Sometimes, it can get too much. It's too time-consuming to check search terms every day, but checking it too infrequently, you face hundreds and thousands of search terms with no way of going through them all.

Since I don't check search terms frequently, I came up with a technique to work with a large search terms report. This post is for those whom, like myself, have scratched their head wondering if there's any way to comb through rows and rows of search terms and understand every searcher's intent.

Before I get to the step-by-step guide, take a step back and imagine looking at a keyword and the search query that triggered it. As you look at the association, how do you determine if the keyword did its job right being triggered for that particular search term? The answer is in intent.

Pamella Neely at SEMrush did an excellent job explaining how you can spot intent in each search term. “Each search query is not just empty words,” she says. “It holds an elusive but critical intent behind it.”

The key is finding the word within a search term that signal intent, because the intent is what determines if someone will convert or not. Looking for the intent keyword in each line is a lot easier than evaluating every word. Let’s see what I mean with an example below.

blue hats for free (4)

free yellow hats (3)

green hats free giveaway (4)

buy one get one free red hat (7)

free coupons for yellow hats (5)

There are 23 words in the five search terms and free appears in every line. What if the search terms were dissected and the words were counted? Free would be at the top of the list.

While all the search terms may point in the same direction - I want free stuff - because the search terms have different modifiers and order, they appear in separate lines. This is the biggest difficulty working with a large volume of search terms; there are countless ways to how people search. Although this example lists similar search terms together, in an actual search terms report, they could be anywhere in a list of thousands of rows. Also, if we don’t know what we are looking for, how are we going know where it is?

Here’s a guide on how you can quickly find these intent keywords in a large search terms report.

Step 1: Download a search terms report in the Keywords Tab.

Make sure the date range is wide enough for a report to have a large volume. Otherwise, it’s more efficient to work in the platform and make changes directly.

Step 2: Copy the search terms and paste them on to a single column in a new tab

TIP: Click the column heading to select the entire column.

Step 3: Select all rows below the heading

Select A2 and follow the instruction on the image to select all occupied cells in Column A.

Step 4: Separate every word in all of the search terms

Go to the Data tab and click on Text to Columns. Select Delimited and then “Space”. This indicates that white spaces are where splits will occur.

Step 5: Use a macro to put all the words into a single column

We are going to use a macro that moves all delimited words to right below the last occupied cell in Column A. If you don’t see the Developer Tab, follow this guide to enable it.

Copy and paste the following macro (Source: MrExcel):

Sub test()

Dim LR As Long, i As Long

For i = 2 To 4

   LR = Cells(Rows.Count, i).End(xlUp).Row

   Range(Cells(1, i), Cells(LR, i)).Copy Destination:=Cells(Rows.Count, 1).End(xlUp).Offset(1)

Next i

Columns("B:Z").Delete

End Sub

Make sure the macro is pasted only on the sheet that has the search terms.

Save the file in the Excel Macro-Enabled Workbook format (*.xlsm).

Step 6: Create a pivot table

Make sure the range includes the entire column (A:A). Filter out (blank), numbers and common words like prepositions in the pivot table and sort the table in descending order from largest to smallest.

And there you have it. This table lists every word in the search terms report separately. When you consider that it’s only one or two words that signal the intent of a search, you can really see the value of arranging all the words like this.

Personally, I’ve used this method to add negative keywords more often that I’ve used it to add keywords. Because I can set the match type to be phrase, adding just a couple of irrelevant negative keywords ensures that I’m blocking all possible variations that include those words.

Conclusion

Users vary a lot in how they search and it can get very overwhelming trying to digest every search term in its entirety. This method will help you find just handful of overarching words that are roots of many different variations. Of course, some queries could lose their meaning when dissected this way and you shouldn’t use this every time you look at a search terms report. Think of the method as an occasional in-depth review.

Keywords in paid search platforms are probably the most important element in your campaign. Think of it this way, even if you have not so good ad copies and a landing page, having all the right keywords could get you some conversions; conversion rate may not be high, but it will still be higher than 0. If you have zero relevant keywords, but the best ad copies and landing page, you won’t get any conversion. This is how important keywords are. Search funnels, customer’s online journeys varying in lengths and complexity, all start with a search. Keyword selection is like picking the paths that you think your customers will travel. It’s important to frequently check that you are on the right paths.

Avoiding Script Timeouts - Part 3

If you've done absolutely all you can to speed up your scripts but they still time out now and again it may not matter.  Depending upon what your script does, if it completes 90% or so of the required actions on a regular basis that may be enough but there's a couple of things you can do to make the best of this situation even if you can't solve the speed issues.

Use a sort order

If you're working to change Keyword positions or optimise for performance it's a good idea to use a sort order to ensure the most important elements are processed first in the script.  In this way if the script does time out you should have hit the bulk of these most important elements already.  For example, if you're optimising for performance you'll likely want to sort by Cost descending so that the elements with the biggest spend get optimised first.  You can do this simply in your selector using the OrderBy method:

var campSelector = AdWordsApp.campaigns()
  .OrderBy("Cost DESC")
  .get();

You can order by any of the same columns you can use in the Condition methods so for example, CTR (note, of course, that in scripts this is written as "Ctr"), AveragePosition, Impressions and so on.

Once you're ordering by some metric, if you want to cover your whole set of elements even if the script times out, you can consider changing this sort order every now and again.  So, for example, you could change the sort order to ascending just for Monday and Thursday, as follows:

var d = new Date();
var theDay = d.getDay();
// day "0" is Sunday
var sortOrder = (theDay == 1 || theDay == 4)? "ASC" : "DESC";
var campSelector = AdWordsApp.campaigns()
  .OrderBy("Cost " + sortOrder)
  .get();

In this way, twice a week your script will run in the "reverse" direction so you should pick up any elements that are frequently missed when it times out before reaching them.  You can, of course, change how often you do this just by editing the day choices.

Check the time

Time outs can be most annoying when the last thing the script does is write results to a spreadsheet, send an email or perform some other final action.  If the script times out this final action won't happen so you could end up with incomplete data or lacking notifications.  Fortunately, there's a solution to this too.

The AdWordsApp object has a useful method called "getExecutionInfo()" that has a method itself called "getRemainingTime()".  Using these methods you can check how long there is to run (in seconds) before the script times out so if you check this figure frequently you can anticipate a time out and force the final action to happen early.  For example:

if(AdWordsApp.getExecutionInfo().getRemainingTime < 45) {

  // code to exit looping and jump to final action

}

So if there's less than 45 seconds remaining, you can jump out of any loops and force the final action.  How you do this will depend upon the coding but it's usually fairly simple.

Whether a log of an incomplete run is useful is of course dependent upon what you're doing, but even a partial report is often more useful than nothing at all (and you could adjust the reporting to show that it's partial).

I hope you find these tips useful!

Avoiding Script Timeouts - Part 2

So, you've done your very best, but that wonderful script that's going to make your life so much easier and your clients so much happier is still timing out every time it runs.  What can you do?

If you're already selecting only the elements you need and your script is as fast as it can be, there's really only one option to allow scripting to process the entire intended scope - you need to split the process up into smaller chunks.  At the time of writing, AdWords Scripts can run as often as once per hour so there is the opportunity to run the same script up to 24 times per day or 12 hours of run time.  If you can't address your entire scope in 12 hours you need to go back to the drawing board!

So, here are some options for splitting up your intended scope:

#1 - Use Labels

Labels are really useful in scripting since they're a dead easy and quick way to identify elements you want to be addressed by the script.  Let's say you have 20 Campaigns you want to process and your script is timing out before it's finished the 7th Campaign so you're guessing you need to run the script 4 times to be absolutely sure.  Simply apply four Labels, for example "Block_1", "Block_2", etc. to the Campaigns in blocks of 5 (or as appropriate so they have roughly the same "load") then use a condition in your Campaign selector as follows:

.withCondition("LabelNames CONTAINS_ANY ['Block_1']")

When the script runs it will only process the Campaigns with the Label "Block_1".  Run the script 3 more times using the appropriate Labels and you're done.

I'll describe how to do this automatically later, but let's move on to another option.

#2 - Set limits

If you're addressing a large Account it's likely you've got a spread of performance metrics across the elements you're looking to process so you could use conditions to split the scripts up, for example:

.withCondition("Impressions > 1000")

(don't forget you'll need a DateRange condition as well).  This option can be useful where the time that the script runs is important.  You may want to ensure your biggest spenders, for example, are processed first in a day and this becomes increasingly important the more runs you need to make.  If, for example, you need the script to run 10 times before it addresses the whole scope, even if you start at 1am you'll still be into the working day before the last elements are processed so using this sort of condition can ensure the most important ones are done first.

#3 - Use Labels (again)

Another use of Labels is in marking elements are "processed".  So, if you were running your script against Keywords you could apply the Label "Processed" to each Keyword you complete and use a selector with the line:

.withCondition("LabelNames CONTAINS_NONE ['Processed']")

in this way, each time the script runs it'll only work on the Keywords not already processed.  The only real downsides to this method are that you need to use another script to remove these processed Labels before each new "day" of script running and unless you also label Campaigns/Ad Groups your script could be inefficient as it searches Campaigns where all the Keywords are already done.

OK, so there's three options for splitting, but how do you get the script to run more than once?  You could use a number of individual scripts pre-set with their conditions and running at different hours, but that's not very efficient and makes it harder to edit the script (since you have to make sure you make exactly the same edits to each one).  I prefer to use systems based upon the hour of the day and just have the same script run every hour.

For example, the following script snippet will return the hour of the day (in your Account time zone):

var theHour = new Date(Utilities.formatDate(new Date(), AdWordsApp.currentAccount().getTimeZone(), 'MMM dd,yyyy HH:mm:ss')).getHours();

If you've used a Label naming convention like the one I have above, you can then build your required Label name using the following:

var labelName = "Block_" + theHour;

So, in your script, you might have the following:

function main() {
  var theHour = new Date(Utilities.formatDate(new Date(),   AdWordsApp.currentAccount().getTimeZone(), 'MMM dd,yyyy HH:mm:ss')).getHours();
  if(theHour > 0 && theHour < 5) {
    var labelName = "Block_" + theHour;
    doProcess(labelName);
  }
}  

function doProcess(labelName) {
  var campIter = AdWordsApp.campaigns()
    .withCondition("LabelNames CONTAINS_ANY ['" + labelName + "']")
    .get();
}

Set that script to run every hour and it will only process the Account in the scripts that run at 1am, 2am, 3am and 4am and will automatically run on the Campaigns with the appropriate Labels.

You can extend this use of Labels and hours to many other situations so go have fun!

In the next post I'll look at some options that'll make the best use of a script you just can't stop timing out.

Avoiding Script Timeouts - Part 1

Scripts are possibly the most useful tool for managing larger Accounts.  Scripts can apply complex processes automatically and reliably across a large number of AdWords elements and allow you to concentrate on what you do best - marketing.  However, ironically, when the Account is very large and/or the script complex, you may find your marvellous script consistently "times out".  At the time of writing, scripts can only run for 30 minutes before automatically terminating; this obviously means there's the potential for an unknown percentage of your AdWords elements to remain unprocessed but it also often means important logging information is lost (as "final" logs are never reached).

This post will hopefully give you some pointers to ensure your scripts run as quickly as possible and the next will give you some options for dealing with scripts that simply cannot be made to run under 30 minutes.  So, let's start with the tips:

#1 - Good structure

All programming benefits from a good structure and with AdWords scripts the structure can be particularly important in terms of run time.  Most scripts will use some form of selector to fetch AdWords elements (Campaigns, Groups, Keywords, etc.) and then process them in some way but how you fetch these elements can have a big impact on how fast a script runs.  Imagine each selector is like sending an office minion out of the building to get something from a local store; the more often you send that person out the more time it's going to take to complete your task.  So, for example, if you want to work on Keywords, rather than selecting Campaigns, then iterating them to select Ad Groups, then iterating those to select Keywords, look for a method to select the Keywords you need in one selector.  Labels are an excellent way to do this (and you could use another script to apply and/or remove the Labels, of course, running before your main one).

Don't forget things like Campaign and Ad Group names are accessible from Keyword objects so you don't need to iterate Campaigns just to retrieve their names for reporting.

As a rule of thumb, your selector should, as far as possible, select the objects you want to work upon directly and if you're not doing that, you should think about how you could.

#2 - Only select what you need

Make sure your selectors use conditions that ensure they return only the elements you need to work upon.  One very common bad practice (in all programming/scripting) is to fetch a large dataset and then examine each returned element for suitability, discarding those that don't fit.  For example, you may select all the Keywords in a Campaign, then use conditional statements to check whether those Keywords are enabled or if they have impressions, etc.  It is much better practice (and far faster) to include these conditions in the original selector so if you want to work only on Keywords that are enabled, ensure you include these conditions in your selector:

.withCondition("Status = 'ENABLED'")
.withCondition("CampaignStatus = 'ENABLED'")
.withCondition("AdGroupStatus = 'ENABLED'")

If you want to check for clicks or impressions, you can do so, as long as you include a time range in the selector, e.g.

.withCondition("Impressions > 0")
.forDateRange("LAST_30_DAYS")

The same thinking can be applied to all other elements you might select; try to use the selector itself to define your returned objects rather than grabbing a huge bundle then spending ages throwing most of it away.

#3 - Use AdWords API reports

The many prebuilt reports in the API are much faster at retrieving AdWords elements than the normal selectors so where possible, use these reports.  The process of using them within scripts is quite different from a selector but there are plenty of examples on the web and once you've used one, the process is similar for all.  These reports aren't appropriate for all script uses but if you can use one, you should.  See here for a list of available reports.

https://developers.google.com/adwords/api/docs/appendix/reports

#4 - Watch your Logging

Logging is, of course, a vital part of scripting, especially in the development/testing stages but logging has a surprisingly big impact on run time, particularly when the logs are frequent and within a loop.  Try to keep your logging to what is absolutely necessary and once you're sure the script is working correctly, consider whether you can remove any.

#5 - Keep loops clean

Avoid including statements within loops that don't need to be calculated in each iteration.  For example, if you wanted to compare each Keyword's CTR against the average for an entire Campaign, you only need to calculate the Campaign average once, don't include that calculation inside the Keyword iterator.  The script within  a loop should be only that needed for the loop to operate correctly.

#6 - Check for "bloat"

It's very easy when writing script to check for certain conditions (average position vs ROAS or CTR vs number of conversions, etc.) to end up with several conditional statements.  It's just as easy for the final version to end up with redundant conditions.  Make sure all the tests/conditions you apply are unique and where they are, consider whether there's a quicker way to write the same set of conditions.

There are other tips for speeding up scripts that can be learned from the many online pages that teach speed improvement for JavaScript and while some can't be applied to AdWords scripts, most can and I'd recommend some quality time with Google search.  You might try starting with a search on "improve javascript speed".

In the next post I'll look at what you can do if despite your best efforts the script still times out.  Good luck!

Shopping Promo Rotation

As we all know, Shopping Campaign Ads can - and should! - contain a short promotional text line to enhance their appeal.  You can read up on it here.  Promotional text can have a big impact on the performance of your Shopping Ads but how do you test different versions?  Although Shopping Ad Groups allow you to create more than one Ad (and therefore more than one version of the promo text) there's currently no option to rotate these Ads or optimise them for conversions or clicks.  I recently wanted to test various texts and came up with this solution, using Scripts, of course.

What does it do?
In this simple form the script attempts to replicate the "rotate evenly" setting available in regular Campaigns.  Given any number of Ads in the Group, when the script runs it pauses any currently enable Ad then enables the one with the least number of impressions "today".  Run once an hour, this should effectively rotate all your Shopping Ads throughout the day allowing you to examine their performance over a reasonable period of time.

Since the Ad chosen to be enabled is always the one with the least impressions, in theory the Ads should rotate in such a way that the same Ad isn't always shown at the same hour of the day, making the test a little more "even", though you could always experiment with programming in another randomising factor.

How do I choose the best Ad?
Once (if) you see a clear "winner" you can either simply remove the script schedule and ensure that winner is enabled, or you can remove the other Ads, it's up to you.

The code
Copy and paste the code below into a new Script, give it an appropriate name, authorise it then PREVIEW the script to check it works as expected.

Note that as provided, the Script will operate on all Shopping Campaigns in the Account.  If you wish to limit which Campaigns it operates upon, you can enable either the name choice condition or the Label choice condition (by removing the // at the start of these lines).  Note also that the script expects there to be at least two Ads in each Campaign.

Ideally I'd suggest scheduling the script to run every hour but you could also try just once a day or even once a week.  Experiment!

As always, the script is provided with no guarantees and you should preview and test it thoroughly before use.

UPDATE: The code below now checks for multiple Ad Groups and uses only those enabled.

UPDATE: Now always rotates to a new Ad, regardless of impressions count of Ad currently showing.

// Shopping Ad Rotation Script (c) Jon Gritton 2015

function main() {
  var sCamp = AdWordsApp.shoppingCampaigns()
    // optional conditions to pick specific Campaigns
    //.withCondition("Name CONTAINS 'XXX'") // use to pick by name
    //.withCondition("LabelNames CONTAINS_ANY ['ShopAdTest']") // use to pick by Label
    .withCondition("Status = 'ENABLED'")
    .get();
  
  while (sCamp.hasNext()) {
    var thisCamp = sCamp.next();
    Logger.log(thisCamp.getName());
    var pAdGs = thisCamp.adGroups()
      .withCondition("Status = 'ENABLED'")
      .get();
  
    while (pAdGs.hasNext()) {
      var thisAdG = pAdGs.next();
      var pAds = thisAdG.ads()
        .withCondition("Status = 'ENABLED'")
        .get();
      while(pAds.hasNext()) {
        var currAd = pAds.next();
      }
      
      var eAd = thisAdG.ads()
      .withCondition("Status = 'PAUSED'")
      .orderBy("Impressions ASC")
      .forDateRange("TODAY")
      .withLimit(1)
      .get().next();
      
      eAd.enable();
      currAd.pause();
      Logger.log("Enabled: " + eAd.getId() + ", Paused: " + currAd.getId());
    }
  }
}