Avoiding Script Timeouts - Part 3

If you've done absolutely all you can to speed up your scripts but they still time out now and again it may not matter.  Depending upon what your script does, if it completes 90% or so of the required actions on a regular basis that may be enough but there's a couple of things you can do to make the best of this situation even if you can't solve the speed issues.

Use a sort order

If you're working to change Keyword positions or optimise for performance it's a good idea to use a sort order to ensure the most important elements are processed first in the script.  In this way if the script does time out you should have hit the bulk of these most important elements already.  For example, if you're optimising for performance you'll likely want to sort by Cost descending so that the elements with the biggest spend get optimised first.  You can do this simply in your selector using the OrderBy method:

var campSelector = AdWordsApp.campaigns()
  .OrderBy("Cost DESC")
  .get();

You can order by any of the same columns you can use in the Condition methods so for example, CTR (note, of course, that in scripts this is written as "Ctr"), AveragePosition, Impressions and so on.

Once you're ordering by some metric, if you want to cover your whole set of elements even if the script times out, you can consider changing this sort order every now and again.  So, for example, you could change the sort order to ascending just for Monday and Thursday, as follows:

var d = new Date();
var theDay = d.getDay();
// day "0" is Sunday
var sortOrder = (theDay == 1 || theDay == 4)? "ASC" : "DESC";
var campSelector = AdWordsApp.campaigns()
  .OrderBy("Cost " + sortOrder)
  .get();

In this way, twice a week your script will run in the "reverse" direction so you should pick up any elements that are frequently missed when it times out before reaching them.  You can, of course, change how often you do this just by editing the day choices.

Check the time

Time outs can be most annoying when the last thing the script does is write results to a spreadsheet, send an email or perform some other final action.  If the script times out this final action won't happen so you could end up with incomplete data or lacking notifications.  Fortunately, there's a solution to this too.

The AdWordsApp object has a useful method called "getExecutionInfo()" that has a method itself called "getRemainingTime()".  Using these methods you can check how long there is to run (in seconds) before the script times out so if you check this figure frequently you can anticipate a time out and force the final action to happen early.  For example:

if(AdWordsApp.getExecutionInfo().getRemainingTime < 45) {

  // code to exit looping and jump to final action

}

So if there's less than 45 seconds remaining, you can jump out of any loops and force the final action.  How you do this will depend upon the coding but it's usually fairly simple.

Whether a log of an incomplete run is useful is of course dependent upon what you're doing, but even a partial report is often more useful than nothing at all (and you could adjust the reporting to show that it's partial).

I hope you find these tips useful!

Avoiding Script Timeouts - Part 2

So, you've done your very best, but that wonderful script that's going to make your life so much easier and your clients so much happier is still timing out every time it runs.  What can you do?

If you're already selecting only the elements you need and your script is as fast as it can be, there's really only one option to allow scripting to process the entire intended scope - you need to split the process up into smaller chunks.  At the time of writing, AdWords Scripts can run as often as once per hour so there is the opportunity to run the same script up to 24 times per day or 12 hours of run time.  If you can't address your entire scope in 12 hours you need to go back to the drawing board!

So, here are some options for splitting up your intended scope:

#1 - Use Labels

Labels are really useful in scripting since they're a dead easy and quick way to identify elements you want to be addressed by the script.  Let's say you have 20 Campaigns you want to process and your script is timing out before it's finished the 7th Campaign so you're guessing you need to run the script 4 times to be absolutely sure.  Simply apply four Labels, for example "Block_1", "Block_2", etc. to the Campaigns in blocks of 5 (or as appropriate so they have roughly the same "load") then use a condition in your Campaign selector as follows:

.withCondition("LabelNames CONTAINS_ANY ['Block_1']")

When the script runs it will only process the Campaigns with the Label "Block_1".  Run the script 3 more times using the appropriate Labels and you're done.

I'll describe how to do this automatically later, but let's move on to another option.

#2 - Set limits

If you're addressing a large Account it's likely you've got a spread of performance metrics across the elements you're looking to process so you could use conditions to split the scripts up, for example:

.withCondition("Impressions > 1000")

(don't forget you'll need a DateRange condition as well).  This option can be useful where the time that the script runs is important.  You may want to ensure your biggest spenders, for example, are processed first in a day and this becomes increasingly important the more runs you need to make.  If, for example, you need the script to run 10 times before it addresses the whole scope, even if you start at 1am you'll still be into the working day before the last elements are processed so using this sort of condition can ensure the most important ones are done first.

#3 - Use Labels (again)

Another use of Labels is in marking elements are "processed".  So, if you were running your script against Keywords you could apply the Label "Processed" to each Keyword you complete and use a selector with the line:

.withCondition("LabelNames CONTAINS_NONE ['Processed']")

in this way, each time the script runs it'll only work on the Keywords not already processed.  The only real downsides to this method are that you need to use another script to remove these processed Labels before each new "day" of script running and unless you also label Campaigns/Ad Groups your script could be inefficient as it searches Campaigns where all the Keywords are already done.

OK, so there's three options for splitting, but how do you get the script to run more than once?  You could use a number of individual scripts pre-set with their conditions and running at different hours, but that's not very efficient and makes it harder to edit the script (since you have to make sure you make exactly the same edits to each one).  I prefer to use systems based upon the hour of the day and just have the same script run every hour.

For example, the following script snippet will return the hour of the day (in your Account time zone):

var theHour = new Date(Utilities.formatDate(new Date(), AdWordsApp.currentAccount().getTimeZone(), 'MMM dd,yyyy HH:mm:ss')).getHours();

If you've used a Label naming convention like the one I have above, you can then build your required Label name using the following:

var labelName = "Block_" + theHour;

So, in your script, you might have the following:

function main() {
  var theHour = new Date(Utilities.formatDate(new Date(),   AdWordsApp.currentAccount().getTimeZone(), 'MMM dd,yyyy HH:mm:ss')).getHours();
  if(theHour > 0 && theHour < 5) {
    var labelName = "Block_" + theHour;
    doProcess(labelName);
  }
}  

function doProcess(labelName) {
  var campIter = AdWordsApp.campaigns()
    .withCondition("LabelNames CONTAINS_ANY ['" + labelName + "']")
    .get();
}

Set that script to run every hour and it will only process the Account in the scripts that run at 1am, 2am, 3am and 4am and will automatically run on the Campaigns with the appropriate Labels.

You can extend this use of Labels and hours to many other situations so go have fun!

In the next post I'll look at some options that'll make the best use of a script you just can't stop timing out.

Avoiding Script Timeouts - Part 1

Scripts are possibly the most useful tool for managing larger Accounts.  Scripts can apply complex processes automatically and reliably across a large number of AdWords elements and allow you to concentrate on what you do best - marketing.  However, ironically, when the Account is very large and/or the script complex, you may find your marvellous script consistently "times out".  At the time of writing, scripts can only run for 30 minutes before automatically terminating; this obviously means there's the potential for an unknown percentage of your AdWords elements to remain unprocessed but it also often means important logging information is lost (as "final" logs are never reached).

This post will hopefully give you some pointers to ensure your scripts run as quickly as possible and the next will give you some options for dealing with scripts that simply cannot be made to run under 30 minutes.  So, let's start with the tips:

#1 - Good structure

All programming benefits from a good structure and with AdWords scripts the structure can be particularly important in terms of run time.  Most scripts will use some form of selector to fetch AdWords elements (Campaigns, Groups, Keywords, etc.) and then process them in some way but how you fetch these elements can have a big impact on how fast a script runs.  Imagine each selector is like sending an office minion out of the building to get something from a local store; the more often you send that person out the more time it's going to take to complete your task.  So, for example, if you want to work on Keywords, rather than selecting Campaigns, then iterating them to select Ad Groups, then iterating those to select Keywords, look for a method to select the Keywords you need in one selector.  Labels are an excellent way to do this (and you could use another script to apply and/or remove the Labels, of course, running before your main one).

Don't forget things like Campaign and Ad Group names are accessible from Keyword objects so you don't need to iterate Campaigns just to retrieve their names for reporting.

As a rule of thumb, your selector should, as far as possible, select the objects you want to work upon directly and if you're not doing that, you should think about how you could.

#2 - Only select what you need

Make sure your selectors use conditions that ensure they return only the elements you need to work upon.  One very common bad practice (in all programming/scripting) is to fetch a large dataset and then examine each returned element for suitability, discarding those that don't fit.  For example, you may select all the Keywords in a Campaign, then use conditional statements to check whether those Keywords are enabled or if they have impressions, etc.  It is much better practice (and far faster) to include these conditions in the original selector so if you want to work only on Keywords that are enabled, ensure you include these conditions in your selector:

.withCondition("Status = 'ENABLED'")
.withCondition("CampaignStatus = 'ENABLED'")
.withCondition("AdGroupStatus = 'ENABLED'")

If you want to check for clicks or impressions, you can do so, as long as you include a time range in the selector, e.g.

.withCondition("Impressions > 0")
.forDateRange("LAST_30_DAYS")

The same thinking can be applied to all other elements you might select; try to use the selector itself to define your returned objects rather than grabbing a huge bundle then spending ages throwing most of it away.

#3 - Use AdWords API reports

The many prebuilt reports in the API are much faster at retrieving AdWords elements than the normal selectors so where possible, use these reports.  The process of using them within scripts is quite different from a selector but there are plenty of examples on the web and once you've used one, the process is similar for all.  These reports aren't appropriate for all script uses but if you can use one, you should.  See here for a list of available reports.

https://developers.google.com/adwords/api/docs/appendix/reports

#4 - Watch your Logging

Logging is, of course, a vital part of scripting, especially in the development/testing stages but logging has a surprisingly big impact on run time, particularly when the logs are frequent and within a loop.  Try to keep your logging to what is absolutely necessary and once you're sure the script is working correctly, consider whether you can remove any.

#5 - Keep loops clean

Avoid including statements within loops that don't need to be calculated in each iteration.  For example, if you wanted to compare each Keyword's CTR against the average for an entire Campaign, you only need to calculate the Campaign average once, don't include that calculation inside the Keyword iterator.  The script within  a loop should be only that needed for the loop to operate correctly.

#6 - Check for "bloat"

It's very easy when writing script to check for certain conditions (average position vs ROAS or CTR vs number of conversions, etc.) to end up with several conditional statements.  It's just as easy for the final version to end up with redundant conditions.  Make sure all the tests/conditions you apply are unique and where they are, consider whether there's a quicker way to write the same set of conditions.

There are other tips for speeding up scripts that can be learned from the many online pages that teach speed improvement for JavaScript and while some can't be applied to AdWords scripts, most can and I'd recommend some quality time with Google search.  You might try starting with a search on "improve javascript speed".

In the next post I'll look at what you can do if despite your best efforts the script still times out.  Good luck!