Scheduled Reports with CRON for App Engine


It is important for web developers to regularly keep track of how their application is being used by their customers. This allows them to have sound decisions on what improvements need to be added for future releases. This post will show how to set up scheduled reports in Google App Engine running in Python. The scheduled reports will contain your application’s usage data, plus other stuff that you may want to measure and monitor over time.

Setting up a Reports Handler

In your application root, create a file that will contain our Python script that measures your application’s usage data. In the case of this post, I’ll set the filename to This file will contain your reports handler and should import classes from your application model. Say we simply want to measure the total number of domains and the number of boards in all the domains, then we need to import the Board and Domain classes from our model. Let’s also import the Mail Python API so we can send our reports in the form of emails.

from google.appengine.api import mail
from google.appengine.ext import db
from google.appengine.api import namespace_manager

import tornado.wsgi
import tornado.locale

from script import BaseHandler
from model import Board, Domain

Writing the Reports Handler

After importing the modules and classes we need, we can now start writing the reports handler. Let’s call our handler ReportsHandler, which should inherit from the base handler in Google App Engine. Afterwards, we can write the code that measures the total number of domains and boards in the application. Let’s assume that our application model has set every domain to its own namespace, i.e. we can only access the Board class within a domain’s namespace. This means that we need to set the namespace for every domain iteration in order to count the total number of boards among all the domains. The code for these measurements are given below:

class ReportsHandler(BaseHandler):

    def get(self):
        total_num_of_domains = Domain.all().count()
        total_num_of_boards = 0
        domains = Domain.all().run(batch_size=200)
        for domain in domains:
            total_num_of_boards += Board.all().count()

Now let’s set up the template for the email containing the reports. We pass the variables total_num_of_domains and total_num_of_boards in our template values and name the file of our email template as report.txt.

class ReportsHandler(BaseHandler):

    def get(self):
        # email address of recipient
        user_address = ""

        if mail.is_email_valid(user_address):
            # email address of sender in the <>
            sender_address = "Your Reporter <>"
            subject = "Your Report Subject"
            template_values = { 
                'total_num_of_domains': total_num_of_domains,
                'total_num_of boards' : total_num_of boards
            body = self.render_string("templates/report.txt", **template_values)
            mail.send_mail(sender_address, user_address, subject, body)

After writing the ReportsHandler contents, we specify the URL that will trigger the report. Let’s use “root_URL/reports” for our example. Put the following code at the bottom of

app = tornado.wsgi.WSGIApplication([
    (r"/reports", ReportsHandler)

Writing the Report Template

We use Tornado to render the template values in our email. Our email contents are located in report.txt, so let’s modify this to contain the following simple template:

This is the scheduled report for your application.

Total number of domains: {{ "{{ total_num_of_domains" }} }}
Total number boards: {{ "{{ total_num_of_unarchived_boards" }} }}

Your loyal reporter

Specifying the Schedule using CRON

Google App Engine uses CRON to run scheduled tasks. You can find the different scheduling formats here. Now create the file cron.yaml in your application root. Let’s say we want to run our report everyday at 8:30 am. Also note that the ReportsHandler is run through the URL “root_URL/reports”. The CRON file must then contain the following code:

- description: 
  url: /reports
  schedule: every day 08:30

We’re done setting up our scheduled reports via email! One can go ahead and customize further the reports handler to include many more measurements in our reports.

Creating PayPal IPN Handlers with Tornado

What is IPN?

Instant Payment Notification (IPN) is PayPal’s message service that sends a notification when a transaction is affected.

Basically, once a PayPal transaction happens, it sends a notification to a handler you specify, with all the information and variables you need to handle that event in your application.

Check out PayPal’s official docs on IPN for more information.

Preparing the PayPal buttons

Before we write our handler, we need a way to send the IPNs. The easiest way to do this is to use one of PayPal’s Payments Standard Buttons. For this article, we’ll be using the “Subscribe” button.

Follow the instructions here to create your button.

Writing the handler

Once we have the button in a page in our app, we can begin writing our IPN handler.

I’ll be using the Tornado web framework for this example, but it should be very similar for other frameworks. Also, I’m deploying on Google App Engine , so there might be some App Engine specific code.

First, determine the URL your handler will use. For example, You’ll need to modify your PayPal button so it knows where to send IPN messages. Just add


to the “Advanced variables” textbox at the bottom of the modify button page. Don’t forget to update the HTML in your app’s subscribe page as well.

Once that’s set up, we can begin writing the code for our IPN handler. I’ll just show the entire handler below and explain the parts.

The code below is basically the code in this blog post , rewritten to work with Tornado.

import logging
import os.path
import urllib
import urllib2
import wsgiref.handlers

import tornado.web import tornado.escape import tornado.wsgi


class IPNHandler(tornado.web.RequestHandler): def verify_ipn(self, data, sandbox=''): # prepares provided data set to inform PayPal we wish to validate the response data["cmd"] = "_notify-validate" params = urllib.urlencode(data)

if sandbox: # sends the data and request to the PayPal Sandbox paypal_url = '' else: paypal_url = ''

req = urllib2.Request(paypal_url, params) req.add_header("Content-type", "application/x-www-form-urlencoded") # reads the response back from PayPal response = urllib2.urlopen(req) status =

# If not verified if not status == "VERIFIED": return False

# if not the correct receiver ID if not sandbox and data['txn_type'] == 'subscr_payment' and not data["receiver_id"] == RECEIVER_ID:'Incorrect receiver_id')['receiver_id']) return False

# if not the correct receiver email if not sandbox and data['txn_type'] != 'subscr_payment' and not data["receiver_email"] == RECEIVER_EMAIL:'Incorrect receiver_email')['receiver_email']) return False

# if not the correct currency if not sandbox and not data.get("mc_currency") == "USD":'Incorrect mc_currency') return False

# otherwise... return True

def subscr_signup(self, data): # handle a 'Signup' IPN message # you can create a User object, for example, # or set a user's plan pass

def subscr_payment(self, data): # handle a 'Payment' IPN message # this message gets sent when you receive a recurring payment # you can re-set your user's plan here pass

def subscr_modify(self, data): # handle a 'Modify' IPN message # the Subscribe button has an option to allow users to modify # their subscription plan # you can upgrade your user's plan here pass

def subscr_eot(self, data): # handle a 'End of Transaction' IPN message # at the end of the subscription period, this message gets sent # you can disable a user here pass

def subscr_cancel(self, data): # handle a 'Cancel' IPN message # when a user cancels his subscription (either in his PayPal page or # in your website), this message gets sent # you can disable a user here pass

def subscr_failed(self, data): # handle a 'Failed' IPN message # sometimes something goes wrong while the IPN is being sent # you can log the error here pass

def post(self, sandbox=''): data = {}

# the values in request.arguments are stored as single value lists # we need to extract their string values for arg in self.request.arguments: data[arg] = self.request.arguments[arg][0]

# If there is no txn_id in the received arguments don't proceed if data['txn_type'] == 'subscr_payment' and not 'txn_id' in data:'IPN: No Parameters') return

# Verify the data received with Paypal if not':')[0] == 'localhost' and not self.verify_ipn(data, sandbox):'IPN: Unable to verify') return'IPN: Verified!')

# Now do something with the IPN data if data['txn_type'] == 'subscr_signup': # initial subscription self.subscr_signup(data) elif data['txn_type'] == 'subscr_payment': # subscription renewed self.subscr_payment(data) elif data['txn_type'] == 'subscr_modify': # subscription plan modified self.subscr_modify(data) elif data['txn_type'] == 'subscr_eot': # subscription expired self.subscr_eot(data) elif data['txn_type'] == 'subscr_cancel': # subscription canceled self.subscr_cancel(data) elif data['txn_type'] == 'subscr_failed': # subscription failed self.subscr_failed(data)


settings = { 'template_path': os.path.join(os.path.dirname(__file__), 'templates'), 'autoescape': None, 'debug': os.environ.get('SERVER_SOFTWARE', '').startswith('Development/'), } app = tornado.wsgi.WSGIApplication([ (r'/ipn', IPNHandler), (r'/ipn/(sandbox)', IPNHandler), ], **settings)

def main(): wsgiref.handlers.CGIHandler().run(app)

if __name__ == '__main__': main()

The “Subscribe” button sends 6 types of transactions (subscr_signup, subscr_payment, subscr_modify, subscr_eot, subscr_cancel, subscr_failed).
Each of those transaction types has its own handler. I’ve placed comments in each of them as guides on what you can do once you receive a specific transaction type.

When the handler receives an IPN (which is a POST request from PayPal), it stores the request’s arguments in the data dict.

We first verify the data by passing it to the verify_ipn method. What it does is resend the exact same arguments to PayPal in order to verify it. Once we receive a “VERIFIED” message (plus various other arguments checks), it is safe to proceed.

We then send data to the specific transaction type handler to make use of.

For a list of all the arguments included in data and the transaction types for other buttons, refer to PayPal’s docs on IPN variables.


Using the Sandbox Tool

PayPal provides a Sandbox tool which simulates sending IPN messages to your IPN handler. Visit to get started. You’ll need to sign up for a separate Sandbox account, then go to “Test Tools” > “Instant Payment Notification (IPN) simulator”. Fill in the URL to your IPN handler (use /ipn/sandbox since the Sandbox uses a different URL), select the transaction type you want, then click submit.

Unfortunately, the IPN simulator doesn’t have the option to send subscription IPNs. I resorted to testing it “live”, by actually clicking on the subscribe button and going through a full cycle.
This can be a pain, since you’ll have to wait for a full cycle to test the other transaction types (payment, eot).

Testing on localhost

You can also test you handler on localhost, but we’ll have to skip the verify_ipn() part, and jump right to handling the transaction type, just assuming the IPN was verified.

Check out my IPN-tester project. It’s a script which sends a POST to http://localhost:8080/ipn with arguments taken from logs from real IPNs.
It only has data for subscr_signup and subsr_cancel for now, but I’ll add other types soon (it’s open source, so anyone can add :P).

It’s the reason why the code above has

if not':')[0] == 'localhost'

so the script only works on the app running on localhost, by skipping the verify_ipn() check.