Serving Different Ads/Content Based on a Users Location (Geo-Targeting)

You probably haven't noticed, but this blog serves up different ads depending on where you're visiting from. Or at least, it'll serve Amazon UK ads if you're near the UK, and Amazon US ads otherwise. Serving up US ads to UK visitors (and vice versa) is pretty pointless, and I've always tried to avoid showing any ads unless they're relevant and at least targeted to the right country.

There are a number of ways to determine where your visitors are coming from, so I spent some time yesterday trying to find the most reliable way (and preferably one that didn't involve having a huge IP database sat alongside my site!). After much hacking and testing, I found what I believe to be the best way. Google.

Google has a JavaScript loader API, which allows developers to load JavaScript libraries from Google with various benefits. That's not really what we're interested in though, it has something more exciting:

google.loader.ClientLocation

It appears that you do not need an API key to use the JavaScript loader, you can simply reference it at http://www.google.com/jsapi. If you look at the JavaScript served up (which is incredibly fast), you'll see something like this:

google.loader.ClientLocation = { "latitude":50.123, "longitude":-2.876, "address": { "city":"Liverpool", "region":"Merseyside", "country":"United Kingdom", "country_code":"GB" } };

Not only do you get the country, but you get the county, city and even lat/lon pair. For me, the location given was within 2-3 miles of where I live, so if you wanted, you could really localise your ads!

On this site, the country is just sent to a script that will serve up some ads based on keywords I've tagged against a post. You might wish to be a bit more exciting and show your users places or people nearby. This could be especially useful for mobile applications/sites, though be sure to read any associated terms and conditions before using it!

Google Wave Notifier Now in 15 Languages!

When I started writing Google Wave Notifier last month, it was really only for my own use. I wouldn't use Google Wave if I had no way of knowing when I had messages (and keeping a browser open is not an acceptable solution!) so I hacked something together to login to Wave and parse the JavaScript objects.

I decided others might benefit from the app, so I cleaned it up a little, gave it an icon and published it on the web. I had no idea that it would get close to 10,000 downloads in little over a month! Nor did I expect so many people to send in translations!

So here we are 6 weeks down the line, and the app has been translated into 15 different languages!

• English
• Czech
• Danish
• Dutch
• French
• German
• Hungarian
• Italian
• Polish
• Portuguese
• Romanian
• Russian
• Spanish
• Swedish
• Turkish

So I'd like to say thanks to everyone that's helped; using, translating and finding bugs in the app. What was a small script written for personal use has turned into something quite useful!

For more information or to download, see the Google Wave Notifier website!

Redirecting Requests from appid.appspot.com to a Custom Domain

If you're running your app engine project on a custom domain (like this blog), you're probably not so happy that people can still access your app at http://appid.appspot.com.

When I started working on my blog, I realised this might be an issue, and did some investigating into how I could stop it. I found solution on SackOverflow that seemed to do what I needed, so I set it up and got it working.

Not long after implementing this code, I found a few problems:

• SSL is not supported on custom domains
• Cron jobs fail when Google invokes them with an appspot.com address and you serve a 301

So with some tweaks, I've managed to get this working as required. The only annoyance is the hard-coded '/admin' check. This is to support cron jobs, task queues etc., which are all protected ("login: admin" in app.yaml). They must work with an appspot.com address, because Google doesn't seem to follow the redirect when invoking them. It's possible you could do an IP address check here, but I'm not sure how consistent cron/task queue IP addresses are.

The code is called like this:

def main():
    dantup.run_app([
        ("/\d*", RootHandler),
        ("/feed", FeedHandler),
        ("/tags/.*", TagHandler),
        ("/archive/.*", ArchiveHandler),
        ("/.*", PostHandler)
    ])

And dantup.py looks like this:

from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

def run_app(url_mapping):
    application = webapp.WSGIApplication(url_mapping, debug=True)
    application = redirect_from_appspot(application)
    run_wsgi_app(application)

def redirect_from_appspot(wsgi_app):
    """Handle redirect to my domain if called from appspot (and not SSL)"""
    from_server = "dantup-blog.appspot.com"
    to_server = "blog.dantup.me.uk"

    def redirect_if_needed(env, start_response):
	
        # If we're calling on the appspot address, and we're not SSL (SSL only works on appspot)
        if env["HTTP_HOST"].endswith(from_server) and env["HTTPS"] == "off":
		
            # Parse the URL
            import webob, urlparse
            request = webob.Request(env)
            scheme, netloc, path, query, fragment = urlparse.urlsplit(request.url)
            url = urlparse.urlunsplit([scheme, to_server, path, query, fragment])
			
            # Exclude /admin calls, since they're used by Cron, TaskQueues and will fail if they return a redirect
            if not path.startswith('/admin'):
                # Send redirect
                start_response("301 Moved Permanently", [("Location", url)])
                return ["301 Moved Peramanently", "Click Here %s" % url]
	
        # Else, we return normally
        return wsgi_app(env, start_response)

    return redirect_if_needed

Hopefully you may find this useful. If you encounter any problems with it, please let me know!

Related Reading

• Developing with Google App Engine by Eugene Ciurana
• Using Google App Engine by Charles Severance
• Google App Engine Cookbook

1-2-3-Testing. Is This Thing On? New Blog!

There's nothing like testing a system after it's gone live!

After lots of frantic hacking over the last few weeks, the blog engine I've been writing for Google App Engine is up and running. It's not complete, but the important things (frontend, feed, posting, etc.) are done so it's at least usable.

Apologies if you just got a flood of old messages in your feed reader. I did investigate keeping the Blogger IDs to avoid this, but it seemed more hassle than it was worth, so I'll forgive you for clicking "Mark All as Read" this one time ;-)

I'll post more info (and code) on the blog as I get time, but now I'm off to sort some food out after a hard day's hacking!

If you notice anything messed up in the transition, please do let me know!

From .NET to Python on App Engine: Building a Blog on Google App Engine

Over the past few weeks I've been playing with Google App Engine. I find the best way to learn a new language/framework/platform is to just jump in and write something in/on it. So that's what I'm doing. I've decided to write my blog in Python for Google App Engine.

As I may have blogged about in the past, I wrote a blog engine in Microsoft ASP.NET MVC not so long ago with the aim of moving away from Blogger. It was around 90% complete when I abandoned it for a variety of reasons (one being Azure pricing).

It's entirely possible the Google App Engine blog engine will also be abandoned, but since the hosting is free it at least stands a good chance of seeing the light of day! It'll also make an interesting comparison to the ASP.NET MVC version.

I started writing code a few nights ago, and currently the blog stands at 159 non-blank lines. I'm actually quite impressed with how little code I've had to write to get up and running. Currently there's no back-end, but the displaying of posts, comments, tags and archives are all working. Here's a quick screenshot to prove it exists! :)

Over the coming weeks I'll blog about how I've built it (including code), the pitfalls and the the experience of moving from .NET and C# to Python and App Engine!

Related Reading

• Developing with Google App Engine by Eugene Ciurana
• Using Google App Engine by Charles Severance

Downloading/Exporting App Engine Logs

Although I've been playing with App Engine for quite a few weeks now, I only found out yesterday how I can download the logs from App Engine for parsing locally. There's no export option in the dashboard, nor any option in the Windows launcher. However, you can do this yourself with appcfg.py.

If, like me, you've only used the launcher up until now, and never done anything with Python outside of GAE, you might find the thought of running python scripts a little scary. Don't worry, it actually turns out to be very easy. I'm hoping when you installed the Launcher you ticked the "Add to PATH" option... :o)

The command you want to run looks something like this:

appcfg.py request_logs appname/ output.txt

You should replace "appname" with the name of your app. This is the folder the contains the app.yaml file for the app you wish to get logs for. That means you need to be in the correct directory (or provide a full path). Output.txt is obviously where the logs should be written.

This command will retrieve everything from the last day. This might not be what you want, so you can increase this with the num_days argument.

appcfg.py --num_days=5 request_logs appname/ output.txt

Note that you can also specify 0 to get all logs:

appcfg.py --num_days=0 request_logs appname/ output.txt

And finally, if you want all logs that you haven't already downloaded, you can use --append, which will scan the last line of the existing file, and download anything since:

appcfg.py --append request_logs appname/ output.txt

However, you'll find this doesn't work on Windows, and you'll end up with a load of duplicate entries. This is in the bugtracker, but I don't know if/when it'll be fixed. For now, I'm having to just download everything (and I'm not sure if this is counting towards my bandwidth quota!).

Another option worth noting, is include_vhost, which will include the hostname used, so you can seperate requests for different versions of your app (or different custom domains). You can use this like so:

appcfg.py --num_days=0 --include_vhost request_logs appname/ "Logs/Logs.txt"

And that's all there is to it. Now you can create nice pretty charts in Microsoft Excel (since Google Spreadsheets sucks!) showing how well (or badly, in my case) your app is doing!

Related Reading

• Developing with Google App Engine by Eugene Ciurana
• Using Google App Engine by Charles Severance

Increased App Engine Quotas, for Free?

While writing my comparison of Azure and App Engine pricing yesterday, I had a thought about how to increase some of your quotas without actually paying anything. I'm not sure whether Google would consider this "gaming the system" and stamp on you (and I certainly have no need to do it with my traffic levels), but I thought I'd post it in the interest of sharing.

If you check the App Engine Quotas page, you'll see the differences between the free quota, and the billing enabled quota. For example, if you don't have billing enabled, you may serve 1,300,000 requests per day. If you have billing enabled, you get 43,000,000 requests per day (other limits, such as CPU/bandwidth still apply).

So, if you've taken a look at how the billing works, you'll know that you get to decide exactly how your budget is split between resources. The minimum budget you can set is $1/day.

Consider what would happen if you enabled billing, but assigned the whole budget to something you know you won't exceed (eg., if you don't use memcache, assign it to memcache). Your app will be given the higher "billing enabled" quotas, but it won't cost you a penny!

As I said earlier - I've no idea whether Google will frown upon this behaviour, so if you do it, it's at your own risk!

Microsoft Windows Azure vs Google App Engine: Pricing

As a .NET developer, I was quite excited to hear about Windows Azure. It sounded like a less painful version of Amazon's EC2, supporting .NET (less painful in terms of server management!). When I saw the pricing, it didn't look too bad either. That was, until I realised that their "compute hour" referred to an hour of your app running, not an hour of actual CPU time. Wow. This changes things. To keep a single web role running, you're looking at $0.12/hour = $2.88/day = $20.16/week = $86.40/month.

Anyone that's bought hosting for a small site/app recently will know that this is not particularly cheap!

So, recently I've been playing around with Google App Engine. It has this massive problem called Python (and an even bigger one called Java ;o)), but it's such a nice framework/engine to work with that I've somehow overlooked this and started coding with it. There's so much to like about it. Everything is so simple to deploy, and it scales "out of the box". Want Cron jobs? No worries, specify them in a file in your app, and when you deploy, App Engine will pick them up and schedule them. Want to queue up work to process later so that your pages return faster? Task queues do just that. What's more, you get a ridiculous free quota every day. It may be Python, but this sounds tempting, no?

So, I thought it'd be interesting to compare the costs of App Engine vs Azure. I understand this isn't really a like-for-like comparison, but both can achieve the same sort of things, and while all programmers will have a preferred language/framework (I'm no exception), many can be swayed by a cool framework or hosting.

First off, let's compare what you get for free. Bear in mind that Azure is free until the end of January, but since this is a CTP and won't end soon, I'm going to exclude it. Google's free quota currently has no time restrictions.

Well, for free, I think we have a clear winner. If you can run your website/app within the limits above, then you can do it for free with Google. It's worth mentioning that Google let you have 10 apps per account (though you may not balance a single site/app across app instances - they are specifically for separate projects).

But what if you're bigger than that? What if you get Slashdotted or Dugg a lot? You might find you quickly break out of the free limits. How do prices compare?

Well, that's interesting. I was going to try and calculate at what point Azure would become cheaper, but looking at those prices, it just isn't going to happen. Now it's worth pointing out that not all of the comparisons are fair. Google bill per actual CPU hour (so if nobody visits your site, it's not costing you), whereas Microsoft are billing for each hour your app is live and able to respond. There's also a significant difference between SQL Server and App Engine Datastore (and depending on what you're doing, one will have advantages over the other).

I really hope Microsoft re-evaluate their pricing for small apps. It's too expensive to play around with small prototypes at those prices, whereas Google's offering will let me get started completely free, until my app is churning a considerable amount of traffic, and even then, it'll work our cheaper for the same processing/transfer.

Sorry Microsoft. I love .NET and Visual Studio, but Google App Engine is just so easy and cheap that it's going to be my "toy of choice" for my hobby coding for the immediate future!

Related Reading

• Cloud Computing by John W. Rittinghouse, James F. Ransome
• Cloud Computing, A Practical Approach by Toby Velte, Anthony Velte, Robert C. Elsenpeter
• Cloud Computing for Dummies by Judith Hurwitz, Robin Bloor, Marcia Kaufman, Fern Halper

Should I move my Blog to Google App Engine?

Over the years, I've written (and re-written) many blog engines with the intention of hosting my blog on my own code. I'm a programmer, that's what we do. We don't like the thought of other people writing our HTML!

If you've been following my blog over the last twelve months, you'll probably know what my language/framework of choice would be when writing a web app, based on my previous posts...

• ASP.NET MVC HandleError Attribute, Custom Error Pages and Logging Exceptions
• Using OpenID in your ASP.NET MVC Application/Blog
• Reducing Duplicate Content with ASP.NET MVC

As a .NET developer by trade, I'm experienced with C#, .NET, ASP.NET MVC etc. and it's always been the logical choice. There's nothing I like to write more than C#/.NET

So why are you hosted on Blogger?

An interesting question :) Unfortuantely the answer is not interesting. The answer is simply: Hosting. Windows hosting is a pain in the ass. I've had many bad Windows hosts that have dodgy control panels, poor performance or a lack of required features. Most of this is addressed with IIS 7 because we can now do most things via XML config files, but it still leaves one issue: Price. Prices seem to vary wildly, and it's such a pain to set up that I'm often put off signing up in case it turns out to be a lemon. This is the reason I have a fully working .NET MVC blog engine ready to roll in my Documents folder that's never made it to the big cloud in the sky.

What about Azure?

A few months ago I was really excited about Azure. It looked ideal for what I wanted, and looking at how EC2 is priced, I suspected it would be cheap if you're only getting a few hundred visitors per day. Then Microsoft announced the prices... To run a no-to-low traffic blog would cost more than getting a virtual server! On top of that, it seemed quite hacky to get MVC working, and it's still only in preview.

Enter, Google App Engine

While playing around in Google Wave, I downloaded the Google App Engine SDK and had a play. While Python isn't exactly my language of choice (I've never written it before!), it struck me as quite a good deal. Five million page views per month... for free! It's hosted on Google's infrastructure and scales well, and there's no way I'd hit those limits anytime soon.

So, now I'm having thoughts... Do I become a hacky Python programmer, putting aside all the .NET experience I have, for free hosting with Microsoft's biggest competitor?

If anyone has any experience with running blogs on Google App Engine, I'd be very interested in hearing from them. Should I run with it, or should I stop being tight-fisted and just pay for some Windows hosting?

To be continued!

Google Wave Notifier now in Spanish, Russian, Swedish

With the help of some users, yesterday I added support for Spanish, Russian and Swedish languages to Google Wave Notifier. All parts of the interface are translated and I'll keep adding languages if people are prepared to help with the translations :)

If you'd like to help translate into your native language, please visit the translation page.