Archive for May 2011

22

May 2011

Google App Engine Price Increases - Bad for Small Apps, and No More Free Scaling :o(

At Google IO it was announced that this year that Google App Engine will come out of "preview" and add additional features and an SLA. Sounds great, but it also seemed like a lot of this hype was to try and downplay something else that was changing... the prices. As the service comes out of preview, it needs to be able to sustain itself financially. That means, price increases! (full details here)

The most significant change to me is the shift from charging for "CPU hours" being actual hours worth of CPU cycles (for a "standard" CPU), to being "your app being live for an hour, even if it uses 0 CPU". This is how most of App Engine's competitors work, and actually the reason I always thought Google App Engine's prices were so reasonable for small apps.

The problem with this model is that if you have a very low traffic application (which, let's face it, is how most apps will start out), it costs you almost nothing. In fact, for the few years this blog has been running on App Engine, I haven't paid a single penny. Even on a day I received 50,000 visitors, I didn't break the free quota despite my application being scaled across 4-5 instances for most of the day. My CPU use was so low, it was measure in minutes!. This made App Engine an ideal place for trying out ideas, because it was really cheap. And let's face it, if your idea takes off and it's already hosted on App Engine, you're unlikely to move away, so it was Win/Win, both for you, and Google.

The alternative model, which App Engine is soon to adopt, is that your low-traffic app is being charged just for being "live". This is the reason I've never set anything up on Azure or EC2. It costs you the same if you get 0 visitors in a month, as a single-instance application getting moderate traffic.

This change has another implication for small apps. No more free scaling. Previously, your application could scale out to a number of instances for periods of the day and you could stay within the free quota. Now that the free quota will be only 24 CPU hours, it's impossible for your application to burst out of a single instance without incurring a cost.

Note: The new pricing page lists "Dynamic Scaling" under the free version, however unless you're prepared for your application to be available for less than 24 hours in a day, I don't see how this could possibly be achieved. It's possible this refers to it being possible, though extra charges will apply.

This might not sound like a big deal - competing services already charge like this anyway (and some don't even have free quotas), however for me, this is more significant. I'm a .NET developer by trade. I'm not a huge fan of Java or Python, but App Engine's pricing was so good, it went some way to outweighing the use of .NET. Now that the pricing is the same, it doesn't really make sense as a .NET developer to give up all the benefits of the development environment/frameworks I'm used to using (and the productivity boost I get from them).

My current thinking is to move my apps over to AppHarbor, but once they're written in .NET, Azure and EC2 are probably also possibilities if AppHarbor doesn't work out. First project, this blog!

21

May 2011

Configuring ELMAH to send emails without putting your password in the config file

Today I was configuring ELMAH to send emails when an exception occurs on my new blog. While looking for the config options, I noticed that a lot of the snippets being posted around suggests people are putting usernames and passwords in their config files to make this work. That's pretty scary! :(

Because SMTP is a bit crap, you don't actually need to authenticate to send email. Most SMTP servers are locked down to stop relaying, but if you deliver directly to the recipients server, the mail will (usually) be accepted anonymously. I sayusually because your email could be rejected based on the From header if something like DomainKeys is set up for that domain.

To configure ELMAH to send mail without authentication, simply look up an MX record for the domain you wish to send the email to, and use that as the SMTP Server. Eg., if you're using Google Apps (like me), you could do something like this:

<errorMail
    from="&quot;Blog Error&quot; &lt;blog+errors@fakedomain.com&gt;"
    to="&quot;Blog Errors&quot; &lt;blog+errors@fakedomain.com&gt;"
    smtpServer="aspmx.l.google.com" />

This means you don't need to put your email username/password in the config file to receive emails about your exceptions.

09

May 2011

Adding Tab-Expansion to Andrew Nurse's PSGet (NuGet Powershell Module)

If you subscribe to my blog or follow me on Twitter you'll probably know I'm quite a fan of NuGet. Recently I was playing around with Andrew Nurse's PSGet module that wraps NuGet.exe for PowerShell (which, by the way, is an awesome idea, and should be added as built-in functionality!). I decided a good way to learn a little more about PowerShell would be to try and add Tab Expansion to PSGet, similar to the functionality in the Visual Studio Package Manager Console.

Note: I forked Andrew Nurse's PSGet project to make these changes, though I'll see if he's interested in pulling them back into his, and updating the NuGet package. That way people will have a single source for PowerShell/NuGet goodness. Until then, if you want my Tab Expansion, you can grab the module from here.

Disclaimer: I've only been using PowerShell for around a week, so I'm sure there are much better ways to do what I've done. I did what I could with the (little) knowledge I have and some Googling. If you can see better ways of doing things, please let me know so I can learn, and update my code (send me a pull request on Bitbucket if you want).

Show me the code!

Tab Expansion in PowerShell works via the TabExpansion method. This function is called and provided with the current line in addition to the last word typed. I couldn't find a way to hook TabExpansion for just the commands we're interested in, so I had to hijack the whole lot. This doesn't seem to be a problem, as falling back to the default behaviour seemed pretty simple by keeping a copy of the function:

# Copy the current Tab Expansion function so we can fall back to it if we're not supposed to handle a command
Copy Function:\TabExpansion Function:\OriginalTabExpansion

function TabExpansion([string] $line, [string] $lastword)
{
    # Only run for "Get-PSPackage" or "Install-PSPackage"
    if (somecondition)
    {
        # Do cool NuGet stuff here
    }
    # Otherwise, fall back to default TabExpansion function
    else
    {
        OriginalTabExpansion $line $lastword
    }
}
Export-ModuleMember -Function TabExpansion

We need our NuGet code to run only for two specific commands, for the first parameter. The easiest way seemed to be to check that $line is equal to one of our commands followed by the $lastword. This probably isn't the best way, but it seems to do the job for the obvious cases.

# Only run for "Get-PSPackage" or "Install-PSPackage" where we're typing the first argument
if ($line -eq "Get-PSPackage $lastword" -or $line -eq "Install-PSPackage $lastword")
{
    # ...
}

Inside this block, we need to query the OData feed for NuGet packages. Rather than hard-coding the URI, we're supposed to go through a Microsoft fwlink (https://go.microsoft.com/fwlink/?LinkID=206669), but because we can't append parameters to the end, we need to resolve this redirect. I found some code in the NuGet issue tracker written by David Ebbo to do just this. I converted it to PowerShell and added a $defaultSource variable:

# Follows redirects and returns the final URI. Used to support fwlink as package source.
function GetRedirectedUri($uri)
{
    $req = [System.Net.HttpWebRequest] [System.Net.WebRequest]::Create($uri)
    $req.AllowAutoRedirect = $FALSE; # Don't follow redirects, to save bandwidth/roundtrip
    $resp = $req.GetResponse()
    if ($resp.StatusCode -eq [System.Net.HttpStatusCode]::Redirect)
    {
        $resp.Headers["Location"]
    }
    else
    {
        $uri;
    }
}
# Set the default NuGet source
$defaultSource = GetRedirectedUri("https://go.microsoft.com/fwlink/?LinkID=206669")

With this done, we just needed to call the feed with some search parameters and parse the results. Luckily, parsing OData is pretty trivial, and with Fiddler and Visual Studio, it was easy to find the syntax for searching for packages with IDs that start with a given string :)

The final step was to pipe through Get-Unique because the OData feed returns multiple entries for packages with multiple versions (but the ID is the same):

# Construct a URI to fetch the top 30 matching packages, sorted by download count
$uri = "$($defaultSource)Packages()?`$filter=startswith(tolower(Id),'$($lastword)')&`$orderby=DownloadCount%20desc,Id&$`skip=0&`$top=30"

# Fetch the data, return as XML
$wc = New-Object Net.WebClient
$data = [xml]$wc.DownloadString($uri)

# Pipe result through Get-Unique because there could be multiple versions with the same ID
$(
    # Loop, just grabbing the ID from Properties
    foreach ($entry in $data.feed.entry)
    {
        $entry.properties.Id
    }
) | Get-Unique

And viola, Tab Expansion! Now when you type "Install-PSPackage x" or "Get-PSPackage x" you can use tab to cycle through the matching packages (sorted by download count, in an attempt to give the most likely result first).

The full module is is up on Bitbucket, but maybe with a little tidying up we can get Andrew Nurse to pull the functionality into his repository and update PSGet on NuGet.

08

May 2011

Extending MarkdownHelper/MarkdownDeep.NET to support Google Code's Prettify Syntax Highlighter

I'm in the process of rewriting my blog in ASP.NET MVC to move it from Google App Engine to AppHarbor. One of the important changes is that all my articles will be stored as Markdown instead of HTML.

I've blogged previously about the MarkdownHelper class I created and put on NuGet to make using Markdown in views as trivial as typing:

@Html.Markdown(Model.Body)

One thing that I didn't take care of was adding syntax highlighting to code blocks. On this blog I currently use Google Code's Prettify syntax highlighter, and this requires a CSS class be applied to code blocks:

<pre class="prettyprint">
// Code here
</pre>

By default, MarkdownDeep (which my MarkdownHelper uses) doesn't add these classes, so I had to make some changes to support it. I didn't want to build this into MarkdownHelper as it's fairly specific to using this particular syntax highlighter. Instead, I decided to take advantage of partial classes to allow the functionality to be customised without having to modify the MarkdownHelper.cs file added by MarkdownHelper/NuGet.

First I amended the MarkdownHelper class, adding "partial" to the declaration. This was pushed to NuGet as MarkdownHelper 1.2:

public static partial class MarkdownHelper
{
// ...
}

With this done, I loaded my blog in Visual Studio and opened the Library Package Manager, which showed the update and replaced the class with one click. With this done, I then created a new file named "MarkdownHelper.CodeFormatting.cs" alongside "MarkdownHelper.cs" in my Helpers folder. This is where my customisations should go, which will allow MarkdownHelper.cs to be updated by NuGet without affecting the customisations.

As it turned out, customising the code blocks with MarkdownDeep was pretty easy. The class has several extension points, the interesting one for me being:

public Func<Markdown, string, string> FormatCodeBlock;

If you set this property, when a code block is encountered, your code will be called instead of the default. To make sure I wasn't losing any functionality, I checked the source code of MarkdownDeep to see exactly what it did with a code block:

b.Append("<pre><code>");
foreach (var line in children)
{
    m.HtmlEncodeAndConvertTabsToSpaces(b, line.buf, line.contentStart, line.contentLen);
    b.Append("\n");
}
b.Append("</code></pre>\n\n");

This seemed simple enough, and actually, the code path when you had a custom method was almost the same, it just didn't put the pre/code blocks in:

var sb = new StringBuilder();
foreach (var line in children)
{
    m.HtmlEncodeAndConvertTabsToSpaces(sb, line.buf, line.contentStart, line.contentLen);
    sb.Append("\n");
}
b.Append(m.FormatCodeBlock(m, sb.ToString()));

This meant the only thing I needed to do was surround the string with the pre/code tags. Adding class="prettyprint" was rather trivial:

/// <summary>
/// Overrides the Markdown formatting for code blocks to inject "prettyprint" classes for syntax highlighting.
/// </summary>
private static string FormatCodeBlock(Markdown md, string code)
{
	// Wrap the code in <pre><code> as the default MarkdownDeep.NET implementation does, but add a class of
	// "prettyprint" which is what Google Code Prettify uses to identify code blocks.
	// http://google-code-prettify.googlecode.com/svn/trunk/README.html
	var sb = new StringBuilder();
	sb.Append("<pre class=\"prettyprint\"><code>");
	sb.Append(code);
	sb.Append("</code></pre>\n\n");
	return sb.ToString();
}

Then in my partial class, I simply assigned this method to the FormatCodeBlock property of the Markdown transformer in a static constructor (since this is a partial class, it has direct access to the "markdownTransform" object):

/// <summary>
/// Static constructor to set MarkdownHelper options.
/// </summary>
static MarkdownHelper()
{
    // Override code formatting to support syntax highlighting.
    markdownTransformer.FormatCodeBlock = FormatCodeBlock;
}

With those changes, all my code blocks have the correct class and highlight correctly :-) You can see a sample of a code block on the homepage of the development site where I'm building the blog. You can also see the full source code on Bitbucket (MarkdownHelper.CodeFormatting.cs).

07

May 2011

Setting up NuGet to Automatically Fetch Packages When Deploying to AppHarbor Without Storing Binaries in Source Control

Over the last few days I've been blogging and tweeting about using NuGet without committing the packages folder to source control. David Ebbo blogged about using a pre-build event to fetch packages at build time.

Unfortunately, this doesn't work on AppHarbor because build events are not supported. If you try, you'll find the pre-build step doesn't fire and the build fails due to the missing dependencies :(

I voted on (and tweeted about, lots) a feature request about supporting NuGet natively on AppHarbor. The request has had quite a few votes, and Friism said they'll probably implement it, but in the meantime, this meant I couldn't deploy to AppHarbor. I didn't want to add my packages folder to source control, because even once I remove it when AppHarbor supports NuGet natively, it'll still be in the history (and therefore every clone), which I'd like to avoid.

I scratched my head a little, and re-read the proposed spec for baking this functionality into NuGet/MSBuild (to be honest, I don't actually understand what there is to improve!). The mentions of MSBuild got me thinking - custom build targets!

I opened up my csproj file in Notepad, and scanned through for the "BeforeBuild" step that's included by default but commented out. I removed the comment, and moved my pre-build event into the target:

<Target Name="BeforeBuild">
<Exec Command="&quot;$(SolutionDir)Tools\NuGet&quot; install &quot;$(ProjectDir)packages.config&quot; -o &quot;$(SolutionDir)packages&quot;" />
</Target>

First I checked whether this worked locally, by deleting my packages folder and running MSBuild from the command line. Success! I added in MarkdownHelper and added some test code to a view:

@Html.Markdown("We can use **Markdown!**")

Everything worked fine locally. An "hg ci" and "hg push" later, Bitbucket notified AppHarbor, AppHarbor checked out, built, and successfully deployed!

To show it working, you can see the test page at dev.dantup.com that will form the basis of the ASP.NET MVC / Razor version of this blog, and view the source code that AppHarbor is pulling to see there's no packages folder!

So, it turned out to be quite simple to get it working. I've also notified David Ebbo to see if he'd update his post with this info for anyone reading there that's using AppHarbor! :-)

« Older posts