Network Programming in .NET

Deploying a website with every check-in is quite a risky buisness, but it’s great for testing, so that your testers can be sure that they are looking at the very latest build of your website.

So, if you are using GitHub as your source control, then you can use Github’s Webhook system to trigger your website to download the latest version of itself from Github, effectively self-updating.

So, if you want to skip to the end, just check out our Github repo at https://github.com/infiniteloopltd/AutoBuild and the corresponding test website at http://autobuild.webtropy.com The website is set to automatically download content from Github on every push.

First, set up a webhook. Go to the repo, and click Settings > WebHooks > Add Webhook

Set up the payload url to point to your website, plus “github.aspx”, and keep the other settings as default.

Next, Add the following three Nuget packages to your project;

Install-Package Octokit

Install-Package SharpZipLib

Install-Package Newtonsoft.Json

and then the corresponding using statements;

using Octokit; // Install-Package Octokit
using System;
using System.Configuration;
using System.IO;
using ICSharpCode.SharpZipLib.GZip; // Install-Package SharpZipLib
using ICSharpCode.SharpZipLib.Tar; // Install-Package SharpZipLib
using System.Threading;
using Newtonsoft.Json.Linq; // Install-Package Newtonsoft.Json

Now, I’ve kept my settings in my web.config, as follows;

<add key=”GITHUB_USERNAME” value=”infiniteloopltd”/>
<add key=”GITHUB_PASSWORD” value=”xxxxxx”/>
<add key=”GITHUB_REPO” value=”AutoBuild”/>
<add key=”DEPLOY_PATH” value=”C:\temp\”/>

And, I’ve set up some public variables as follows

protected GitHubClient client = new GitHubClient(new ProductHeaderValue(“Autobuild”));
string username = ConfigurationManager.AppSettings[“GITHUB_USERNAME”];
string password = ConfigurationManager.AppSettings[“GITHUB_PASSWORD”];
string repository = ConfigurationManager.AppSettings[“GITHUB_REPO”];
string output = ConfigurationManager.AppSettings[“DEPLOY_PATH”];

The ProductHeaderValue seems arbirary, but I’ve called it AutoBuild.

Now, the magic happens when I start using Octokit;

private bool DownloadArchive(string expectedCommit)
{
var tDownload = client.Repository.Content.GetArchive(username, repository);
var bArchive = tDownload.Result;
var stream = new MemoryStream(bArchive);
Stream gzipStream = new GZipInputStream(stream);
Stream gzipStream2 = new GZipInputStream(stream);
TarInputStream tarIn = new TarInputStream(gzipStream);
TarEntry tarEntry;
var strRoot = “”;
while ((tarEntry = tarIn.GetNextEntry()) != null)
{
string name = tarEntry.Name.Replace(‘/’, Path.DirectorySeparatorChar);
if (strRoot == “”)
{
strRoot = name;
if (!strRoot.Contains(expectedCommit)) return false;
}
if (tarEntry.IsDirectory)
continue;
if (Path.IsPathRooted(name))
name = name.Substring(Path.GetPathRoot(name).Length);
name = name.Replace(strRoot, “”);
string outName = Path.Combine(output, name);
string directoryName = Path.GetDirectoryName(outName);
Directory.CreateDirectory(directoryName);
FileStream outStr = new FileStream(outName, System.IO.FileMode.Create);
tarIn.CopyEntryContents(outStr);
outStr.Close();
}
tarIn.Close();
gzipStream.Close();
stream.Close();
return true;
}

What happens here, is that I request the latest archive of my repo from GitHub, which is in TAR.GZ format. What that actually means is that it is a TAR container of files, that has been compressed into a GZip stream.

I unzip the byte array, and pass it into a TarInputStream from SharpZipLib. GitHub wraps all your files into a folder that’s named

infiniteloopltd-AutoBuild-51d1c48\

i.e. {username}-{repo}-{commit}, this means that each file needs to be extracted one by one, since this wrapper folder needs to be removed.

However, the wrapper folder comes in very handy later on.

Now, that’s where everything is nice and sensible, then it gets ugly….

Unfortunately, if you call GetArchive too soon after the webhook is called, then you get the previous commit, and that’s useless, your files don’t update.

So, therefore, I need to read the parameters sent to the webhook to ensure that the commit I am expecting matches the one in the Archive, luckily, there is plenty of data sent to the webhook by github in Json format as follows;

{
  "ref": "refs\/heads\/master",
  "before": "b75dcf060de3710bfd7c054e4869397cc75380ba",
  "after": "90d9567997f96203853e071500136a16b1335c3a",
  "created": false,
  ....

The first 7 chars of this commit guid match the code shown on the website

… and also match the code contained in the wrapper folder, which allows you match the archive with the code. And if it doesn’t match, I just wait 5 seconds and try again.

So my Page_Load is defined as (With some logging)

protected void Page_Load(object sender, EventArgs e)
{
var strLog = “Started at ” + DateTime.Now;
client.Credentials = new Credentials(username, password);
var strJsonPayload = Request.Form[“Payload”];
string strExpectedCommit = “”;
if (!string.IsNullOrEmpty(strJsonPayload))
{
strLog += “\nGitHub payload ” + strJsonPayload;
var jPayload = JObject.Parse(strJsonPayload);
strExpectedCommit = jPayload[“after”].ToString().Substring(0, 7);
strLog += “\nExpected Commit ” + strExpectedCommit;
}
for(int i=0;i<10;i++)
{
var blnOK = DownloadArchive(strExpectedCommit);
if (blnOK) break;
Thread.Sleep(TimeSpan.FromSeconds(5));
strLog += “\nRetry ” + i;
}
File.WriteAllText(output + “buildlog.txt”, strLog);
}

Et Voila … every time you push a code change to GitHub. github calls this webhook, which downloads a zip of the code, and deploys it ontop of itself, creating a self-updating website! 🙂