Amazon Glacier is a great place to store data cheaply that you have no urgent requirement to recover quickly. This could be data that you need to store for “safe keeping”, or backups-of-backups. It’s not a replacement for S3, since you can’t retrieve the data quickly.

The CLI examples on the Amazon site are quite complex, since they recommend you break the data into 1MB chunks before uploading. This approach is simpler, but will only work for files up to 4GB.

First, you’ll need to install the AWS CLI and run AWS configure, and enter your IAM access key, secret, and region (i.e. eu-west-1).

You can create a vault via the AWS web interface, this just confirms that you have access to them via the CLI;

aws glacier list-vaults –account-id –

Which returns;

{
“VaultList”: [
{
“VaultARN”: “arn:aws:glacier:eu-west-1:xxxx:vaults/AI-String
-xxxx”,
“VaultName”: “AI-String-xxx”,
“CreationDate”: “2019-08-20T14:19:59.345Z”,
“NumberOfArchives”: 0,
“SizeInBytes”: 0
},
{
“VaultARN”: “arn:aws:glacier:eu-west-1:xxxxxx:vaults/xxxxx
-www-backup”,
“VaultName”: “xxxxx-www-backup”,
“CreationDate”: “2017-08-27T14:45:10.021Z”,
“LastInventoryDate”: “2017-08-28T09:09:29.270Z”,
“NumberOfArchives”: 3,
“SizeInBytes”: 8880671766
}
]
}

Now, you can upload a file as follows;

aws glacier upload-archive –vault-name AI-String-Processing –accou
nt-id – –archive-description “AI String xxxx” –body AI-String-xxxx
.zip

Which returns

{
“location”: “/xxxxx/vaults/AI-String-xxxx/archives/….”,
“checksum”: “…..f”,
“archiveId”: “…Q”
}

Now, if you wait about 4 hours, you should see the archive counter increase in the AWS Web User interface.

Cyprus has the highest rate of car ownership in the world, with 742 cars per 1,000 people, which in 2009, was 651,149 cars out of a total population of 854,802. If your company is in the automotive trade, and is selling cars, car services, or car parts to Cyprus, then knowing the exact make and model of a Cypriot car from it’s license plate, will make your website that much easier to use.

We’ve developed an API at www.api.com.cy that allows you determine the make, model, age and engine size of a Cypriot car from it’s number plate. The website is in Greek, but don’t worry if you don’t speak greek, you can access the same API by clicking any of the other country versions, and access the /CheckCyprus API directly.

Cyprus support

Car registration plates in Cyprus use the /CheckCyprus  endpoint and return the following information:

• Make / Model
• Engine Size
• Age
• Representative image

Sample Registration Number: 

KXJ391

Sample Json:

{

  “Description”: “DODGE CALIBER”,

  “CarMake”: {

    “CurrentTextValue”: “DODGE”

  },

  “CarModel”: {

    “CurrentTextValue”: “CALIBER”

  },

  “MakeDescription”: {

    “CurrentTextValue”: “DODGE”

  },

  “ModelDescription”: {

    “CurrentTextValue”: “CALIBER”

  },

  “EngineSize”: {

    “CurrentTextValue”: “1998”

  },

  “Power”: “115”,

  “RegistrationDate”: “17/03/2010”,

  “ManufactureDate”: “01/01/2006”,

  “Convertible”: “False”,

  “ImageUrl”: “http://www.api.com.cy/image.aspx/@RE9ER0UgQ0FMSUJFUg==”

}

If you want to determine the country from an IP address, then there are a million and one APIs that you can use, but they tend to have free usage limits, plus there is a performance hit of making a network call every time.

You can donwload a free database from Maxmind that you can use to do the lookup offline, but it ads another complexity – that the fact that IP addresses change ownership, and can map to a different country, if you don’t keep the database updated.

So, not only does this demo determine the Country from an IP address from an offline database, but it also has code to automatically download and update the data every month.

So, if you don’t want to read further, and just jump to the code, here is the repo;

https://github.com/infiniteloopltd/MaxMindDemo

So, the basics first. If you are happy with a rough lookup, then just pull the MaxMind Nuget package as follows

install-package MaxMind.GeoIP2

Then download and unzip the GeoLite2-Country.mmdb file from https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz and place it in your bin folder.

Then all you need is this;

var reader = new DatabaseReader(“GeoLite2-Country.mmdb”);
var response = reader.Country(“8.8.8.8”);
Console.WriteLine(response.Country.IsoCode);

Which should say that IP 8.8.8.8 is in the US (It’s Google)

Now, the next fun part is how to update this mmdb file automatically. We can download the TAR.GZ as follows;

var wc = new WebClient();
var bData = wc.DownloadData(“https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz”);
var zippedStream = new MemoryStream(bData);

Which give’s us a Tar GZ file, Which is a file that is in Tar format (*Uncompressed, but a format where multiple files are stored as one) and Gzipped (Compressed). So, we need to Gzip the file, and copy it into a memory stream –

var gzip = new GZipStream(stream, CompressionMode.Decompress);
var mTar = new MemoryStream();
gzip.CopyTo(mTar);
mTar.Seek(0, SeekOrigin.Begin);

Now, with a TAR stream, we have to separate this into a list of objects that define as follows;

public class TarEntry
{
public string FileName { get; set; }
public byte[] Contents { get; set; }
}

The code to parse the TAR file is as follows;

private static List<TarEntry> ExtractTarEntries(Stream stream)
{
var lTarEntries = new List<TarEntry>();
var buffer = new byte[100];
while (true)
{
stream.Read(buffer, 0, 100);
var name = Encoding.ASCII.GetString(buffer).Trim(‘\0’);
if (String.IsNullOrWhiteSpace(name))
break;
stream.Seek(24, SeekOrigin.Current);
stream.Read(buffer, 0, 12);
var size = Convert.ToInt64(Encoding.ASCII.GetString(buffer, 0, 12).Trim(‘\0’), 8);
stream.Seek(376L, SeekOrigin.Current);
var buf = new byte[size];
stream.Read(buf, 0, buf.Length);
lTarEntries.Add(new TarEntry
{
Contents = buf,
FileName = name
});

var pos = stream.Position;

var offset = 512 – (pos % 512);
if (offset == 512)
offset = 0;

stream.Seek(offset, SeekOrigin.Current);
}
return lTarEntries;
}

finally, the code to check the age is as follows;

var fi = new FileInfo(“GeoLite2-Country.mmdb”);
if (!fi.Exists || (DateTime.Now – fi.LastWriteTime).TotalDays > 30)
{
DownloadGeoliteDB();
}

DNS is a very simple protocol, which runs over UDP port 53. It’s primary role is to determine the IP address that is related to a domain. So for example, DNS-JS.com resolves to 95.154.244.106, but it’s also used to determine what server handles the email for a given domain, and lots of other ‘glue’ that holds the internet together.

The issue is, you can’t make a low level packet requests using Javascript alone, so this library helps you make DNS requests using Browser-side javascript.

So, a simple example would be;

DNS.Query(“dns-js.com”,
DNS.QueryType.A,
function(data) {
console.log(data);
});

Which makes a DNS “A” type request for “dns-js.com”, and will return the result as a parameter to your callback function, in this case as “data”, and written to the console.

The full list of Query types are as follows;

A : Address Mapping record
NS : Name Server record
MD : A mail destination (OBSOLETE - use MX)
MF : A mail forwarder (OBSOLETE - use MX)
CNAME : Canonical Name record
SOA : Marks the start of a zone of authority
MB : A mailbox domain name (EXPERIMENTAL)
MG : A mail group member (EXPERIMENTAL)
MR : A mailbox rename domain name (EXPERIMENTAL)
NULL : A Null resource record (EXPERIMENTAL)
WKS : A well known service description
PTR : Reverse-lookup Pointer record
HINFO : Host information
MINFO : Mailbox or mail list information
MX : Mail exchanger record
TXT : Text Record 
RP : Responsible Person
AFSDB : AFS Data Base location
AAAA : IP Version 6 Address record
SRV : Service Location
SSHFP : A SSH Fingerprint resource record
RRSIG : RRSIG rfc3755
AXFR : DNS zone transfer request.
ANY : Generic any query 
URI : A Uniform Resource Identifier (URI) resource record
CAA : A certification authority authorization

and, you can see other demos at https://www.dns-js.com/

When you try to serialize an object that contains an IPAddress, you get the error message Error getting value from ‘ScopeId’ on ‘System.Net.IPAddress’.

So, you have to override how Json.NET (Newtonsoft) serializes this type. Which means you create a class that converts a this problematic type to and from a string youself.

public class IPConverter : JsonConverter<IPAddress>
{
public override void WriteJson(JsonWriter writer, IPAddress value, JsonSerializer serializer)
{
writer.WriteValue(value.ToString());
}

public override IPAddress ReadJson(JsonReader reader, Type objectType, IPAddress existingValue, bool hasExistingValue, JsonSerializer serializer)
{
var s = (string)reader.Value;
return IPAddress.Parse(s);
}
}

Now, you pass this class into JsonSerializerSettings, like so;

var jsonSettings = new JsonSerializerSettings();
jsonSettings.Converters.Add(new IPConverter());
var json = JsonConvert.SerializeObject(result, Formatting.Indented, jsonSettings);

Where results is the object that contains the IP address.

If your emergency backup system for your website, is a standby server, and the idea is to boot up the standby server, and switch DNS if your main server ever goes down, then this is all fine … until the day when the server goes down, and every second costs money in downtime,  And logging into Route53, and changing those DNS records manually seems to take ages, especially, if you have lots of websites, subdomains, MX records … etc.

Here is where creating a script for AWS Route53 in advance can save precious seconds, and makes sure that you can have an orderly and predictable changeover of DNS records, from live to backup, and then backup to live again.

AWS has a command line interface (CLI), and you can download the tools here; https://aws.amazon.com/cli/

Now, you’ll have to set up AWS CLI by typing aws configure, and providing your access key etc.

You’ll need to get the zone ID of each domain you want to configure, which you can get a list of them all by typing aws route53 list-hosted-zones

Next, you’ll need to create a JSON file for each domain you want to change, with details of the new records you need to change. Here is a sample JSON file:

{
“Comment”: “Update A record”,
“Changes”: [
{
“Action”: “UPSERT”,
“ResourceRecordSet”: {
“Name”: “www.domain.com”,
“Type”: “A”,
“TTL”: 300,
“ResourceRecords”: [
{
“Value”: “xxx.xxx.xxx.xxx”
}
]
}
},
{
“Action”: “UPSERT”,
“ResourceRecordSet”: {
“Name”: “domain.com”,
“Type”: “A”,
“TTL”: 300,
“ResourceRecords”: [
{
“Value”: “xxx.xxx.xxx.xxx”
}
]
}
}
]
}

Obviously, domain.com is updated with your domain, and xxx.xxx.xxx.xxx with your new server IP. You should set the TTL low on the switch-to-backup phase, so that the DNS change is held only temporarily by clients.

You should also create the reverse of this file (switch back to live) for when your main server recovers.  The TTL on the switch-to-live file can be longer, as long as you don’t expect your main server to crash again anytime soon!

Now, create a batch file with the command

aws route53 change-resource-record-sets –hosted-zone-id XXXXXX –change-batch file://update.json

Where XXXXX is your zone ID from earlier, and update.json is the file above.

In a real-world example, this batch file, should have lists of lots of domains and subdomains to be changed, and a corresponding batch file that reverses all the changes.

This means, that if your live server ever goes down, you boot up the backup, and run this batch, and the DNS will start directing traffic at your backup server. Once the main server is fixed, you can quickly reverse it, so that traffic is returned to your main server again.

One of the benifits of this, is that, during the stressful event of an outage, you are not wasting time configuring DNS, but can change everything at once, even non-essential websites or domains, that would probably be let crash during an outage.

This is probably the start of a much larger project, but starting today, I launched TweetJS.com which is a browser-based Javascript library, that lets you display tweets on your website. No authentication needed.

Currently, it has two methods, ListTweetsOnUserTimeline and Search, where TweetJs.ListTweetsOnUserTimeline takes two parameters, the username, and a callback function. It will return data on tweets posted by a given user. and TweetJs.Search takes two parameters, the search query, and a callback function. It will return data on tweets that contain the search query.

They are called like this

TweetJs.ListTweetsOnUserTimeline("PetrucciMusic",
function (data) {
    console.log(data);
});

Under the hood, this makes an ajax call back to our server, which then makes the call to Twitter, using our authentication keys.

Since Tweetsharp is no longer being maintained, there is a fork called tweetmoasharp that is actively maintained. It is pretty much a drop-in replacement for Tweetsharp, with minor differences.

Here’s some code that I wrote to retrieve the tweets from a given user’s timeline

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

var service = new TwitterService(_consumerKey, _consumerSecret);
service.AuthenticateWith(_accessToken, _accessTokenSecret);

var xt = service.ListTweetsOnUserTimeline(new ListTweetsOnUserTimelineOptions() { ScreenName = “PetrucciMusic” });
var strJson = service.Response.Response;

This returns tweets from the account https://twitter.com/PetrucciMusic

The TLS1.2 setting may not be needed, depending on what version of .NET you are using, but I was using an old version, which needed it.

Google forms give you a really easy way of asking your users questions, and storing the answers in a google sheet. The big downside, is that you loose alot of control over the way your page is displayed, and you can end up with an iframe in the middle of the page that looks really out of place.

This is where http://googleformrestyler.apixml.net/ can help.

You simply follow these three steps.

Step 1.

Create your google form, and get the embed script by pressing Send » Send via < > » Copy. Then take the “src” parameter out of the iframe element. Which should appear something like this;

https://docs.google.com/forms/..../viewform?embedded=true

Step 2.

Add the following script tag on to your page instead of the iframe element:

http://googleformrestyler.apixml.net/GoogleFormStyler.js

Step 3.

Edit the CSS as you wish

<style>
.freebirdFormviewerViewFooterFooterContainer {
    display: none !important
}
</style>

Now, if you want to change the functionality of the page, you can also hack the javascript. For example, by default, when you submit a Google form, it redirects back to Google. If you want to keep the user on your page, and do something else, here’s some Javascript that you can use to change this behaviour;

var ifrm = document.createElement("iframe");
ifrm.setAttribute("name", "swallow");
document.body.appendChild(ifrm);
ifrm.style.display = 'none';
document.forms[0].target="swallow";
var button = document.querySelectorAll("div[role='button']")[0];
button.addEventListener("click", function()
{
	alert('Here you can intercept the response');
}, true);

What this javascript does, is that it creates an invisible iframe called “swallow”, then changes the form target to output to this hidden iframe. It then attaches to the click event of the submit button and can run custom javascript once the submit is pressed.

Now, that’s the end-user description, How does this system work, let’s look at the inner workings, specifically, the Javascript that gets loaded:

http://googleformrestyler.apixml.net/GoogleFormStyler.js

This contains the following script

(function (XHR) {
    var open = XHR.prototype.open;
    XHR.prototype.open = function (method, url, async, user, pass) {
        this._url = url;
        if (url.indexOf("gstatic.com") !== -1 ||
            url.indexOf("docs.google.com") !== -1) {
            url = "http://googleformrestyler.apixml.net/corsProxy.aspx?base64Url=" + btoa(url);
        }
        open.call(this, method, url, async, user, pass);
    };
})(XMLHttpRequest);
(function() {
    var script = document.currentScript ||
        /*Polyfill*/ Array.prototype.slice.call(document.getElementsByTagName('script')).pop();
    var URL = script.getAttribute('form');
    var xhr = new XMLHttpRequest();
    xhr.open("GET", URL);
    xhr.onload = function() {
        document.write(xhr.response);
    };
    xhr.send();
})();

The first part, hijacks the Ajax functionality on the page, and if the Ajax request is bound for Google (gstatic.com or docs.google.com), and will route it instead via a page called corsProxy.aspx

The second part, makes an ajax request to fetch the contents of the url specified in the form attribute of the script tag, and then document.write’s it out to the page.

CorsProxy.aspx is a ASP.NET script that proxies the request, and returns the result, with the CORS header set.

protected void Page_Load(object sender, EventArgs e)
{
var base64Url = Request.QueryString[“base64Url”];
var data = Convert.FromBase64String(base64Url);
var url = Encoding.ASCII.GetString(data);
var wc = new WebClient();
Response.AppendHeader(“Access-Control-Allow-Origin”, “*”);
var bResponse = wc.DownloadData(url);
Response.BinaryWrite(bResponse);
}