Network Programming in .NET

I wanted to download the description, screenshots, and icons from apps in from the Windows Phone store, which seemed quite easy initially, until I discovered that simply using WebClient to download the HTML from the webpage fired back an error: “Your request appears to be from an automated process.”, so, I had to think of a more ‘natural’ way to extract the data from the page, that’s when I remembered PhantomJS cloud, a hosted service for running PhantomJS, the headless browser software.

I’ve left my API Key out of the code below, you’ll need to sign up for a key yourself, 

var strWPUrl = “http://www.windowsphone.com/en-gb/store/app/barcelona_metro/36c88182-9a6d-e011-81d2-78e7d1fa76f8”;
var strUrl =
“http://api.phantomjscloud.com/single/browser/v1/YOUR_KEY_HERE/?targetUrl=” +
strWpUrl;
const string strScreenshotRegex = @”/images/(?<ScreenshotGuid>[\w-]+)\?imageType=ws_screenshot_large”;
const string strIconGuidRegex = @”/images/(?<IconGuid>[\w-]+)\?imageType=ws_icon_large”;
const string strDescriptionRegex = @”itemprop…description.{27}(?<Description>.*)./pre”;
const string strAppNameRegex = @”app/(?<Name>\w+)”;
var strAppName = Regex.Match(strUrl, strAppNameRegex).Groups[“Name”].Value;
var strPath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + @”\WP\” + strAppName;
if (!Directory.Exists(strPath))
{
Directory.CreateDirectory(strPath);
Directory.CreateDirectory(strPath + @”\Screenshots”);
}

var strHtml = wc.DownloadString(strUrl);
Console.WriteLine(“Downloaded WP url:” + strAppName);
var strDescription = Regex.Match(strHtml, strDescriptionRegex).Groups[“Description”].Value;
var fs = new FileStream(strPath + @”\” + strAppName + “.txt”, FileMode.Create);
var sw = new StreamWriter(fs);
sw.Write(strDescription);
sw.Flush();
sw.Close();
fs.Close();
var strIconGuid = Regex.Match(strHtml, strIconGuidRegex).Groups[“IconGuid”].Value;
var strIconUrl = @”http://cdn.marketplaceimages.windowsphone.com/v8/images/&#8221; + strIconGuid;
wc.DownloadFile(strIconUrl, strPath + @”\Icon.png”);
var strScreenshotGuids = Regex.Matches(strHtml, strScreenshotRegex).Cast<Match>().Select(m => m.Groups[“ScreenshotGuid”].Value);
foreach (var strScreenshotGuid in strScreenshotGuids)
{
var strScreenhotUrl = @”http://cdn.marketplaceimages.windowsphone.com/v8/images/&#8221; + strScreenshotGuid;
wc.DownloadFile(strScreenhotUrl, strPath + @”\Screenshots\” + strScreenshotGuid + “.png”);
}

 

This downloads the HTML via PhantomJS cloud, runs some regexes to get the description of the app, and downloads the icon and screenshots (in original size), and saves them to disk, in a folder called WP on the desktop

If you’re looking at TcpClient to make HTTP requests, without totally exhausting the functionality of HttpWebRequest or WebClient, then you’re making a massive mistake. However, in my case, I had to deal with the case where a website required a cookie with a comma in it, which is against the HTTP protocol, and thus an illegal action when trying to construct a new System.Net.Cookie object, I was therefore forced to a lower level, and use TCP/IP to simulate my HTTP stream.

Starting with the lowest level, my TCP/IP stream is handled through this function, which has HTTPS handling built-in:

private static string SendWebRequest(string url, string request, bool isHttps)
{
using (var tc = new TcpClient())
{
tc.Connect(url, isHttps ? 443 : 80);
using (var ns = tc.GetStream())
{
if (isHttps)
{
// Secure HTTP
using (var ssl = new SslStream(ns, false, ValidateServerCertificate, null))
{
ssl.AuthenticateAsClient(url, null, SslProtocols.Tls, false);
using (var sw = new System.IO.StreamWriter(ssl))
{
using (var sr = new System.IO.StreamReader(ssl))
{
sw.Write(request);
sw.Flush();
return sr.ReadToEnd();
}
}
}
}
// Normal HTTP
using (var sw = new System.IO.StreamWriter(ns))
{
using (var sr = new System.IO.StreamReader(ns))
{
sw.Write(request);
sw.Flush();
return sr.ReadToEnd();
}
}
}
}
}

The SSL handling requires the following delegate to be defined, which simply accepts all server certs

// The following method is invoked by the RemoteCertificateValidationDelegate.
private static bool ValidateServerCertificate(
object sender,
X509Certificate certificate,
X509Chain chain,
SslPolicyErrors sslPolicyErrors)
{
return true; // Accept all certs
}

Then, my GET and POST methods were as follows:

/// <summary>
/// Requests the specified URL.
/// </summary>
/// <param name=”url”>The URL.</param>
/// <param name=”cookies”>The cookies.</param>
/// <returns></returns>
public string Request(string url, List<string> cookies )
{
var lCookieValues = cookies.Select(cookie => cookie.Left(cookie.IndexOf(“;”, StringComparison.Ordinal))).ToArray();
var strCookieValues = String.Join(“;”, lCookieValues);
var address = new Uri(url);
var strHttpRequest = “GET ” + address.PathAndQuery + ” HTTP/1.1\r\n”;
strHttpRequest += “Accept: application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*\r\n”;
strHttpRequest += “Accept-Language: en-US\r\n”;
strHttpRequest += “User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)\r\n”;
strHttpRequest += “Cookie: ” + strCookieValues + “\r\n”;
strHttpRequest += “Connection: close\r\n”;
strHttpRequest += “Host: ” + address.Host + “\r\n\r\n”;

return SendWebRequest(address.Host, strHttpRequest, address.Scheme == Uri.UriSchemeHttps);
}

Then the POST request is as follows:

/// <summary>
/// Requests the specified URL.
/// </summary>
/// <param name=”url”>The URL.</param>
/// <param name=”postdata”>The post data.</param>
/// <param name=”cookies”>The cookies.</param>
/// <returns></returns>
public string Request(string url, string postdata, List<string> cookies)
{
var lCookieValues = cookies.Select(cookie => cookie.Left(cookie.IndexOf(“;”, StringComparison.Ordinal))).ToArray();
var strCookieValues = String.Join(“;”, lCookieValues);
var address = new Uri(url);
var strHttpRequest = “POST ” + address.PathAndQuery + ” HTTP/1.1\r\n”;
strHttpRequest += “Accept: application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*\r\n”;
strHttpRequest += “Accept-Language: en-US\r\n”;
strHttpRequest += “User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)\r\n”;
strHttpRequest += “Cookie: ” + strCookieValues + “\r\n”;
strHttpRequest += “Connection: close\r\n”;
strHttpRequest += “Content-Type: application/x-www-form-urlencoded\r\n”;
strHttpRequest += “Content-Length: “+ postdata.Length +”\r\n”;
strHttpRequest += “Host: ” + address.Host + “\r\n\r\n”;
strHttpRequest += postdata + “\r\n”;
return SendWebRequest(address.Host, strHttpRequest, address.Scheme == Uri.UriSchemeHttps);
}

One new features of Windows Phone 8.1 is the FileOpenPicker, which allows you select any file from the phone, and even from skydrive (ahem, OneDrive).

I tried following one of the examples on-line for how to use the FileOpenPicker, but as soon as I tried to
override OnActivated(IActivatedEventArgs e), It reported no suitable method to override. Which confused me
immensely, but I learned after a bit of messing about that Windows Phone 8.1 apps come in two flavours,
XAML and Silverlight. If you upgrade / retarget a Windows Phone 8 app, you get a Silverlight WP8 app, whereas
if you create one from scratch, you get a XAML based one, and they are VERY VERY different. This
article refers to the SILVERLIGHT flavour, not the XAML version.

Also, a very quirky thing about the FileOpenPicker, is that you’d expect an event or delegate to callback
when the operation has completed. But in fact what happens is that your application is suspended when the
FileOpenPicker opens, and is resumed when the FileOpenPicker closes. This is all to save device memory.

So, what you need to do, is create a handler for “ContractActivated” in App.xaml.cs as follows:

In App.xaml.cs
(Top level)
public FileOpenPickerContinuationEventArgs FilePickerContinuationArgs { get; set; }

(In InitializePhonrApplication)
PhoneApplicationService.Current.ContractActivated += Application_ContractActivated;

// Code to execute when the application is activated (brought to foreground)
// This code will not execute when the application is first launched
private void Application_ContractActivated(object sender, IActivatedEventArgs e)
{
var filePickerContinuationArgs = e as FileOpenPickerContinuationEventArgs;
if (filePickerContinuationArgs != null)
{
this.FilePickerContinuationArgs = filePickerContinuationArgs;
}
}

Then in your page, you add a button called btnSelectFile and kick off the FileOpenPicker as follows

In [YourPage].xaml.cs

private void btnSelectFile_Click(object sender, RoutedEventArgs e)
{
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.FileTypeFilter.Add(“*”);
openPicker.PickSingleFileAndContinue();
}

In order to handle callback from the FileOpenPicker once the app resumes, then you have to override the OnNavigatedTo method, and check to see if the FilePickerContinuationArgs field has been populated.

protected override void OnNavigatedTo(NavigationEventArgs e)
{
var app = App.Current as App;
if (app.FilePickerContinuationArgs != null)
{
this.ContinueFileOpenPicker(app.FilePickerContinuationArgs);
}
}

This then calls some code to pop up a messagebox with the name of the file, and copy the file to local storage for further
processing.

public void ContinueFileOpenPicker(FileOpenPickerContinuationEventArgs args)
{
MessageBox.Show(“Picked file: ” + args.Files[0].Name);
CopyPickedFile(args.Files[0]);
}

private async void CopyPickedFile(StorageFile file)
{
await file.CopyAsync(ApplicationData.Current.LocalFolder, file.Name, NameCollisionOption.ReplaceExisting);
// now the picked file is in local storage
}

All this code is used in the Windows Phone 8.1 version of http://www.printfromWindowsPhone.com and the app
can be downloaded from Windows Phone Store http://www.windowsphone.com/en-gb/store/app/print/da1074a1-8fbe-4f87-9592-52b2d67d966d

This was a horrible unhelpful message when I tried to reference a DLL to a Windows Phone 8 project, that I had downloaded off the internet. The solution is simple, find the DLL in windows explorer, right click on it, and select “unblock”. 

The same error occurred in a Windows Phone 7 project in VS 2010, but the error message was much more helpful, Great to see Microsoft going backwards.

One of the limitations of an iPhone or iPad, is that you can’t directly connect a printer to it, and often, trying to connect to a wifi printer does not work, or is far too complicated. This simple new system that we’ve developed called “Print from iPad” see http://www.printfromipad.com allows you to print from your iPad by simply forwarding an email to your account at @printfromipad.com

Print any document or image directly from your iPad. Works with any printer, WiFi or USB. Follow this 3-step simple setup:

• Create an account with us to set up your free 10Mb online print queue.
• Download our Print From iPad printer spooler for Windows PC.
• Print anything from your iPad by emailing it to you@PrintFromIPad.com

A key exercise in app store optimisation (ASO) is to ensure that your app title is in the language searched for
by your user. Therefore, my app “Free SMS”, should be written “Бесплатные СМС” in Russian.

This tool makes it really easy to do so. (http://engine-designs.com/2011/08/17/wp7-appreslib-dll-generator/)
Following the instructions are easy, but the steps are as follows:
1. Modify Sample.xml to your app name, translated into all supported languages:
– example:
<Language Name="French" LocaleID="040c“>
Mon App
Mon Tile

2. Run WPAppResLib.exe Sample.exe, and a folder will be generated with a dll and mui files. Copy all these files into your project root, and in Visual Studio, select View all files (refresh), Include in project, and change the build action to content.

3. Test it by switching the language to French on the emulator, and the app title should change.

Luhn Check algorithm in Javascript

An integral part of the generation of IMEI numbers is the Luhn Check algorithm, which determines the final digit in an IMEI number

 // Javascript code copyright 2009 
 // This code may be used freely, as long as this copyright notice is intact.
 function Calculate(Luhn)
 {
	var sum = 0;
    for (i=0; i<luhn.length; for="" i="" (i="Luhn.length-1;" (0,1,2,3,4,-4,-3,-2,-1,0);="" array="" delta="new" var="" }="" +="parseInt(Luhn.substring(i,i+1));" sum="" {="" )="" i++="">0; i-=2 )
    {		
		var deltaIndex = parseInt(Luhn.substring(i,i+1));
		var deltaValue = delta[deltaIndex];	
		sum += deltaValue;
	}	
	var mod10 = sum % 10;
	mod10 = 10 - mod10;	
	if (mod10==10)
	{		
		mod10=0;
	}
	return mod10;
 }

 function Validate(Luhn)
 {
	var LuhnDigit = parseInt(Luhn.substring(Luhn.length-1,Luhn.length));
	var LuhnLess = Luhn.substring(0,Luhn.length-1);
	if (Calculate(LuhnLess)==parseInt(LuhnDigit))
	{
		return true;
	}	
	return false;
 }

Pretty cool stuff to see C# code run natively on Android, using Xamarin and Mono. Not sure if I’ll be moving from Phonegap yet, but, it’s got great potential.

Here’s the Main Activity in C#:

using System;

using Android.App;
using Android.Content;
using Android.Runtime;
using Android.Views;
using Android.Widget;
using Android.OS;

namespace XamarinHelloWorld
{
[Activity (Label = “Xamarin-HelloWorld”, MainLauncher = true, Icon = “@drawable/icon”)]
public class MainActivity : Activity
{
int count = 1;

protected override void OnCreate (Bundle bundle)
{
base.OnCreate (bundle);

// Set our view from the “main” layout resource
SetContentView (Resource.Layout.Main);

// Get our button from the layout resource,
// and attach an event to it
Button button = FindViewById<Button> (Resource.Id.myButton);

button.Click += delegate {
button.Text = string.Format (“{0} clicks!”, count++);
};
}
}
}

 

And, I had to tweak the AndroidManifest.xml:

<?xml version=”1.0″ encoding=”utf-8″?>
<manifest xmlns:android=”http://schemas.android.com/apk/res/android” android:versionCode=”1″ android:versionName=”1.0″ package=”Xamarin_HelloWorld.Xamarin_HelloWorld”&gt;
<uses-sdk android:targetSdkVersion=”19″ android:minSdkVersion=”19″ />
<application android:label=”Xamarin-HelloWorld”></application>
</manifest>

This was because I was using an API level 19 Emulator, and got this “minimum android version not supported by device” xamarin error whenever I tried to kick it off.