Network Programming in .NET

A relatively recent change to Webworks apps on BB10, was that you had to include the file:

<script src=”local:///chrome/webworks.js” type=”text/javascript”></script> in order to access the native functionality, such as browser invoke etc. 

Interestingly, if you enable remote web inspector, you can actually see the contents of this file (listed here), and the underlying “gubbins” are exposed.

What webworks.js is actually doing, is that it converts calls such as blackberry.invoke.invoke({target: “sys.browser”,uri: “xxxx”}); to an AJAX call to localhost on port 8472, which does not appear to be accessible from outside the device itself.

The full protocol is unlikely to be published, and it’s not a good idea to use this protocol directly, it’s more for academic interest.

Here is an example AJAX request

1. Request URL:
   http://localhost:8472/blackberry.invoke/invoke
2. Request Method:
   POST
3. Status Code:

   

   200 OK
4. Request Headersview source
   1. Accept-Language:
      en-GB,en;q=0.8
   2. Content-Type:
      application/json
   3. Origin:
      local://
   4. User-Agent:
      Mozilla/5.0 (BB10; Touch) AppleWebKit/537.35+ (KHTML, like Gecko) Version/10.2.1.2141 Mobile Safari/537.35+
5. Request Payloadview source
    

   {,…}

   1. request: “%7B%22target%22%3A%22sys.browser%22%2C%22uri%22%3A%22http%3A%2F%2Fwww.google.com%2Fpatents%2FUSD293241%3Fdq%3DResearch%2Bin%2BMotionw%26source%3Duds%22%7D”
6. Response Headersview source
   1. Access-Control-Allow-Headers:
      Content-Type
   2. Access-Control-Allow-Origin:
      *, local://

I was looking for a way to run automated tests on Javascript, and one of the key requirements of automated tests, are that the results are communicated back. As long as I’m just looking for erroneous / exceptional events then email is a great way to get my attention. But I wanted to find an online service that would provide this for me, rather than writing my own server side code, so I tried a few, and came up with mailgun.net

function ObjToQs(obj)
{
str = ”;
for(key in obj) {
str += key + ‘=’ + obj[key] + ‘&’;
}
str = str.slice(0, str.length – 1);
return str;
}

var page = require(‘webpage’).create(),
url = ‘https://api.mailgun.net/v2/sandboxed7c45b35fc04752931f87faecae4428.mailgun.org/messages&#8217;,
data = {
“from” : “Mailgun Sandbox <postmaster@sandboxed7c45b35fc04752931f87faecae4428.mailgun.org>”,
“to” : “xxxx@gmail.com”,
“subject” : “Hello!”,
“text”: “Test from mailgun / phantomjs”
};

page.customHeaders={‘Authorization’: ‘Basic ‘+btoa(‘api:key-xxxxxx’)};
page.open(url, ‘post’, ObjToQs(data), function (status) {
if (status !== ‘success’) {
console.log(‘FAIL to load the log’);
console.log(status);
} else {
console.log(‘Log success’);

var result = page.evaluate(function () {
return document.body.innerText;
});

console.log(“log Result: ” + result);
phantom.exit();
}
});

I’ve left out my API key, you can get your own at mailgun.net – 

A successful result should be as follows:

C:\research\phantomjs>phantomjs errorreport.js
Log success
log Result: {
“message”: “Queued. Thank you.”,
“id”: “<20140422121333.32180.12028@sandboxed7c45b35fc04752931f87faecae4428.mai
lgun.org>”
}

Plenty of API’s have usage restrictions, either calls per minute, or calls per day etc. But perhaps many of your users search for similar data, and that data remains static. If I’m looking data on the longitude and latitude of a town, then this data is not going to change, then “cloud” caching would be ideal. This solution is not designed for data that may change, i.e. the current weather,

I’ve decided to use Parse.com as the backend, because it has generous free usage limits, (1 milion queries per month), it’s got a Javascript API, and it’s well known, and you can export your data if you ever wanted to move to another system.

So, lets take a simple ajax get request, in the form $.get(url, callback) and write a function with the same signature 

function cachedAjaxGet(url,callback)
{
var query = new Parse.Query(AjaxCache);
query.equalTo(“url”, url);
query.first({
success: function(object) {
if (object == undefined)
{
console.log(“cache-miss”);
$.get(url,function(data)
{
var ajaxCache = new AjaxCache();
ajaxCache.save({data: JSON.stringify(data), url: url},
{
success: function(object)
{
console.log(“stored as ” + object.id);
}
});
callback(data);
});
}
else
{
console.log(“cache-hit”);
var result = JSON.parse(object.get(‘data’));
callback(result);
}
}});
}

Then, don’t forget to initialize parse.com as follows

Parse.initialize(“….”, “…”);
var AjaxCache = Parse.Object.extend(“AjaxCache”);

And of course, include the Parse.com library from the CDN:

<script src=”http://www.parsecdn.com/js/parse-1.2.8.min.js”></script&gt;;

App crashes are bad. They lead to negative app reviews, and often, the user does not, or cannot describe the exact actions taken just before the app crashed.

There are plenty of good app error reporting tools out there, such as exceptional, and airbrake, but lets say, we just want to have an email whenever a Javascript app crashes on a phonegap app. We want it free, and we want it as simple as possible. – But you’d like a stack trace of the error, and a decent description. – Oh, and let’s do it without requiring server-side code.

So, I set up a free account on FormToEmail.com, pressed the generate code button, and got my user_id and form_id, then I drop this function into my app:

function sendCrashReport(message, url , linenumber, column, errorObj)
{
var stack = “”;
if(errorObj !== undefined) //so it won’t blow up in the rest of the browsers
stack = errorObj.stack;
$.post(‘http://formtoemail.com/user_forms.php&#8217;,
{
“user_id” : “……”,
“form_id” : 1,
“message” : message,
“url” : url,
“linenumber” : linenumber,
“stack”: stack
},function(data)
{
console.log(data);
});
}

Then, as close as possible to the initialisation of the page, I write the code:

window.onerror = sendCrashReport;

– Of course, this may not catch compile errors or business logic mess ups, but It’ll catch serious runtime errors that could otherwise crash your app,

AngularJS, and in particular, the ionic framework is a great way to develop Javascript apps, with best-practices baked in. It gives y0u two-way data-binding out of the box, and encourages you to separate logic and display, as well as keeping the global namespace clean. – So, I decided to give it a go, and develop an app using the ionic framework, which is based on AngularJS.

So, after getting it up and running on iOS and Android, after a few tweaks, my last platform was BlackBerry, and when firing it up on the simulator, I was disappointed to see that I couldn’t click on any of the links on the list; which was a simple rendering as follows:

<ion-list>
<ion-item ng-repeat=”item in data” type=”item-text-wrap” href=”#/tab/company/{{item.cacheId}}”>
{{item.title}}
</ion-item>
</ion-list>

Nothing special there, and there was no sign of the bug in Chrome, so I couldn’t debug it locally. I needed a way to debug it directly on the device. Then I remembered the remote web inspector. I’d heard of it, but never used it.

Firstly, to enable it, open the browser on the Z10 emulator, and press … => settings => developer tools => web inspector => ON. It shows a IP address, and port, you type this into any webkit browser on the same local network, and then select your application from the list.

When using bbwp to build your bar file, use the -d switch to enable remote web inspector.

You can then see console.log output, and run jquery commands against the UI. I ran the command $(“.item-content”) to see how  AngularJS had rendered the list, and I saw the problem immediately. AngularJS had “sanitized” the URL links like so href=”unsafe:local://index.htm#…” which meant that they would be blocked. The solution I found to this, could be improved, but it worked for me:

setTimeout(function () {
console.log(“SearchResultCtrl – Desanitizing… BB10 requirement”);
$( “.item-content” ).each( function( index, element ){
$(this).attr(“href”,$(this).attr(“ng-href”));
});
}, 2000);

And, then when I re-built and deployed the app back on the z10 simulator, it worked like a charm.

In hindsight, perhaps in future, I’ll just wrap my android APKs to run on the BB10 devices, but for the moment, I’m still going to stick with Webworks.

In previous versions of Phonegap/Cordova (i.e. 2.x), the InAppBrowser was baked into Cordova.js proper, and was not considered a plugin, it was simply called via “Window.open”, and you got Phonegap’s ‘augmented’ version of the browser control.

However, after upgrading a project to Cordova 3.5.0, the inAppBrowser baffled me, since, although window.open was still working, it wasn’t using the inAppBrowser, so therefore, once window.open was called, the user had left the app, and was then on the web.

Anyway, I finally figured out that you needed to download the InAppBrowser as a plugin. Copy the js files into the src folder (talking about Android here), a couple of images for the forward and back button, and a js file into the www folder.

However that *bloody* js file didn’t work, not sure why, but calls to “require” and “modules”, were simply not there, so, since it was a short file, I decided to pick it apart, and just use the little functionality I required from it.

Now, I’m not advocating this approach, just if you can’t face it any longer, this hack may save you alot of time.

So, instead of Window.open(url, … ), I use a new function AltWindowOpen as follows:

function altWindowOpen(strUrl, strWindowName, strWindowFeatures)
{
console.error(“altWindowOpen()”);
var cb = function(event) {
console.error(“inappBrowser – ” + event.type);
if(event.type == “loadstart”)
{
browserRef_loadstart(event);
}
};
cordova.exec(cb, cb, “InAppBrowser”, “open”, [strUrl, strWindowName, strWindowFeatures]);
}

As you see, I’m only interested in the “loadstart” event, and I want to call browserRef_loadstart() whever that happens. The event object has the property “event.url” so you can see what it’s loaded.

In order to close the browser, you call

cordova.exec(null, null, “InAppBrowser”, “close”, []);

I’ve used console.error instead of console.log so it shows up in logcat more visibly. – Just another trick I discovered tonight.

The Google search API is only really useful for searching one site, or a list of related sites. It also has a stingy limit of 100 queries per day, so I guess, Google isn’t really making it easy for people to use it.

Anyway, if those caveats are acceptable, then here’s some code that can be used to query Google in JSON format, once again, this code is designed for Phonegap – or similar trusted environments.

function GoogleSearch(query,callback)
{
var strUrl = “https://www.googleapis.com/customsearch/v1?&#8221;;
strUrl += “q=” +encodeURIComponent(query);
strUrl+= “&cx=xxxxxx&key=xxxxxxx”;
$.get(strUrl,function(data)
{
console.log(data);
callback(data);
});
}

The callback from this function can be rendered simply using mustache thus:

<div id=”results”></div>
<script id=”resultsTpl” type=”text/template”>
{{#items}}
<a href=”{{link}}”>{{title}}</a><br>
{{/items}}
</script>

Of course, there’s plenty more data there, but all I needed was the link and title.

EDIT: NB: This is no longer supported, please see this instead;

https://blog.dotnetframework.org/2017/04/24/bing-image-search-using-cognitive-search/

Once again, bing dissapoints, I spent an hour this morning getting the new Bing API to work. It’s great that the old documentation isn’t marked as deprecated. Grrr. Yup, if you see anything requiring an AppId, it’s obsolete.

Anyway, here is the new API, which on the free level, gives you 5,000 requests per month to play with. Unfortunately, for my needs, it wasn’t suitable, but the code is here anyway.

A word of note, is that you have to wrap your query in quotes for this to work, and yet again, I’m presuming a high trust environment for this code, such as Phonegap / Webworks / Windows Store / Firefox OS etc. Not the web.

function bingSearch(query,callback)
{
console.log(“bingSearch(” + query + “)”);
$.ajax
({
type: “GET”,
url: “https://api.datamarket.azure.com/Bing/SearchWeb/Web?$format=json&Query=&#8221; + encodeURIComponent(query),
dataType: ‘json’,
async: false,
headers: {
“Authorization”: “Basic ” + btoa(“:YOUR-PASSWORD”)
},
success: function (data){
console.log(“bingSearch() – success:”);
console.log(data);
callback(data);
}
});
}

Hope this is useful to someone.

This is very much on the same vein as my last post on Facebook OAuth, and demonstrates how a very similar method can be used to read and display linkedin Connections from a user.

As with all OAuth systems, the manual flow is as follows (a) Send the user to a login page (b) get a code back (c) exchange this code for an access token (d) use the access token to request data from the provider’s API.

In this example, it is designed for use in situations where XDR is allowed, such as Phonegap, Windows store apps, Firefox os, etc – not a normal website. However, the flow could be adapted using JSONP.

So, step 1. is to send the user to the login page

function openLoginDialog()
{
console.log(“openLoginDialog()”);
var strUrl = “https://www.linkedin.com/uas/oauth2/authorization?&#8221;;
strUrl += “response_type=code”;
strUrl += “&client_id=xxxxxxxxxxxxxx”;
strUrl += “&scope=r_fullprofile%20r_emailaddress%20r_network”;
strUrl += “&state=DCEEFWF45453sdffef424”;
strUrl += “&redirect_uri=http://xxxxxxxxxxxxxxx”;
location.href=strUrl;
}

You will need to set up an “App” at developers.linkedin.com, and you will also need access to a web server, in the above code, you’ll need to replace client_id and redirect_uri. “state” appears to be an extra security measure, but quite an ineffective one, in my mind, it can be any random string.

Once the user logs in, then LinkedIn will refer the user back to your redirect_uri with the “code” parameter added. You have about 10 seconds to exchange this for an access token, any delay at this point, and the key exchange will fail.

function ExchangeCodeForAccessToken(code,callback)
{
console.log(“ExchangeCodeForAccessToken(” + code + “)”);
var strUrl = “https://www.linkedin.com/uas/oauth2/accessToken?&#8221;;
strUrl += “grant_type=authorization_code”;
strUrl += “&code=” + code;
strUrl += “&redirect_uri=http://xxxxxxxxxxxxxxxx”;
strUrl += “&client_id=xxxxxxxx”;
strUrl += “&client_secret=xxxxxxxxxx”;
$.get(strUrl,function(data){
console.log(“ExchangeCodeForAccessToken() ” + data);
if (typeof(data)==”string”)
{
data = eval(“(” + data + “)”);
}
OAUTH_Access_token = data.access_token;
var currentDate = new Date();
OAUTH_Expiry = new Date(currentDate.getTime() + 60*24*60*60*1000);
localStorage.setItem(“OAUTH_Access_token”,OAUTH_Access_token);
localStorage.setItem(“OAUTH_Expiry”,OAUTH_Expiry);
callback();
});
}

Unlike facebook, all tokens have a lifetime of 60 days, it doesn’t vary, and the response of the key exchange is in JSON.  You’ll need to replace the client_id and client_secret from the developer console on Linkedin for your app.

Once you have the access_token, then you can call the linkedin API, here, for example, I am getting my connections, and displaying them in a HTML table using mustache

function logged_in()
{
console.log(“logged_in()”);

var strUrl = “https://api.linkedin.com/v1/people/~/connections?&#8221;;
strUrl += “format=json”;
strUrl += “&oauth2_access_token=” + OAUTH_Access_token;
$.get(strUrl,function(data){
console.log(“logged_in() ” + data);
if (typeof(data)==”string”)
{
data = eval(“(” + data + “)”);

}
connections = data;
var template = $(“#friendlistTpl”).html();
var html = Mustache.to_html(template, data);
$(“#friendlist”).html(html);
});
}

Where my Mustache template is as follows:

<div id=”friendlist”></div>

<script id=”friendlistTpl” type=”text/template”>
<table>
{{#values}}
<tr>
<td>
<img src=”{{pictureUrl}}”>
</td>
<td>
{{firstName}} {{lastName}} <br>
{{headline}}
</td>
</tr>
{{/values}}
</table>
</script>

There is plenty more data returned from this call, and much more to explore in the LinkedIn API.

Although Facebook recommend that you use their SDK to provide access to their data, and only suggest using the manual login flow for Windows 8 apps, or desktop apps, lets just take a look at how you do it. – This particular code is designed for use within Phonegap mobile apps, where cross domain ajax is permitted.

As with all OAuth systems, you never get to see the user’s username and password, you are only provided with proof that the user has logged in.

So, the first step, is to send the user to the Facebook login page.

function openLoginDialog()
{
console.log(“openLoginDialog()”);
var strUrl = buildLoginDialog(OAUTH_App_id,OAUTH_Redirect_Url);
location.href=strUrl;
}

function buildLoginDialog(app_id, redirect_uri)
{
console.log(“buildLoginDialog(” + app_id + “,” + redirect_uri + “)”);
var strUrl = “https://www.facebook.com/dialog/oauth?&#8221;;
strUrl += “client_id=” + app_id;
strUrl += “&redirect_uri=” + redirect_uri;
return strUrl;
}

When openLoginDialog is called, then the user will be redirected to the login page on facebook. You will need to have a App_id set up in Facebook developers. The redirect URL is any page online, as long as it’s authorised for your facebook app.

Once facebook returns the user to your URL, it will append a “code” in the querystring. You need to now exchange this code for an access_key, this is because the code is very easily spoofed, so it needs to be checked.

function ExchangeCodeForAccessToken(code,callback)
{
console.log(“ExchangeCodeForAccessToken(” + code + “)”);
/*
https://graph.facebook.com/oauth/access_token?
client_id={app-id}
&redirect_uri={redirect-uri}
&client_secret={app-secret}
&code={code-parameter}
*/
var strUrl = “https://graph.facebook.com/oauth/access_token?&#8221;;
strUrl += “client_id=” + OAUTH_App_id;
strUrl += “&redirect_uri=” + OAUTH_Redirect_Url;
strUrl += “&client_secret=” + OAUTH_App_Secret;
strUrl += “&code=” + code;
$.get(strUrl,function(data)
{
console.log(data);
OAUTH_Access_token = getQueryVariable(“access_token”,data);
var expiry = getQueryVariable(“expires”,data);
var currentDate = new Date();
OAUTH_Expiry = new Date(currentDate.getTime() + expiry*1000);
localStorage.setItem(“OAUTH_Access_token”,OAUTH_Access_token);
localStorage.setItem(“OAUTH_Expiry”,OAUTH_Expiry);
callback();
});
}

This ideally should be placed on a server, since the App_secret is now visible to anyone who decompiles your code.

Once you have the access_token, then you can make authenticated requests against Facebook, one of the most flexible ways to query facebook’s data is using FQL, which is based on SQL, as it’s name suggests.

Here is some code I used to run a FQL query against facebook, and display the results using mustache

function logged_in()
{
console.log(“logged_in()”);
executeFql(“SELECT name, uid FROM user WHERE uid in (SELECT uid2 FROM friend WHERE uid1 = me())”,function(data)
{
fqlResponse = data;
console.log(JSON.stringify(data));
var template = $(“#friendlistTpl”).html();
var html = Mustache.to_html(template, {data: data});
$(“#friendlist”).html(html);
});
}

function executeFql(query,callback)
{
console.log(“executeFql(” + query + “)”);
var strUrl = “https://api.facebook.com/method/fql.query?&#8221;;
strUrl += “query=” + encodeURI(query);
strUrl += “&access_token=” + OAUTH_Access_token;
strUrl += “&format=json”;
$.get(strUrl,callback);
}

I hope all that info helps someone!