Network Programming in .NET

Greece has a population of 10.72 million, and a 5.28 million passenger cars which is a 49% car ownership rate, which is quite average for an EU country. Based on Eurostat data from 2018, graph below.

Today, we’ve launched our Greek API on the domain πινακιδα.gr which means “License plate” in Greek. The API endpoint is available at http://www.πινακιδα.gr/api/reg.asmx?op=CheckGreece

Car registration plates in Greece use the /CheckGreece  endpoint and return the following information:

• Make / Model
• Age
• Doors
• Engine size
• Representative image

Sample Registration Number: 

POT1309

Sample Json:

{
  "Description": "KIA Picanto",
  "CarMake": {
    "CurrentTextValue": "KIA"
  },
  "CarModel": {
    "CurrentTextValue": "Picanto"
  },
  "MakeDescription": {
    "CurrentTextValue": "KIA"
  },
  "ModelDescription": {
    "CurrentTextValue": "Picanto"
  },
  "Variant": "1.0i 12v Lx",
  "VehicleType": "ΕΙΧ",
  "Doors": 5,
  "EngineBand": 7,
  "EngineSize": {
    "CurrentTextValue": 999
  },
  "RegistrationDate": "01/07/2008",
  "Region": "Dodecanese - Rhodes",
  "ImageUrl": "http://www.πινακιδα.gr/image.aspx/@S0lBIFBpY2FudG8="
}

This issue affects Windows users, specifically Visual Studio with the AWS Toolkit installed, and it happens when the AWS Toolkit keeps a copy of old AWS credentials that may no longer be valid.

Here is the scenario.

I set up my AWS credentials using AWS configure, this stores the credentials in a file /user/.aws/credentials – I run Visual Studio, and AWS Toolkit keeps a copy of the credentials. Now, I decide to change the AWS credentials, and using the CLI again, I change the credentials on my machine. The CLI works fine, but Visual Studio is using old credentials, and breaks with an error like;

UnrecognizedClientException – The security token included in the request is invalid

at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleException(IExecutionContext executionContext, HttpErrorResponseException exception)
at Amazon.Runtime.Internal.ExceptionHandler`1.Handle(IExecutionContext executionContext, Exception exception)
at Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext executionContext, Exception exception)
at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)

Well, what how to clear the AWSToolkit cache?

Go to

C:\Users\<your user>\AppData\Local\AWSToolkit

and

erase RegisteredAccounts.json

Then restart Visual Studio, and it will find the correct credentials.

There are many services out there, that can give you a rotating proxy service, using a pool of IP addresses, and some of those pools are probably huge. However, here is an interesting approach discovered by Black Hills Information Security that can use AWS API Gateway as a proxy – A proxy that can only be used for one domain, but it rotates the Egress IP address with every request, and you’re using the IP address pool of an entire AWS Availability zone, and technically, you could set up multiple region end points, if you wanted.

So, the solution is called FireProx, and it’s a Python script that sets up an API gateway for you, such that you can make a request to the AWS gateway URL, and the request will be proxied to the destination,

For example;

https://3eqe4t8sxh.execute-api.eu-west-1.amazonaws.com/fireprox/

Goes to “http://icanhazip.com&#8221; – and nearly demonstrates the rotating proxy IP.

To run it, in Windows (Assuming you have Python and VirtualEnv already installed, and AWS already configured):

git clone https://github.com/ustayready/fireprox
cd fireprox
py -m virtualenv -p python3 .
env\Scripts\activate
pip install -r requirements.txt
python fire.py

Then you can create an api gateway using;

python fire.py --command create --url http://icanhazip.com

The GitHub repo is here; so you can verify the source yourself; https://github.com/ustayready/fireprox

If you need a quick check to determine the country of origin of your website visitor using Javascript, then you can use this little snippet of code using the ip-api service;

  var s = document.createElement("script");
  s.src = "http://ip-api.com/json/?callback=ipFound";
  document.body.appendChild(s);
  function ipFound(ipInfo)
  {
	  alert("You are from " + ipInfo.countryCode);
  }

It’s free to use (for non-commercial purposes), and uses no API Key, It’s using JSONP which is supported on most browsers.

Evidently, if you want more foolproof check, then a server-side check would be better, but it’s good to help guide your users to the right country, or preselect some options for them.

The AWS Toolkit for Visual Studio is great, when it works, but when it breaks, it’s a headache. The AWS Toolkit allows you publish your .NET code to AWS Lambda, with a API gateway already setup for you. It does this with a Cloud Formation stack under the hood, and this encapsulates the steps required to publish the Lambda, connect and configure the API gateway, and give you a public URL to play with.

Now, as of the 1st of March 2021, AWS deprecated the policy AWSLambdaFullAccess and replaced it with the similarly named AWSLambda_FullAccess – which is one underscore, but perhaps an hour of work to change.

First off, the error isn’t very helpful;

AspNetCoreFunctionRole

UPDATE_FAILED

1 validation error detected: Value ‘AWSLambdaFullAccess‘ at ‘policyArn’ failed to satisfy constraint: Member must have length greater than or equal to 20 (Service: AmazonIdentityManagement; Status Code: 400; Error Code: ValidationError; Request ID: cec22c28-2aae-4de8-9ef7-51880070081d; Proxy: null)

What does that mean, well, after googling, I found out that the policy had been deprecated, and set about editing the cloudformation stack.

So, I went into AWS, found the cloud formation stack, and tried to edit the template. But I couldn’t edit it, just make a copy of it (perhaps there is a better way here!, please leave a comment!) – and then I created a new Cloud Formation Stack, with a new name, but with the policy AWSLambdaFullAccess changed to AWSLambda_FullAccess

Back in Visual Studio, I clicked on the “Publish to AWS Lambda”, then selected the new stack from the drop down list, and pressed publish. Finally it worked!

It gave me a new API endpoint URL, which I didn’t want, and I had to reconfigure some permissions and the VPC that I had applied to the lambda manually.

Not a good experience at all, and I would really like to slap the AWS engineer that for making that change.

---

Following up from this post yesterday, I figured out how to update the existing Cloud Formation Stack, which importantly, allowed me to keep the same URL, and maintained the VPC and custom permissions.

So, First off, you go into the stack – press the Template tab, then View in Designer.

If you flick the language settings from JSON to YAML and back again, you get the JSON in a pretty format, so you can search for the AWSLambdaFullAccess and replace it with AWSLambda_FullAccess – Once finished, you can save the template locally, which you will upload again later., by pressing File > Save

Back in the Stack, press the Change Sets tab, and press Create Change set, select “Replace current template”, then “upload a template file”, and upload the file from the previous step. Press Next > Next > Next, then create change set.

Once you’ve done that, you’ll need to execute the change set.

However, if you’re like me, your stack will be in an invalid state, which you have to fix first, so you need to rollback the changeset, while ignoring the error that stopped the automatic rollback in the first place.

(from: https://aws.amazon.com/premiumsupport/knowledge-center/cloudformation-update-rollback-failed/)

If you want to skip FAILED resources during rollback, complete the following:

1.    From the Stack name column, select the stack that’s stuck in UPDATE_ROLLBACK_FAILED status.

2.    Choose Stack Actions, and then choose Continue update rollback.

3.    In the Continue update rollback dialog box, expand Advanced troubleshooting.

4.    In the Resources to skip – optional section, select the resources that you want to skip.

5.    Choose Continue update rollback.

Once you’ve successfully rolled back the stack, then you can execute the change set, and your cloudformation stack should be back working again, and you can deploy again from Visual Studio.