Archive

Posts Tagged ‘net’

Porting a PHP OAuth Spotler Client to C#: Lessons Learned

Recently I had to integrate with Spotler’s REST API from a .NET application. Spotler provides a powerful marketing automation platform, and their API uses OAuth 1.0 HMAC-SHA1 signatures for authentication.

They provided a working PHP client, but I needed to port this to C#. Here’s what I learned (and how you can avoid some common pitfalls).


🚀 The Goal

We started with a PHP class that:

✅ Initializes with:

  • consumerKey
  • consumerSecret
  • optional SSL certificate verification

✅ Creates properly signed OAuth 1.0 requests

✅ Makes HTTP requests with cURL and parses the JSON responses.

I needed to replicate this in C# so we could use it inside a modern .NET microservice.


🛠 The Port to C#

🔑 The tricky part: OAuth 1.0 signatures

Spotler’s API requires a specific signature format. It’s critical to:

  1. Build the signature base string by concatenating:
    • The uppercase HTTP method (e.g., GET),
    • The URL-encoded endpoint,
    • And the URL-encoded, sorted OAuth parameters.
  2. Sign it using HMAC-SHA1 with the consumerSecret followed by &.
  3. Base64 encode the HMAC hash.

This looks simple on paper, but tiny differences in escaping or parameter order will cause 401 Unauthorized.

💻 The final C# solution

We used HttpClient for HTTP requests, and HMACSHA1 from System.Security.Cryptography for signatures. Here’s what our C# SpotlerClient does:

✅ Generates the OAuth parameters (consumer_key, nonce, timestamp, etc).
✅ Creates the exact signature base string, matching the PHP implementation character-for-character.
✅ Computes the HMAC-SHA1 signature and Base64 encodes it.
✅ Builds the Authorization header.
✅ Sends the HTTP request, with JSON bodies if needed.

We also added better exception handling: if the API returns an error (like 401), we throw an exception that includes the full response body. This made debugging much faster.


🐛 Debugging tips for OAuth 1.0

  1. Print the signature base string.
    It needs to match exactly what Spotler expects. Any stray spaces or wrong escaping will fail.
  2. Double-check timestamp and nonce generation.
    OAuth requires these to prevent replay attacks.
  3. Compare with the PHP implementation.
    We literally copied the signature generation line-by-line from PHP into C#, carefully mapping rawurlencode to Uri.EscapeDataString.
  4. Turn off SSL validation carefully.
    During development, you might disable certificate checks (ServerCertificateCustomValidationCallback), but never do this in production.

using System.Security.Cryptography;
using System.Text;

namespace SpotlerClient
{
 
    public class SpotlerClient
    {
        private readonly string _consumerKey;
        private readonly string _consumerSecret;
        private readonly string _baseUrl = "https://restapi.mailplus.nl";
        private readonly HttpClient _httpClient;

        public SpotlerClient(string consumerKey, string consumerSecret, bool verifyCertificate = true)
        {
            _consumerKey = consumerKey;
            _consumerSecret = consumerSecret;

            var handler = new HttpClientHandler();
            if (!verifyCertificate)
            {
                handler.ServerCertificateCustomValidationCallback = (sender, cert, chain, sslPolicyErrors) => true;
            }

            _httpClient = new HttpClient(handler);
        }

        public async Task<string> ExecuteAsync(string endpoint, HttpMethod method, string jsonData = null)
        {
            var request = new HttpRequestMessage(method, $"{_baseUrl}/{endpoint}");
            var authHeader = CreateAuthorizationHeader(method.Method, endpoint);
            request.Headers.Add("Accept", "application/json");
            request.Headers.Add("Authorization", authHeader);

            if (jsonData != null)
            {
                request.Content = new StringContent(jsonData, Encoding.UTF8, "application/json");
            }

            var response = await _httpClient.SendAsync(request);

            if (!response.IsSuccessStatusCode)
            {
                var body = await response.Content.ReadAsStringAsync();
                return body;
            }

            return await response.Content.ReadAsStringAsync();
        }

        private string CreateAuthorizationHeader(string httpMethod, string endpoint)
        {
            var timestamp = DateTimeOffset.UtcNow.ToUnixTimeSeconds().ToString();
            var nonce = Guid.NewGuid().ToString("N");

            var paramString = "oauth_consumer_key=" + Uri.EscapeDataString(_consumerKey) +
                              "&oauth_nonce=" + Uri.EscapeDataString(nonce) +
                              "&oauth_signature_method=" + Uri.EscapeDataString("HMAC-SHA1") +
                              "&oauth_timestamp=" + Uri.EscapeDataString(timestamp) +
                              "&oauth_version=" + Uri.EscapeDataString("1.0");

            var sigBase = httpMethod.ToUpper() + "&" +
                          Uri.EscapeDataString(_baseUrl + "/" + endpoint) + "&" +
                          Uri.EscapeDataString(paramString);

            var sigKey = _consumerSecret + "&";

            var signature = ComputeHmacSha1Signature(sigBase, sigKey);

            var authHeader = $"OAuth oauth_consumer_key=\"{_consumerKey}\", " +
                             $"oauth_nonce=\"{nonce}\", " +
                             $"oauth_signature_method=\"HMAC-SHA1\", " +
                             $"oauth_timestamp=\"{timestamp}\", " +
                             $"oauth_version=\"1.0\", " +
                             $"oauth_signature=\"{Uri.EscapeDataString(signature)}\"";

            return authHeader;
        }

        private string ComputeHmacSha1Signature(string data, string key)
        {
            using var hmac = new HMACSHA1(Encoding.UTF8.GetBytes(key));
            var hash = hmac.ComputeHash(Encoding.UTF8.GetBytes(data));
            return Convert.ToBase64String(hash);
        }
    }
}

✅ The payoff

Once the signature was constructed precisely, authentication errors disappeared. We could now use the Spotler REST API seamlessly from C#, including:

  • importing contact lists,
  • starting campaigns,
  • and fetching campaign metrics.

📚 Sample usage

var client = new SpotlerClient(_consumerKey, _consumerSecret, false);
var endpoint = "integrationservice/contact/email@gmail.com";
var json = client.ExecuteAsync(endpoint, HttpMethod.Get).GetAwaiter().GetResult();

🎉 Conclusion

Porting from PHP to C# isn’t always as direct as it looks — especially when it comes to cryptographic signatures. But with careful attention to detail and lots of testing, we managed to build a robust, reusable client.

If you’re facing a similar integration, feel free to reach out or clone this approach. Happy coding!

Categories: Uncategorized Tags: , , , ,

🚫 Why AWS SDK for S3 No Longer Works Smoothly with .NET Framework 4.8 — and How to Fix It

In 2024, more .NET developers are finding themselves in a strange situation: suddenly, tried-and-tested .NET Framework 4.8 applications that interact with Amazon S3 start throwing cryptic build errors or runtime exceptions. The culprit? The AWS SDK for .NET has increasingly shifted toward support for .NET Core / .NET 6+, and full compatibility with .NET Framework is eroding.

In this post, we’ll explain:

  • Why this happens
  • What errors you might see
  • And how to remove the AWS SDK altogether and replace it with pure .NET 4.8-compatible code for downloading (and uploading) files from S3 using Signature Version 4.

🧨 The Problem: AWS SDK & .NET Framework 4.8

The AWS SDK for .NET (like AWSSDK.S3) now depends on modern libraries like:

  • System.Text.Json
  • System.Buffers
  • System.Runtime.CompilerServices.Unsafe
  • Microsoft.Bcl.AsyncInterfaces

These dependencies were designed for .NET Core and later versions — not .NET Framework. While it was once possible to work around this with binding redirects and careful version pinning, the situation has become unstable and error-prone.


❗ Common Symptoms

You may see errors like:

Could not load file or assembly ‘System.Text.Json, Version=6.0.0.11’

Or:

Could not load file or assembly ‘System.Buffers, Version=4.0.5.0’

Or during build:

Warning: Unable to update auto-refresh reference ‘system.text.json.dll’

Even if you install the correct packages, you may end up needing to fight bindingRedirect hell, and still not get a working application.


✅ The Solution: Remove the AWS SDK

Fortunately, you don’t need the SDK to use S3. All AWS S3 requires is a properly signed HTTP request using AWS Signature Version 4, and you can create that yourself using standard .NET 4.8 libraries.


🔐 Downloading from S3 Without the AWS SDK

Here’s how you can download a file from S3 using HttpWebRequest and Signature Version 4.

✔️ The Key Points:

  • You must include the x-amz-content-sha256 header (even for GETs!)
  • You sign the request using your AWS secret key
  • No external packages required — works on plain .NET 4.8

🧩 Code Snippet


public static byte[] DownloadFromS3(string bucketName, string objectKey, string region, string accessKey, string secretKey)
{
var method = "GET";
var service = "s3";
var host = $"{bucketName}.s3.{region}.amazonaws.com";
var uri = $"https://{host}/{objectKey}";
var requestDate = DateTime.UtcNow;
var amzDate = requestDate.ToString("yyyyMMddTHHmmssZ");
var dateStamp = requestDate.ToString("yyyyMMdd");
var canonicalUri = "/" + objectKey;
var signedHeaders = "host;x-amz-content-sha256;x-amz-date";
var payloadHash = HashSHA256(string.Empty); // Required even for GET

var canonicalRequest = $"{method}\n{canonicalUri}\n\nhost:{host}\nx-amz-content-sha256:{payloadHash}\nx-amz-date:{amzDate}\n\n{signedHeaders}\n{payloadHash}";
var credentialScope = $"{dateStamp}/{region}/{service}/aws4_request";
var stringToSign = $"AWS4-HMAC-SHA256\n{amzDate}\n{credentialScope}\n{HashSHA256(canonicalRequest)}";

var signingKey = GetSignatureKey(secretKey, dateStamp, region, service);
var signature = ToHexString(HmacSHA256(signingKey, stringToSign));

var authorizationHeader = $"AWS4-HMAC-SHA256 Credential={accessKey}/{credentialScope}, SignedHeaders={signedHeaders}, Signature={signature}";

var request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = method;
request.Headers["Authorization"] = authorizationHeader;
request.Headers["x-amz-date"] = amzDate;
request.Headers["x-amz-content-sha256"] = payloadHash;

try
{
    using (var response = (HttpWebResponse)request.GetResponse())
    using (var responseStream = response.GetResponseStream())
    using (var memoryStream = new MemoryStream())
    {
        responseStream.CopyTo(memoryStream);
        return memoryStream.ToArray();
    }
}
catch (WebException ex)
{
    using (var errorResponse = (HttpWebResponse)ex.Response)
    using (var reader = new StreamReader(errorResponse.GetResponseStream()))
    {
        var errorText = reader.ReadToEnd();
        throw new Exception($"S3 request failed: {errorText}", ex);
    }
}

}
🔧 Supporting Methods


private static string HashSHA256(string text)
{
using (var sha256 = SHA256.Create())
{
return ToHexString(sha256.ComputeHash(Encoding.UTF8.GetBytes(text)));
}
}
private static byte[] HmacSHA256(byte[] key, string data)
{
using (var hmac = new HMACSHA256(key))
{
return hmac.ComputeHash(Encoding.UTF8.GetBytes(data));
}
}
private static byte[] GetSignatureKey(string secretKey, string dateStamp, string region, string service)
{
var kSecret = Encoding.UTF8.GetBytes("AWS4" + secretKey);
var kDate = HmacSHA256(kSecret, dateStamp);
var kRegion = HmacSHA256(kDate, region);
var kService = HmacSHA256(kRegion, service);
return HmacSHA256(kService, "aws4_request");
}
private static string ToHexString(byte[] bytes)
{
return BitConverter.ToString(bytes).Replace("-", "").ToLowerInvariant();
}


📝 Uploading to S3 Without the AWS SDK
You can extend the same technique for PUT requests. The only differences are:

You calculate the SHA-256 hash of the file content

You include a Content-Type and Content-Length header

You use PUT instead of GET

Let me know in the comments if you’d like the full upload version — it follows the same Signature V4 pattern.

✅ Summary
Feature AWS SDK for .NET Manual Signature V4
.NET Framework 4.8 support ❌ Increasingly broken ✅ Fully supported
Heavy NuGet dependencies ✅ ❌ Minimal
Simple download/upload ✅ ✅ (with more code)
Presigned URLs ✅ 🟡 Manual support

Final Thoughts
If you’re stuck on .NET Framework 4.8 and running into weird AWS SDK issues — you’re not alone. But you’re not stuck either. Dropping the SDK and using HTTP + Signature V4 is entirely viable, especially for simple tasks like uploading/downloading S3 files.

Let me know if you’d like a full upload example, presigned URL generator, or if you’re considering migrating to .NET 6+.

Storing data directly in GPU memory with #CLOO in C#

Although I’m not entirely sure of a practical application for this. This application, using C# and CLOO can store arbitrary data in the GPU memory. In this case, I’m picking a large file off the disk, and putting it in GPU memory.

In the case of this NVIDIA Geforce card, the memory is dedicated to the GPU, and not shared with the system, ordinarily.

TL;DR; The Github repo is here – https://github.com/infiniteloopltd/GpuMemoryDemo

The core function is here;

 static void Main()
        {
            var platform = ComputePlatform.Platforms[0];
            var device = platform.Devices.FirstOrDefault(d => d.Type.HasFlag(ComputeDeviceTypes.Gpu));
            var context = new ComputeContext(ComputeDeviceTypes.Gpu, new ComputeContextPropertyList(platform), null, IntPtr.Zero);
            var queue = new ComputeCommandQueue(context, device, ComputeCommandQueueFlags.None);

            const string largeFilePath = "C:\\Users\\fiach\\Downloads\\datagrip-2024.3.exe";
            var contents = File.ReadAllBytes(largeFilePath);

            var clBuffer = Store(contents, context, queue);

            var readBackBytes = Retrieve(contents.Length, clBuffer, queue);

            Console.WriteLine($"Original String: {contents[0]}");
            Console.WriteLine($"Read Back String: {readBackBytes[0]}");
            Console.WriteLine($"Strings Match: {contents[0] == readBackBytes[0]}");

            
            // Memory leak here. 
            //Marshal.FreeHGlobal(readBackPtr);
            //Marshal.FreeHGlobal(buffer);
            
        }

        public static ComputeBuffer<byte> Store(byte[] stringBytes, ComputeContext context, ComputeCommandQueue queue)
        {
            var buffer = Marshal.AllocHGlobal(stringBytes.Length);

            Marshal.Copy(stringBytes, 0, buffer, stringBytes.Length);

            var clBuffer = new ComputeBuffer<byte>(context, ComputeMemoryFlags.ReadWrite, stringBytes.Length);

            queue.Write(clBuffer, true, 0, stringBytes.Length, buffer, null);
            
            return clBuffer;
        }

        public static byte[] Retrieve(int size, ComputeBuffer<byte> clBuffer, ComputeCommandQueue queue)
        {
            var readBackPtr = Marshal.AllocHGlobal(size);

            queue.Read(clBuffer, true, 0, size, readBackPtr, null);

            var readBackBytes = new byte[size];

            Marshal.Copy(readBackPtr, readBackBytes, 0, size);

            return readBackBytes;
        }
    }

we’ll walk through a C# program that demonstrates the use of OpenCL to store and retrieve data using the GPU, which can be beneficial for performance in data-heavy applications. Here’s a breakdown of the code:

1. Setting Up OpenCL Context and Queue

The program begins by selecting the first available compute platform and choosing a GPU device from the platform:

csharpCopy codevar platform = ComputePlatform.Platforms[0];
var device = platform.Devices.FirstOrDefault(d => d.Type.HasFlag(ComputeDeviceTypes.Gpu));
var context = new ComputeContext(ComputeDeviceTypes.Gpu, new ComputeContextPropertyList(platform), null, IntPtr.Zero);
var queue = new ComputeCommandQueue(context, device, ComputeCommandQueueFlags.None);
  • ComputePlatform.Platforms[0]: Selects the first OpenCL platform on the machine (typically corresponds to a GPU vendor like NVIDIA or AMD).
  • platform.Devices.FirstOrDefault(...): Finds the first GPU device available on the platform.
  • ComputeContext: Creates an OpenCL context for managing resources like buffers and command queues.
  • ComputeCommandQueue: Initializes a queue to manage commands that will be executed on the selected GPU.

2. Reading a Large File into Memory

The program then loads the contents of a large file into a byte array:

csharpCopy codeconst string largeFilePath = "C:\\Users\\fiach\\Downloads\\datagrip-2024.3.exe";
var contents = File.ReadAllBytes(largeFilePath);

This step reads the entire file into memory, which will later be uploaded to the GPU.

3. Storing Data on the GPU

The Store method is responsible for transferring the byte array to the GPU:

csharpCopy codevar clBuffer = Store(contents, context, queue);
  • It allocates memory using Marshal.AllocHGlobal to hold the byte array.
  • The byte array is then copied into this allocated buffer.
  • A ComputeBuffer<byte> is created on the GPU, and the byte array is written to it using the Write method of the ComputeCommandQueue.

Note: The Store method utilizes Marshal.Copy to handle memory copying between managed memory (RAM) and unmanaged memory (GPU).

4. Retrieving Data from the GPU

The Retrieve method is responsible for reading the data back from the GPU into a byte array:

csharpCopy codevar readBackBytes = Retrieve(contents.Length, clBuffer, queue);
  • The method allocates memory using Marshal.AllocHGlobal to hold the data read from the GPU.
  • The Read method of the ComputeCommandQueue is used to fetch the data from the GPU buffer back into the allocated memory.
  • The memory is then copied into a managed byte array (readBackBytes).

5. Verifying the Data Integrity

The program prints the first byte of the original and retrieved byte arrays, comparing them to verify if the data was correctly transferred and retrieved:

csharpCopy codeConsole.WriteLine($"Original String: {contents[0]}");
Console.WriteLine($"Read Back String: {readBackBytes[0]}");
Console.WriteLine($"Strings Match: {contents[0] == readBackBytes[0]}");

This checks whether the first byte of the file content remains intact after being transferred to and retrieved from the GPU.

6. Memory Management

The program has a commented-out section for freeing unmanaged memory:

csharpCopy code//Marshal.FreeHGlobal(readBackPtr);
//Marshal.FreeHGlobal(buffer);

These lines should be used to free the unmanaged memory buffers allocated with Marshal.AllocHGlobal to avoid memory leaks, but they are commented out here, leaving room for improvement.

Potential Improvements and Issues

  • Memory Leaks: The program does not properly free the unmanaged memory allocated via Marshal.AllocHGlobal, leading to potential memory leaks if run multiple times.
  • Error Handling: The program lacks error handling for situations like missing GPU devices or file read errors.
  • Large File Handling: For large files, this approach may run into memory constraints, and you might need to manage chunked transfers for efficiency.

In summary, this program demonstrates how to work with OpenCL in C# to transfer data between the host system and the GPU. While it shows the core functionality, handling memory leaks and improving error management should be considered for a production-level solution.