The Misleading IndexOutOfRangeException That Means “Your List Isn’t Thread-Safe”

May 11, 2026 Infinite Loop Development Ltd Leave a comment

If you’ve ever seen a stack trace like this in a .NET application:

			
System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Collections.Generic.List`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.WhereListIterator`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at YourCode.SomeMethod(...)

		

…and you stared at the offending line — something innocuous like myList.Where(x => x.IsActive).ToList() — wondering how on earth a LINQ query could be indexing outside an array, you’ve run into one of the most misleadingly-named exceptions in the framework.

The message says “index out of bounds.” The actual problem is a race condition.

What the stack trace is telling you

Read the frames from the bottom up. Your code called ToList(). ToList() calls the List<T> constructor that takes an IEnumerable<T>. That constructor enumerates the Where iterator, which in turn enumerates the underlying List<T> via its Enumerator. And it’s Enumerator.MoveNext() that throws.

So the iterator is walking your list, and at some point it tries to read element N, and N is past the end of the internal array.

How can that happen? The list got smaller — or its internal buffer got swapped — while the enumerator was mid-walk.

List<T> is implemented as a wrapper around a T[] array plus a _size field. The enumerator captures a reference to the list at construction time, then on each MoveNext() it increments an index and reads list._items[index]. If another thread calls Remove, Clear, or triggers a resize via Add between two MoveNext() calls, the array your enumerator is reading from may have been replaced with a smaller one, or the items may have been shuffled. The result: an index that was valid when you started iterating is no longer valid, and you get IndexOutOfRangeException.

Why you don’t get the “nice” exception

List<T> does have a version-check mechanism. Every mutation increments an internal _version field, and the enumerator records the version it started with. If MoveNext() notices the version has changed, it throws InvalidOperationException: Collection was modified; enumeration operation may not execute — the exception most .NET developers have seen at least once and immediately recognise as a concurrency or mid-loop-mutation bug.

The catch: that version check happens after the index increment and array access. If the racing thread mutates the list in a way that shrinks the array between those two operations, you hit the raw IndexOutOfRangeException before the version check ever runs. Same root cause, much worse error message.

This is also why the bug is so frustrating to reproduce. It depends on precise interleaving of two threads down to the instruction level. You can hammer it in a test loop and not see it for a million iterations, then it triggers twice in five minutes in production.

The minimal reproduction

			
var list = new List<int>();
for (int i = 0; i < 1000; i++) list.Add(i);
// Thread A — repeatedly enumerate
Task.Run(() =>
{
    while (true)
    {
        var copy = list.Where(x => x > 0).ToList();
    }
});
// Thread B — repeatedly mutate
Task.Run(() =>
{
    var rng = new Random();
    while (true)
    {
        if (list.Count > 0) list.RemoveAt(rng.Next(list.Count));
        list.Add(rng.Next());
    }
});

		

Run that for a few seconds and you’ll get either InvalidOperationException or IndexOutOfRangeException — sometimes both, on different runs. The exception you get is essentially a coin flip determined by exactly when the second thread’s mutation lands relative to the first thread’s bounds check.

The diagnostic giveaway

The single most useful signal in the stack trace is this frame:

   at System.Collections.Generic.List`1.Enumerator.MoveNext()

IndexOutOfRangeException originating from List<T>.Enumerator.MoveNext almost always means concurrent modification. The enumerator’s own code is straightforward — there’s no realistic way for it to compute a bad index on its own. Something outside the enumerator changed the list while it was looking away.

If your stack trace shows that frame, stop looking for off-by-one errors in your LINQ predicate. Start looking for which other thread is writing to the same list.

Fixing it

The fix depends on the access pattern. In rough order of preference:

Use the right collection type. System.Collections.Concurrent has thread-safe equivalents tuned for different shapes of access:

ConcurrentBag<T> — many writers, occasional bulk drain. No keyed lookup or removal by item.
ConcurrentQueue<T> / ConcurrentStack<T> — FIFO / LIFO producer-consumer pipelines.
ConcurrentDictionary<TKey, TValue> — by far the most generally useful. Supports thread-safe add, remove, lookup, and snapshot enumeration via .Keys and .Values. If you’re keying items by an ID, this is almost always what you want.

Crucially: .Values on a ConcurrentDictionary returns a snapshot, so iterating it is safe even if other threads are mutating the dictionary at the same time. No exceptions, no locks.

Lock around access. If you’re stuck with List<T> — maybe the API surface is fixed, or the contention is low enough that the overhead doesn’t matter — wrap every read and every write in a lock on a dedicated private object. Every read and every write. Missing one is enough to bring the bug back.

			
private static readonly object _lock = new object();
private static readonly List<Thing> _items = new List<Thing>();
public static List<Thing> GetActiveThings()
{
    lock (_lock)
    {
        return _items.Where(t => t.IsActive).ToList();
    }
}
public static void AddThing(Thing t)
{
    lock (_lock) { _items.Add(t); }
}

		

The ToList() inside the lock is deliberate — it forces the enumeration to complete before the lock is released, so the returned list is a safe, isolated copy the caller can work with at leisure.

Snapshot, then iterate. A halfway measure for read-mostly workloads:

var snapshot = Volatile.Read(ref _items).ToArray();

…paired with copy-on-write semantics for the writers. This is the pattern behind ImmutableList<T> from System.Collections.Immutable, which is worth knowing about for scenarios with many readers and rare writers.

What I’d take away from this

Two things.

First: when you see IndexOutOfRangeException coming out of a LINQ chain on a List<T>, your first hypothesis should be “another thread is writing to this list,” not “my predicate has a bug.” The stack trace looks like a logic error and it almost never is.

Second: List<T> being non-thread-safe is one of those facts every .NET developer knows in the abstract and still trips over in practice, because the framework gives you no help at all until something explodes. There’s no ThrowIfShared mode, no Roslyn analyzer that flags static List<T> fields, no runtime check at write time. The only feedback you get is a confusing exception from deep inside an enumerator, possibly weeks after deployment.

The fix is almost always “use the right type from System.Collections.Concurrent.” It costs you nothing in code clarity and saves you from a class of bug that’s genuinely painful to track down once it’s loose in production.

Categories: Uncategorized Tags: ai, artificial-intelligence, cloud, java, technology

Thread Pool Exhaustion in ASP.NET: The Async Database Trap

May 7, 2026 Infinite Loop Development Ltd Leave a comment

If you’ve ever migrated a working ASP.NET application from synchronous database calls to async, and suddenly found yourself hitting connection pool timeouts under load, you’ve likely fallen into one of the most subtle and destructive traps in the .NET ecosystem: sync-over-async deadlock.

The Symptom

Everything works fine in development. You push to production, traffic picks up, and then:

			
Timeout expired. The timeout period elapsed prior to obtaining a connection 
from the pool. This may have occurred because all pooled connections were in 
use and max pool size was reached.

Your database isn’t overloaded. Your queries are fast. But connections are being swallowed and never returned.

What Actually Happens

To understand the deadlock, you first need to understand two things: the ASP.NET synchronization context, and what blocking on an async method actually does.

The Synchronization Context

In classic ASP.NET (WebForms and MVC on the traditional pipeline), each request runs with a synchronization context that ensures continuations — the code that runs after an await — resume on the same thread that started the request. This is a design choice that simplifies state management, but it has a fatal implication when you block.

The Deadlock Sequence

Consider this code:

csharp

			
// Somewhere in a sync method:
var result = GetDataAsync().Result; // ← the problem

Here’s what happens step by step:

Thread A handles the request and calls GetDataAsync().Result
Thread A is now blocked — it’s sleeping, waiting for the Task to complete
GetDataAsync() runs its SQL query asynchronously and completes
The async machinery looks for a thread to resume on — but the synchronization context says it must resume on Thread A
Thread A is blocked waiting for the task. The task is waiting for Thread A. Neither can proceed.

This is a classic deadlock. The thread never releases, the SQL connection it holds is never returned to the pool, and every subsequent request that hits the same code path adds another frozen thread and another stranded connection.

Why It Only Surfaces Under Load

With light traffic, the thread pool has spare threads. The continuation sneaks onto a different free thread and completes before the pool runs dry. As concurrency increases, all available threads become blocked, no free thread exists to run any continuation, and the whole system seizes.

This is why the bug can pass development and staging entirely undetected.

The Broken Pattern

csharp

			
public DataTable GetUserData(string userId)
{
    // Blocking on an async method — dangerous in ASP.NET
    return GetUserDataAsync(userId).Result;
}
public async Task<DataTable> GetUserDataAsync(string userId)
{
    using var conn = new SqlConnection(connectionString);
    using var cmd = new SqlCommand("sp_GetUser @1", conn);
    cmd.Parameters.AddWithValue("@1", userId);
    await conn.OpenAsync();
    using var reader = await cmd.ExecuteReaderAsync();
    var dt = new DataTable();
    dt.Load(reader);
    return dt;
}

		

The async method itself is fine. The problem is the caller blocking on it with .Result.

The Fix: Async All the Way Down

The only correct solution is to await the entire call chain without any blocking calls. There must be no .Result, .Wait(), or .GetAwaiter().GetResult() anywhere in the path from the entry point to the database.

csharp

			
// ✅ Correct: full async chain
public async Task<DataTable> GetUserDataAsync(string userId)
{
    using var conn = new SqlConnection(connectionString);
    using var cmd = new SqlCommand("sp_GetUser @1", conn);
    cmd.Parameters.AddWithValue("@1", userId);
    await conn.OpenAsync();
    using var reader = await cmd.ExecuteReaderAsync();
    var dt = new DataTable();
    dt.Load(reader);
    return dt;
}

		

And the caller:

csharp

var data = await GetUserDataAsync(userId); // ✅ not .Result

The WebForms Special Case

WebForms Page_Load is synchronous by signature, which tempts developers to block. The correct bridge is RegisterAsyncTask:

csharp

			
protected void Page_Load(object sender, EventArgs e)
{
    RegisterAsyncTask(new PageAsyncTask(DoWorkAsync));
}
private async Task DoWorkAsync()
{
    var data = await GetUserDataAsync(userId);
    // ... use data
}

		

RegisterAsyncTask is ASP.NET’s own sanctioned mechanism for running async work from a sync page lifecycle event. It does not block, does not hold threads, and allows the page pipeline to handle async completion correctly.

Coexisting Sync and Async

A pragmatic migration strategy — rather than converting everything at once — is to maintain both sync and async versions of database methods, and use each only from the appropriate call path:

csharp

			
// Sync version — for legacy sync call paths
public static DataTable BoundPopulateDataTable(string command, string[] parameters)
{
    using var conn = new SqlConnection(ConnectionString);
    using var cmd = new SqlCommand(command, conn);
    cmd.Parameters.AddRange(ConvertSqlParameters(parameters).ToArray());
    conn.Open();
    using var reader = cmd.ExecuteReader();
    var dt = new DataTable();
    dt.Load(reader);
    return dt;
}
// Async version — only called from async paths
public static async Task<DataTable> BoundPopulateDataTableAsync(string command, string[] parameters)
{
    using var conn = new SqlConnection(ConnectionString);
    using var cmd = new SqlCommand(command, conn);
    cmd.Parameters.AddRange(ConvertSqlParameters(parameters).ToArray());
    await conn.OpenAsync();
    using var reader = await cmd.ExecuteReaderAsync();
    var dt = new DataTable();
    dt.Load(reader);
    return dt;
}

		

The discipline required is simple: never call the async version from a sync context, and never block on it.

Quick Diagnostic Checklist

If you’re seeing connection pool timeouts after an async migration, scan your codebase for these patterns:

Pattern	Risk
`someTask.Result`	❌ Deadlock
`someTask.Wait()`	❌ Deadlock
`someTask.GetAwaiter().GetResult()`	❌ Deadlock
`await someTask`	✅ Safe
`RegisterAsyncTask(...)`	✅ Safe WebForms bridge

Summary

The async deadlock in ASP.NET is invisible under low load, catastrophic under real traffic, and trivially easy to introduce during a migration. The root cause is always the same: blocking a thread on an async operation inside a synchronization context that needs that thread to resume.

The rule is simple and absolute: if you make a method async, every caller must also be async, all the way to the top of the call stack. There are no shortcuts. .Result is not a shortcut — it’s a time bomb.

Done correctly, async database access is genuinely more scalable. Done incorrectly, it’s worse than sync in every way.

Categories: Uncategorized Tags: ai, artificial-intelligence, chatgpt, llm, technology

The dangers of Parallel.ForEach(… , async (item)) in IIS

April 28, 2026 Infinite Loop Development Ltd Leave a comment

A single, trivial exception — one that your code already has a catch block for — shouldn’t be able to bring down your entire IIS web server. But it can, and it will, if you combine Parallel.ForEach with an async lambda. This post explains exactly why it happens, how to spot it in the Windows Event Log, and how to fix it permanently.

The Setup

You have a method that needs to perform the same async operation against multiple items — calling a set of external APIs, processing a batch of records, sending a collection of requests. You reach for Parallel.ForEach because it sounds like the right tool: parallel work, multiple items, run them all at once. You even add a try/catch inside the lambda because you’re being responsible. It looks like this:

			
Parallel.ForEach(items, async (item) =>
{
    try
    {
        var result = await ProcessItemAsync(item);
        lock (results) { results.Add(result); }
    }
    catch (ItemNotFoundException)
    {
        // item not found - fine, skip it
    }
    catch (Exception ex)
    {
        lock (errors) { errors.Add(ex); }
    }
});

		

This looks safe. It has error handling. It uses async/await. It compiles without a single warning. And it will crash your IIS worker process (w3wp.exe) the moment any exception is thrown after an await.

Why It Crashes: The async void Trap

Parallel.ForEach was designed before async/await existed in C#. It expects a synchronous Action<T> delegate. When you pass it an async lambda, something subtle and dangerous happens: the compiler silently treats the lambda as returning void rather than Task.

This is the async void anti-pattern, and it has one devastating property: any exception thrown inside it cannot be caught by any caller. It escapes directly to the thread’s synchronisation context — and on a raw ThreadPool thread, that means it goes completely unhandled.

Here is the exact sequence of events that kills your server:

Parallel.ForEach fires the lambda for each item in the collection
Each lambda hits the first await and suspends, returning control immediately
Parallel.ForEach sees each lambda return (as void) and considers its job done
Parallel.ForEach exits — the method returns to its caller — everything looks fine
Milliseconds later, the awaited operations complete and the continuations resume on raw ThreadPool threads
An exception is thrown inside one of those continuations
The try/catch inside the lambda? It only catches exceptions thrown before the first await. After the await, the lambda has already returned as far as Parallel.ForEach is concerned
The exception has no owner, no observer, no catch block — it propagates to the ThreadPool itself
In .NET 4.0 and later, an unhandled exception on a ThreadPool thread terminates the process
w3wp.exe crashes. IIS restarts the application pool. All in-flight requests are lost

The particularly insidious part is that the try/catch gives you a false sense of security. You can see it right there in the code. But it doesn’t work the way you expect once an await is involved.

A Minimal Reproduction

You don’t need a complex codebase to reproduce this. The following is all it takes:

			
public static void CrashIIS()
{
    Parallel.ForEach(new[] { 1, 2, 3 }, async (item) =>
    {
        await Task.Delay(100); // simulate any async I/O
        throw new Exception("This kills w3wp.exe");
        // After the await, this runs on an orphaned ThreadPool thread
        // The process terminates
    });
    // Parallel.ForEach has already returned here
    // The crash happens 100ms later
}

		

Call that from any ASP.NET request handler — a controller action, an HttpHandler, anywhere — and your application pool will crash within moments. The caller gets no exception. The HTTP response may even succeed before the crash occurs. The next user to make any request gets a 503.

Even wrapping the call in a try/catch at the call site doesn’t help:

			
try
{
    Parallel.ForEach(new[] { 1, 2, 3 }, async (item) =>
    {
        await Task.Delay(100);
        throw new Exception("Crash");
    });
}
catch (Exception ex)
{
    // This NEVER fires.
    // The exception doesn't happen until after Parallel.ForEach
    // has already exited this try block entirely.
    Log(ex);
}

		

The catch block is long gone by the time the exception is thrown. This is what makes the pattern so dangerous — it looks exception-safe at every level, and isn’t.

How It Appears in the Windows Event Log

When this crash occurs, it leaves a very specific fingerprint in the Windows Event Log. Open Event Viewer → Windows Logs → Application and look for two entries appearing within seconds of each other.

Entry 1: .NET Runtime — Unhandled Exception

Source: .NET Runtime
Event ID: 1026

			
Application: w3wp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: YourNamespace.YourException
   at YourClass.YourMethod()
   at SomeClass+<SomeMethod>d__3.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task)
   at SomeClass+<>c__DisplayClass4_0+<<YourParallelMethod>b__0>d.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.ExecutionContext.RunInternal(...)
   at System.Threading.ExecutionContext.Run(...)
   at System.Threading.QueueUserWorkItemCallback.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

		

The key indicators are at the bottom of the stack trace:

QueueUserWorkItemCallback.ExecuteWorkItem()
ThreadPoolWorkQueue.Dispatch()

These tell you the exception surfaced on a raw ThreadPool work item with no managed owner — the classic signature of an orphaned async continuation. You will also see compiler-generated state machine names like <YourParallelMethod>b__0>d.MoveNext() in the trace, confirming the exception came from inside an async lambda. The angle brackets and the b__ notation are the C# compiler’s naming convention for anonymous methods and lambdas.

Entry 2: Application Error — w3wp.exe Fault

Source: Application Error
Event ID: 1000

			
Faulting application name: w3wp.exe
Faulting module name: KERNELBASE.dll
Exception code: 0xe0434352

Exception code 0xe0434352 is the Windows error code for a managed (.NET) exception that has escaped to the Win32 layer. It’s the OS-level record of a .NET exception killing a process. When you see this code combined with KERNELBASE.dll as the faulting module, a .NET unhandled exception is the cause.

What to Look For — Summary

Signal	Where	What it means
`ThreadPoolWorkQueue.Dispatch()` at bottom of stack	Event ID 1026, .NET Runtime	Exception from orphaned async continuation
Compiler-generated names like `b__0>d.MoveNext()`	Event ID 1026, .NET Runtime	Exception came from inside an async lambda
Exception code `0xe0434352`	Event ID 1000, Application Error	.NET exception killed the process
Faulting module: `KERNELBASE.dll`	Event ID 1000, Application Error	Managed exception, not a native crash
Both entries within seconds of each other	Application log	Single event caused immediate process termination

The Effect on IIS

When w3wp.exe terminates due to an unhandled exception, IIS detects the process death and marks the application pool as faulted. Depending on your Rapid Fail Protection settings (found in IIS Manager → Application Pools → Advanced Settings), IIS will either:

Restart the worker process automatically — users experience a brief outage and then service resumes, with the first request after restart being slow due to application warm-up
Disable the application pool if failures occur too frequently within the Rapid Fail Protection window (default: 5 failures in 5 minutes) — this results in a persistent 503 until an administrator manually starts the pool again

This is worth understanding because thread exhaustion and this crash pattern look identical from the outside — both produce 503 errors — but they behave very differently. Thread exhaustion self-recovers when load drops. A crashed application pool requires either automatic restart (if Rapid Fail Protection hasn’t tripped) or manual intervention. If your team is regularly performing IISResets to recover from outages, a crash like this is a more likely culprit than thread exhaustion.

The Fix

The correct replacement for Parallel.ForEach with async work is Task.WhenAll, which is async-native and properly propagates exceptions back to the awaiting caller:

			
public static async Task<IReadOnlyList<Result>> ProcessAllAsync(IEnumerable<Item> items)
{
    var tasks = items.Select(async item =>
    {
        try
        {
            return await ProcessItemAsync(item);
        }
        catch (ItemNotFoundException)
        {
            return Result.Empty;
        }
    });
    // All items processed in parallel.
    // Exceptions surface here, as AggregateException, to a proper awaiter.
    return await Task.WhenAll(tasks);
}

		

With Task.WhenAll:

All items are processed in parallel — no performance regression
Every async continuation is properly tracked by the Task infrastructure
Exceptions are collected and re-thrown as AggregateException when awaited — to a caller that can handle them
The process does not terminate

As an immediate safety net while refactoring, you can also add a global handler in Global.asax that prevents process termination from unobserved task exceptions:

			
// In Application_Start (Global.asax)
TaskScheduler.UnobservedTaskException += (sender, args) =>
{
    Logger.Error("Unobserved task exception", args.Exception);
    args.SetObserved(); // prevents process termination
};

		

This is a safety net, not a fix — the underlying orphaned tasks still exist and their results are still lost. But it prevents a single unhandled background exception from taking down your entire server while you work through a proper refactor.

The Rule

The rule to remember is simple: never pass an async lambda to Parallel.ForEach. The two are fundamentally incompatible. Parallel.ForEach has no understanding of Task, does not await the work it fires, and any exception thrown after the first await inside your lambda will be orphaned on the ThreadPool. In .NET 4.0 and later, that means process termination.

The pattern is particularly easy to introduce because it compiles cleanly, looks reasonable, and even appears to have proper error handling. The only sign something is wrong is your server going down.

When you need parallel async work, use Task.WhenAll. It was designed for exactly this purpose.

Found this useful? If you’re diagnosing IIS instability, check your application pool’s Rapid Fail Protection settings and review Event Viewer’s Application log for Event ID 1026 with ThreadPoolWorkQueue.Dispatch() at the bottom of the stack trace — that’s the fingerprint that points directly to this pattern.

Categories: Uncategorized Tags: ai, c, cloud, net, technology

Taiwan motorcycle plate lookup via #API

April 17, 2026 Infinite Loop Development Ltd Leave a comment

Our vehicle API network expands into Taiwan with full motorcycle registration lookups — make, age, engine size, and emissions test history, all in one call.

Taiwan has one of the highest motorcycle densities in the world, with over 14 million registered two-wheelers on its roads. Today, we’re pleased to announce that the /CheckTaiwan endpoint is live — bringing motorcycle registration data from Taiwan’s national vehicle database into our global API network.

What the endpoint returns

A single call to /CheckTaiwan with a Taiwanese plate number returns structured vehicle data covering identity, registration history, and emissions compliance records:

Make — Manufacturer name (Chinese & romanised)
Age / Registration year — Year of manufacture and license issue date
Engine size — Displacement in cc and engine cycle type
Inspection records — Full emissions test history with HC, CO, and CO₂ readings

Sample lookup: MWN-0076

Here’s what a real response looks like for a 2018 Kymco (光陽) motorcycle:

			
{
  "Description": "光陽",
  "RegistrationYear": "2018",
  "CarMake": { "CurrentTextValue": "光陽" },
  "EngineSize": "149",
  "ManufactureDate": "01/08/2018",
  "LicenseIssueDate": "05/09/2018",
  "EngineCycle": "四行程",
  "TestRecords": [
    {
      "LicensePlate": "MWN-0076",
      "InspectionType": "定期檢驗",
      "HC_ppm": "102",
      "CO_pct": "0.1",
      "CO2_pct": "14.8",
      "Result": "合格",
      "TestDate": "20240822"
    },
    {
      "LicensePlate": "MWN-0076",
      "InspectionType": "定期檢驗",
      "HC_ppm": "315",
      "CO_pct": "0.1",
      "CO2_pct": "14.9",
      "Result": "合格",
      "TestDate": "20240718"
    }
  ]
}

		

The TestRecords array is particularly valuable — it provides a full chronological emissions test history, with pass/fail status (合格 = pass), hydrocarbon and carbon monoxide readings, and the serial number of each inspection. This supports fleet compliance monitoring, insurance underwriting, and second-hand vehicle verification use cases.

API endpoint

The endpoint is live now at:

https://www.chepaiapi.tw/api/reg.asmx?op=CheckTaiwan

Full documentation and interactive testing are available at chepaiapi.tw.

Expanding Chinese-language coverage

This launch also deepens our Chinese-language vehicle data coverage. Alongside Taiwan, our mainland China vehicle lookup service at chepaiapi.cn continues to serve customers requiring PRC plate data — together forming a comprehensive Chinese-language API offering across both sides of the strait.

Use cases

The Taiwan endpoint is well-suited to:

Insurers pricing two-wheeler policies
Logistics platforms operating scooter fleets
Used vehicle marketplaces verifying provenance
KYC and compliance workflows touching Taiwanese vehicle assets

The inclusion of emissions test records is a differentiator that goes beyond simple registration confirmation — providing genuine due diligence depth for any platform that needs it.

Taiwan joins our network of 55+ country vehicle lookup APIs. We’ll continue expanding coverage across Asia Pacific throughout 2026.

Visit chepaiapi.tw to get started →

Categories: Uncategorized

Batch AI Processing: Why Multithreading is the Wrong Instinct

April 17, 2026 Infinite Loop Development Ltd Leave a comment

When developers first encounter a large-scale AI classification job — say, two million records that each need to be sent to an LLM for analysis — the instinct is immediately familiar: spin up threads, parallelise the work, saturate the API. It’s the same pattern that works for database processing, file I/O, HTTP scraping. More threads, more throughput.

With LLM APIs, that instinct leads you straight into a wall. And the wall has a name: TPM.

The Problem with Multithreading LLM Calls

Most LLM APIs — OpenAI included — impose a Tokens Per Minute (TPM) limit. This is a rolling window, not a per-request limit. Every token you send in a prompt, and every token the model returns, counts against it.

The naive multithreaded approach burns through this budget in a way that’s both wasteful and hard to control:

The system prompt repeats on every request. If your prompt is 700 tokens and you’re running 20 threads firing one request each, you’re spending 14,000 tokens per second just on prompt overhead — before the model has classified a single record. With a 200,000 TPM limit, you’ve consumed 4.2 minutes of budget in one second.

Burst behaviour triggers rate limits unpredictably. The TPM limit is a rolling window. Twenty threads firing simultaneously create a spike that can exceed the per-minute budget in seconds, even if your average rate would be well within limits. The API returns 429 errors, your retry logic kicks in, those retries themselves consume tokens, and the situation compounds.

Thread count is a blunt instrument. Dialling concurrency up and down doesn’t map cleanly to token consumption because request latency varies. A batch that takes 500ms doesn’t consume the same tokens as one that takes 1,500ms, but both hold a thread slot for their duration.

The Better Model: Semantic Batching

The insight that changes everything is this: the system prompt is a fixed overhead, and you should amortise it across as many classifications as possible per API call.

Instead of:

			
Thread 1: [system prompt 700 tokens] + [address 1: 15 tokens] → [result: 15 tokens]
Thread 2: [system prompt 700 tokens] + [address 2: 15 tokens] → [result: 15 tokens]
...× 20 threads
Total: 14,000 tokens for 20 classifications

You send:

			
[system prompt 700 tokens] + [addresses 1-20: 300 tokens] → [results 1-20: 100 tokens]
Total: 1,100 tokens for 20 classifications

That’s a 12× reduction in token consumption for the same work. Suddenly your 200,000 TPM budget — which could only sustain ~270 single-record requests per minute — supports ~3,600 classifications per minute. No extra threads needed.

Key Implementation Details

1. Include an ID in Both Request and Response

The most important correctness detail in batch processing is never rely on positional alignment.

If you send 20 addresses and ask the model to return 20 results, it might return 19. Now you don’t know which one it dropped. If you’re matching by position, records from item 7 onwards get silently misclassified.

The fix is to include a unique identifier in both directions:

			
User message:
id=548033: product X
id=548034: product Y
...
System prompt format instruction:
Reply ONLY with a JSON array. Format: [{"id":548033,"c":"E"}, ...]

		

Now you build a dictionary from the response keyed on id, and match each input item explicitly. A missing id means that specific record gets skipped and retried on the next run. Everything else classifies correctly regardless of what the model dropped.

2. Resolve Labels Locally

The model doesn’t need to return the full label text. "Prime City Professionals" costs tokens on every response item. A single letter costs one token.

Keep a static dictionary in your code:

csharp

			
private static readonly Dictionary<string, string> Labels = new()
{
    { "A", "Prime Product"   },
    { "B", "Budget Product"        },
    // ...
};

		

The model returns "c":"A", you look up the label locally. This also eliminates a class of hallucination errors where the model invents a label name slightly different from your taxonomy.

Note: even "category" vs "c" matters at scale. In the OpenAI tokenizer, "category" is 3 tokens; "c" is 1. Across 100,000 batch calls, that’s 200,000 tokens — small but free.

3. Track TPM with a Rolling Window, Not Concurrency

Rather than trying to infer safe concurrency from trial and error, measure what you’re actually consuming and throttle directly on that signal.

csharp

			
// On each successful response, record tokens used with a timestamp
tokenWindow.Enqueue((DateTime.UtcNow, inputTokens + outputTokens));
// Before each request, prune entries older than 60 seconds and sum the rest
var cutoff = DateTime.UtcNow.AddSeconds(-60);
while (window.Peek().t < cutoff) window.Dequeue();
long tpmUsed = window.Sum(x => x.tok);
// Throttle graduated to usage
if (tpmUsed > tpmLimit * 0.98) Thread.Sleep(2000);
else if (tpmUsed > tpmLimit * 0.95) Thread.Sleep(800);
else if (tpmUsed > tpmLimit * 0.85) Thread.Sleep(300);

		

This gives you automatic, self-correcting throttling that responds to real consumption rather than guessing from thread counts. If a batch of records happens to have longer addresses, the window fills faster and the delay kicks in sooner. No manual tuning required.

4. Resumability via Cursor Pagination

For a job that takes hours or days, stopping and restarting must be safe and cheap. The key is two things working together:

Write results immediately after each batch, not at the end of a page. If you crash mid-page, you’ve lost one batch (20 records), not a thousand.

Use a NULL-check filter combined with cursor pagination. The query for unclassified records looks like:

sql

WHERE segment_category IS NULL AND id > {lastId} ORDER BY id LIMIT 1000

On restart, lastId resets to 0, but the IS NULL filter automatically skips everything already classified. The cursor (id > lastId) keeps the query fast on large tables — OFFSET pagination slows to a crawl at millions of rows because the database still has to scan all preceding rows to find the offset position.

5. Handle Partial Batches Gracefully with Skip vs Error

Not all failures are equal. Distinguish between:

Error: something went wrong that warrants logging (HTTP 500, persistent 429 after retries, DB connection failure). These need attention.
Skip: the record wasn’t returned in this batch response. Leave it NULL in the database, it will be picked up automatically on the next run. No log noise needed.

This distinction keeps your error output meaningful. If every missing batch item logs as an error, a run with 0.1% skip rate produces thousands of error lines that mask real problems.

The Result

What started as a job estimated at 16–67 days with a naive multithreaded approach settled to around 7 hours using semantic batching — processing two million records through a rate-limited API without a single configuration change to the API account.

The throughput improvement didn’t come from more concurrency. It came from being smarter about what gets sent in each request.

The general principle applies beyond LLM classification: whenever you have a fixed overhead per API call (authentication, context, schema), the correct optimisation is to amortise that overhead across as much work as possible per call, not to fire more calls in parallel.

Summary of Patterns

Pattern	Naive approach	Better approach
Throughput	More threads	Larger batches
Rate limiting	Catch 429, retry	Track TPM rolling window, throttle proactively
Result matching	Positional array index	ID-keyed dictionary
Label resolution	Ask model for full text	Return code, resolve locally
Resumability	Track page offset	NULL-check filter + cursor pagination
Failure handling	All failures are errors	Skip vs Error distinction
DB resilience	Crash on connection drop	Exponential backoff retry

The instinct to parallelise is correct in principle — you want to keep the API busy. But with token-limited LLM APIs, the right parallelism is within a single request, not across many simultaneous ones.

Categories: Uncategorized Tags: ai, artificial-intelligence, chatgpt, llm, technology

Fixing Chrome’s “Aw, Snap!” STATUS_ACCESS_VIOLATION in CDP Automation

March 26, 2026 Infinite Loop Development Ltd Leave a comment

How a race condition between page navigation and JavaScript execution causes STATUS_ACCESS_VIOLATION crashes — and how to fix it properly.

C# · .NET·Puppeteer / CDP·Chrome Automation 💥

😬 Aw, Snap! Something went wrong displaying this page.

Chrome’s infamous crash page — and the Windows error behind it: STATUS_ACCESS_VIOLATION (0xC0000005). If you’re automating Chrome via CDP and seeing this, a race condition is likely the culprit.

If you’ve built a browser automation system using the Chrome DevTools Protocol — whether through Puppeteer, Playwright, or a custom CDP client — you may have encountered Chrome processes dying with a STATUS_ACCESS_VIOLATION error. The browser just vanishes. No clean exception, no useful log output. Just Chrome’s “Aw, Snap!” page, or worse, a completely dead process.

This error code (0xC0000005 on Windows) means the process attempted to read or write memory it doesn’t own. It’s a hard native crash, well below the level where .NET exception handling can help you. A try/catch around your CDP call won’t save you.

The Usual Suspects

Most writeups on this error point to GPU driver issues, sandbox misconfiguration, or DLL injection from antivirus software — and those are all valid causes. The standard advice is to throw flags like --disable-gpu, --no-sandbox, and --disable-dev-shm-usage at the problem until it goes away.

But there’s another cause that gets far less attention: injecting JavaScript into a page that’s mid-navigation. This is a timing issue, and it’s surprisingly easy to introduce.

The Race Condition

Consider a common automation pattern: click a button, wait for the resulting page to load, then execute some JavaScript on the new page. A naive implementation might look like this:

problematic// Click a button that triggers navigation
await ExecuteJavascript("document.querySelector('button').click();");

// Arbitrary delay, then poll readyState
await Task.Delay(1000);
while (true)
{
    await Task.Delay(500);
    var readyState = await ExecuteJavascript("document.readyState");
    if (readyState == "complete") break;
}

This pattern has a fundamental flaw. After clicking the button, Chrome begins tearing down the current document and loading a new one. There is a window — however brief — where the old document is gone but the new one isn’t yet attached. If ExecuteJavascript fires a CDP Runtime.evaluate command into that gap, Chrome is asked to execute JavaScript in a context that no longer exists.

The result isn’t a clean error. Chrome’s internal state becomes inconsistent, and the access violation follows.

Why does it only crash sometimes? Because the race condition is non-deterministic. On a fast machine or a fast network, the new page loads before the poll fires and everything works. On a slower day, the poll lands in the gap and Chrome crashes. This makes the bug look intermittent and hardware-dependent, when it’s actually a logic error.

What the Timeline Looks Like

Button click dispatched

CDP sends Runtime.evaluate with the click expression. Navigation begins.

⚠ Danger zone begins

Old document is being torn down. New document not yet attached to the frame.

document.readyState polled

If Runtime.evaluate fires here, Chrome has no valid document context to evaluate against.

STATUS_ACCESS_VIOLATION

Chrome dereferences a null or freed pointer. Process dies. “Aw, Snap!”

New document attached (if we were lucky)

If the poll happened to land here instead, readyState returns “loading” or “complete” and everything works fine.

The Fix: Let Chrome Tell You

The correct solution is to stop guessing when navigation is complete and instead subscribe to Chrome’s own navigation events. CDP exposes Page.loadEventFired precisely for this purpose — it fires when the new page’s load event has completed, meaning the document is fully attached and ready for JavaScript execution.

fixedprivate async Task WaitForPageLoad(ChromeSession session, int timeoutMs = 30000)
{
    var tcs = new TaskCompletionSource<bool>();

    session.Subscribe<LoadEventFiredEvent>(e => tcs.TrySetResult(true));

    var timeoutTask = Task.Delay(timeoutMs);
    var completed = await Task.WhenAny(tcs.Task, timeoutTask);

    if (completed == timeoutTask)
        throw new TimeoutException("Page load timed out");
}

Critically, the subscription must be set up before triggering navigation — not after. Otherwise there’s a small window where the event fires before you’re listening:

usage// Subscribe FIRST, then trigger navigation
var pageLoad = WaitForPageLoad(chromeSession);

await ExecuteJavascript("document.querySelector('button').click();");

// Now wait — Chrome will signal when it's actually ready
await pageLoad;

// Safe to execute JavaScript on the new page
await ExecuteJavascript("/* your code here */");

Why Not Just Catch the Exception?

A STATUS_ACCESS_VIOLATION is a native Windows exception originating inside the Chrome process itself. It is entirely outside the .NET runtime. Wrapping your CDP calls in try/catch does nothing — there is no managed exception to catch. The Chrome process simply dies.

Similarly, adding more Task.Delay calls doesn’t fix the race condition — it just makes it less likely to trigger on any given run, while leaving the underlying problem completely intact.

Applies to Puppeteer and Other CDP Clients Too

This issue isn’t specific to C# or any particular CDP library. The same race condition can occur in Node.js with Puppeteer, Python with pyppeteer, or any system that drives Chrome via the DevTools Protocol. Puppeteer’s page.waitForNavigation() and page.waitForLoadState() exist precisely to solve this problem — they’re wrappers around the same loadEventFired event.

If you’re rolling a custom CDP client in any language, the principle is the same: never rely on arbitrary delays or polling to determine when a page is ready for JavaScript execution. Subscribe to Page.loadEventFired or Page.frameStoppedLoading, and let Chrome do the signalling.

Summary

Root cause: JavaScript injected via CDP (Runtime.evaluate) during a page transition hits Chrome in an inconsistent internal state, causing a STATUS_ACCESS_VIOLATION native crash.

Why it’s intermittent: The race condition depends on timing — fast loads mask it, slow loads expose it.

The fix: Subscribe to Page.loadEventFiredbefore triggering navigation, and await the event before executing any JavaScript. Never use Task.Delay or document.readyState polling as a substitute for proper navigation events. Chrome DevTools Protocol · Browser Automation · STATUS_ACCESS_VIOLATION

Categories: Uncategorized Tags: ai, books, coding, javascript

Cracking the Code: Estimating a Car’s Age from Its Argentine Licence Plate

March 21, 2026 Infinite Loop Development Ltd Leave a comment

Technical Deep-dive · Vehicle Data

How sequential plate issuance, a little combinatorics, and 200,000 training records let you estimate a registration year from seven characters.By Infinite Loop Development · March 2026

Try it live

The techniques described in this post are implemented in ar.matriculaapi.com — an API that returns full vehicle details for any Argentine licence plate, including an estimated registration year for plates where the exact date is unknown.

Every Argentine licence plate encodes its own approximate birth year. You just have to know how to read it.

Argentina has issued plates in two distinct sequential formats over the past three decades. Because they are allocated in strict national order, a plate’s position in that sequence maps — with surprising precision — to the year the vehicle was registered. This post explains the technique: the encoding, the boundary estimation, and the confidence model.

Two Formats, One Principle

Argentina uses two plate formats, each covering a different era.

OZY040

Pre-Mercosur · ABC123≈ 1990 – 2016

AC875MD

Mercosur · AB123CD2016 – present

The Pre-Mercosur format (ABC123) consists of three letters followed by three digits. It was Argentina’s standard from the early 1990s through approximately 2016, when the country transitioned to the regional Mercosur standard.

The Mercosur format (AB123CD) uses two letters, three digits, then two more letters. It began with AA000AA and has been incrementing steadily ever since, shared across the Mercosur bloc — which is why plates from Brazil, Uruguay, and Paraguay share the same structure.

The critical insight is that both formats were issued sequentially at a national level. A plate allocated in 2019 will always have a higher sequence number than one from 2018. This makes year estimation a matter of finding where a given plate falls in the sequence.

Encoding a Plate to a Single Integer

To compare plates across their sequence, we convert each plate to a single integer using mixed-radix encoding — the same idea as a number system that switches base partway through.

Mercosur encoding

The Mercosur plate AB123CD has four alphabetic components (each 0–25) and one numeric component (0–999). Treating letters as base-26 and the number as base-1000:

Python

def encode_mercosur(plate: str) -> int:
    """
    Encode an AB123CD Mercosur plate to a sequence integer.
    AA000AA = 0, AA000AB = 1, ... AZ999ZZ = 17,575,999, BA000AA = 17,576,000 ...
    """
    p = plate.upper().replace(" ", "").replace("-", "")
    assert len(p) == 7, "Mercosur plates are 7 characters"

    l1 = ord(p[0]) - ord('A')  # 0-25
    l2 = ord(p[1]) - ord('A')  # 0-25
    n  = int(p[2:5])             # 0-999
    l3 = ord(p[5]) - ord('A')  # 0-25
    l4 = ord(p[6]) - ord('A')  # 0-25

    return (l1 * 26 + l2) * 676_000 \
         + n              * 676     \
         + l3             * 26      \
         + l4

A few spot checks: AA000AA → 0. AA000AB → 1. AA001AA → 676. AB000AA → 676,000. The encoding is monotonically increasing — every plate that comes later in the alphabet maps to a strictly higher integer.

Pre-Mercosur encoding

The earlier ABC123 format encodes similarly, but with three leading letters instead of two:

Python

def encode_pre_mercosur(plate: str) -> int:
    """
    Encode an ABC123 pre-Mercosur plate to a sequence integer.
    AAA000 = 0, AAA001 = 1, ... ZZZ999 = 17,575,999
    """
    p = plate.upper().replace(" ", "").replace("-", "")
    assert len(p) == 6, "Pre-Mercosur plates are 6 characters"

    l1 = ord(p[0]) - ord('A')  # 0-25
    l2 = ord(p[1]) - ord('A')  # 0-25
    l3 = ord(p[2]) - ord('A')  # 0-25
    n  = int(p[3:])             # 0-999

    return (l1 * 676 + l2 * 26 + l3) * 1000 + n

Learning the Boundaries from Real Data

Encoding gives us integers. To turn those integers into years, we need to know which sequence ranges correspond to which years. This is where training data comes in.

Using over 200,000 Argentine plates with known registration years, we computed the mean sequence number per year. This gives a representative centroid for each year’s plate population:

The year boundary cut points are simply the midpoints between adjacent means. This produces clean, non-overlapping ranges — every sequence integer maps to exactly one year:

Seq range	Estimated year	Cut point derivation
< 671,619	2016	(294k + 1,048k) / 2
671,619 – 1,467,170	2017	(1,048k + 1,885k) / 2
1,467,170 – 2,207,136	2018	(1,885k + 2,528k) / 2
2,207,136 – 2,719,720	2019	(2,528k + 2,910k) / 2
2,719,720 – 3,088,306	2020	(2,910k + 3,265k) / 2
3,088,306 – 3,472,769	2021	(3,265k + 3,679k) / 2
3,472,769 – 3,888,534	2022	(3,679k + 4,097k) / 2
3,888,534 – 4,307,005	2023	(4,097k + 4,516k) / 2
4,307,005 – 4,773,859	2024	(4,516k + 5,030k) / 2
≥ 4,773,859	2025	(open upper bound)

The naive approach used min/max ranges per year — but these overlap badly. Using the mean and splitting at midpoints gives clean, unambiguous boundaries.

The Full Estimator in Python

Python

import re
from dataclasses import dataclass
from typing import Optional

# ── Mercosur year boundaries (midpoints between annual means) ──────────────
# Derived from 200k+ Argentine plates with known registration years.
# Each January, recalculate the mean for the new year and add one entry:
#   new_cut = (mean_prev_year + mean_new_year) / 2
# ──────────────────────────────────────────────────────────────────────────
MERCOSUR_CUTS = [
    (671_619,   2016),
    (1_467_170, 2017),
    (2_207_136, 2018),
    (2_719_720, 2019),
    (3_088_306, 2020),
    (3_472_769, 2021),
    (3_888_534, 2022),
    (4_307_005, 2023),
    (4_773_859, 2024),
    (float('inf'), 2025),
    # Add 2026 here in January 2027:
    # (cut_2025_2026, 2025), (float('inf'), 2026),
]

# Pre-Mercosur: two-letter prefix → dominant year
# Derived from same training dataset. Stable — no new plates since ~2016.
PRE_MERCOSUR_PREFIX = {
    # A block 1994-1996
    'AA': lambda n: 1994 if n < 500 else 1995,
    'AB': 1995, 'AC': 1995, 'AD': 1995, 'AE': 1995, 'AF': 1995,
    'AG': 1995, 'AH': 1995, 'AI': 1995, 'AJ': 1995, 'AK': 1995,
    'AL': 1995, 'AM': 1995, 'AN': 1995,
    'AO': lambda n: 1995 if n < 400 else 1996,
    'AP': 1996, 'AQ': 1996, 'AR': 1996, 'AS': 1996, 'AT': 1996,
    'AU': 1996, 'AV': 1996, 'AW': 1996, 'AX': 1996, 'AY': 1996, 'AZ': 1996,
    # B block 1996-1998
    'BA': 1996, 'BB': 1996,
    'BC': lambda n: 1996 if n < 300 else 1997,
    'BD': 1997, 'BE': 1997, 'BF': 1997, 'BG': 1997, 'BH': 1997,
    'BI': 1997, 'BJ': 1997, 'BK': 1997, 'BL': 1997, 'BM': 1997,
    'BN': 1997, 'BO': 1997,
    'BP': lambda n: 1997 if n < 500 else 1998,
    'BQ': 1998, 'BR': 1998, 'BS': 1998, 'BT': 1998, 'BU': 1998,
    'BV': 1998, 'BW': 1998, 'BX': 1998, 'BY': 1998, 'BZ': 1998,
    # C-P blocks follow same pattern; see full table at ar.matriculaapi.com
    # ... (abbreviated for readability)
}

PRE_MERCOSUR_RE = re.compile(r'^[A-Z]{3}\d{3}$')
MERCOSUR_RE     = re.compile(r'^[A-Z]{2}\d{3}[A-Z]{2}$')


@dataclass
class PlateEstimate:
    input_plate:    str
    format:         str         # 'MERCOSUR' | 'PRE-MERCOSUR' | 'MERCOSUR-IMPORT' | 'UNKNOWN'
    estimated_year: Optional[int]
    confidence:     str         # 'HIGH' | 'MEDIUM' | 'LOW'
    sequence_num:   Optional[int]
    notes:          Optional[str] = None


def estimate_year(plate: str) -> PlateEstimate:
    p = plate.upper().replace(" ", "").replace("-", "")

    # ── Mercosur AB123CD ─────────────────────────────────────────────────────
    if MERCOSUR_RE.match(p):
        seq = encode_mercosur(p)

        if seq < 7:
            return PlateEstimate(plate, 'MERCOSUR-IMPORT', None, 'LOW', seq,
                notes='Sequence predates Argentine rollout; likely import')

        year = next(y for cut, y in MERCOSUR_CUTS if seq < cut)

        # Confidence: MEDIUM if within 5% of the nearest boundary
        confidence = 'HIGH'
        cuts = [c for c, _ in MERCOSUR_CUTS[:-1]]
        gaps = [MERCOSUR_CUTS[i+1][0] - MERCOSUR_CUTS[i][0]
                for i in range(len(cuts))]
        for cut, gap in zip(cuts, gaps):
            if abs(seq - cut) < gap * 0.05:
                confidence = 'MEDIUM'
                break

        return PlateEstimate(plate, 'MERCOSUR', year, confidence, seq)

    # ── Pre-Mercosur ABC123 ───────────────────────────────────────────────────
    if PRE_MERCOSUR_RE.match(p):
        prefix = p[:2]
        num    = int(p[3:])
        entry  = PRE_MERCOSUR_PREFIX.get(prefix)

        if entry is None:
            return PlateEstimate(plate, 'PRE-MERCOSUR', None, 'LOW', None,
                notes=f'Prefix {prefix!r} not in training data')

        year = entry(num) if callable(entry) else entry

        confidence = ('LOW'    if prefix[0] in 'RSTUV'
                 else 'MEDIUM' if prefix <= 'DZ'
                 else 'HIGH')

        return PlateEstimate(plate, 'PRE-MERCOSUR', year, confidence,
                              encode_pre_mercosur(p))

    return PlateEstimate(plate, 'UNKNOWN', None, 'LOW', None,
        notes='Does not match any known Argentine plate format')


# ── Quick demo ────────────────────────────────────────────────────────────
for test in ['AC601QQ', 'AC875MD', 'OZY040', 'GDA123', 'AB 172UC']:
    r = estimate_year(test)
    print(f"{r.input_plate:10} → {r.estimated_year}  [{r.confidence}]  {r.format}")

Why Pre-Mercosur Is Trickier

You might expect pre-Mercosur plates to work the same way as Mercosur — encode to an integer, find the range. But the raw data tells a different story: the sequence number ranges for adjacent years overlap almost completely.

The reason is that Argentina’s provinces received independent plate allocations. Buenos Aires, Córdoba, and Mendoza were all issuing plates simultaneously from their own provincial ranges. A national sequence number alone therefore can’t pinpoint a year — the number space was being consumed by 23 provinces in parallel.

What does cluster cleanly by year is the two-letter prefix. The national allocation advanced through the alphabet over time, so GA–GT plates are overwhelmingly from 2007, HT–HZ from 2009, and so on. The training data confirms this: for the densest prefixes, over 90% of plates share a single dominant year.

At prefix boundaries — AO, BC, GT, HS, and others — the numeric suffix provides a secondary signal. A plate in the GT prefix with a low number (GT100) likely precedes one with a high number (GT850) by several months, straddling a year boundary.

Confidence and Outliers

The estimator returns one of three confidence levels:

HIGH The plate sits comfortably within a year band — more than 5% away from any boundary. For Mercosur plates from 2017 onwards (where training density is highest) this is the common case.

MEDIUM The plate is within 5% of a year cut point, or belongs to a pre-Mercosur prefix with moderate training data. The estimate is most likely correct but the adjacent year is plausible — late registrations, delivery delays, and data entry lag all introduce real ambiguity near boundaries.

LOW The prefix is sparse (old R/S/T/U/V plates, typically pre-1995) or the plate is flagged as a likely import. Imports arise when a Mercosur-format plate has a sequence number that predates Argentina’s 2016 rollout — these vehicles were most likely registered in Brazil, Uruguay, or Paraguay and subsequently imported.

Keeping It Fresh

The Mercosur boundaries need updating once a year. The process is three steps:

1. Collect a fresh batch of plates with known years.
2. Compute the mean sequence number for the new year’s cohort.
3. Set the new cut point as (mean_prev + mean_new) / 2 and append it to the table.

No existing cut points change — you’re only ever adding one new entry to the bottom of the list. The pre-Mercosur prefix table is stable and needs no maintenance, since no new plates in that format have been issued since approximately 2016.

Beyond Year Estimation

Year estimation is useful on its own — for insurance quoting, fleet valuation, or fraud detection — but it becomes much more powerful when combined with a full vehicle lookup.

The Argentine Matricula API takes any plate (pre-Mercosur or Mercosur) and returns the complete vehicle record: make, model, year, engine, and more. For records where the registration year is missing or uncertain, the sequence-based estimate described here fills the gap automatically, with a confidence flag so downstream consumers know how much to trust it.

Argentine vehicle lookup API

Try a plate lookup at ar.matriculaapi.com. The API is available for integration via vehicleapi.com and supports batch queries, JSON responses, and per-format confidence scoring.

Categories: Uncategorized

The Hidden Cost of ORDER BY NEWID()

March 19, 2026 Infinite Loop Development Ltd Leave a comment

Fetching a random row from a table is a surprisingly common requirement — random banner ads, sample data, rotating API credentials. The instinctive solution in SQL Server is elegant-looking but conceals a serious performance trap.

			
-- Looks innocent. Isn't.
SELECT TOP 1 * FROM LARGE_TABLE ORDER BY NEWID()

What SQL Server actually does

NEWID() generates a fresh GUID for every single row in the table. SQL Server must then sort the entire result set by those GUIDs before it can hand back the top one. On a table with a million rows you are generating a million GUIDs, sorting a million rows, and discarding 999,999 of them.

The problem: On large tables, ORDER BY NEWID() performs a full table scan and a full sort — O(n log n) work — regardless of how many rows you need. It cannot use any index for ordering.

A faster alternative: seek, don’t sort

The key insight is to convert the “random sort” into a “random seek”. If we can generate a random Id value cheaply and then let the clustered index do the work, we avoid scanning the table entirely.

			
DECLARE @Min INT = (SELECT MIN(Id) FROM LARGE_TABLE)
DECLARE @Max INT = (SELECT MAX(Id) FROM LARGE_TABLE)
SELECT TOP 1 *
FROM LARGE_TABLE
WHERE Id >= @Min + ABS(CHECKSUM(NEWID()) % (@Max - @Min + 1))
ORDER BY Id ASC

		

MAX(Id) and MIN(Id) are single index seeks on the primary key. CHECKSUM(NEWID()) generates a random integer without sorting anything. The WHERE Id >= clause then performs a single index seek from that point forward, and ORDER BY Id ASC TOP 1 picks up the very next row.

The result: Two index seeks to get the range, one index seek to find the row. Constant time regardless of table size.

Performance at a glance

Approach	Reads	Sort	Scales with table size?
ORDER BY NEWID()	Full scan	Full sort	O(n log n)
CHECKSUM seek	3 index seeks	None	O(1)

Three caveats to know

1. Id gaps cause mild bias. If rows have been deleted, gaps in the Id sequence mean rows immediately after a gap are slightly more likely to be selected. For most use cases — sampling, rotation, A/B testing — this is an acceptable trade-off.

2. Ids may not start at 1. This is why we use @Min rather than hardcoding zero. If your identity seed started at 1000, NEWID() % MAX(Id) would generate values 0–999, which would never match any row and you’d always get the first row in the table.

3. CHECKSUM can return INT_MIN. ABS(INT_MIN) overflows back to negative in SQL Server. The fix is to apply the modulo before the ABS, keeping the intermediate value safely within range.

When you don’t need randomness at all

For round-robin rotation across a fixed set of rows — such as alternating between API credentials or cookie sessions — true randomness is unnecessary overhead. A deterministic slot based on the current second is even cheaper:

			
-- Rotates across N accounts, one per second, no writes required
WHERE Slot = DATEPART(SECOND, GETUTCDATE()) % TotalAccounts

This resolves to a constant integer comparison — effectively a single index seek — and scales to any number of accounts automatically. No tracking table, no writes, no contention.

The takeaway: whenever you reach for ORDER BY NEWID(), ask whether you actually need true randomness or just approximate distribution. In most production scenarios, a cheap seek beats an expensive sort by several orders of magnitude.

Categories: Uncategorized Tags: ai, database, sql, sql-server, technology

Enrich Your Qualtrics Surveys with Real-Time Respondent Data Using AvatarAPI

March 4, 2026 Infinite Loop Development Ltd Leave a comment

Qualtrics is excellent at capturing what respondents tell you. But what if you could automatically fill in what you already know — or can discover — the moment they enter their email address?

AvatarAPI resolves an email address into rich profile data in real time: a profile photo, full name, city, country, and the social network behind it. By embedding this lookup directly into your Qualtrics survey flow, you collect more information about each respondent without asking a single extra question.

What Data Does AvatarAPI Return?

When you pass an email address to the API, it returns the following fields — all of which can be mapped into Qualtrics Embedded Data and used anywhere in your survey:

Field	Description
`Image`	URL to the respondent’s profile photo
`Name`	Resolved full name
`City`	City of residence
`Country`	Country code
`Valid`	Whether the email address is real and reachable
`IsDefault`	Whether the avatar is a fallback/generic image
`Source.Name`	The social network the data came from
`RawData`	The complete JSON payload

Watch the Video Walkthrough

Before diving into the written steps, watch this complete tutorial — from configuring the Web Service element to rendering the avatar photo on a results page:

Step-by-Step Integration Guide

You can either follow these steps from scratch, or import the ready-made AvatarAPI.qsf template file directly into Qualtrics (see Step 8).

Step 1 — Get Your AvatarAPI Credentials

Sign up at avatarapi.com to obtain a username and password. A free demo account is available for evaluation — use the credentials demo / demo to test before going live.

The API endpoint you will call is:

https://avatarapi.com/v2/api.aspx

Step 2 — Create an Email Capture Question

In your Qualtrics survey, add a Text Entry question with a Single Line selector. This is where respondents will enter their email address.

Note the Question ID assigned to this question (e.g. QID3) — you will reference it when configuring the Web Service. You can find the QID by opening the question’s advanced options.

Tip: Add email format validation via Add Validation → Content Validation → Email to ensure the value passed to the API is always well-formed.

Step 3 — Add a Web Service Element to Your Survey Flow

Navigate to Survey Flow (the flow icon in the left sidebar). Click Add a New Element Here and choose Web Service. Position this element after the block containing your email question and before your results block.

Configure it as follows:

URL: https://avatarapi.com/v2/api.aspx
Method: POST
Content-Type: application/json

Step 4 — Set the Request Body Parameters

Under Set Request Parameters, switch to Specify Body Params and add these three key-value pairs:

			
{
  "username": "your_username",
  "password": "your_password",
  "email": "${q://QID3/ChoiceTextEntryValue}"
}

		

The Qualtrics piped text expression ${q://QID3/ChoiceTextEntryValue} dynamically inserts whatever email the respondent typed. Replace QID3 with the actual QID of your email question if it differs.

Step 5 — Map the API Response to Embedded Data Fields

Scroll down to Map Fields from Response. Add one row for each field you want to capture. The From Response column is the JSON key returned by AvatarAPI; the To Field column is the Embedded Data variable name.

From Response (JSON key)	To Field (Embedded Data)
`Image`	`Image`
`Name`	`Name`
`Valid`	`Valid`
`City`	`City`
`Country`	`Country`
`IsDefault`	`IsDefault`
`Source.Name`	`Source.Name`
`RawData`	`RawData`

Note: Qualtrics stores these variables automatically — you don’t need to pre-declare them as Embedded Data elsewhere in the flow, though doing so in the survey flow header keeps things organised.

Step 6 — Display the Avatar Photo on a Results Page

Add a Descriptive Text / Graphic question in a block placed after the Web Service call in your flow.

In the rich-text editor, switch to the HTML source view and paste this snippet:

			
<img
  src="${e://Field/Image}"
  alt="Profile Picture"
  style="width:100px; height:100px; border-radius:50%;"
/>

		

The expression ${e://Field/Image} inserts the profile photo URL at runtime. The border-radius: 50% gives it a circular crop for a polished appearance.

You can display other fields using the same pattern:

			
Name:    ${e://Field/Name}
City:    ${e://Field/City}
Country: ${e://Field/Country}
Source:  ${e://Field/Source.Name}

Step 7 — Test with the Demo Account

Before going live, test the integration using the demo credentials. Enter a well-known email address (such as a Gmail address you know has a Google profile photo) to verify the image and data return correctly.

After a test submission, check the Survey Data tab — all mapped fields (Image, Name, City, Country, etc.) should appear as columns alongside your standard question responses.

Rate limits & production use: The demo credentials are shared and rate-limited. Swap in your own account credentials before publishing a live survey to ensure reliable performance.

Step 8 — Import the Ready-Made QSF Template

Rather than building from scratch, you can import the AvatarAPI.qsf file directly into Qualtrics. This gives you a pre-configured survey with the email question, Web Service flow, and image display block already set up.

To import: go to Create a new project → Survey → Import a QSF file and upload AvatarAPI.qsf. Then update the Web Service credentials to your own username and password, and you’re ready to publish.

How the Survey Flow Works

Once configured, your survey flow has this simple three-part structure:

Block — Respondent enters their email address
Web Service — Silent POST to avatarapi.com/v2/api.aspx; response fields mapped to Embedded Data
Block — Results page displays the avatar photo and enriched profile data

The respondent experiences a seamless survey: they enter their email on page one, the API call fires silently between pages, and they see a personalised result — including their own profile photo — on page two.

Practical Use Cases

Lead enrichment surveys — Capture a prospect’s email and automatically resolve their name, city, and country without asking. Append this data to your CRM export from Qualtrics.

Event registration flows — Display the registrant’s photo back to them as a confirmation step, increasing engagement and reducing drop-off.

Email validation checkpoints — Use the Valid flag in a branch logic condition to route respondents with unresolvable addresses to a correction screen or alternative path.

Research panels — Enrich responses with geographic signals without asking respondents to self-report location, reducing survey length and improving data quality.

Get Started

API documentation & sign-up: avatarapi.com
API endpoint: https://avatarapi.com/v2/api.aspx
Demo credentials: username demo / password demo
Video tutorial: Watch on YouTube

Categories: Uncategorized Tags: ai, artificial-intelligence, digital-marketing, llm, technology

Building a Wildcard Catch-All POP3 Mail Server on Ubuntu

February 12, 2026 Infinite Loop Development Ltd Leave a comment

Receive mail for any address on any subdomain — no per-account configuration required

Introduction

This guide walks through setting up a wildcard catch-all mail server on Ubuntu using Postfix and Dovecot. The goal is to receive email sent to any address on any subdomain of your domain — for example, anything@abc.yourdomain.com or test@xyz.yourdomain.com — without having to configure individual mailboxes in advance.

This is particularly useful for testing, disposable address systems, API integrations, and mail sink setups where you want to capture inbound mail programmatically. The server will not send mail — only receive it. Mail older than 24 hours is automatically purged.

Architecture Overview

The stack consists of three components working together:

Postfix — receives inbound SMTP and delivers to a local virtual mailbox
Dovecot — serves POP3 access to the mailbox
A single catch-all mailbox — all mail for all subdomains and addresses funnels into one Maildir

Rather than creating individual accounts, everything is routed to a single mailbox. A POP3 client connects with one username and password to retrieve all mail regardless of which address or subdomain it was sent to.

Part 1 — DNS Configuration

How Wildcard MX Records Work

MX records must point to a hostname, not an IP address directly. This means two DNS records are needed: an MX record pointing to a mail hostname, and an A record resolving that hostname to your server’s IP address.

Create the following records in your DNS provider (AWS Route 53 or equivalent):

Record Name	Type / Value
*.yourdomain.com	MX — 10 mail.yourdomain.com
mail.yourdomain.com	A — your.server.ip.address

The wildcard MX record *.yourdomain.com matches any single-level subdomain lookup. When a sending mail server looks up the MX record for abc.yourdomain.com, it matches the wildcard and is directed to mail.yourdomain.com, which in turn resolves to your server’s IP via the A record.

Note that the wildcard covers one subdomain level deep. Mail to anything@abc.yourdomain.com is covered. A deeper level such as anything@a.b.yourdomain.com would require a separate record.

Verifying DNS Records

From a Windows machine, use nslookup to verify records have propagated:

# Check the MX recordnslookup -type=MX abc.yourdomain.com
# Check the A record for the mail hostnslookup mail.yourdomain.com
# Query AWS nameservers directly (before public propagation)nslookup -type=NS yourdomain.comnslookup -type=MX abc.yourdomain.com ns-123.awsdns-45.com

You can also use dnschecker.org to check propagation across multiple global resolvers simultaneously.

Part 2 — Server Setup

Install Postfix and Dovecot

sudo apt updatesudo apt install postfix dovecot-pop3d -y

During the Postfix installation prompt, select Internet Site and enter your domain name (e.g. yourdomain.com) when asked for the mail name.

Configure Postfix

Edit the main Postfix configuration file:

sudo nano /etc/postfix/main.cf

Add or update the following values:

myhostname = mail.yourdomain.commydomain = yourdomain.com
# Leave mydestination empty — we use virtual mailboxes insteadmydestination =
# Accept mail for any subdomain matching the wildcardvirtual_mailbox_domains = regexp:/etc/postfix/virtual_domainsvirtual_mailbox_base = /var/mail/vhostsvirtual_mailbox_maps = regexp:/etc/postfix/virtual_mailboxvirtual_minimum_uid = 100virtual_uid_maps = static:5000virtual_gid_maps = static:5000
# Required to prevent open relaysmtpd_relay_restrictions = permit_mynetworks, reject_unauth_destination

Create the virtual domains file — this regexp matches any subdomain of your domain:

sudo nano /etc/postfix/virtual_domains

/^\.+\.yourdomain\.com$/ OK

Create the virtual mailbox map — this catches all addresses and routes them to a single catchall mailbox:

sudo nano /etc/postfix/virtual_mailbox

/^.+@.+\.yourdomain\.com$/ catchall/

Rebuild the aliases database (required to avoid a startup warning):

newaliases

Create the Virtual Mail User and Mailbox

Postfix delivers mail as a dedicated system user (vmail). Create the user, group, and mailbox directory:

sudo groupadd -g 5000 vmailsudo useradd -u 5000 -g 5000 -d /var/mail/vhosts -s /sbin/nologin vmailsudo mkdir -p /var/mail/vhosts/catchallsudo chown -R vmail:vmail /var/mail/vhosts

Configure Dovecot for POP3

Enable the POP3 protocol in the main Dovecot config:

sudo nano /etc/dovecot/dovecot.conf

protocols = pop3

Set the mail location to the catchall Maildir:

sudo nano /etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:/var/mail/vhosts/catchall

Allow plaintext authentication (suitable for internal/trusted use — see the TLS note at the end for public-facing deployments):

sudo nano /etc/dovecot/conf.d/10-auth.conf

disable_plaintext_auth = noauth_mechanisms = plain login
passdb { driver = passwd-file args = /etc/dovecot/users}
userdb { driver = static args = uid=5000 gid=5000 home=/var/mail/vhosts/catchall}

Create the Dovecot users file with your chosen credentials:

sudo nano /etc/dovecot/users
# Format: username:{PLAIN}passwordmailuser:{PLAIN}yourpasswordhere

Start the Services

sudo systemctl restart postfixsudo systemctl restart dovecot

Verify Postfix is running:

postfix status

Check the mail log for any errors:

tail -30 /var/log/mail.log

Part 3 — Firewall Configuration

Cloud Firewall (Linode / AWS / equivalent)

Open the following inbound ports in your cloud provider’s firewall. On Linode this is found under Networking > Firewalls in the dashboard. Changes apply immediately with no reboot required.

Port / Protocol	Purpose
22 TCP	SSH (ensure this is always open)
25 TCP	SMTP inbound (receiving mail)
110 TCP	POP3 (retrieving mail)

UFW on the Ubuntu Instance

sudo ufw allow 22/tcpsudo ufw allow 25/tcpsudo ufw allow 110/tcpsudo ufw enablesudo ufw status

Always confirm port 22 is allowed before enabling UFW to avoid locking yourself out of SSH.

Part 4 — Testing

Test SMTP Locally

From the server itself, connect to Postfix on port 25 and send a test message. Use 127.0.0.1 rather than localhost to avoid IPv6 connection issues:

telnet 127.0.0.1 25

You should immediately see the greeting banner:

220 mail.yourdomain.com ESMTP Postfix

Then send a test message interactively:

EHLO test.comMAIL FROM:<test@test.com>RCPT TO:<anything@abc.yourdomain.com>DATASubject: Test mail
Hello this is a test.QUIT

Each step should return a 250 OK response. The RCPT TO line is the critical one — if the wildcard regexp is configured correctly, Postfix will accept any subdomain address. After QUIT, verify the mail landed in the mailbox:

tail -20 /var/log/mail.logls -la /var/mail/vhosts/catchall/new/

You should see a file in the new/ directory — that is the email in Maildir format.

Test POP3 Locally

telnet 127.0.0.1 110

Dovecot should respond with:

+OK Dovecot (Ubuntu) ready.

Then authenticate and list messages:

USER mailuserPASS yourpasswordhereLISTRETR 1QUIT

A successful LIST response showing message count confirms the full chain is working: inbound SMTP via Postfix, delivery to virtual Maildir, and POP3 retrieval via Dovecot.

Part 5 — Automatic Mail Purge

To automatically delete mail older than 24 hours, add a cron job:

sudo crontab -e

Add the following line:

0 * * * * find /var/mail/vhosts/catchall -type f -mmin +1440 -delete

This runs every hour and removes any file in the catchall mailbox older than 1440 minutes (24 hours).

Optional — Silence the Backwards Compatibility Warning

Postfix logs a harmless warning about backwards-compatible default settings. To silence it:

postconf compatibility_level=3.6postfix reload

Security Notes

Port 110 transmits credentials in plaintext. For any public-facing deployment, configure Dovecot with TLS and use POP3S on port 995 instead.
The smtpd_relay_restrictions = permit_mynetworks, reject_unauth_destination setting prevents your server from acting as an open relay — do not remove this.
Consider rate limiting inbound SMTP connections if the server is publicly accessible to reduce spam load.
The vmail system user has no login shell (nologin) and cannot be used to access the system interactively.

Summary

With Postfix and Dovecot configured as described above, your server will:

Accept inbound SMTP for any address on any subdomain of your domain
Deliver all mail into a single catch-all Maildir with no per-account configuration
Expose all received mail via POP3 using a single username and password
Automatically purge mail older than 24 hours
Require no restart or reconfiguration when new subdomains or addresses are used

Categories: Uncategorized Tags: devops, linux, security, technology, ubuntu

Older Entries

What the stack trace is telling you

Why you don’t get the “nice” exception

The minimal reproduction

The diagnostic giveaway

Fixing it

What I’d take away from this

Share this:

The Symptom

What Actually Happens

The Synchronization Context

The Deadlock Sequence

Why It Only Surfaces Under Load

The Broken Pattern

The Fix: Async All the Way Down

The WebForms Special Case

Coexisting Sync and Async

Quick Diagnostic Checklist

Summary

Share this:

The Setup

Why It Crashes: The async void Trap

A Minimal Reproduction

How It Appears in the Windows Event Log

Entry 1: .NET Runtime — Unhandled Exception

Entry 2: Application Error — w3wp.exe Fault

What to Look For — Summary

The Effect on IIS

The Fix

The Rule

Share this:

What the endpoint returns

Sample lookup: MWN-0076

API endpoint

Expanding Chinese-language coverage

Use cases

Share this:

The Problem with Multithreading LLM Calls

The Better Model: Semantic Batching

Key Implementation Details

1. Include an ID in Both Request and Response

2. Resolve Labels Locally

3. Track TPM with a Rolling Window, Not Concurrency

4. Resumability via Cursor Pagination

5. Handle Partial Batches Gracefully with Skip vs Error

The Result

Summary of Patterns

Share this:

The Usual Suspects

The Race Condition

What the Timeline Looks Like

Button click dispatched

⚠ Danger zone begins

document.readyState polled

STATUS_ACCESS_VIOLATION

New document attached (if we were lucky)

The Fix: Let Chrome Tell You

Why Not Just Catch the Exception?

Applies to Puppeteer and Other CDP Clients Too

Summary

Share this:

Two Formats, One Principle

Encoding a Plate to a Single Integer

Mercosur encoding

Pre-Mercosur encoding

Learning the Boundaries from Real Data

The Full Estimator in Python

Why Pre-Mercosur Is Trickier

Confidence and Outliers

Keeping It Fresh

Beyond Year Estimation

Share this:

What SQL Server actually does

A faster alternative: seek, don’t sort

Performance at a glance

Three caveats to know

When you don’t need randomness at all

Share this:

What Data Does AvatarAPI Return?

Watch the Video Walkthrough

Step-by-Step Integration Guide