Home > Uncategorized > The dangers of Parallel.ForEach(… , async (item)) in IIS

The dangers of Parallel.ForEach(… , async (item)) in IIS

A single, trivial exception — one that your code already has a catch block for — shouldn’t be able to bring down your entire IIS web server. But it can, and it will, if you combine Parallel.ForEach with an async lambda. This post explains exactly why it happens, how to spot it in the Windows Event Log, and how to fix it permanently.


The Setup

You have a method that needs to perform the same async operation against multiple items — calling a set of external APIs, processing a batch of records, sending a collection of requests. You reach for Parallel.ForEach because it sounds like the right tool: parallel work, multiple items, run them all at once. You even add a try/catch inside the lambda because you’re being responsible. It looks like this:

Parallel.ForEach(items, async (item) =>
{
try
{
var result = await ProcessItemAsync(item);
lock (results) { results.Add(result); }
}
catch (ItemNotFoundException)
{
// item not found - fine, skip it
}
catch (Exception ex)
{
lock (errors) { errors.Add(ex); }
}
});

This looks safe. It has error handling. It uses async/await. It compiles without a single warning. And it will crash your IIS worker process (w3wp.exe) the moment any exception is thrown after an await.


Why It Crashes: The async void Trap

Parallel.ForEach was designed before async/await existed in C#. It expects a synchronous Action<T> delegate. When you pass it an async lambda, something subtle and dangerous happens: the compiler silently treats the lambda as returning void rather than Task.

This is the async void anti-pattern, and it has one devastating property: any exception thrown inside it cannot be caught by any caller. It escapes directly to the thread’s synchronisation context — and on a raw ThreadPool thread, that means it goes completely unhandled.

Here is the exact sequence of events that kills your server:

  1. Parallel.ForEach fires the lambda for each item in the collection
  2. Each lambda hits the first await and suspends, returning control immediately
  3. Parallel.ForEach sees each lambda return (as void) and considers its job done
  4. Parallel.ForEach exits — the method returns to its caller — everything looks fine
  5. Milliseconds later, the awaited operations complete and the continuations resume on raw ThreadPool threads
  6. An exception is thrown inside one of those continuations
  7. The try/catch inside the lambda? It only catches exceptions thrown before the first await. After the await, the lambda has already returned as far as Parallel.ForEach is concerned
  8. The exception has no owner, no observer, no catch block — it propagates to the ThreadPool itself
  9. In .NET 4.0 and later, an unhandled exception on a ThreadPool thread terminates the process
  10. w3wp.exe crashes. IIS restarts the application pool. All in-flight requests are lost

The particularly insidious part is that the try/catch gives you a false sense of security. You can see it right there in the code. But it doesn’t work the way you expect once an await is involved.


A Minimal Reproduction

You don’t need a complex codebase to reproduce this. The following is all it takes:

public static void CrashIIS()
{
Parallel.ForEach(new[] { 1, 2, 3 }, async (item) =>
{
await Task.Delay(100); // simulate any async I/O
throw new Exception("This kills w3wp.exe");
// After the await, this runs on an orphaned ThreadPool thread
// The process terminates
});
// Parallel.ForEach has already returned here
// The crash happens 100ms later
}

Call that from any ASP.NET request handler — a controller action, an HttpHandler, anywhere — and your application pool will crash within moments. The caller gets no exception. The HTTP response may even succeed before the crash occurs. The next user to make any request gets a 503.

Even wrapping the call in a try/catch at the call site doesn’t help:

try
{
Parallel.ForEach(new[] { 1, 2, 3 }, async (item) =>
{
await Task.Delay(100);
throw new Exception("Crash");
});
}
catch (Exception ex)
{
// This NEVER fires.
// The exception doesn't happen until after Parallel.ForEach
// has already exited this try block entirely.
Log(ex);
}

The catch block is long gone by the time the exception is thrown. This is what makes the pattern so dangerous — it looks exception-safe at every level, and isn’t.


How It Appears in the Windows Event Log

When this crash occurs, it leaves a very specific fingerprint in the Windows Event Log. Open Event Viewer → Windows Logs → Application and look for two entries appearing within seconds of each other.

Entry 1: .NET Runtime — Unhandled Exception

Source: .NET Runtime
Event ID: 1026

Application: w3wp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: YourNamespace.YourException
at YourClass.YourMethod()
at SomeClass+<SomeMethod>d__3.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task)
at SomeClass+<>c__DisplayClass4_0+<<YourParallelMethod>b__0>d.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Threading.ExecutionContext.RunInternal(...)
at System.Threading.ExecutionContext.Run(...)
at System.Threading.QueueUserWorkItemCallback.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

The key indicators are at the bottom of the stack trace:

  • QueueUserWorkItemCallback.ExecuteWorkItem()
  • ThreadPoolWorkQueue.Dispatch()

These tell you the exception surfaced on a raw ThreadPool work item with no managed owner — the classic signature of an orphaned async continuation. You will also see compiler-generated state machine names like <YourParallelMethod>b__0>d.MoveNext() in the trace, confirming the exception came from inside an async lambda. The angle brackets and the b__ notation are the C# compiler’s naming convention for anonymous methods and lambdas.

Entry 2: Application Error — w3wp.exe Fault

Source: Application Error
Event ID: 1000

Faulting application name: w3wp.exe
Faulting module name: KERNELBASE.dll
Exception code: 0xe0434352

Exception code 0xe0434352 is the Windows error code for a managed (.NET) exception that has escaped to the Win32 layer. It’s the OS-level record of a .NET exception killing a process. When you see this code combined with KERNELBASE.dll as the faulting module, a .NET unhandled exception is the cause.

What to Look For — Summary

SignalWhereWhat it means
ThreadPoolWorkQueue.Dispatch() at bottom of stackEvent ID 1026, .NET RuntimeException from orphaned async continuation
Compiler-generated names like b__0>d.MoveNext()Event ID 1026, .NET RuntimeException came from inside an async lambda
Exception code 0xe0434352Event ID 1000, Application Error.NET exception killed the process
Faulting module: KERNELBASE.dllEvent ID 1000, Application ErrorManaged exception, not a native crash
Both entries within seconds of each otherApplication logSingle event caused immediate process termination

The Effect on IIS

When w3wp.exe terminates due to an unhandled exception, IIS detects the process death and marks the application pool as faulted. Depending on your Rapid Fail Protection settings (found in IIS Manager → Application Pools → Advanced Settings), IIS will either:

  • Restart the worker process automatically — users experience a brief outage and then service resumes, with the first request after restart being slow due to application warm-up
  • Disable the application pool if failures occur too frequently within the Rapid Fail Protection window (default: 5 failures in 5 minutes) — this results in a persistent 503 until an administrator manually starts the pool again

This is worth understanding because thread exhaustion and this crash pattern look identical from the outside — both produce 503 errors — but they behave very differently. Thread exhaustion self-recovers when load drops. A crashed application pool requires either automatic restart (if Rapid Fail Protection hasn’t tripped) or manual intervention. If your team is regularly performing IISResets to recover from outages, a crash like this is a more likely culprit than thread exhaustion.


The Fix

The correct replacement for Parallel.ForEach with async work is Task.WhenAll, which is async-native and properly propagates exceptions back to the awaiting caller:

public static async Task<IReadOnlyList<Result>> ProcessAllAsync(IEnumerable<Item> items)
{
var tasks = items.Select(async item =>
{
try
{
return await ProcessItemAsync(item);
}
catch (ItemNotFoundException)
{
return Result.Empty;
}
});
// All items processed in parallel.
// Exceptions surface here, as AggregateException, to a proper awaiter.
return await Task.WhenAll(tasks);
}

With Task.WhenAll:

  • All items are processed in parallel — no performance regression
  • Every async continuation is properly tracked by the Task infrastructure
  • Exceptions are collected and re-thrown as AggregateException when awaited — to a caller that can handle them
  • The process does not terminate

As an immediate safety net while refactoring, you can also add a global handler in Global.asax that prevents process termination from unobserved task exceptions:

// In Application_Start (Global.asax)
TaskScheduler.UnobservedTaskException += (sender, args) =>
{
Logger.Error("Unobserved task exception", args.Exception);
args.SetObserved(); // prevents process termination
};

This is a safety net, not a fix — the underlying orphaned tasks still exist and their results are still lost. But it prevents a single unhandled background exception from taking down your entire server while you work through a proper refactor.


The Rule

The rule to remember is simple: never pass an async lambda to Parallel.ForEach. The two are fundamentally incompatible. Parallel.ForEach has no understanding of Task, does not await the work it fires, and any exception thrown after the first await inside your lambda will be orphaned on the ThreadPool. In .NET 4.0 and later, that means process termination.

The pattern is particularly easy to introduce because it compiles cleanly, looks reasonable, and even appears to have proper error handling. The only sign something is wrong is your server going down.

When you need parallel async work, use Task.WhenAll. It was designed for exactly this purpose.


Found this useful? If you’re diagnosing IIS instability, check your application pool’s Rapid Fail Protection settings and review Event Viewer’s Application log for Event ID 1026 with ThreadPoolWorkQueue.Dispatch() at the bottom of the stack trace — that’s the fingerprint that points directly to this pattern.

Categories: Uncategorized Tags: , , , ,
  1. No comments yet.
  1. No trackbacks yet.

Leave a comment