Using Parallel.For

Overview

A normal for loop runs its iterations one after another on a single thread. If the loop body is slow and each iteration is independent of the others, the machine is only using one of its processor cores while the rest sit idle.

Parallel.For(), found in the System.Threading.Tasks namespace, solves this problem. It takes a range of integer values and runs the loop body for every value in that range, spreading the work over several threads. The range includes the starting value and excludes the ending value, which is exactly how array and list indexes work, so Parallel.For(0, items.Length, ...) visits every element exactly once.

The method splits the range into chunks and hands each chunk to a different thread, so the iterations do not run in order. The call itself is blocking: the thread that calls Parallel.For() waits until every iteration has finished (or until the loop is cancelled).

This article follows an easy-to-harder path with five examples:

1. Step 1: the simplest loop over a range of indexes.
2. Step 2: limiting how many iterations may run together with ParallelOptions.MaxDegreeOfParallelism.
3. Step 3: stopping the loop from the outside with a CancellationToken.
4. Step 4: a bad example with a shared += counter and a good example with Interlocked.Add, showing that the loop body is not synchronized for you.
5. Step 5: a normal for loop and a Parallel.For loop doing the same CPU-heavy work, with their run times compared.

Features and Design

The simplest overload of the method looks like this:

1. fromInclusive – the first index of the range; this value is used.
2. toExclusive – the loop stops before this value; this value is not used.
3. body – an Action<int> delegate that is called exactly once for every index in the range.

The method returns a ParallelLoopResult structure. Its IsCompleted property is true when every iteration ran and the loop was not stopped early. The delegate can also take a second, optional parameter of type ParallelLoopState, which allows the loop to exit early and to detect unhandled exceptions.

An overload that takes a ParallelOptions instance lets you change the default behavior. The class exposes three properties:

1. MaxDegreeOfParallelism – the maximum number of iterations allowed to run at the same time. The default is -1, which means "no upper limit, let .NET decide". Setting it to 1 makes the loop behave like a sequential for loop.
2. CancellationToken – a token that can stop the loop from the outside. The default is CancellationToken.None, which means the loop is never cancelled.
3. TaskScheduler – a custom scheduler, or null to use the default scheduler.

Two behaviors are important to remember:

1. When the token is cancelled, Parallel.For() throws an OperationCanceledException directly; it is not wrapped in an AggregateException. Exceptions thrown by the loop body, on the other hand, are collected into an AggregateException.
2. The Parallel class synchronizes its own internal bookkeeping, but it provides no synchronization at all for the data your delegate touches. Any shared variable, collection or file accessed from the loop body still needs a lock, an Interlocked method, or a thread-safe collection.

When Parallel.For Helps

Parallel.For() is designed for ranges of independent, CPU-bound work. Typical use cases are data transformation over large sets, matrix computations, image processing, simulations, data aggregation and analysis, rendering, and large-scale mathematical computations such as searching for prime numbers.

Step 1: Code Example – The Simplest Parallel.For Loop

How It Works

1. An array of five strings is created. Index 0 writes to slot 0, index 1 to slot 1, and so on, so two iterations never touch the same memory. This is why no lock is needed.
2. Parallel.For(0, labels.Length, ...) produces the indexes 0, 1, 2, 3 and 4. The end value 5 is never passed to the delegate.
3. Each iteration records the id of the thread that ran it. Several different ids appear, and the main thread (usually id 1) also takes part, because the calling thread helps with the work instead of just waiting.
4. The array is printed after the loop. Since Parallel.For() blocks until all iterations are done, every slot is already filled.
5. result.IsCompleted reports True because nothing stopped the loop early.

Sample Output

The thread ids change on every run and on every machine; the index order in the printed list is always 0 to 4, because the printing happens after the loop.

Step 2: Code Example – Limiting Parallelism with ParallelOptions

How It Works

1. A ParallelOptions instance with MaxDegreeOfParallelism = 2 is passed to the loop, so at most two iterations may be inside the delegate at any moment.
2. Each iteration increases a shared runningCount when it starts and decreases it when it ends. Interlocked.Increment and Interlocked.Decrement change the value in one uninterruptible step, so no update is lost.
3. InterlockedMax remembers the largest value runningCount ever reached. It uses Interlocked.CompareExchange, which writes the new value only if the field still holds the value the method last read; otherwise it tries again with the fresh value.
4. The 100 ms sleep represents work. Without it, an iteration would finish before the next one started and the overlap could not be measured.
5. The reported maximum is 2, which proves the limit was respected. Removing the options object would let .NET use as many threads as it wants, and the value would be higher.

Sample Output

Step 3: Code Example – Cancelling a Parallel.For Loop

How It Works

1. A CancellationTokenSource creates the token and decides when the cancellation happens. CancelAfter(150) starts a timer that cancels the token after 150 milliseconds.
2. The token is placed into ParallelOptions.CancellationToken. From that moment the loop checks the token while it works and refuses to start further iterations once it is cancelled.
3. The loop is asked to run 100 iterations, each sleeping 100 ms, two at a time. That would take roughly five seconds, so the cancellation always wins and only a handful of iterations complete.
4. Parallel.For() then throws OperationCanceledException. Note that it is caught directly, not through an AggregateException. Iterations already running are not killed; they finish their current work first.
5. The counter shows how many iterations completed before the loop stopped. This number changes from run to run, which is normal for concurrent code.

Sample Output

Step 4: Code Example – Shared Counter, Plain += (bad) vs Interlocked (good)

How It Works

1. Both loops do exactly the same job: add 1 to a shared variable 100000 times, so the correct answer is always 100000.
2. The first loop uses unsafeTotal += 1. This single line is really three machine steps: read the current value, add one, write the result back. Two threads can read the same value before either of them writes, and one of the two updates disappears.
3. The printed result is therefore far below 100000, and it is different on every run. This is a classic race condition, and it happens even though Parallel.For() itself is perfectly correct: the class synchronizes its own bookkeeping, never your data.
4. The second loop uses Interlocked.Add(ref safeTotal, 1). The processor performs the read-add-write as one uninterruptible operation, so no update can be lost and the total is always exact.
5. A lock statement would also give the correct answer, but for a simple counter Interlocked is faster because it does not block any thread.

Sample Output

The first number is different every time you run the program, and it may occasionally even be correct on a slow machine. That unpredictability is exactly what makes race conditions dangerous.

Step 5: Code Example – Normal for Loop vs Parallel.For

How It Works

1. The unit of work is CountPrimesBelow(200000), a purely CPU-bound calculation that uses only local variables, so it is safe to run on many threads at the same time.
2. The first loop is an ordinary for loop: the eight work items run one after another on a single thread, and a Stopwatch measures the total time.
3. The second loop is a Parallel.For over the same range. .NET splits the eight indexes into chunks and runs them on several thread pool threads, so multiple processor cores work at the same time.
4. Each iteration writes only to its own array slot, so no synchronization is needed. This is the ideal shape for Parallel.For(): independent iterations and no shared state.
5. SequenceEqual confirms both loops produced identical values. Parallel execution changes the timing and the order of the work, never the results of an independent calculation.

Sample Output

The measured times depend on the machine. The speed-up is limited by the number of processor cores, and it can never be larger than that number, because there are only so many cores to share the work.

When to Use

Use Parallel.For() when:

1. The loop iterates over a numeric range, most often the indexes of an array or a list.
2. The iterations are independent: one iteration does not need the result of another and the order does not matter.
3. The work in each iteration is CPU-bound and heavy enough that splitting it is worth the extra cost of scheduling threads.

Avoid it when:

1. The loop body is very short (for example a simple addition). Starting and coordinating threads then costs more than the loop itself, and the parallel version becomes slower.
2. The iterations depend on each other or must run in a fixed order.
3. The work is I/O-bound, such as calling web services or reading files. In that case async/await with Task.WhenAll is a better fit, because it does not block threads while waiting.
4. Every iteration hits the same shared object anyway. If almost all of the work happens inside a lock, the threads simply wait for each other and nothing is gained.

Related members are Parallel.ForEach() for collections that are not indexed by numbers, Parallel.Invoke() for a fixed set of different actions, and PLINQ (AsParallel()) for query-style processing.