Autonomous Parallel Execution for .NET Array Code | ILNumerics Accelerator

SUPPORTDOCUMENTATIONGetting Started<br>Computing EngineSetup & Rules<br>Arrays<br>Data I/O<br>Configuration<br>Built-in Functions<br>Toolboxes<br>HDF5<br>Low Level Expert Functions<br>Threading Model<br>Accelerator CompilerDemos, Benchmarks<br>Configuration<br>Accelerated Features<br>Roadmap

Compatibility

Visualization Engine<br>Visual Studio Tools

Tutorials<br>Find Support<br>CHANGELOG

Strong scaling .NET array code by autonomous execution

Modern hardware is massively parallel, but software production is still largely sequential. Compilers can often use SIMD vector units automatically. Beyond that fine granularity, parallel execution across cores, accelerator devices, and larger program regions usually requires manual engineering, because global static dependency analysis becomes too complex to solve safely and profitably in general.

ILNumerics closes this gap by replacing global static dependency analysis with local runtime dependency checks. Instead of searching a sequential program for safe places to parallelize, ILNumerics treats every array instruction as an autonomous execution unit that may run freely and concurrently, coordinating only with the instructions that directly produce its required inputs.

The core idea: strong scaling without rewriting your .NET code

In ordinary execution, the main thread walks through the program and executes array instructions in source-code order. This creates unnecessary waiting: the thread may spend time in long-running operations, while other independent work could already be prepared or executed elsewhere.

ILNumerics changes this model. The main thread no longer has to perform the full numerical workload itself. Its sole task becomes discovering upcoming array instructions, capturing their minimal dependency structure, and releasing them for autonomous execution.

The more work can leave the main thread, the more execution time can overlap. Sequential program time is compressed into concurrent and pipelined runtime behavior: independent instructions may run concurrently, while dependent instructions preserve only the local order required for correct results.

This is strong scaling applied to high-level .NET scientific array code: the same sequential program can complete faster as more CPU cores, vector units, memory bandwidth, and accelerator resources are brought to bear. Instead of requiring developers to rewrite algorithms around explicit threads, tasks, or device kernels, ILNumerics exposes parallel execution opportunities automatically from the program’s array operations and their runtime dependencies.

In practice, ILNumerics Accelerator can achieve speedups of 3x to more than 100x for high-level .NET numerical code, depending on workload and hardware configuration (benchmarks).

Autonomous array instructions

ILNumerics array instructions are not passive operations waiting for a central scheduler. Each instruction is an active execution unit with its own decision logic.

An instruction attempts to execute its workload as early as possible. At runtime, it decides when to run, where to run, and how to run, based on the operations it handles, its direct dependencies, data location, and available execution resources.

This decentralized model reduces the need for global synchronization. The main thread does not have to orchestrate every execution decision. It exposes the necessary dependency information, while autonomous instructions coordinate locally.

The execution net

The execution net is the moving window of array instructions that are currently active. It is established and constantly updated by the main thread as the program is visited in normal sequential order.

For each visited instruction, the execution net captures only the dependency structure required for correctness: which local inputs must be available before which work can proceed. It does not preserve the complete rigid source-code order when that order is not required by data dependencies.

At any time, the execution net may contain hundreds or thousands of active instructions. Within this window, instructions can execute, overlap, and move to suitable compute resources as soon as their local dependencies permit. Completed instructions leave the execution net, cleaning-up after themselves.

The main thread builds the minimum dependency structure; autonomous instructions turn it into parallel execution. Workload and scheduling are parallelized.

Array pipelining

Dependent instructions do not always need to wait for complete upstream results. Often, partial information is enough to begin useful work.

For example, the output size of a segment can often be determined from the shapes of its input arrays. Device selection may only require information about input shape and data location in order to estimate transfer and compute costs. These decisions can start before all input values have been fully produced.

By starting downstream preparation and execution...