Install Software

Note

Be sure to follow the hardware setup instructions to install your MX3 M.2.

Important

By downloading and installing the MemryX SDK, you agree to the terms of its licenses.

Optional Software Recommendations

Here we offers a set of recommended operating systems and distributions for use with MemryX accelerators across various host platforms. While the SDK is designed to work across many environments, the OS options listed here are those we’ve validated internally or commonly seen in successful deployments.

These are recommendations, not requirements. The SDK supports more than what’s shown on this page.

x86 (Intel/AMD)

On x86 (Intel/AMD) hosts, if you’re unsure which OS to choose, we recommend Ubuntu 24.04. It’s a widely supported and well-documented Linux distribution known for its stability and ease of use.

ARM

For ARM-based hosts, the recommended OS will vary depending on the specific board. In all cases, we suggest using Linux distributions based on the Ubuntu/Debian family.

• Raspberry Pi 5: Raspberry Pi OS (64-bit only) — install via the official Raspberry Pi Imager

• Orange Pi 5 Plus / Max: Ubuntu 24.04 from ubuntu-rockchip

• Rockchip RK3588 SBCs (e.g., Rock5B, BPI-M7): Ubuntu 24.04 from ubuntu-rockchip

• Other ARM-based SBCs: Armbian

RISC-V

• Banana Pi BPI-F3: Bianbu OS v2.0

• Other RISC-V SBCs: Armbian

Installation

Depending on your platform and workflow, complete one or more of the following:

Linux

• Install Runtime (Linux)

• Install Tools

Windows

• Install Runtime (Windows)

• Install Tools

The SDK supports both x86_64 and ARM64 architectures across major Linux distributions and Windows.

SDK Components

The MemryX SDK is organized into two main components: Tools and Runtime Software. Together, they provide everything needed to convert, simulate, deploy, and run AI models on MemryX accelerators.

1. Tools

The SDK includes a suite of tools that support model compilation, bit-accurate simulation, and benchmarking. These are distributed through the memryx Python package and can be installed via pip. For more details, see the Tools section.

Key Components Packaged

• Neural Compiler

  Compiles neural network models into a MemryX Dataflow Program (DFP) runtime format to run on MXA. See: Neural Compiler

• Simulator

  Simulates MXA behavior on host systems for model validation, bit-accurate performance measurement, and debugging. See: Simulator

• Python Runtime API

  Included in the memryx package, allowing Python-based applications to run inference on MemryX hardware. See: Accelerator API (Python)

• Python Benchmark Tool

  A standalone performance benchmarking tool built on top of the memryx Python runtime API. mx_bench is designed to evaluate model performance on MemryX hardware, providing detailed measurements of inference throughput, latency, and system utilization. More details: Python Benchmark

2. Runtime Software

The runtime software runs on the target system where MemryX hardware is deployed. It provides the execution environment and communication infrastructure required to recognize hardware devices, manage them efficiently, and perform inference. For more details, see the Runtime Software section.

Key Components Packaged

• Device Driver (memx-drivers)

  Includes the kernel module that enables the host system to detect and communicate with MemryX PCIe-based hardware. It also manages firmware updates and facilitates runtime communication with the hardware during inference.

• Runtime Daemon (mxa-manager)

  A background service responsible for resource management and runtime orchestration between user applications and MemryX hardware.

• C++ Runtime API (memx-accl)

  A high-performance C++ library that provides direct access to MemryX accelerator capabilities. The API allows developers to load compiled models, configure execution parameters, manage input and output buffers, and run inference tasks with minimal latency. It supports both single-stream and multi-stream execution models, enabling efficient use of hardware resources across diverse application scenarios. For detailed usage, refer to: Accelerator API (C++)

• C++ Benchmark Tool (acclbench)

  A standalone performance benchmarking tool built on top of the memx-accl API. acclbench is designed to evaluate model performance on MemryX hardware, providing detailed measurements of inference throughput and thermals. More details: C++ Benchmark

Platform Support

The following table outlines the platforms currently supported by the SDK, specifying which components are available on each platform:

Note

• On Windows, the Compiler and Simulator are supported only via WSL.

• The Benchmark tool (`acclBench`) is supported natively on Windows.

• The Driver and C++ Runtime are supported natively on Windows, but only the C++ API is available (Python runtime is not supported).

• On Linux ARM64, the Simulator is currently not supported.

Component	Linux x86_64	Linux ARM64	Windows
Tools
└── Compiler	✅ Supported	✅ Supported	✅ Supported (WSL Only)
└── Simulator	✅ Supported	❌ Not Supported	✅ Supported (WSL Only)
└── Benchmark	✅ Supported	✅ Supported	✅ Supported (Native & acclBench Only)
Driver & Runtime	✅ Supported	✅ Supported	✅ Supported (Native & C++ Only)

Note

• Tools include Compiler, Simulator, and Python Runtime and Benchmark.

• Runtime includes Driver and C++ Runtime support.