Runtime Usage#

The MemryX dataflow architecture is designed to achieve high-throughput inference performance. Since it is a “pipelined dataflow”–different from traditional CPU/GPU architectures–proper use of the MemryX Runtime is essential to achieve optimal performance when integrating into user applications.

Under the hood, there is also the Linux and Windows drivers (installed in Get Started), but these are not directly interacted with by the user.

Instead, the Python and C++ APIs provide a high-level interface to the MemryX hardware, allowing users to focus on their application logic rather than low-level hardware details.

This section provides deeper details on:

Callback Functions

How to define and use async Callback functions with the MemryX Runtime.

callbacks.html

Multi-Stream

Multiple Streams (e.g. multiple cameras) and how to use them with the Runtime in your application.

streams.html

Pre & Post Processing

How to connect cropped pre- and post-processing models in your application, and commentary on performance.

prepost.html

MXA-Manager and Shared / Local Modes

Overview of the MXA-Manager daemon, and Shared and Local modes: two ways to control application access to MXA hardware.

shared_local.html

Multi-Device

How to use multiple accelerator modules on the same system, with automatic load balancing.

multi_device.html

Multi-DFP

Using multiple DFPs at the same time on the same device, within applications or across multiple running processes.

multi_dfp.html