MXA-Manager and Shared / Local#

MXA Manager#

Before discussing Shared and Local modes, it’s important to understand the role of the MXA Manager. This is a system process that manages access to the MXA hardware: scheduling DFPs, coordinating between processes, monitoring hardware stats, and more.

It is installed as a system service on both Linux and Windows, and is automatically started when the system boots up.

Protocols#

There are two methods for applications to interact with the MXA Manager: UNIX Socket Files and TCP/IP Networking. To configure which the server should use, edit the /etc/memryx/mxa_manager.conf file on Linux or the C:\Program Files\memryx\mxa_manager.conf file on Windows.

mxa_manager.conf
# Socket/address to listen on
#
# * use a folder path to use UNIX sockets
# * use 127.0.0.1 for IP networking, limited to localhost
# * set to 0.0.0.0 or an interface address to accept external network traffic
#
LISTEN_ADDRESS="/run/mxa_manager/"

# The daemon will use 3 ports/socket files in order starting with this number.
# For example, "10000" will use 10000, 10001, and 10002
BASE_PORT=10000

# Socket/address to listen on
#
# * use a folder path to use UNIX sockets
# * use 127.0.0.1 for IP networking, limited to localhost
# * set to 0.0.0.0 or an interface address to accept external network traffic
#
LISTEN_ADDRESS="127.0.0.1"

# The daemon will use 3 ports/socket files in order starting with this number.
# For example, "10000" will use 10000, 10001, and 10002
BASE_PORT=10000

Protocol Info#

UNIX Domain Sockets

On Linux, the MXA-Manager service uses UNIX Domain Socket files by default. This is the preferred option for most applications, as it provides fast and secure communication, and also makes Docker configuration much easier.

By default, the socket files will be created as /run/mxa_manager/1000{0,1,2}.sock, and managed by systemd.

TCP/IP Networking

Available on both Linux and Windows, the MXA-Manager can also use TCP/IP networking. Although potentially more complicated to set up when using Docker or specialized network configurations, there should be little to no performance difference between IP and UNIX sockets.

Shared / Local#

Applications using the MemryX runtime can execute DFPs in one of two modes:

  • Shared Mode: multiple processes can access the same MXA hardware simultaneously, and multiple DFPs can be run at a time – the mxa-manager system process handles the coordination

  • Local Mode: a process acquires exclusive access to the given MXA device(s), and can directly communicate with the hardware while running a single DFP

The desired mode is set when creating an Accelerator object in either the Python or C++ Runtime APIs.

Shared Mode#

graph TB p1(("Process 1")) --- sr["mxa-manager"] p2(("Process 2")) --- sr sr --- hw["MX3 Hardware"] style p1 fill:#CFE8FD, stroke:#595959 style p2 fill:#CFE8FD, stroke:#595959 style hw fill:#FFE699, stroke:#595959 style sr fill:#A9D18E, stroke:#595959

In Shared Mode, multiple processes can run one or more DFPs at the same time, and the MXA-Manager service handles the scheduling and execution on the actual hardware. Client processes submit DFPs to execute, then stream input and output feature maps to/from MXA-Manager.

Important

Shared Mode is required for the following scenarios:

  • Multiple processes using the same DFP on the same MXA device

  • Multiple processes using different DFPs on the same MXA device

  • Multiple DFPs or Accl objects in a single process (not needed for multiple models within the same DFP)

  • Multiple DFPs or Accl objects from multiple processes

Note

Since Shared Mode is needed for many use cases, it is the default mode for the MemryX Runtime when using the Accelerator APIs. When you create an Accelerator object, it will automatically use Shared Mode unless you specify otherwise.

Local Mode#

graph TD p1(("Process 1")) --> hw["MX3 Hardware"] p2(("Process 2")) -.-x hw style p1 fill:#CFE8FD, stroke:#595959 style p2 fill:#CFE8FD, stroke:#595959 style hw fill:#FFE699, stroke:#595959 linkStyle 1 stroke:#FF0000, stroke-width:2, stroke-dasharray:4

While Shared Mode is the most versatile, it can incur some overhead due to the need for inter-process communication and scheduling by the MXA-Manager service. On moderately powerful systems, this overhead is usually negligible, but for weak host CPUs or applications that require maximum FPS, it can be beneficial to use Local Mode.

In Local Mode, a single application process (more specifically, a single Accelerator object) is granted exclusive control over the requested MXA device(s). The MXA-Manager then cedes control of that device directly to that object instance, allowing it to communicate directly with the hardware without the overhead of the MXA-Manager.

Important

Only use Local Mode in your application if:

  1. There is a single Accelerator object in your application

  2. Only 1 DFP is being used

  3. No other processes will need to access the same MXA device(s)

Note

If you have multiple MXA devices in your system, you can still run multiple apps / DFPs in Local Mode at the same time, as long as each is assigned to different devices.

Note

The Benchmark tools default to Local Mode for maximum performance, but you can use Shared mode in them with the -s flag.

Usage#

Hint

It’s pretty easy to decide which mode to use:

  • Multiple DFPs and/or applications running at the same time?

    • Use Shared.

  • Single DFP and single application?

    • Use Local for a performance boost (but it still works in Shared).

  • Not sure?

    • Use Shared so you have flexibility for different scenarios.

See also

Check out the Tutorials section for examples of both modes in action.

Troubleshooting#

On Linux, you can check the status of the MXA Manager service with:

sudo systemctl status mxa-manager

If it looks like the service is reporting errors, you can always restart it with:

sudo systemctl restart mxa-manager
Advanced Debugging

For assistance with debugging, you can enable verbose logging for the service.

First, run this command:

sudo systemctl edit mxa-manager

This will open a text editor. Add the following new [Service]/Environment lines to the file:

### Editing /etc/systemd/system/mxa-manager.service.d/override.conf
### Anything between here and the comment below will become the contents of the drop-in file

[Service]
Environment="SPDLOG_LEVEL=debug"

### Edits below this comment will be discarded


### /usr/lib/systemd/system/mxa-manager.service
# [Unit]
# Description=The MemryX MX3 device management daemon.
#
# [Service]
# TimeoutStartSec=30
# ExecStartPre=/bin/sleep 2
# ExecStart=/usr/bin/mxa_manager
# Restart=on-failure
# RestartSec=2
# RuntimeDirectory=mxa_manager
# RuntimeDirectoryMode=0777
# User=mxa-manager
#
# [Install]
# WantedBy=multi-user.target

Then save and exit the editor. Finally, restart the service with:

sudo systemctl restart mxa-manager

This will enable verbose logging, which can be useful for debugging issues with the MXA Manager or the MemryX Runtime. The logs can now be monitored in real-time with:

sudo journalctl -u mxa-manager -f

See also