Inference Over IP Networks#

Introduction#

One of the “bonus” features of mxa-manager and Shared Mode is that connections between client and management server do not have to be on the same physical machine. One computer can have MXA(s) connected to it, and one or more clients can request to use them in Shared Mode, via typical TCP/IP networking over ethernet, etc.

This tutorial explains how to set up such a system.

Warning

This feature is not the intended way to use the MXA, and will have performance gaps versus running on the same physical machine. Data is sent uncompressed between client and the mxa-manager server, so network traffic congestion can be quite severe.

However, minimal changes are required for programs to function over the network, so this tutorial is provided as an “interesting experiment” users may try.

Runtime Installation#

Both the server and the client(s) need to have the runtime installed. No special steps are necessary.

Configure Server#

Edit Address Config#

You can configure the mxa-manager daemon to listen either on all interfaces, or a specific network interface. As network configurations can vary greatly based on your infrastructure, we will assume you know which you want to use.

To listen on all interfaces, use 0.0.0.0 in the following config.

To listen on a specific interface, use its IP, such as 192.168.1.1 in the following config.

Edit /etc/memryx/mxa_manager.conf in your favorite text editor, and replace the LISTEN_ADDRESS with your chosen IP. E.g., replace this:

LISTEN_ADDRESS="127.0.0.1"

With this:

LISTEN_ADDRESS="0.0.0.0"

Edit Port Config (optional)#

By default, mxa-manager will use ports 10000, 10001, and 10002. You can configure the base port in the config file, then the server will use the base, base+1, and base+2 ports. For example, a base port of 400 will use ports 400, 401, and 402.

To change this config, edit the /etc/memryx/mxa_manager.conf file:

# Replace with your desired base port
BASE_PORT=10000

Update Firewall Rules#

Lastly, make sure your firewall allows incoming connections on your chosen ports. This is very OS and environment dependent, but if you’re using ufw on Ubuntu, the commands would be the following for the default ports:

sudo ufw allow 10000
sudo ufw allow 10001
sudo ufw allow 10002

Restart the Service#

Now that everything’s configured, restart the service:

sudo systemctl restart mxa-manager.service

Connect Clients#

In your client applications, now we simply specify the IP (+port) of the server when setting up your MxAccl or MxAcclMT object, like you would for Docker.

For example, if your server is running on a computer with IP 192.168.1.100 and the default ports, use:

MxAccl* accl = new MxAccl(true, "192.168.1.100");
accl->connect_dfp("my_model.dfp");
...

And if you want to specify the base port as 400 as well, replace the first line with:

MxAccl* accl = new MxAccl(true, "192.168.1.100", 400);

Summary#

Congrats! You are now running inference remotely over the network, with one server and one or more clients. As previously noted, the performance will be low and network traffic quite heavy, but this feature might enable some niche applications such as mini MXA server farms.

On this page