Simulator#

class Simulator(dfp='model.dfp', sim_directory: str = None, verbose=0, no_progress_bar=False, timeout=inf, **kwargs)#

MemryX hardware simulator.

The Simulator provides a bit-accurate / approximately timed model for MXAs. It is built using SystemC and models the arithmetic operations / finite-state machines (FSM) of the chip to accurately simulate chip performance. After loading a Dataflow Program, the simulator consume provided input feature maps, run inference, and produces output feature maps. It will also measure the estimated FPS and latency.

Parameters:

dfpstring or pathlib.Path or Dfp: Path to dfp or a Dfp object
verboseint: How loud the simulator and its function calls will be.
sim_directorystring or pathlib.Path: Path for working directory. The simulator will create several files while simulating. Defaults to a tempfile.TemporaryDirectory.
no_progress_barbool: Supress progress bar display.

Examples

from memryx import Simulator

# 1. Use a compiled dfp
s = Simulator(dfp=dfp)

# 2. Use the specified DFP file
s = Simulator(dfp='/path/to/model.dfp')

infer(inputs, model_idx=0)#

Runs inference on the Simulator

Simulate the configured dfp with the given inputs. A running directory will be generated (defaults to tempfile.TemporaryDirectory) This directory will remain after the simulator finishes if sim_directory was specified. This is a blocking call and is liable to take quite a while to simulate. For a non-blocking version see Simulator.start().

Note

The Simulator running directory should not be tampered with while the simulator is executing.

Parameters:

inputsnp.array() or list of np.array(): The array shape should be model’s input_shape. If there are multiple inputs to the DFP (e.g. multi-model) then a list of (appropriately shaped) numpy arrays should be provided in the same order as indicated by argument model_idx
model_idxint or list of ints.: to be provided in case of multi-model dfp eg. model_idx = 0, inputs provided are only for model 0 model_idx = 2, inputs provided are only for model 2 model_idx = [0,1,2] inputs for model 0,1,2 are provided in the same order

Returns:

outputslist of np.array(): Produces the NN output, this data is returned as a list of np.array(). Each of the returned arrays will have the model’s output_shape.

Raises:

SimulatorError: When the simulator fails to run.
ValueError, TypeError: When the input is incorrectly configured.
FileNotFoundError: When the running directory is tampered w/ during exectution.

Examples

Example 1:: model input = (1, 224, 224, 3)

inputs = np.ones([1, 224, 224, 3])
outputs = s.infer(inputs, model_idx=0)  
# outputs[0].shape = (1000, 1)

Example 2:: Batched input = (8, 128, 128, 3)

# outputs will also contain a batch of 8
inputs = np.ones([8, 128, 128, 3])
outputs = s.infer(inputs, model_idx=0)
# outputs[0].shape = (8, 1000, 1)

Example 3:: Multi-model input

# model_0 with inputs [None, 32, 32, 3], [None, 224, 224, 3]
# model_1 with input  [None, 48, 48, 3]

# To get output of only model_1
inputs = [np.random.random([1, 48, 48, 3])]
outputs = s.infer(inputs, model_idx=1)

# To get outputs of model_0 and model_1 together
inputs = [
    np.random.random([1, 32, 32, 3]),
    np.random.random([1, 224, 224, 3]),
    np.random.random([1, 48, 48, 3])
]
outputs = s.infer(inputs, model_idx=[0, 1])

benchmark(frames=4)#

Run the simulator to benchmark the performance

Simulate the configured dfp random data to obtain latency and fps A running directory will be generated (defaults to tempfile.TemporaryDirectory). This directory will remain after the simulator finishes if sim_directory was specified. This is a blocking call and is liable to take quite a while to simulate. For a non-blocking version see Simulator.start().

Parameters:

framesint

Number of frames to run.

frames >= 4 to get estimated fps, if frames < 4 fps returned will be 0

Returns:

latency, fpsfloat, float: Reports the latency and FPS. If called while the simulator is executing (i.e blocked) it will return (None, None)

Note

The latency is timed from the moment the first data of the input is consumed until the last data of the output produced. The FPS is calculated as the time between output frames.

Examples

# multi-frame, random data
latency,fps = s.benchmark(frames=4)

start(inputs, model_idx=0)#

Start the simulator (non-blocking)

Same functionality as infer() except this function uses multiprocessing to spawn a child process for the simulator. The simulation progress can be monitored using Simulator.get_progess() and Simulator.is_running(). Once the simulator is finished, results can be read back using Simulator.get_results().

Examples

import numpy as np
import time
from tensorflow import keras
from memryx import NeuralCompiler, Simulator

dfp = NeuralCompiler(models=keras.applications.MobileNet(), verbose=1).run()
s   = Simulator(dfp)

# start a non-blocking simulation
image = np.zeros([4,224,224,3]) # running 4 frames of inputs!
s.start(inputs=image)
while s.is_running():
    print("{:.1f}".format(s.get_progress())); time.sleep(0.1);

outputs,latency,fps = s.get_results()

# spawn many simulators (4) to process many frames in parallel
images = [image for i in range(4)]
sims = [Simulator(dfp=dfp, sim_directory="simdir_{}".format(i)) for i in range(4)]
[s.start(inputs=image) for s,image in zip(sims,images)]

while(any([s.is_running() for s in sims])):
    print([round(s.get_progress(),1) for s in sims], end='\r')
    time.sleep(0.1)

outputs = np.concatenate([s.get_results()[0] for s in sims])

is_running()#

Get if the Simulator is running.

Returns the running status of the Simulator.

Returns:

running_statusbool: Return True if the simulator is still running.

get_progress()#

Get Simulator progress.

Returns the approximate progress of the simulation run.

Returns:

progressfloat: Returns the simulator progress (between 0-100).

get_results()#

Get the Simulator results.

Return the simulator results (if it is finished).

Returns:

outputs, latency, fpsnp.array() or list of np.array(), float, float: Produces the NN output and reports the latency and FPS. The latency is timed from the moment the first data of the input is consumed until the last data of the output produced. The FPS is calculated as the time between output frames. The output data is returned as a list of np.array(). The arrays will have the shape [N + output_shape] for each output where N is the number of frames simulated. If called while the simulator is already executing it will return (None, None, None).

get_config()#

Get the Simulator config.

Returns the current config set for the Simulator.

Returns:

configdict: dictionary of Simulator configurations

reset_config()#

Reset the Simulator config.

Clears any previously set config in the Simulator instance and sets them to the default values (defined in the constructor).

set_config(**kwargs)#

Configure the Simulator.

Configure the Simulator with the keyword arguments listed in the constructor.

Parameters:

**kwargs: Keyword args used to configure the simulator.