Neural Compiler#

Resource Errors

Resource errors are identified by the prefix memryx.errors.ResourceError:, followed by a detailed message. This indicates insufficient resources on the target chip/module for compiling your neural network models.

Troubleshooting Steps:

  • Confirm you are targeting the correct number of chips on your module. For the m.2 module, the chip count is 4, which can be specified with the -c 4 argument. For further details on using the -c argument, refer to the Neural Compiler documentation.

  • Consider using the --autocrop argument to attempt running your model. Cropping pre-processing and/or post-processing layers may enable better mapping to the accelerator. More information on the --autocrop argument can be found in the Neural Compiler documentation.

Hint

Pre-processing and post-processing layers are much lighter in computational demand compared to the core of the model and can often be cropped without adversely affecting performance. These layers, being less resource-intensive, can be efficiently executed on the host through the Accelerator API.

  • Explore the possibility of utilizing more chips with the -c num_chips argument, but proceed with caution. Although the simulator can support any number of chips, it’s crucial to adhere to the hardware’s chip capacity. This method can provide insights into the necessary chip count for your model. Alternatively, compiling with -c min allows the compiler to determine the minimum required number of chips.

See also

PyTorch-specific Errors

Deprecated since version v0.8.0: Direct support for PyTorch 1

PyTorch models can be utilized by exporting them to ONNX (for more information see tutorial on exporting to ONNX). Direct support for Pytorch 2 will be available at a later date.

PyTorchIF: torch.fx symbolic tracer failed with this message: [msg]

PyTorch is unique compared to the other frameworks (TF[/lite], Keras, ONNX) because it is intended to be run dynamically: a python script executes line-by-line and calls PyTorch functions that run on a CPU or GPU. But for many accelerators, including ours, a ‘static graph’ of the neural network model is required. This graph has the model parameters (kernel shapes, layer connections, etc.) fixed instead of interpreted at runtime by Python.

PyTorch offers a tracing framework called torch.fx, that can create static graphs from pytorch+python code. However, it isn’t perfect, and will often reach cases where is doesn’t know what to do.

If you know how to use torch.fx, and know the arguments that need to be passed to the symbolic tracer, you can try using the --pt_trace_args argument to the Compiler.

If you are unsure what args to give to torch.fx, or if the tracer still failed even with --pt_trace_args, we recommend that you export the model to ONNX and try compiling the resulting .onnx file.

For more info on PyTorch->ONNX, see here.