Automating DFP Generation with the Compiler API#

The MemryX Neural Compiler offers a versatile command-line interface (CLI Tool) to compile models and generate DFPs for the MemryX Accelerator (MXA). In addition to the CLI, MemryX provides a robust Compiler API (compiler API), allowing for full automation of DFP generation.

In this step-by-step guide, we will demonstrate how to leverage the Compiler API to build an automation tool that reads input parameters from a JSON file and generates one or multiple DFPs programmatically.

See the sections below for a step-by-step detailed explanation.

1. Defining the Input JSON Format#

The first step is to define a structured input format that includes all necessary parameters for the Compiler API. Below is a sample JSON file that can be used as input.

{
  "dfps": [
      {
      "models_dir": "yolov7Tiny416_2chip",
      "models": [
          "path/to/model/yolov7-tiny_416.onnx"
      ],
      "num_chips": 2,
      "chip_gen": "mx3",
      "autocrop": true,
      "effort": "normal",
      "dfp_fname":"yolo_dfp",
      "verbose": 1
      }
  ]
  }

This JSON structure enables defining DFP groups with the model name and its corresponding parameters. Let us try to understand this a little more in detail. The models_dir is used to specify the directory in which the models are stored. You may specify as many arguments to the compiler via the JSON file as needed. num_chips, models and autocrop are some examples of arguments in the example above.

For multi-model compilations, the model paths can be provided as a list to the models argument as seen in the example below. The compiler will handle them according to the information in the documentation compiler API.

Here’s an example of a multi-model input JSON:

{
      "dfps": [
      {
          "models_dir": "yolov7Tiny416_multimodelinputtest",
          "models": [
          "path/to/model/yolov7-tiny_416.onnx",
          "path/to/model/yolov7-tiny_416.tflite"
          ],
          "num_chips": 4,
          "chip_gen": "mx3",
          "autocrop": false,
          "effort": "normal",
          "dfp_fname":"testingfname.dfp",
          "verbose": 4,
          "show_optimization": true
      }
      ]
  }

Additionally, to support generating DFPs multiple times in a single run, you can include several DFP groups in the JSON file, as illustrated below:

{
  "dfps": [
      {
          "models_dir": "yolov7Tiny416_firsttime",
          "models": [
              "path/to/model/yolov7-tiny_416.onnx"
          ],
          "num_chips": 2,
          "chip_gen": "mx3",
          "autocrop": true,
          "effort": "normal",
          "dfp_fname":"testingfname.dfp",
          "verbose": 4,
          "show_optimization": true
      },
      {
          "models_dir": "yolov7Tiny416_secondtime",
          "models": [
          "path/to/model/yolov7-tiny_416.onnx"
          ],
          "num_chips": 2,
          "chip_gen": "mx3",
          "autocrop": false,
          "effort": "normal",
          "verbose": 4,
          "show_optimization": true
      }
  ]
  }

2. Automating the Process with Python#

Now that the input format is defined, we can move on to writing a Python script that parses this input file and invokes the Compiler API based on the provided configurations.

First, import the required Python libraries along with the MemryX libraries.

import json
import argparse
import os
import shutil
from memryx import NeuralCompiler

Next, parse the DFP group data from the input file and set the input configuration dynamically.

def compile_models_recursively(nc, groups_data):

    # Base case: if no dfp groups left to compile then enbd and return
    if not groups_data:
        return

    # Compile the first dfdp group of models
    group_data = groups_data[0]
    compile_group(nc, group_data)

    # Recursively compile the remaining dfp groups
    compile_models_recursively(nc, groups_data[1:])

def compile_groups(json_file):
    # Load the dfp groups and their parameters from the input JSON file
    with open(json_file, 'r') as file:
        groups_data = json.load(file)["dfps"]
    
    #Initialize the NeuralCompiler
    nc = NeuralCompiler()

    # Start the recursive group compilation
    compile_models_recursively(nc, groups_data)

if __name__ == "__main__":
    # Parse CLI arguments to accept the input JSON file path
    parser = argparse.ArgumentParser(description="Compile neural network models using configurations given in a JSON file")
    parser.add_argument("json_file", type=str, help="Path to the input JSON configuration file.")
    
    # Get the file path from the command line input
    args = parser.parse_args()

    # Compile models based on the provided JSON file
    compile_groups(args.json_file)

The script will iterate through each DFP group and compile the models as specified. Each DFP group will be processed in its own folder, and the output files will be placed accordingly. The DFP file name defaults to the group name. For example, if your JSON file contains "models_dir": "yolov7Tiny416_firsttime" and you have not specified the "dfp_fname" argument, the DFP file generated will be saved as yolov7Tiny416_firsttime.dfp. You can override this by using the dfp_fname parameter in the input JSON to name the file.

def set_compiler_config(nc, params):
    for key, value in params.items():
        if isinstance(value, dict):  # If a nested dictionary, call recursively
            set_compiler_config(nc, value)
        else:
            # Set the parameter if it exists in the config
            nc.set_config(**{key: value})

def compile_group(nc, group_data):
    
    # Reset configuration of the neural compiler for the new dfp group
    nc.reset_config()

    # Set group-level configurations
    group_params = {key: value for key, value in group_data.items() if key not in ["models", "models_dir"]}
    set_compiler_config(nc, group_params)

    # Compile all models in the group together
    models = group_data["models"]
    nc.set_config(models=models)

    # Set DFP file name from json or dfp_fname if specified
    dfp_fname = group_data.get("dfp_fname", f"{group_data['models_dir']}.dfp")
    nc.set_config(dfp_fname=dfp_fname)

    # Verify and print the current configuration
    config = nc.get_config()
    print(f"Compiling group: {group_data.get('models_dir')} with models: {models}")
    
    print("printing config after final set ")
    print(config)
    
    dfp_dir = group_data['models_dir']

    # Create a directory in the name of the dfp group name given will replace old directory
    os.makedirs(dfp_dir, exist_ok=True)


# Change directory into newly created directory
    original_dir = os.getcwd()
    os.chdir(dfp_dir)

    try:
        # Run the compiler and generate the DFP file
        dfp = nc.run()
        if 'dfp_fname' in group_data:
            print(f"DFP file saved to {group_data['dfp_fname']}.dfp")
        else:
            print(f"DFP file saved to {group_data['models_dir']}.dfp")

    finally:
        #Move back to original directory
        os.chdir(original_dir)

Summary#

In this tutorial, we covered how to programmatically use the MemryX Compiler API to automate DFP generation for one or multiple models.