Using a Pre-Trained ONNX Model for Inferencing – InApps Technology is an article under the topic Software Development Many of you are most interested in today !! Today, let’s learn Using a Pre-Trained ONNX Model for Inferencing – InApps Technology in today’s post !

Read more about Using a Pre-Trained ONNX Model for Inferencing – InApps Technology at Wikipedia

You can find content about Using a Pre-Trained ONNX Model for Inferencing – InApps Technology from the Wikipedia website

In the previous part of this series, I introduced the Open Neural Network Exchange (ONNX) and the ONNX Runtime as the interoperable toolkit and platform for machine learning and deep models.

In this tutorial, we will explore how to use an existing ONNX model for inferencing. In just 30 lines of code that includes preprocessing of the input image, we will perform the inference of the MNIST model to predict the number from an image.

The objective of this tutorial is to make you familiar with the ONNX file format and runtime.

Setting up the Environment

To complete this tutorial, you need Python 3.x running on your machine. We will start by creating a Python3 virtual environment to isolate it from the main Python environment on the machine.

With the virtual environment in place, let’s install the Python modules needed by our program. The following command will install ONNX, ONNX Runtime, and OpenCV in your environment.

Let’s download and expand the MNIST pre-trained model trained in Microsoft CNTK Toolkit from the ONNX Model Zoo.

The above command results in a new directory called mnist that has the model and the test data serialized into ProtoBuf files. We are not going to use the test data for the tutorial.

We can now examine the model through the Netron tool by opening the model.onnx file.

The MNIST model from the ONNX Model Zoo uses maxpooling to update the weights in its convolutions as shown in the graph from Netron.

The model has two convolutional layers, two maxpool layers, one dense layer, and an output layer that can classify one of the 10 values representing the labels used in the MNIST dataset.

Writing Inference Code for Prediction

We will now write code for performing inference on the pre-trained MNIST model.

Let’s start by importing the right Python modules.

Notice that we are using ONNX, ONNX Runtime, and the NumPy helper modules related to ONNX.

The ONNX module helps in parsing the model file while the ONNX Runtime module is responsible for creating a session and performing inference.

Next, we will initialize some variables to hold the path of the model files and command-line arguments.

In the next step, we will load the image and preprocess it with OpenCV.

The above code snippet is responsible for converting the image to grayscale and resizing it to 28X28 array. This array will be used as an input to the model.

Read More:   Brendan Eich on Creating JavaScript in 10 Days, and What He’d Do Differently Today – InApps Technology 2022

We will now convert the image into a NumPy array of type float32.

We are now ready to pass the data to the model for inference.

We need to use the same name as the input layer and the output layer of the neural network. You can easily retrieve them from the session.getinputs() and session.getoutputs() methods. The output from the above snippet matches the input and output node names shown by Netron.

Let’s pass the input to the session and print the prediction.

We apply the argmax function of NumPy to retrieve the value with the highest probability.

Try running the code by passing an image of a handwritten number. It predicts that with good probability.

Here is the complete code for your reference:

In the next part of this tutorial, we will learn how to export a PyTorch model and converting that into a TensorFlow saved model file. Stay tuned.

Read More:   De-Risk a Technology Startup – InApps 2022

Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at

Feature image by DavidRockDesign from Pixabay.


Rate this post
As a Senior Tech Enthusiast, I bring a decade of experience to the realm of tech writing, blending deep industry knowledge with a passion for storytelling. With expertise in software development to emerging tech trends like AI and IoT—my articles not only inform but also inspire. My journey in tech writing has been marked by a commitment to accuracy, clarity, and engaging storytelling, making me a trusted voice in the tech community.

Let’s create the next big thing together!

Coming together is a beginning. Keeping together is progress. Working together is success.

Let’s talk

Get a custom Proposal

Please fill in your information and your need to get a suitable solution.

    You need to enter your email to download


      Success. Downloading...