Archive for Image Processing

Porting a Keras trained model to Jetson Xavier

June 14, 2023 at 9:56 am · Filed under Image Processing, NVIDIA Jetson, tensorflow ·Tagged

I have been training resnet models on my Windows desktop machine with big GPUs. Now I need to deploy these models on a Jetson Xavier for real-time predictions.

My model format historically has been the Keras .h5 file. So now I need to convert the .h5 model file into the onnx exchange format.

There are two steps:

1) convert the .h5 to a frozen .pb format – this step is done on the desktop machine

2) convert the .pb format to the onnx format – this step is done on the Jetson.

On the PC

# python 

from keras.models import load_model

# you can either load an existing h5 model using whatever you normally use for this
#  purpose, or just train a new model as save using the template below

model = load_model('model.h5',custom_objects=models.custom_objects())

# save in pb froozen pb format, this format saves in  folder 
#  and you are supplying the folder name

model.save('./outputs/models/saved_model/')  # SavedModel format

After running this python script, the folder saved_model will contain:

On Jetson

Assuming the tensorflow2 (Jetpack-5.0) runtime has already been installed on the Jetson, and python3 with pip3, follow the install procedure in the tensorflow-onnx project, but use pip3:

https://github.com/onnx/tensorflow-onnx

pip3 install onnxruntime
pip3 install -U tf2onnx

Then copy the entire folder saved_model folder from the PC to the Jetson workspace.

From a command line in Jetson, run the following command: 


python3 -m tf2onnx.convert --saved-model path_to/saved_model --output model.onnx

Now, in the location where the command was run, there will be new file called model.onnx

Comments off

NVIDIA 2D Image and Signal Performance Primitives (NPP)

May 23, 2023 at 4:13 pm · Filed under Cuda, Image Processing, NVIDIA Jetson

Based on the lack of examples and discussion in the forums, I assume the NPP are under-utilized and under-appreciated.  Since I discovered these, it has been a game changer for me in my image processing work. Since machine vision camera resolutions are now at 12 Mega-pixels and higher, its required to accelerate processing with a GPU. No longer do I need to create many of my own Cuda algorithms for 2D image processing – many of them already exist.

For example, resizing an image (x,y re-scale) is fully supported on any pixel data type and with multiple filter types, all accelerated with Cuda parallel operations (see my post and example project on an image resize implementation here).

The NVIDIA documentation is a bit sparse, the shear number of functions and sub-libraries are daunting. I suggest starting with this page.

https://docs.nvidia.com/cuda/npp/modules.html

Within this page, open the topics and drill down, I think you will be impressed with the number of Cuda functions available.

Comments off

create tensorflow data set from a list of numpy images

May 21, 2023 at 2:06 pm · Filed under Image Processing, Numpy, tensorflow

It took me a while to figure out the most optimal way to do this, so I thought I would share it.

I was originally using tf.stack() to create a tensor from a python list of numpy images. This operation was taking 3.37 seconds to stack a list of 40 images of 256x256x3 of type uint8.

Then I found tf.convert_to_tensor() and this reduced the operation down to 7 milliseconds.

    for image_file in images_list:
        img = cv2.imread(image_file)
        height,width= img.shape[:2]

        # my model design is based on creating 256x256 patches from larger images
        patches = patch_processor.extract_patches(img)


        tensor_list=[]
        for patch in patches:
            # convert 8 bit RGB image to floating point 0,1
            np_image_data = np.asarray(patch,dtype=np.float32)
            np_image_data = np_image_data / 127.5 - 1
            rgb_tensor = tf.convert_to_tensor(np_image_data, dtype=tf.float32)
            tensor_list.append(np.expand_dims(rgb_tensor, axis=0))
        
        # make one multi-demisional tensor that contains all the tensor patches for batch prediction


        ## this was taking 3.37 seconds for 36 images of 256x256
        #patches_tensor = tf.stack(tensor_list) 

        # convert python list to np array of image patches

        patches_tensor = np.array(tensor_list) 

        # create a tensorflow dataset from the array of image patches
        
        dataset1 = tf.data.Dataset.from_tensor_slices(patches_tensor) 
 
        # predict the patches

        predictions = model.predict( dataset1)

Comments off