Archive for tensorflow

Porting a Keras trained model to Jetson Xavier

June 14, 2023 at 9:56 am · Filed under Image Processing, NVIDIA Jetson, tensorflow ·Tagged

I have been training resnet models on my Windows desktop machine with big GPUs. Now I need to deploy these models on a Jetson Xavier for real-time predictions.

My model format historically has been the Keras .h5 file. So now I need to convert the .h5 model file into the onnx exchange format.

There are two steps:

1) convert the .h5 to a frozen .pb format – this step is done on the desktop machine

2) convert the .pb format to the onnx format – this step is done on the Jetson.

On the PC

# python 

from keras.models import load_model

# you can either load an existing h5 model using whatever you normally use for this
#  purpose, or just train a new model as save using the template below

model = load_model('model.h5',custom_objects=models.custom_objects())

# save in pb froozen pb format, this format saves in  folder 
#  and you are supplying the folder name

model.save('./outputs/models/saved_model/')  # SavedModel format

After running this python script, the folder saved_model will contain:

On Jetson

Assuming the tensorflow2 (Jetpack-5.0) runtime has already been installed on the Jetson, and python3 with pip3, follow the install procedure in the tensorflow-onnx project, but use pip3:

https://github.com/onnx/tensorflow-onnx

pip3 install onnxruntime
pip3 install -U tf2onnx

Then copy the entire folder saved_model folder from the PC to the Jetson workspace.

From a command line in Jetson, run the following command: 


python3 -m tf2onnx.convert --saved-model path_to/saved_model --output model.onnx

Now, in the location where the command was run, there will be new file called model.onnx

Comments off

create tensorflow data set from a list of numpy images

May 21, 2023 at 2:06 pm · Filed under Image Processing, Numpy, tensorflow

It took me a while to figure out the most optimal way to do this, so I thought I would share it.

I was originally using tf.stack() to create a tensor from a python list of numpy images. This operation was taking 3.37 seconds to stack a list of 40 images of 256x256x3 of type uint8.

Then I found tf.convert_to_tensor() and this reduced the operation down to 7 milliseconds.

    for image_file in images_list:
        img = cv2.imread(image_file)
        height,width= img.shape[:2]

        # my model design is based on creating 256x256 patches from larger images
        patches = patch_processor.extract_patches(img)


        tensor_list=[]
        for patch in patches:
            # convert 8 bit RGB image to floating point 0,1
            np_image_data = np.asarray(patch,dtype=np.float32)
            np_image_data = np_image_data / 127.5 - 1
            rgb_tensor = tf.convert_to_tensor(np_image_data, dtype=tf.float32)
            tensor_list.append(np.expand_dims(rgb_tensor, axis=0))
        
        # make one multi-demisional tensor that contains all the tensor patches for batch prediction


        ## this was taking 3.37 seconds for 36 images of 256x256
        #patches_tensor = tf.stack(tensor_list) 

        # convert python list to np array of image patches

        patches_tensor = np.array(tensor_list) 

        # create a tensorflow dataset from the array of image patches
        
        dataset1 = tf.data.Dataset.from_tensor_slices(patches_tensor) 
 
        # predict the patches

        predictions = model.predict( dataset1)

Comments off