{"id":515,"date":"2023-06-14T09:56:43","date_gmt":"2023-06-14T14:56:43","guid":{"rendered":"https:\/\/davidwdrell.net\/wordpress\/?p=515"},"modified":"2023-06-14T09:57:17","modified_gmt":"2023-06-14T14:57:17","slug":"porting-a-keras-trained-model-to-jetson-xavier","status":"publish","type":"post","link":"https:\/\/davidwdrell.net\/wordpress\/?p=515","title":{"rendered":"Porting a Keras trained model to Jetson Xavier"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">I have been training resnet models on my Windows desktop machine with big GPUs. Now I need to deploy these models on a Jetson Xavier for real-time predictions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">My model format historically has been the Keras .h5 file. So now I need to convert the .h5 model file into the onnx exchange format.  <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There are two steps: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1) convert the .h5 to a frozen .pb format &#8211; this step is done on the desktop machine<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">2) convert the .pb format to the onnx format &#8211; this step is done on the Jetson.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">On the PC<\/h2>\n\n\n\n<pre class=\"wp-block-code\"><code># python \n\nfrom keras.models import load_model\n\n# you can either load an existing h5 model using whatever you normally use for this\n#  purpose, or just train a new model as save using the template below\n\nmodel = load_model('model.h5',custom_objects=models.custom_objects())\n\n# save in pb froozen pb format, this format saves in  folder \n#  and you are supplying the folder name\n\nmodel.save('.\/outputs\/models\/saved_model\/')  # SavedModel format<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">After running this python script, the folder saved_model will contain:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><a href=\"https:\/\/davidwdrell.net\/wordpress\/wp-content\/uploads\/2023\/06\/image.png\"><img loading=\"lazy\" decoding=\"async\" width=\"611\" height=\"83\" src=\"https:\/\/davidwdrell.net\/wordpress\/wp-content\/uploads\/2023\/06\/image.png\" alt=\"\" class=\"wp-image-516\" srcset=\"https:\/\/davidwdrell.net\/wordpress\/wp-content\/uploads\/2023\/06\/image.png 611w, https:\/\/davidwdrell.net\/wordpress\/wp-content\/uploads\/2023\/06\/image-300x41.png 300w\" sizes=\"auto, (max-width: 611px) 100vw, 611px\" \/><\/a><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">On Jetson<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Assuming the tensorflow2 (Jetpack-5.0) runtime has already been installed on the Jetson, and python3 with pip3, follow the install procedure in the tensorflow-onnx project, but use pip3:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/onnx\/tensorflow-onnx\">https:\/\/github.com\/onnx\/tensorflow-onnx<\/a><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip3 install onnxruntime\npip3 install -U tf2onnx<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then copy the entire folder saved_model folder from the PC to the Jetson workspace.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">From a command line in Jetson, run the following command: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>\npython3 -m tf2onnx.convert --saved-model path_to\/saved_model --output model.onnx\n<\/code><\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Now, in the location where the command was run, there will be new file called model.onnx<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I have been training resnet models on my Windows desktop machine with big GPUs. Now I need to deploy these [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[5,9,11],"tags":[15],"class_list":["post-515","post","type-post","status-publish","format-standard","hentry","category-image-processing","category-nvidia-jetson","category-tensorflow","tag-tensorflow"],"_links":{"self":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/515","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=515"}],"version-history":[{"count":1,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/515\/revisions"}],"predecessor-version":[{"id":517,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/515\/revisions\/517"}],"wp:attachment":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=515"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=515"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=515"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}