{"id":493,"date":"2023-05-21T14:06:28","date_gmt":"2023-05-21T19:06:28","guid":{"rendered":"https:\/\/davidwdrell.net\/wordpress\/?p=493"},"modified":"2023-05-22T06:04:44","modified_gmt":"2023-05-22T11:04:44","slug":"create-tensorflow-data-set-from-a-list-of-numpy-images","status":"publish","type":"post","link":"https:\/\/davidwdrell.net\/wordpress\/?p=493","title":{"rendered":"create tensorflow data set from a list of numpy images"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">It took me a while to figure out the most optimal way to do this, so I thought I would share it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I was originally using tf.stack() to create a tensor from a python list of numpy images. This operation was taking 3.37 seconds to stack a list of 40 images of 256x256x3 of type uint8.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then I found tf.convert_to_tensor() and this reduced the operation down to 7 milliseconds.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>    for image_file in images_list:\n        img = cv2.imread(image_file)\n        height,width= img.shape&#91;:2]\n\n        # my model design is based on creating 256x256 patches from larger images\n        patches = patch_processor.extract_patches(img)\n\n\n        tensor_list=&#91;]\n        for patch in patches:\n            # convert 8 bit RGB image to floating point 0,1\n            np_image_data = np.asarray(patch,dtype=np.float32)\n            np_image_data = np_image_data \/ 127.5 - 1\n            rgb_tensor = tf.convert_to_tensor(np_image_data, dtype=tf.float32)\n            tensor_list.append(np.expand_dims(rgb_tensor, axis=0))\n        \n        # make one multi-demisional tensor that contains all the tensor patches for batch prediction\n\n\n        ## this was taking 3.37 seconds for 36 images of 256x256\n        #patches_tensor = tf.stack(tensor_list) \n\n        # convert python list to np array of image patches\n\n        patches_tensor = np.array(tensor_list) \n\n        # create a tensorflow dataset from the array of image patches\n        \n        dataset1 = tf.data.Dataset.from_tensor_slices(patches_tensor) \n \n        # predict the patches\n\n        predictions = model.predict( dataset1)<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>It took me a while to figure out the most optimal way to do this, so I thought I would [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[5,10,11],"tags":[],"class_list":["post-493","post","type-post","status-publish","format-standard","hentry","category-image-processing","category-numpy","category-tensorflow"],"_links":{"self":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/493","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=493"}],"version-history":[{"count":1,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/493\/revisions"}],"predecessor-version":[{"id":494,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/493\/revisions\/494"}],"wp:attachment":[{"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=493"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=493"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/davidwdrell.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=493"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}