Archive for March, 2023

Example implementation of nppiResize_32f_C1R_Ctx()

Project Source Code

The project source can be found here:

https://github.com/daviddrell/image_proc_samples

The project structure is Visual Studio 2019 with Cuda 11.7 installed. If you are using a different version of Cuda, I find the easiest was to solve this is to edit the Visual Studio project file in a text editor and change the version number there.

Overview

This is an example of re-scaling the size of the image in gray-scale floating point format accelerated using cuda on a GPU.

This example creates a simulated image of 2048×2048. In actual image processing applications you will have an image that comes from a jpeg or tiff file and must be decoded, often into an array of RGB bytes or directly into a gray-scale format. Many image processing operations occur on a gray-scale version of the image encoded as floating point, typically of values 0 to 1, or -1 to +1.

NVIDIA cuda comes with a library of basic image processing functions which are accelerated with parallel operations on the GPU, that run on top of the cuda library.

One of these functions is nppiResize_32f_C1R_Ctx(). The file resize.cpp implements all the memory operations necessary to resize an image using nppiResize_32f_C1R_Ctx().

The file resize.h provides a simple entry point for an image resize function which can be called from a c program with no knowledge of cuda programming.

Code Details

Refer to the gitlab project link. The sample entry point from a c programing perspective is given in main.cu. The example implementation of nppiResize_32f_C1R_Ctx() is given in resize.cpp.

Sample Results

In a machine learning application, I needed to analyze a biological image (cells growing into vessel structures imaged under a microscope). The scientist provided images that were sized at 5995 x 6207 pixels. This size is too extreme for the requirements of extracting the structures. Additionally, the AI models were trained on images typically in the range of 1000×1000 to 2000×2000 pixels. So I scale down the images using the resize_Cuda() function demonstrated in the example project.

Here is the original image that is too large:

Original Image at 5995 x 6207 pixels.

Here is the downsized image at 2000 x 2070 (the width was set to be 2000, our max AI trained size, the height was calculated to be 2070 to maintain the aspect ratio):

downsized image at 2000 x 2070 pixels

Here is the result of the analysis showing the branches and loops detected:

final analysis output

Comments off

Save Framos IMX 253 images as Jpeg on NVIDIA Jetson

I was recently working on a demonstration of a Framos IMX 253 mono-chrome camera with a 12-bit sensor supported on a Jetson Xavier. For the demo, I needed to save the images as jpeg format. I thought it may be useful for others to see the inner-workings of a jpeg compression implementation Jetson, using the hardware assisted jpeg compressor.

The image came from the camera driver as 12 bit format in 16 bit integer array format. The sensor is mono-chrome, but the Jetson jpeg compressor only takes a single YUV format as input.

For this demo, I skip the step of re-mapping the luminance values from a 12 bit range to an 8 bit range and simply take the lower 8-bits. A production implementation needs a scheme for this re-mapping of luminance range. The implementation on a Jetson should take advantage of the hardware assist.

Step 1: get things setup


// Info is a structure provided from the caller that contains info about the frame 

// prepare to time execution of the jpeg compression    
auto start = std::chrono::steady_clock::now();

// prepare the output file
std::string outFile="/path/to/file.jpg";
std::ofstream* outFileStr = new std::ofstream(outFile);
if(!outFileStr->is_open())
        return false;

// create an instance of the nvidia jetson jpeg encoder

NvJPEGEncoder* jpegenc = NvJPEGEncoder::createJPEGEncoder("jpenenc");

// the jpeg output buffer size is 1.5 times the width*height 
unsigned long out_buf_size = Info.Width * Info.Height * 3 / 2;
unsigned char *out_buf = new unsigned char[out_buf_size];

Step 2: create an nvidia native buffer

// V4L2_PIX_FMT_YUV420M =  is the only format which appears to be supported by the Jetson jpeg encoder

// allocate the buffer    
NvBuffer buffer(V4L2_PIX_FMT_YUV420M, Info.Width, Info.Height , 0);

buffer.allocateMemory();

NvBuffer::NvBufferPlane* plane = &buffer.planes[0];

//convert the image luminance from uint16 to 8 bits and copy into the nvidia buffer
for(int y=0; y < Info.Height;y++)
{
    for(int x=0; x < Info.Width;x++)
    {
        plane->data[x+(y*plane->fmt.stride)] = (unsigned char) (m_img[x+(y*Info.Width)]);
    }
}

plane->bytesused = 1 * plane->fmt.stride * plane->fmt.height;

Step 3: the Framos camera driver provides a mono-chrome image, so make the image actually mono-chrome by setting the UV vectors to neutral color (127d).

 // initialize the Cb plan to the 127d value which means 0 color

plane = &buffer.planes[1];
char* data = (char *) plane->data;
plane->bytesused = 0;
for (int j = 0; j < plane->fmt.height; j++)
{
    memset(data,127,plane->fmt.width);
    data += plane->fmt.stride;
}
plane->bytesused = plane->fmt.stride * plane->fmt.height;

// initialize the Cr plan to the 127d value which means 0 color

plane = &buffer.planes[2];
data = (char *) plane->data;
plane->bytesused = 0;
for (int j = 0; j < plane->fmt.height; j++)
{
    memset(data,127,plane->fmt.width);
    data += plane->fmt.stride;
}
plane->bytesused = plane->fmt.stride * plane->fmt.height;

Step 4: run the actual jpeg encode function, save the file, and measure the results:

jpegenc->encodeFromBuffer(buffer, JCS_YCbCr, &out_buf, out_buf_size, 95);

auto end = std::chrono::steady_clock::now();

outFileStr->write((char *) out_buf, out_buf_size);
outFileStr->close();

printf( "Jpeg Encode Elapsed time in nanoseconds: %d\n",std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count());

delete[] out_buf;
delete outFileStr;

Result:

I am seeing roughly 25 milliseconds for the encode and save on an image 3840 x 2160 pixels.

Comments off