Raspberry Pi AI Camera

Easily integrate artificial intelligence capabilities into your Raspberry Pi projects!

Thanks to the Sony IMX500 sensor, a revolutionary sensor that redefines image capture standards in the world of modern cameras!

The announcement of the AI camera for Raspberry Pi took place during Embedded World, a major conference dedicated to embedded technologies. This announcement caught the attention of developers and engineers due to its advanced onboard processing capabilities.

The Sony IMX500 sensor is the first of its kind to integrate an artificial intelligence processor that analyzes images in real time without the need for an external processor or online service.

How is the Sony IMX500 sensor structured?

The IMX500 uses a stacked sensor, combining an image sensor with a powerful DSP and dedicated onboard SRAM. This gives you high-speed AI processing without the need to go back and forth to a server.

According to Sony, placing the AI-dedicated layer and memory layer directly below the pixel layer offers multiple advantages. The sensor can process data and perform “internal” AI analysis almost instantly. Sony reports an object recognition time of 3.1 milliseconds.

With a resolution of 12.3 megapixels, this 7.857 mm sensor (type 1/2.3) stands out with its 1.55 µm × 1.55 µm pixels, optimized for detailed and sharp image capture.

Performance-wise, the Sony IMX500 offers an impressive resolution of 4056 x 3040 pixels, perfect for applications where every detail counts. It features an IR filter for optimal light management, and provides adjustable manual focus, offering remarkable flexibility with a focus range from 20 cm to infinity. Its 4.74 mm focal length and horizontal (66 ± 3 degrees) and vertical (52.3 ± 3 degrees) fields of view ensure balanced and immersive image coverage.

With an F1.79 aperture, this sensor promises excellent low-light performance while minimising noise for clear, bright images. Despite its lack of infrared sensitivity, the Sony IMX500 delivers high quality images with Bayer RAW10 and YUV/RGB outputs, as well as rich metadata and full Regions of Interest (ROI) support.

This sensor also excels in processing thanks to its ability to handle input tensors up to 640 x 640 pixels with 'int8' or 'uint8' data types. It has 83,884,880 bytes of memory dedicated to firmware processing, network weights and memory management, ensuring smooth performance.

On the video side, the Sony IMX500 offers capture at 30 frames per second in 2x2 binned mode (2028×1520 pixels, 10-bit) and 10 fps at full resolution (4056×3040 pixels, 10-bit), making it an excellent choice for applications requiring both speed and detail.

The sensor is compact, measuring 25 × 24 × 11.9 mm, and is easily connected via a 200 mm ribbon cable compatible with 15 × 1 mm FPC or 22 × 0.5 mm FPC connectors. Finally, it is designed to operate over a temperature range of 0°C to 50°C, making it ideal for use in a variety of environments.

The Sony IMX500 is not just a sensor: it's an innovative and high-performance image capture solution that will meet the needs of the most demanding professionals.

What are the benefits in use?

As mentioned earlier, all calculations are performed locally, without the need to pass through an external service. As a result, this architecture offers a gain in responsiveness and performance.

The camera enables advanced onboard processing, compatible with many neural network models. This feature optimizes performance by offloading the Raspberry Pi's main processor, making it available for other tasks.

This also has an impact on bandwidth and power consumption. Instead of sending the entire image, the device sends only its metadata, which is a few bytes instead of several megabytes.

Where can the IMX500 be used?

The AI camera is particularly ideal for real-time analysis of people and goods flows. In industry, many use AI to detect anomalies on production lines.

Useful resources

Configuring and testing AI Camera

Introducing the AI Camera