Skip to content

4Kp30 Multi-Sensor Camera with AI Inference Solution System Example Design for Agilex™ 5 Devices - Features

  • Inputs:
    • Two MIPI D-PHY interfaces connected to two Framos FSM:GO IMX678 color camera modules with Sony Starvis2 8MP IMX678 sensor and Framos PixelMate MIPI CSI-2 interface.
    • Optional Framos GMSL3 5m Link extender
    • Internal frontend Test Pattern Generator (TPG) supporting color bars and solid colors.
  • Input selection switch
  • Frontend video pipeline (CFA image):
    • Clipper IP.
    • ISP IP: Black Level Statistics (BLS), Defective Pixel Correction (DPC), Adaptive Noise Reduction (ANR), Black Level Correction (BLC), Vignette Correction (VC), White Balance Statistics (WBS), White Balance Correction (WBC), and Demosaic (DMS).
  • AI inference engine:
    • Supports YOLOv8 nano detection and pose inference models.
    • 640x384 size input images with high accuracy and fast 30FPS inference rate.
    • Inference result overlay.
  • Backend video pipeline (RGB image):
    • Histogram Statistics (HS) IP.
    • 3D LUT IP.
    • Tone Mapping Operator (TMO) IP.
    • Un-Sharp Mask (USM) IP.
    • Mixer IP:
      • TPG for uniform background (also supports color bars).
      • ISP Output overlay.
      • AI Inference Result overlay.
      • Logo overlay.
    • 1D LUT IP – for OETF operations such as Gamma, HLG, and PQ.
  • Up to 4Kp30 10-bit Multi-rate DisplayPort output:
    • Selects best resolution and color depth supported by the connected Monitor. Supported modes:
      • 720p30, 1080p30, 4Kp30, 8-bit and 10-bit color depth.
  • Video frame capture (3 color plane 16-bit color depth, 48-bit lossless TIFF image):
    • Sensor Raw Output (Grayscale RGB).
    • ISP Output (RGB).

Grayscale RGB

The single color plane of the sensor CFA image is copied into all 3 color planes on a per-pixel basis to produce an RGB image.

  • Software stack running on HPS (quad-core ARM Cortex CPU):
    • Based on Poky Linux with additional Yocto layers supporting Altera® FPGAs.
    • Uses the Yocto build system.



Back


Last update: December 10, 2025
Created: October 6, 2025
Ask in the Forum