Robotics Camera System Example Design — Features

This design targets robotics perception workloads on Altera® FPGA devices.

Key features

• FRAMOS IMX678 sensor ingestion over MIPI CSI-2.
• ISP Lite pipeline in FPGA fabric for image conditioning.
• DisplayPort video output for local monitoring.
• Hard Processor System (HPS) running Linux for higher-level robotics software (for example ROS 2).
• Modular Design Toolkit (MDT) project generation from XML (AGX_5E_Modular_Devkit_HPS_ISP_CAM_ROB.xml).

Vision pipeline features

• MIPI ingress path: the FRAMOS IMX678 sensor connects through MIPI D-PHY and CSI-2 receive IP, delivering raw Bayer frames into FPGA fabric.
• ISP In + ISP Lite conditioning: Bayer input selection, test-pattern debug support, and first-stage image processing (black-level correction, white balance, demosaic, and color correction).
• ISP Robotics processing path: clip/scale branches, grayscale extraction, frame-buffering, and mixer support for robotics-oriented perception workflows.
• Display path output: ISP Lite Out and DisplayPort TX provide local real-time monitoring for bring-up, tuning, and validation.

Compute and control features

• Nios® V pipeline control: embedded soft processor configures MIPI/ISP/Display pipeline CSRs and services subsystem interrupts.
• HPS Linux orchestration: HPS-side software controls ISP Robotics registers and manages data movement through HPS-FPGA bridges and MSGDMA.
• DDR-backed frame access: FPGA EMIF interfaces enable frame writer/reader buffering for downstream processing and software access.

Software and integration features

• Yocto/KAS build flow: Linux image generation from agilex-ed-robotics/sw using kas-camera.yml.
• Additional Yocto layer(s) supporting Docker (meta-altera-fpga).
• ROS 2-ready deployment model: supports containerized workflows with Altera ROS 2, enabling topic publication and visualization tooling.
• MDT-first hardware flow: reproducible create/build flow driven by HPS_ISP_CAM_ROBOTICS and Creating and Building the Design based on Modular Design Toolkit (MDT)..
• Reference alignment: architecture and IP usage are consistent with VVP IP Suite and related Agilex camera example designs.

Deliverables

• Prebuilt assets from Release Tag: SD card image (wic.gz, wic.bmap), QSPI image (top.hps.jic), and programmable files (top.core.rbf, top.sof).
• Optional pre-created Quartus archive ROBOTICS_ISP_CAMERA.qar for faster rebuild and iteration.

Back to Documentation

---

Last update: June 25, 2026
Created: June 25, 2026