Robot Controller with Vision System Example Design — Features¶

This design combines robotics vision processing with a 6-axis (3×2) Drive-on-Chip motor control system.

Key features¶

6-axis FOC motor control: three NiosV/g instances, each controlling two axes (simulated motor model included).
Robotics vision ISP pipeline in FPGA fabric with optional MIPI sensor input (default test-pattern source) and DisplayPort output.
ArUco marker injection in the video path to emulate visual feedback and manipulation workflows without a physical camera or robot arm.
HPS Linux for higher-level robotics stacks (for example ROS 2 integration).
Modular Design Toolkit (MDT) project generation from XML AGX_5E_Modular_Devkit_HPS_ISP_VIS_DOC3x2_ROB.xml.

ISP Lite conditioning: Bayer-to-RGB conversion pipeline (black-level correction, white balance, demosaic, and color correction).
Robotics ISP path: grayscale derivation, clipping/scaling branches, frame buffering through FPGA EMIF, and ArUco overlay/mixing.
DisplayPort output: local monitoring and validation of the processed video path directly from FPGA.

Drive-on-Chip 3×2 topology: six-axis control implemented as three dual-axis drive/control slices.
Real-time motor loops in fabric: deterministic current-speed-position control with simulated motor model for development and debug.
NiosV/g control firmware: embedded software for low-level control orchestration and register interaction.
HPS bridge access: HPS-side visibility and control of selected drive and ISP Robotics registers.

Linux image via KAS: Yocto/KAS flow with kas-vision-doc.yml for reproducible software builds.
Additional Yocto Layer(s) supporting Docker (meta-altera-fpga).
ROS 2 integration: validated container flow with Altera ROS 2.
ROS control path: compatible with the ROS Control hardware interface approach used by Drive-on-Chip based flows.
MDT-first project generation: scalable create/build flow from HPS_ISP_VIS_DOC3x2_ROBOTICS and Creating and Building the Design based on Modular Design Toolkit (MDT)..

Prebuilt assets from Release Tag: SD card image (wic.gz, wic.bmap), QSPI image (top.hps.jic), and programmable files (top.core.rbf, top.sof).
Optional pre-created Quartus archive ROBOTICS_ISP_VISION_DOC.qar for faster rebuild and experimentation.

Last update: June 25, 2026
Created: June 25, 2026