Skip to content

4Kp30 Camera Lite Solution System Example Design for Agilex™ 3 Devices

The design is compatible with Altera® Quartus® Prime Pro Edition version 25.3 Linux

Overview

The 4Kp30 Camera Lite Example Design for Agilex™ 3 Devices, demonstrates a practical, area and cost-efficient glass-to-glass camera system solution. The support for industry-standard Mobile Industry Processor Interface (MIPI) D-PHY and MIPI CSI-2 interface on Agilex™ 3 FPGAs provide a powerful platform for camera product development.

The MIPI interface supports up to 2.5Gbps per lane and up to 8x lanes per MIPI interface, enabling seamless data reception from a 4K image sensor to the FPGA fabric for further processing. The MIPI CSI-2 IP instance converts pixel data to AXI4-Streaming outputs, enabling connectivity to other IP cores within Altera®'s Video and Vision Processing (VVP) Suite.

The design is a hardware-software co-design, where:

  • The hardware component includes a sensor feeding into an Image Signal Processing (ISP) subsystem. The ISP subsystem is a video processing pipeline incorporating many VVP IP cores such that the raw sensor image data can be processed into RGB video data. The design drives the resulting 4Kp30 streaming video output data through an Altera® DisplayPort IP.
  • The software stack consists of a baremetal application running on an embedded Nios® V soft processor. The backend part of the software application interrogates the hardware, discovers the IP components dynamically and configures them. Multiple feedback loops monitor the hardware and keep various hardware components in lockstep, e.g. DisplayPort Hot Plug Detect (HPD). The frontend of the software creates a terminal-based user interface and runs it over a JTAG-UART.

The following diagram provides an overview of the interaction between the software running on the Nios® V and the hardware components running in the programmable logic parts of the device.


VVP_ISP_ED_Top_level

High-Level Block Diagram of the Camera Lite Solution System Example Design


The rest of this documentation has been organized in the following sections:


Features and Specifications

The 4Kp30 Camera Lite Example Design provides:

  • Support for the following Input and Output video interfaces
    • Input: MIPI CSI-2
    • Output: DisplayPort 1.4
  • Support for the following sensor:
    • Raspberry PI HQ IMX477
  • Support for the following progressive video resolutions on the input and output interfaces:
    • Fixed: 3840x2160 30Hz (4K30p)
  • Video pipeline subsystem:
    • 12-bit raw data from the sensor up to the Demosaic IP.
    • 10-bit RGB for video processing.
    • VVP ISP IP cores included: Black Level Correction, White Balance Correction, Demosaic, Color Correction Matrix and 1D LUT.
    • Video Frame Buffer
    • Support for an input video switch:
      • 2 inputs and 1 output
      • Input ports receive the data from Sensor and a TPG
      • The selected input data is broadcast to the ISP-Lite video pipeline
    • Support for an output video mixer:
      • Icon Generator
      • Input video stream
      • TPG

Camera Lite Video Pipeline Functional Description

Pre-requisites

Hardware Requirements

Agx3-DK

Agilex™ 3 FPGA and SoC C-Series Development Kit


Software Requirements


Repository and Assets Release Tag Requirements

The sources listed in this table are the most current and highly recommended for Quartus® 25.3 builds. Users are advised to utilize the updated versions of these building blocks in production environments. Please note that this is a demonstration design and is not suitable for production or final deployment.


Component Location Branch
Assets Release Tag https://github.com/altera-fpga/agilex3-ed-camera/releases/tag/rel-25.3 rel-25.3
Repository https://github.com/altera-fpga/agilex3-ed-camera rel-25.3
Modular Design Toolkit https://github.com/altera-fpga/modular-design-toolkit rel-25.3


Getting Started

Follow the instructions provided in this section to run the 4K30p Camera Solution System Example Design on the Altera® Agilex™ 3 Development Kit.

Download the pre-built FPGA SOF Binary Files

  • Download the binaries for your specific Agilex™ 3 development kit:


Source Link Description Device Part Number
SOF golden_agx3c_soc_devkit_isp_lite_top.sof SOF file for the Agilex™ 3 FPGA and SoC C-Series Development Kit A3W135BM16AEA
SOF golden_agx3c_fpga_devkit_isp_lite_top.sof SOF file for the Agilex™ 3 FPGA C-Series Development Kit A3Y135BM16AEA


Setting Up the Development Kit

Warning

Handle ESD-sensitive equipment (boards, microSD cards, camera sensors, etc.) only when properly grounded and at an ESD-safe workstation

  • Configure the Agilex™ 3 FPGA and SoC C-Series Development Kit switches and jumpers to their factory default settings, as per to the user guide instructions


board-top

Agilex™ 3 Development Kit - Default Switch Positions (Top View)


board-bottom

Agilex™ 3 Development Kit - Default Switch Positions (Bottom View)


  • Proceed to connect the following items to the development kit, as shown in the following figures:
    • USB-C power supply (J8)
    • MIPI sensor cable ( MIPI Bank 3A - J21)
    • DisplayPort Transmit cable (J12)
    • USB-C JTAG cable (J2)


connect-1

Sensor and Development Kit Connections (#1)


connect-2

Sensor and Development Kit Connections (#2)


connect-3

Sensor and Development Kit Connections (#3)


Running the Demonstration

Programming the Development Kit

  • To program the FPGA using a SOF File:

    • Power up the board.
    • Launch the Quartus® Programmer and Configure the "Hardware Setup..." settings as follows:


hw-setup-set

Programmer GUI Hardware Settings


  • Click "Auto Detect", select the device A3CW135BM16A (for the Devkit with SoC) or A3CY135BM16A (for the Devkit without SoC), and press "Change File.."


programmer-agx3

Programmer after "Auto Detect"


  • Select your SOF file, e.g. golden_agx3c_soc_devkit_isp_lite_top.sof. Check the "Program/Configure" box and press the "Start" button (see below). Wait until the programming has been completed.


programmer-agx3-3

Programming the FPGA with a SOF file


Demonstrations

  • After the FPGA has been programmed, you could proceed to open a JTAG-UART terminal, using the following command:

    • Linux: 
      juart-terminal --device 1 --instance 0
    • Windows: 
      juart-terminal.exe --device 1 --instance 0
  • If successful, you should see the following message on the terminal:

board-mipi


  • After pressing h, you should see the menu for this demo:

board-mipi


  • The menu provides a list of different debug and configuration options in this example design. Each option has been mapped to a specific alphanumeric character, and you can control them from a JTAG-UART terminal interface via keyboard input. A brief description of all the options is provided below:
  • Pressing s, will toggle between MIPI Rx and Input TPG. The Input TPG has been pre-configured to support four patterns:
    • Color Bars
    • Solid Colors: Blue, Green, and Red

board-mipi


  • Pressing r, will restore the ISP video pipeline to its default configuration.

board-mipi


  • Pressing t, will toggle the Icon on and off.

board-mipi


  • Pressing 1 will enable the BLC, while Pressing 2 will put the BLC in bypass mode.

board-mipi


  • Pressing 3 will enable the WBC, while Pressing 4 will put the WBC in bypass mode.
    • WBC enabled allows incremental changes from 3000K to 9000K on 1000K steps

board-mipi


  • Pressing 5 will enable the Demosaic, while Pressing 6 will put the Demosaic in bypass mode.

board-mipi


  • Pressing 7 will enable the CCM, while Pressing 8 will put the CCM in bypass mode.
    • CCM enabled allows incremental changes from 3000K to 9000K on 1000K steps

board-mipi


  • Pressing 9 will enable the 1D-LUT, while Pressing 0 will put the 1D-LUT in bypass mode.
    • 1D-LUT enabled produces a BT-709 gamma curve

board-mipi


  • Pressing m will increase the exposure gain of the sensor, while Pressing n will decrease the exposure gain of the sensor.
    • For incremental changes, the SW App first increases the digital gain, and once the value reaches its maximum limit, it then increases the analog gain.
    • For decremental changes, the SW App first decreases the analog gain, and once the value reaches zero, it then decreases the digital gain.

board-mipi


Generating the Example Design from Scratch Using the MDT-Flow

The hardware design for the 4Kp30 Camera Lite Solution System Example Design uses the Modular Design Toolkit (MDT). The MDT is a method of creating and building Platform Designer (PD) based Quartus® projects from a single .xml file.

The main advantages of using MDT are:

  • Enforces a hierarchical design approach (single level deep).
  • Encourages design reuse through a library of off-the-shelf subsystems.
  • Enables simple porting of designs to different development boards and FPGA devices.
  • Provides consistent folder structure and helper scripts.
  • Uses TCL scripting for the PD Quartus® project.

The MDT flow consists of 2 separate main steps; a create step and a build step.

The create step:

  • Parses the design .xml file.
  • Creates a Quartus® project.
  • Creates a PD system for the project.
  • Copies all the project files and adds all the MDT generated files to the Quartus® project.

The build step:

  • Compiles the Nios® V Software and generates a .hex file.
  • Runs the Quartus® compilation flow.
  • Post processes .sof files.

Related Information

Modular Design Toolkit


Create the design using the Modular Design Toolkit (MDT)

Follow the next steps to create the Quartus® and Platform Designer Project for the 4Kp30 Camera Lite Solution System Example Design:

  • Create your workspace and clone the repository using --recurse-submodules:
cd <workspace>
git clone -b rel-25.3 --recurse-submodules https://github.com/altera-fpga/agilex3-ed-camera.git agilex3-ed-camera
  • Define a <project> location of your choice, creating a directory structure where necessary.
  • Navigate to the agilex3-ed-camera/agx3c_devkit directory containing the cloned repository and create your project, selecting the XML variant for the project.
cd agilex3-ed-camera/agx3c_devkit/designs
quartus_sh -t ../modular-design-toolkit/scripts/create/create_shell.tcl -xml_path ./AGX_3C_SoC_Devkit_ISP_Lite.xml -proj_path <project> -o
  • The previous command will create your Quartus® Prime and Platform Designer Project in <project>, with a folder structure that is consistent with the MDT methodology.


qar-content2

**Project Directory Structure.**


  • The following table provides a brief explanation regarding the content of each of the top-level folders:
    Directory Description
    non_qpds_ip Contains the source code (RTL) of the design’s custom IP that is not part of Quartus® Prime.
    quartus Contains the base files for the Quartus® Project including the top.qpf, top.qsf.
    rtl Contains the sources files to build the project.
    scripts Contains a collection of TCL scripts from "Modular Design Toolkit" to build and compile the design software and hardware.
    sdc Contains the .sdc files for the subsystems to compile the project.
    software Contains all the files for building the application for the Nios® V.


Build this hardware design using the Modular Design Toolkit (MDT)

Follow the next steps to build and generate the SOF file for the 4Kp30 Camera Lite Solution System Example Design:

  • Navigate to the <project>/scripts directory and build your project:
quartus_sh -t build_shell.tcl -full_compile
  • The -full_compile MDT build option performs not just the full Quartus® compilation, but also compiles any Nios® V software, and merges it into the .hex ROM file built into the project during compilation.
  • Wait for the compilation to finish.
  • The generated FPGA programming file is located in the <project>/quartus/output_files directory:
    • top.sof

  • In case you need to recompile the Nios® V SW App, please follow the additional steps:

    • The design includes an initial version of settings.bsp that contains parameters to run the design. If you modify the Platform Designer's hardware, ensure you keep the integrity of the settings.bsp file fully synchronized with the Plaform Designer project, i.e. top_qsys.qsys.

    • Compile the software application and merge the SOF file with the newly generated HEX file with the following command:

        cd <project>/scripts/
  quartus_sh -t build_shell.tcl -update_sof
    • The -update_sof MDT build option compiles the Nios® V software and merges it into the .hex ROM file built into the project, generating a new top.sof file with the latest SW App updates.


Extra Resources


Other Repositories Used

Component Location Branch
Modular Design Toolkit https://github.com/altera-fpga/modular-design-toolkit rel-25.3


Useful User Manuals and Reference Materials


Notices & Disclaimers

Altera® Corporation technologies may require enabled hardware, software or service activation. No product or component can be absolutely secure. Performance varies by use, configuration and other factors. Your costs and results may vary. You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Altera or Intel products described herein. You agree to grant Altera Corporation a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document, with the sole exception that you may publish an unmodified copy. You may create software implementations based on this document and in compliance with the foregoing that are intended to execute on the Altera or Intel product(s) referenced in this document. No rights are granted to create modifications or derivatives of this document. The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Altera disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. You are responsible for safety of the overall system, including compliance with applicable safety-related requirements or standards. © Altera Corporation. Altera, the Altera logo, and other Altera marks are trademarks of Altera Corporation. Other names and brands may be claimed as the property of others.

OpenCL* and the OpenCL* logo are trademarks of Apple Inc. used by permission of the Khronos Group™.


Ask in the Forum