4Kp30 Multi-Sensor Camera with AI Inference Solution System Example Design for Agilex™ 5 Devices - SOF Modular Design Toolkit (MDT) Flow¶

Pre-requisites¶

Software Requirements to build¶

Linux OS installed.
72 GB free storage (~2GB for Quartus® Build and ~70GB for YOCTO/KAS build).
Python/PIP/KAS for Yocto Build (or a suitable container) see KAS.
Altera® Quartus® Prime Pro Edition version 25.1 Linux.
- Altera® Quartus® Agilex™ 5 Support.
FPGA NiosV Open-Source Tools 25.1 (installed with Quartus® Prime).

License Requirements to build¶

Note licenses must be downloaded and installed where needed.

OpenCore Plus (OCP) IP evaluation license.

Free licenses:
- Nios® V Processor.

Full or Free licenses:
- Altera® FPGA AI Suite.

Hardware Requirements¶

Agilex™ 5 FPGA E-Series 065B Modular Development Kit.

Agilex™ 5 FPGA E-Series 065B Modular Development Kit

1 or 2 Framos FSM:GO IMX678C Camera Modules, with:
- Wide 110deg HFOV Lens, or
- Medium 100deg HFOV Lens, or
- Narrow 54deg HFOV Lens.
(Optional) Mount/Tripod
- Framos Tripod Mount Adapter.
- Tripod.
A Framos cable for PixelMate MIPI-CSI-2 for each Camera Module:
- 150mm flex-cable, or
- 300mm micro-coax cable.
(Optional) Framos GMSL3 5m (if the Camera Solution System Example Design supports this feature).
MicroSD card (minimum 8Gb).
DP cable or HDMI cable with DP to HDMI Adapter (recommend 4Kp60 capable).
USB Micro B cable x2 (for QSPI programming and HPS serial console access).
Ethernet cable (for HPS network connection).
PC monitor or TV: 4Kp30 capable required.

Software Requirements to run¶

Host PC with:
- 8 GB of RAM (less if not rebuilding binaries).
- Linux/Windows OS installed.
- Serial terminal (such as GtkTerm or Minicom on Linux, and TeraTerm or PuTTY on Windows).
  - FTDI FT232R USB UART drivers (for a Windows host).
- Tool to write images for removable USB drives or microSD cards such as Win32DiskImager on Windows or "dd" command on Linux.
- Altera® Quartus® Prime Pro Edition version 25.1 Programmer and Tools.
- Ethernet connection (either direct from Host PC to development board, or via a switch or router).
  - Note, you may need to disconnected/disabled VPN if it is installed on the Host PC.
- Web browser.

Download and Compile the AI Models¶

The Altera® FPGA AI Suite IP in the Camera with AI Inference Solution System Example Design, is optimized to run both the ultralytics YOLOv8 nano detection and pose inference models, switching between them at runtime. The End User must go to ultralytics website to review and accept licensing and copyright information, before downloading the YOLOv8 nano models. The models must then be compiled for the FPGA AI Suite IP. This will only need to be done once:

Visit the ultralytics YOLO website.
Review and accept the licensing and copyright terms.
Download the YOLOv8 nano detection inference model yolov8n.pt
Download the YOLOv8 nano pose inference model yolov8n-pose.pt
Download the model_compiler to your <workspace> directory
Compile the models for FPGA AI Suite IP:

Note

The downloaded YOLOv8 nano models must be placed in the directory specified

mkdir -p <workspace>
cd <workspace>
git clone [https://github.com/altera-fpga/agilex-ed-camera-ai] .
cd yolo_cnn

echo "Download the YOLOv8 nano inference models from ultralytics website into directory yolo_cnn"
wget <url>

mkdir -p compile
cd compile
echo "This step can take some time to extract Altera® FPGA AI Suite"
cmake -G Ninja ..
echo "This step can take some time to generate a python virtual environment"
ninja
cd output
tree

.
├── generated_arch.arch
│   ├── yolov8n-pose_dla_m2m_compiled_640_384.bin
│   └── yolov8n_dla_m2m_compiled_640_384.bin
├── yolov8n-pose_categories.txt
└── yolov8n_categories.txt

1 directory, 4 files

The model_compiler generates the following YOLOv8 nano inference model binaries:

Detection generated_arch.arch/yolov8n_dla_m2m_compiled_640_384.bin
Pose generated_arch.arch/yolov8n-pose_dla_m2m_compiled_640_384.bin

Additionally, the model_compiler generates the following category identifier files: * Detection yolov8n_categories.txt * Pose yolov8n-pose_categories.txt

Getting Started - build and run new binaries¶

Follow the instructions provided in this section to build the Camera Solution System Example Design for the Agilex™ 5 FPGA E-Series 065B Modular Development Kit. You will build a new SOF file using the Modular Development Toolkit, and a new microSD Card image using KAS and Yocto (suitable for use with the OpenCore License).

Reference pre-built Binaries¶

You can use the pre-built binaries for reference:

Boot Source	Link
Pre-built microSD Card Image for SOF MDT Flow	fpga-first-vvp-isp-demo-image-agilex5_mk_a5e065bb32aes1.wic.gz
Pre-built FPGA First `.sof` file	fsbl_agilex5_modkit_vvpisp_time_limited.sof

HW Compilation¶

Use the SOF Modular Design Toolkit (MDT) Flow to create and build the FPGA Design.

SW Compilation¶

Use the Create microSD card image (.wic.gz) using YOCTO/KAS flow to create the microSD card image.

Note

use KAS_MACHINE=agilex5_mk_a5e065bb32aes1 and kas/agilex_camera_ff.yml configuration

Programming¶

Setting Up the Development Kit¶

Warning

Handle ESD-sensitive equipment (boards, microSD cards, camera sensors, etc.) only when properly grounded and at an ESD-safe workstation

Configure the Agilex™ 5 FPGA E-Series 065B Modular Development Kit switches as shown:

Modular Development Kit - Default Switch Positions

FPGA configuration modes used by the Camera Solution System Example Design

The Camera Solution System Example Design can use one of the following FPGA configuration modes: JTAG or ASx4(QSPI).
The MSEL switch (S4) on the SOM board instructs the FPGA device on which configuration mode to use.

JTAG mode: Modular Development Kit SOM Board S4=OFF:OFF, or
ASx4 (QSPI) mode: Modular Development Kit SOM Board S4=ON:ON

The pre-built version of the design uses ASx4 (QSPI) mode, S4=ON:ON.

Make the required connections between the Host PC and the Modular Development Kit as shown in the following diagram:

Host PC and Modular Development Kit Connections diagram

Connect micro USB cable between the Modular Development Kit Carrier Board (J35) and the Host PC. This will be used for QSPI programming / FGPA configuration over JTAG. Look at what serial ports are enumerated on your Host computer. There should be a series of four.
Connect micro USB cable between the Modular Development Kit SOM Board (J2, HSP_UART) and the Host PC. This will be used to access HPS serial console. Look at what ports are enumerated on your Host computer. There should be a series of four. Use the 3^rd one in the list as the HPS serial port.
Connect an RJ45 cable between the ethernet port on the Modular Development Kit SOM Board (J6, ETH 1G HPS) and make sure it is on the same network as your Host PC. You can check the eth0 IP address after boot using the Linux ip a command.

Modular Development Kit Connector Locations

Burn the microSD Card Image¶

Either use your own or download the pre-built <name>.wic.gz image.
If required, extract <name>.wic image from the compressed download file.
- On Linux, use the dd utility:
```
tar -xzf `<name>.wic.gz`
```
- On Windows, use the 7-Zip program (or similar):
  - Right click <name>.wic.gz file, and select "Extract All..."

Write the <name>.wic image to the microSD card using a USB writer:
- On Linux, use the dd utility:
```
# Determine the device associated with the SD card on the host computer.
cat /proc/partitions
# This will return for example /dev/sd<x>
# Use dd to write the image in the corresponding device
sudo dd if=<name>.wic of=/dev/sd<x> bs=1M
# Flush the changes to the microSD card
sync
```
- On Windows, use the Win32DiskImager program (or similar):
  - Click browse icon and select "*.*" filter:
  Navigate to your download and select <name>.wic in the "Disk Imager" tool
  - Write the image (note your Device may be different to that shown):
  Write the microSD Card using the "Disk Imager" tool

Turn off the Modular Development Kit and insert the microSD card in the microSD card slot located on the Modular Development Kit SOM Board.

Copying the Compiled AI Models to the microSD Card¶

The compiled models must be copied onto the microSD card for the Application Software to use at runtime:

scp and ssh the compiled models directly to the Development Kit (using its <ip address>):
- Power up the Modular Development Kit (if not already powered) and set up the serial terminal emulator (minicom, TeraTerm, PuTTY, etc.):
  - Select the correct COMx port. (The Modular Development Kit presents 4 serial COM ports over a single connection and the Linux system uses the 3^rd port in order). Set the port configuration as follows:
    - 115200 baud rate, 8 Data bits, 1 Stop bit, CRC and Hardware flow control disabled.
  - The Linux OS will boot.
  - Take note of the Modular Development Kit IP address.
    - The IP address can also be found using the terminal by logging in as root (no password required) and querying the Ethernet controller:
```
root
ifconfig
```
    - eth0 provides the IPv4 or IPv6 address to connect to.
- Using a Linux terminal (or Windows equivalent like PowerShell) on your Host, copy the files from the output directory to the Development Kit:
```
cd compile/output
scp -r * root@<ip address>:
```
  * Ensure sdcard has stored the files
```
ssh root@<ip address>
sync
ls *_categories.txt *.arch
```
  Outputs:
```
yolov8n-pose_categories.txt yolov8n_categories.txt

generated_arch.arch:
yolov8n-pose_dla_m2m_compiled_640_384.bin yolov8n_dla_m2m_compiled_640_384.bin
```
  * The Development Kit can be powered down, and restarted to load the models.

Running¶

Setting Up the Camera Solution¶

Warning

Handle ESD-sensitive equipment (boards, microSD Cards, Camera sensors, etc.) only when properly grounded and at an ESD-safe workstation

Make the required connections between the Host PC and the Agilex™ 5 FPGA E-Series 065B Modular Development Kit as detailed in the Setting Up the Modular Development Kit section.
Connect the Framos cable(s) between the Framos Camera Module(s) and the MIPI connector(s) on the Modular Development Kit Carrier Board taking care to align the cable(s) correctly with the connector(s) (pin 1 to pin 1). When using a single camera module, either MIPI connector can be used.

Modular Development Kit Carrier Board MIPI Connector Locations

Modular Development Kit Carrier Board with MIPI Framos Flex Cable Connected

Framos Camera with Flex Cable Connected

If using the optional Framos GMSL3 solution (and the Camera Solution System Example Design supports it):
- Connect the Framos FFA-GMSL-SER-V2A Serializer module back-to-back to the Framos Camera module.
- Using the Framos 150mm flex-cable connect the Framos FFA-GMSL-DES-V2A Deserializer module to the MIPI0 connector on the Modular Development Kit Carrier Board taking care to align the cable correctly with the connector (pin 1 to pin 1).
  - Note the System Example Design only supports one GMSL3 link on the MIPI0 port. However, a second Framos Camera module can be connected directly to the MIPI1 port using a Framos 150mm flex-cable.
- Connect the serializer module to the deserializer module using the GMSL3 5m coax cable.
- Connect the power supply to the deserializer module.
  - Note the GMSL3 deserializer module must be powered up before the Modular Development Kit.
- Power up the Modular Development Kit and ensure the deserializer modules Lock LED is illuminated green.

GMSL Connections

Connect the Modular Development Kit Carrier Board DisplayPort Tx connector to the Monitor using a suitable cable (and the adapter if you are using an HDMI cable).

Modular Development Kit with Connections

Program the FPGA SOF¶

To program the FPGA using SOF with first stage bootloader (fsbl):
- Power down the Modular Development Kit. Set MSEL=JTAG by setting the S4 dip switch on the Modular Development Kit SOM Board to OFF-OFF.
  - This prevents the starting of any bootloader and FPGA configuration after power up and until the SOF is programmed over JTAG.
- Power up the Modular Development Kit.
- Either use your own or download the pre-built fsbl SOF image, and program the FPGA with either the command:
```
quartus_pgm -c 1 -m jtag -o "p;fsbl_agilex5_modkit_vvpisp_time_limited.sof"
```
- or, optionally use the Quartus® Programmer GUI:
  - Launch the Quartus® Programmer and Configure the "Hardware Setup..." settings as follows:

Programmer GUI Hardware Settings

Click "Auto Detect", select the device A5EC065BB32AR0 and press "Change File.."

Programmer after "Auto Detect"

Select your fsbl_agilex5_modkit_vvpisp_time_limited.sof file. Check the "Program/Configure" box and press the "Start" button (see below). Wait until the programming has been completed.

Programming the FPGA with SOF file

Connecting with a Web Browser¶

Power up the Modular Development Kit (if not already powered) and set up the serial terminal emulator (minicom, TeraTerm, PuTTY, etc.):
- Select the correct COMx port. (The Modular Development Kit presents 4 serial COM ports over a single connection and the Linux system uses the 3^rd port in order). Set the port configuration as follows:
  - 115200 baud rate, 8 Data bits, 1 Stop bit, CRC and Hardware flow control disabled.
The Linux OS will boot and the Camera Solution System Example Design Software Application should run automatically.
A few seconds after Linux boots, the Software Application will detect the attached Monitor and the ISP processed output will be displayed using the best supported format.
Take note of the Modular Development Kit's IP address.
- The IP address can also be found using the terminal by logging in as root (no password required) and querying the Ethernet controller:
```
root
ifconfig
```
- eth0 provides the IPv4 or IPv6 address to connect to.

An Example ifconfig Output for a DHCP Network

An Example ifconfig Output for a Network with no DHCP support or is using a direct connection

Connect your web browser to the boards IP address so you can interact with the Camera Solution System Example Design using the GUI.
- To connect using IPv6 in the example address shown above, you would use http://[fe80::a8bb:ccff:fe55:6688] (note the square brackets)
- To connect using IPv4 for the DHCP example shown above, you would use http://192.168.0.1

An Example Web Browser URL for an IPv6 Address

An Example Web Browser URL for an IPv4 address

During connection, you will see the Altera® splash screen, after which you will be presented with the Web GUI.

An Example Camera Solution System Example Design GUI

Back

Last update: January 23, 2026
Created: December 10, 2025