Agilex™ 5 2x10GbE Precision Time Protocol System Example Design¶

Introduction¶

The Precision Time Protocol (IEEE1588v2) synchronizes clocks across networked devices to maintain a unified and precise time reference. In systems where components must coordinate actions — such as logging, data exchange, or event triggering — PTP ensures consistent timing, enabling deterministic operations and enhancing overall system integrity.

Agilex™ 5 is designed to operate as a PTP network node configured as an Ordinary Clock, Boundary Clock, or Transparent Clock, as defined by IEEE 1588-2008—Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems.

The details of the Precision Time Protocol are beyond the scope of this document. For comprehensive information, refer to 1588-2008 - IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems.

System Example Design Overview¶

The Agilex™ 5 2x10GbE Precision Time Protocol System Example Design includes two Ethernet ports with built-in two-step hardware PTP timestamping capabilities. The integrated Agilex™ 5 Hard Processor System (HPS) runs a PTP software stack that complements the hardware-based timestamping functionality.

The System Example Design (SED) provides the necessary drivers and user applications to support the Linux Network stack, the Linux PTP stack, and Layer 2 Packet Switch through the Linux kernel Traffic Control (TC) system.

The system's primary components include:

Golden Hardware Reference Design (GHRD)
Reference HPS software including:
- Arm Trusted Firmware
- U-Boot
- Linux Kernel
- Linux Drivers
- User Space Applications

The high-level hardware setup for the system example design is shown below:

Figure 1. Agilex™ 5 Precision Time Protocol system example design hardware setup

Glossary¶

Acronym	Full Form
PTP	Precision Time Protocol
SED	System Example Design
ToD	Time of Day
mSGDMA	Modular Scatter-Gather DMA
QoS	Quality of Service
AVST	Avalon Streaming
AXI	Advanced eXtensible Interface
CDC	Clock Domain Crossing
ETS	Egress Timestamp
ITS	Ingress Timestamp
TS	Timestamp
FP	Fingerprint
NVM	Non-Volatile Memory
GHRD	Golden Hardware Reference Design
GSRD	Golden Software Reference Design

Prerequisites¶

This system example design builds upon the Golden System Reference Design (GSRD) for the Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES). It is recommended that you familiarize yourself with the GSRD development flow before proceeding with this document.

The following items are required to fully utilize the SED:

Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES) (DK-A5E065BB32AES1) × 2.
- HPS IO48 OOBE daughter card × 2.
- Micro USB cable for serial output × 2.
- USB Type B cable for on-board FPGA Download Cable II × 2.
- Micro SD card (4GB or greater) × 2.
- Mini USB cable for OOBE daughter card serial port × 2.
- 100G/40G QSFP Cable. Design tested with:
  - FS (Q28-PC01) - 1m (3ft) 100G QSFP28 Passive Direct Attach Copper Twinax Cable.
  - FS(Q28-AO03) - 3m (10ft) 100G QSFP28 Active Optical Cable.
  - FS(QSFP-40G-AO03) - 3m (10ft) 40G QSFP+ Active Optical Cable.

A Host PC with:
- OS:Ubuntu 22.04 LTS. The system example design source files were compiled using Ubuntu 22.04 LTS; other versions and distributions may also be compatible.
- Serial terminal software (e.g., Minicom on Linux, Tera Term, or PuTTY on Windows) is required.
- Micro SD card slot or Micro SD card writer/reader
- Altera Quartus® Prime Pro 25.3

U-Boot and Linux compilation, Yocto build, and SD card image creation requires a Linux host PC. All other operations can be performed on either a Windows or a Linux host.

Release Contents¶

Binaries¶

Release notes and pre-built binaries are available under the GitHub repository release.

File	Description
Images.zip	Pre-compiled bitstream files for system example design devices.
sdimage.tar.gz	SD binary image containing Linux boot files.
Source code (zip)	System example design source files provided as a ZIP archive
Source code (tar.gz)	System example design source files provided as a TAR GZ archive

Sources¶

Component	Location	Branch	Commit ID/Tag
Hardware Design	https://github.com/altera-fpga/agilex5-ed-ptp/tree/rel/25.3/a5e065b-prem-devkit-exp-es/src/hw	rel/25.3	SED-2X10GE_PTP-agilex5_dk_a5e065bb32aes1-Q25.3-Rel1.1
Linux	https://github.com/altera-fpga/linux-socfpga	socfpga-6.12.19-lts-ethernet-sed	SED-2X10GE_PTP-agilex5_dk_a5e065bb32aes1-Q25.3-Rel1.1
Arm Trusted Firmware	https://github.com/altera-fpga/arm-trusted-firmware	socfpga_v2.13.0	116f2f97fa533e3540be97a2d9ec828f2a2b68aa
U-Boot	https://github.com/altera-fpga/u-boot-socfpga	socfpga_v2025.07	e5f40a8ed1ec65f20c4e2491bfe8e738efce6d94
Yocto Project: poky	https://git.yoctoproject.org/poky/	rel/25.3	d1c25a3ce446a23e453e40ac2ba8f22b0e7ccefd
Yocto Project: meta-intel-fpga	https://git.yoctoproject.org/meta-intel-fpga/	rel/25.3	9714ae1ef8f22302bac60b7d2081bbdf3199ca70
Yocto Project: meta-intel-fpga-refdes	https://github.com/altera-fpga/meta-intel-fpga-refdes/	rel/25.3	bffc5bc012f1653beb58878b54b44e74b0f27404
Yocto Project: meta-agilex5-sed	https://github.com/altera-fpga/agilex5-ed-ptp/tree/rel/25.3/a5e065b-prem-devkit-exp-es/src/sw/yocto/meta-agilex5-sed	rel/25.3	SED-2X10GE_PTP-agilex5_dk_a5e065bb32aes1-Q25.3-Rel1.1
GSRD Build Script: gsrd-socfpga	https://github.com/altera-fpga/agilex5-ed-ptp/tree/rel/25.3/a5e065b-prem-devkit-exp-es/src/sw/yocto/build.sh	rel/25.3	SED-2X10GE_PTP-agilex5_dk_a5e065bb32aes1-Q25.3-Rel1.1

Release Notes¶

Agilex™ 5 2x10GbE Precision Time Protocol System Example Design Release Notes.

Agilex™ 5 2x10GbE Precision Time Protocol System Example Design Architecture¶

Hardware Architecture¶

Figure 2 illustrates the high-level architecture of the system example design. The main components include:

HPS Subsystem
DMA Subsystem
Packet Switch Subsystem
Ethernet Subsystem
Main Time Of the Day Subsystem
Subordinate Time Of the Day Subsystem
Ethernet Packet Generators

Figure 2. Agilex™ 5 Precision Time Protocol SED High Level Hardware Architecture

HPS Subsystem¶

The HPS Subsystem, built around the Agilex™ 5 Hard Processor System (HPS) and supporting logic, manages PTP synchronization and handles Time of Day (ToD) adjustments. It also provides access to status and control registers for other system components.

The subsystem communicates with onboard components via its peripherals, using an I2C bus to monitor and configure the QSFP28 module. It also controls the Clock IC Skyworks Si5518 PTP & SyncE Network Synchronizer over the same bus, performing phase and frequency adjustments to maintain system-wide timing accuracy.

Ethernet Subsystem¶

The Ethernet Subsystem is a flexible and high-performance solution for connecting your system to a network. It is designed for easy integration, expansion, and reliable synchronous operation. It consists of the following components, - Ethernet QHIP (includes PTP Tile Adaptor): Handles Ethernet data and provides precise time synchronization. - System Clock, SRC: Manages clock signals. - CSR Access: Allows Control and Status Registers (CSR) access directly to the Host/HPS, as detailed in the system memory map.

The Ethernet Subsystem connects to the QSFP and clock cleaner on the board to create a complete Synchronous Ethernet link. It provides network packet access and includes Ethernet Layer 1 and Layer 2 components such as MAC, PCS, FEC, and PMA, which interface with an external Ethernet PHY.

The Ethernet Subsystem instantiates the Agilex™ 5 GTS Ethernet Hard IP . Refer to the GTS Ethernet Hard IP User Guide Agilex™ 5 FPGAs and SoCs for details.

Figure 3. Ethernet Subsystem High-level Architecture

DMA Subsystem¶

The DMA Subsystem uses mSGDMA engines to transfer data between the HPS and the Ethernet Subsystem. It includes four DMA Ports, 2-transmit (TX) and 2-receive (RX) for each Ethernet port. These channels natively handle PTP Timestamps and Tx Fingerprints.

The subsystem groups DMA Ports into sets of two, assigning each group to one Ethernet port in the Ethernet Subsystem. It also translates protocols between Avalon® Streaming (AVST) and AXI-Stream (AXI-ST) interfaces, and performs clock domain crossing between the HPS Subsystem and Ethernet Subsystem clock domains. Figure 3 shows a high-level architecture diagram of one of the DMA Subsystem ports.

Figure 4. DMA Subsystem Port High-level Architecture

Packet Switch Subsystem¶

The Packet Switch Subsystem implements an L2–L4 Ethernet packet switch that arbitrates among four client interfaces per port, connecting two DMA engines and a traffic generator to the transmit path. On the receive path, packet routing between ports and clients is handled by a TCAM (refer Section 3.2.2.), with rules dynamically configurable via software. By default, packets without a TCAM match are dropped for security reasons. Matched entries, the Packet Switch subsystem routes packets to either a DMA Port or a User Port (Traffic Generator).

The TX datapath arbitration ignores Ethernet packet type and uses a weighted priority round-robin scheme to manage requests from DMA and User Ports. Figure 5 illustrates the high-level architecture of the Packet Switch transmitter path. On the transmit path, the Ethernet Subsystem returns the egress timestamp (ETS) for each packet along with its corresponding fingerprint for tracking.

The RX datapath does not implement priority-based arbitration. Instead, traffic priority to the HPS is software-defined, with each DMA Port represented as a queue and assigned a configurable priority level. Figure 6 illustrates the high-level architecture of the Packet Switch receiver path.

Figure 5. Packet Switch Subsystem TX Datapath High Level Architecture

Figure 6. Packet Switch Subsystem RX Datapath High Level Architecture

TCAM Data Structure¶

The TCAM supports lookups using Ethernet frame headers, protocol headers, and IEEE 1588-specific fields.

The table below lists the 492-bit TCAM key fields. If a packet lacks a corresponding header field, the key field is set to 0. Otherwise, the Packet Switch parser populates the field with extracted data. Unused bits from shorter headers are also zeroed.

Field	Width (bits)	Description
rsvd	32	Reserved.
flagField	16	flagField field in PTP header.
messageType	4	messageType field in PTP header.
ip_protocol	8	IP header protocol field, defined as Protocol in IPv4 or next_header in IPv6.
ethtype	16	Ethernet header ethtype
		dot2q: (eth.{da,sa}) / (vlana.{tpid,tci}) / (vlanb.{tpid,tci}) / ethtype dot1q: (eth.{da,sa}) / (vlana.{tpid,tci}) / ethtype eth: (eth.{da,sa,ethtype}) - ethtype = ethtype
tci_vlana	16	TCI field for VLAN A in IEEE 802.1Q frames
		dot2q: (eth.{da,sa}) / (vlana.{tpid,tci}) / (vlanb.{tpid,tci}) / ethtype - tci_vlana = vlana.tci dot1q: (eth.{da,sa}) / (vlana.{tpid,tci}) / ethtype - tci_vlana = vlana.tci eth: (eth.{da,sa,ethtype}) - tci_vlana = '0
tci_vlanb	16	TCI field for VLAN B in IEEE 802.1Q frames
		dot2q: (eth.{da,sa}) / (vlana.{tpid,tci}) / (vlanb.{tpid,tci}) / ethtype - tci_vlanb = vlanb.tci dot1q: (eth.{da,sa}) / (vlana.{tpid,tci}) / ethtype - tci_vlanb = '0 eth: (eth.{da,sa,ethtype}) - tci_vlanb = '0
l4_src_port	16	L4 header source port.
		- l4_src_port = udp.sport - l4_src_port = tcp.sport
l4_dst_port	16	L4 header destination port.
		- l4_dst_port = udp.dport - l4_dst_port = tcp.dport
src_ip	128	L3 source address field.
		IPv4: - src_ip[31:0] = ipv4.src_ip, src_ip[127:32] = '0 IPv6: - src_ip[127:0] = ipv6.src_ip
dst_ip	128	L3 destination address field.
		IPv4: - dst_ip[31:0] = ipv4.dst_ip, dst_ip[127:32] = '0 IPv6: - dst_ip[127:0] = ipv6.dst_ip
src_mac	48	Ethernet header source MAC address.
		- src_mac = eth.sa
dst_mac	48	Ethernet header destination MAC address.
		- dst_mac = eth.da

Table 1. TCAM Key Fields

The Linux packetswitch user application provides access to the TCAM key registers from the OS, removing the complexity of doing low level access to the Packet Switch IP.

The table below defines the structure of a TCAM query result. This data is used to route in-transit packets to the destination port specified by the matching TCAM rule.

Field	Width (bits)	Description
rsvd	27	Reserved.
drop	1	Drop packet.
egr_port	4	Selects which egress port to send traffic. 4’d0: MSGDMA Channel 0 4’d1: MSGDMA Channel 1 4’d2 – 4’d7: reserved 4’d8: User 4’d8 – 4’d15: reserved

Table 2. TCAM Query Result Fields

Time Of the Day Subsystem¶

The Main ToD (Time of Day) module uses a 96-bit counter to represent the current time. Software synchronizes this counter with the PTP network time, using data from the Subordinate PTP port. The Main ToD operates with a 156.25 MHz clock sourced from the onboard clock cleaner, causing the counter to increment every 6.4 ns. An additional IOPLL can be added to fine-tune the ToD clock, enhancing the accuracy of the 1PPS (one pulse per second) signal, especially since the counter may not align perfectly with one-second intervals. The 1PPS signal is primarily used for precise timing across various applications. For further details, refer to the ToD User Guide.

The Main ToD Subsystem wraps the IEEE 1588 Time of Day Clock FPGA IP and its support logic, serving as the system’s local ToD reference. The IP is configured for Accuracy Advanced mode and uses the IOPLL Reconfig FPGA IP, as described in the IOPLL and TOD Setup using IOPLL Reconfig IP chapter of the Ethernet Design Example Components User Guide.

Figure 7. Board High Level Clocking Architecture

The Subordinate ToD Subsystem instantiates a dedicated IEEE 1588 Time of Day Clock FPGA IP per Ethernet interface and integrates an IEEE 1588 TOD Synchronizer FPGA IP to present timestamps in the Ethernet clock domain, as described in the Connect the Precision Time Protocol Interface chapter of the GTS Ethernet Hard IP User Guide.

Ethernet Packet Generators¶

Two system blocks can generate Ethernet packets. The HPS produces PTP packets when the ptp4l service is enabled and can optionally generate synthetic traffic via ping or iperf3. Additionally, two hardware traffic generators can saturate Ethernet bandwidth with synthetic traffic when enabled by HPS software.

SED Custom IP¶

Block	Entity Name	Description_
AVST to AXI Bridge	avst_axist_bridge	AVST to AXI bridge facilitating data transfer between the DMA channels and the Ethernet Subsystem. The bridge operates bidirectionally, providing AXI to AVST translation for data flowing from the Ethernet Subsystem to the DMA channels. It is a single block servicing both TX and RX DMA channels.
TX DMA Fifo	tx_dma_fifo	Top-level wrapper for custom blocks in the TX DMA Datapath.
TX ETS Adapter	hssi_ets_ts_adapter	Adapts the egress timestamp and fingerprint from the Ethernet Subsystem to a format manageable by the TX DMA channel.
TX DMA PKT FIFO	cdc_packet_fifo	Dual clock FIFO for clock domain crossing of packet information from the DMA channel to the Ethernet Subsystem.
TX FP Generator	tx_dma_fifo	Sequential fingerprint generator. A fingerprint will be generated for all packets going out of the system. This is combinational logic inside the 'tx_dma_fifo' module.
TX TS Valid	tx_dma_fifo	Logic that inserts egress timestamps into the MSGDMA prefetcher. This is combinational logic inside the 'tx_dma_fifo' module.
TX TS/FP FIFO	fp_resp_fifo/ts_fifo	Two independent FIFOs to store the returned egress timestamp and its corresponding fingerprint.
TX Completion	ts_chs_compl	Timestamp completion follow-up module. Captures FP and TS from the HSSI subsystem and forwards them to the TX DMA FIFO module if they are valid.
FP Compare	ts_chs_compl	Combinational logic that tracks fingerprints returned by the HSSI Subsystem to validate and return the associated egress timestamp.
TX TS FIFO	ts_fifo	FIFO to store the returned egress timestamp.
RX DMA Fifo	rx_dma_fifo	Top-level wrapper for custom blocks in the RX DMA Datapath.
RX TS Valid	rx_dma_fifo	Logic that inserts ingress timestamps into the MSGDMA prefetcher. This is combinational logic inside the 'rx_dma_fifo' module.
RX DMA PKT FIFO	cdc_packet_fifo	Dual clock FIFO for clock domain crossing of packet information from the Ethernet Subsystem to the DMA channel.
RX TS FIFO	ts_fifo	Single clock FIFO to store ingress timestamps from the Ethernet Subsystem.
Main ToD	master_tod	Wrapper for the Ethernet IEEE 1588 Time of Day Clock FPGA IP. Includes a state machine that flags when the Main ToD Subsystem output is valid for the ToD Subordinate Subsystem to consume.
Packet Generator	eth_f_packet_client_top	Generic Ethernet packet generator/checker. Packet generation parameters are configurable at runtime via software.
Packet Generator Adaptor	eth_f_packet_client_top_axi_adaptor	Provides translation services between AVST and AXI-ST for the system Packet Generators.
Packet Switch	packet_switch_subsys	Top level wrapper for TCAM, arbitration and routing logic to handle the system TX/RX data path

Board Level Clocking Architecture¶

At the board level, the system clocking architecture includes the following components:

Skyworks Si5518A PTP & SyncE Network Synchronizer (U67)
Oven Controlled Crystal Oscillator (OCXO) (X2)
Agilex™ 5 A5ED065BB32AE6SR0 (U5)
Si5332A Low-Jitter Clock Generator (U411,U412)

Figure 8. Board High Level Clocking Architecture

In the default clock configuration for the Development Kit, SI5518A OUT2 is set to 125 MHz. User needs to configure SI5518A clock IC with new settings for this solution. The steps to carry out this configuration are shown in the Section.

FPGA Design Clocking Architecture¶

The clock frequencies for the GTS Ethernet Hard IP ports i_clk_tx, i_clk_rx, i_clk_ref_p, i_clk_sys, o_clk_pll, o_clk_rec_div, i_clk_tx_tod, i_clk_rx_todand i_clk_ptp_sample follow the guidelines in the Section 4.1. Implement Required Clocking of GTS Ethernet Hard IP User Guide Agilex™ 5 FPGAs and SoCs.

Figure 9 shows the high-level system clock distribution tree. For clarity, only one Ethernet port (P0) and one DMA Subsystem port are shown. All DMA ports share the same clock connections. Ports connected to Ethernet port 1 use its output clocks.

Figure 9. FPGA High Level Clocking Architecture Reduced Diagram

Software Architecture¶

The system example design uses an HPS-first boot flow, where the HPS initializes before configuring the FPGA fabric. U-Boot is loaded from SPI flash or via a partial RBF. The second-stage boot loader loads the Linux kernel and full FPGA bitstream from the SD card. U-Boot enables the HPS bridges and programs the FPGA via the SDM. Once configured, the HPS boots into Linux.

The solution includes drivers and user-space tools for the Linux network stack, PTP stack, and QoS via Traffic Control (TC). GTS Ethernet Hard IP drivers support standard tools such as ethtool. Drivers for the IEEE 1588 TOD Clock IP, Skyworks Si5518A synchronizer, and DMA ports interface are also provided.

Egress QoS is managed by the Linux kernel’s Traffic Control (TC) system. The Ethernet driver uses device tree data to enumerate DMA channels for each physical port. For each channel, it registers a TX buffer ring and exposes it as a separate hardware queue. TC applies queue disciplines (qdiscs) to control packet enqueue/dequeue behavior per queue. The driver integrates with TC to enable per-queue priorities and flow control.

For each DMA ingress channel, the Ethernet driver registers an RX buffer. Ingress QoS is controlled by the Packet Switch IP and the packetswitch application, which defines filtering and routing rules. Packets matching a rule are directed to a specific DMA RX channel, queue, or User Port; unmatched packets are dropped by default.

The HPS polls RX queues based on interrupt priority, with higher-priority channels mapped to higher-priority interrupts.

Intel Agilex™ 5 SoC FPGA Ethernet Drivers¶

Driver	Description_	File
Ethernet Driver	The Ethernet driver exposes a standard `netdev` API to the kernel, enabling DMA channel discovery, hardware Time-of-Day (ToD) access, and `ethtool` enabling	`/drivers/net/ethernet/altera/intel_fpga_eth_main.c`
ToD Driver	Provides access to the configuration and status registers of the Ethernet IEEE 1588 Time of Day Clock FPGA IP.	`/drivers/net/ethernet/altera/intel_fpga_tod.c`
HSSI Driver	Provides access to the Ethernet subsystem configuration and status registers.	`/drivers/net/ethernet/altera/intel_fpga_hssiss.c`
GTS Ethernet IP Driver	Provides access to the GTS Ethernet Hard IP configuration and status registers.	`/drivers/net/ethernet/altera/intel_fpga_hssigl_gts_driver.c`
QSFP Driver	The QSFP driver interfaces with the onboard QSFP module, handling configuration registers reads and controlling power and interrupt pins.	`/drivers/net/phy/qsfp-mem-core.c`
Si5518A Driver	The Si5518A driver enables frequency steering for Time-of-Day (ToD) adjustment via the onboard Si5518A SyncE & IEEE 1588 Network Synchronizer.	`/drivers/net/ethernet/altera/dpll/intel_freq_ctrl_si5518_i2c.c`
Freq control Driver	This driver enables frequency steering for Time-of-Day (ToD) adjustment via the onboard Si5518A SyncE & IEEE 1588 Network Synchronizer.	`/drivers/net/ethernet/altera/intel_freq_control.c`

User Space Applications¶

ptp4l¶

ptp4l is IEEE 1588 PTP software implementation from the The Linux PTP Project, included in the HPS image. It offers extensive configuration options for system setup. Refer to the ptp4l man page for details

phc2sys¶

phc2sys is an open-source utility that synchronizes system clocks, typically aligning the system clock with a PTP Hardware Clock (PHC) managed by ptp4l. For configuration details, refer to the phc2sys man page.

ethtool¶

ethtool is an open-source utility for querying and configuring network driver and hardware settings. For usage details, refer to the ethtool man page.

packetgenerator¶

This application configures the Packet Generator IP core in the FPGA to generate synthetic Ethernet traffic for validating data path integrity and line rates. It can also modulate bandwidth to test system QoS policies.

Packet generation is customizable via parameters such as:

Source and destination MAC addresses
Frame sizes
Idle packet gaps

Syntax

packetgenerator [--device] [/dev/uioX] [options]

Parameters

--help: Print this help contents
--device: UIO device name
--dump: Dump all register contents
--register-offset offset: 32-bit aligned register offset to do direct register read/write
--register-value value: 32-bit value to be written to the register
--dest-mac: Destination MAC address in the packet
--src-mac: Source MAC address in the packet
--traffic bool: Enable or disable traffic
--one-shot bool: Enable or disable one-shot mode
--soft-reset: Trigger a soft reset
--packet-checker bool: Enable or disable packet checker
--cntr-snapshot bool: Take a counter snapshot
--cntr-clear bool: Clear all counter CSRs
--cntr-internal-clear bool: Clear all internal counters
--fixed-gap bool: Enable or disable fixed gap between packets
--pkt-len-mode value: Set packet generation length mode (Fixed/Incremental) [1,2]
--num-idle-cycles value: Number of idle cycles to insert [0...255]
--tx-pkt-size value: TX packet size [64...9216]
--tx-max-pkt-size value: Maximum TX packet size [64...9216]
--num-packets value: Number of packets to generate [0...0xFFFFFFFF]

System example design packet generators are mapped to /dev/uio0 and /dev/uio1.

Basic Usage

Synthetic Traffic Configuration – The command below sets up the packet generator with parameters including dynamic packet mode, fixed gap, packet length mode, idle cycles, packet checker, one-shot mode, and packet sizes before initiating traffic generation.

packetgenerator --device /dev/uio0 --traffic false --fixed-gap true --pkt-len-mode 0x01 --num-idle-cycles 22 --packet-checker true --one-shot false --tx-pkt-size 1024 --tx-max-pkt-size 1024

Start Packet Generator – The command below initiates traffic generation based on the current configuration parameters.

packetgenerator --device /dev/uio0 --traffic 1

Configuration and Status Report – The command below captures a snapshot of all internal configuration and status registers in the packet generator hardware.

packetgenerator --device /dev/uio0 --dump

packetswitch¶

The packetswitch application configures the hardware Packet Switch IP, which routes incoming packets to one of two DMA channels per Ethernet interface based on user-defined (Packet switch) rules. Packets that do not match any rule are dropped by default.

Rule priority is determined by index number; higher index means higher priority. If multiple rules match, the rule with the highest index is applied. To ensure correct behavior, generic rules should be programmed first followed by more specific rules at higher indices.

A maximum of 32 keys can be programmed (0-31).

Syntax

packetswitch [--device] [/dev/uioX] [Options]

Parameters

--help: Print this help contents
--device: *UIO device name
--dump: Dump all register contents
--set-key: Set Key. Requires Key fields to be provided
--remove-key: Remove Key using key-index
--flush-all-keys: Flush all Key entries from the system
--flush-all-counters: Flush all debug counters value to 0
--show-key: Search for Keys fulfilling a search criteria for a port
--register-rw: Do a direct register read write
--key-index: Key index to work on
--dest-mac: Key - Destination MAC. packetswitch can resolve MAC addresses from Ethernet interface names, e.g. eth1.
--src-mac: Key - Source MAC. packetswitch can resolve MAC addresses from Ethernet interface names, e.g. eth1.
--dest-ip: Key - Destination IP Address
--src-ip: Key - Source IP address
--dest-port: Key - Destination L4 port
--src-port: Key - Source L4 port
--vlanb: Key - VALNB
--vlana: Key - VLANA
--ethtype: Key - Ethernet type
--protocol: Key - IP Protocol type
--message: Key - IP Message type
--flag: Key - Flag field
--result: Defines the DMA or user port to which the Ethernet packet will be routed if the rule evaluation is true. The mapping for this parameter is as follows:
- 0x0: route packet to DMA-0
- 0x1: route packet to DMA-1
- 0x8: route packet to User Port (Packet Generator)
port: Ethernet interface to which the rule will apply.
- 0: Apply rule to eth1 Ethernet interface
- 1: Apply rule to eth2 Ethernet interface
register-offset: Register offset to read/write to. Refer to the Packet switch Register Map for direct registers base address and offsets.
register-value: Register value to write. Can be comma separated to write multiple values.
length: Number of registers to read
mask: Set Mask properties for fields manually

System example design Packet Switch is mapped to /dev/uio2.

Basic Usage

Route all incoming traffic to DMA 0:

packetswitch --port 0 --set-key --key-index 0 --result 0x0

--port 0: This rule applies to Ethernet interface eth1.
--key-index 0: This rule is stored in key index 0, setting the lowest priority for the rule.
--result 0x0: Packets fulfilling the rule will be routed to DMA-0.

The above command defines the following rule:

Route traffic based on destination MAC address

packetswitch --port 0 --set-key --key-index 0 --dest-mac "eth1" --result 0x1

The above command defines the following rule:

--port 0: This rule applies to Ethernet interface eth1.
--key-index 0: This rule is stored in key index 0, setting the lowest priority for the rule.
--dest-mac "eth1": This is the filter established by the rule. The rule will return a hit if the evaluated Ethernet frame has the eth1 interface MAC address in the MAC address destination field.
--result 0x1: Packets fulfilling the rule will be routed to DMA-1.

Route traffic based on VLAN and DF flag

packetswitch --port 0 --set-key --key-index 2 --ethtype 0x0800 --protocol 0x01 --vlana 100 --vlanb 200 --flag 0x2 --result 0x1

The above command defines the following rule:

--port 0: This rule applies to Ethernet interface eth1.
--key-index 2: This rule is stored in key index 2.
--ethtype 0x0800: Ethernet Type is set to IPv4.
--protocol 0x01: The IP protocol is set to ICMP.
--vlana 100: The primary VLAN ID is 100.
--vlanb 200: The secondary VLAN ID is 200.
--flag 0x2: The fragment flag is set to 0x2.
--result 0x1: Packets fulfilling the rule will be routed to DMA-1.

Address Map Details¶

Address Map¶

Subordinate Name	Component	Agilex™ HPS H2F AXI Manager	Register Description
top.qhip_port0_s0	GTS Ethernet Hard IP - Port 0	0x4030_0000 - 0x403f_ffff	Link
top.qhip_port1_s0	GTS Ethernet Hard IP - Port 1	0x4050_0000 - 0x405f_ffff	Link
top.eth_f_packet_client_top[0]	User Port-0(Packet Client)	0x5000_0000 - 0x5000_0fff	Link
top.eth_f_packet_client_top[1]	User Port-1(Packet Client)	0x5000_1000 - 0x5000_1fff	Link
top.packet_switch_subsys_csr_s0	PTP Packet Switch	0x5001_0000 - 0x5001_ffff
top.master_tod_csr	Main ToD Subsystem	0x4405_0000 - 0x4405_003f	Link
top.soc_inst.cdc_tod_125_100M	CDC TOD FIFO	0x4405_0000 - 0x4405_03ff
top.qsfp_cntlr_csr_s0	QSFP Controller-0/1	0x4404_0000 - 0x4404_ffff
top.top_user_space_csr_s0	User Space CSR	0x4020_0000 - 0x4020_0fff
top.soc_inst.subsys_msgdma_p0_eth_ch0.csr	DMA Subsystem Port 0 -Channel 0	0x4500_0000 - 0x4500_00ff	Link
top.soc_inst.subsys_msgdma_p0_eth_ch1.csr	DMA Subsystem Port 0 -Channel 1	0x4500_0100 - 0x4500_01ff	Link
top.soc_inst.subsys_msgdma_p1_eth_ch0.csr	DMA Subsystem Port 1 -Channel 0	0x4500_0200 - 0x4500_02ff	Link
top.soc_inst.subsys_msgdma_p1_eth_ch1.csr	DMA Subsystem Port 1 -Channel 1	0x4500_0300 - 0x4500_03ff	Link

Table 3. Qsys_top Platform Designer system address map.

Packet Generator Register Description¶

Refer to section Packet Client Register Map in the MACsec FPGA System Design User Guide for the register description.

DMA SubSystem Port Memory Map¶

Subordinate Name	Component	[rx/tx]_dma_csr	Register Description
tx_dma_prefetcher	DMA prefetcher	0x0000 - 0x001F	Register Map of mSGDMA
tx_dma_dispatcher	DMA dispatcher	0x0020 - 0x003F	Register Map of mSGDMA
dma_fifo_0	DMA FIFO	0x0040 - 0x005F	Register Map of mSGDMA
rx_dma_prefetcher	DMA prefetcher	0x0080 - 0x009F	Register Map of mSGDMA
rx_dma_dispatcher	DMA dispatcher	0x00A0 - 0x00BF	Register Map of mSGDMA

Table 4. DMA Subsystem channel memory map.

Packet Switch Register Map¶

Module	Start Address	End Address
Ingress Arbiter 0	0x0	0x8
Ingress Arbiter 1	0xC	0x14
Egress RX Demux 0	0x60	0x70
Egress RX Demux 1	0x88	0x98
Ingress RX Width Adapter 0	0x1A0	0x1A8
Ingress RX Width Adapter 1	0x1AC	0x1B4
TCAM_0 (16KB)	0x200	0x41FC
TCAM_1 (16KB)	0x4200	0x81FC
Egress RX Width Adapter 0 (User Port)	0x8200	0x8208
Egress RX Width Adapter 1 (User Port)	0x820C	0x8214

Table 5. Packet Switch Register Description

Packet Switch Ingress Arbiter Register Description¶

Register Name	Offset	Field	Width (bits)	Type	HW Reset Value	Description
scratch_reg	0x00	scratch	32	RW	32'h0	Scratch Register.
cfg_priority_dma	0x04	reserved	[31:8]	RO	16'h0	Reserved.
		ch_1	[7:4]	RW	4'h2	Configured priority level for DMA channel 1. 0: highest priority, 3: lowest priority, other values are reserved. This register along with cfg_priority_user register (0x8) configures the ingress arbiter priority levels. Values across both registers must have unique priority values.
		ch_0	[3:0]	RW	4'h0	Configured priority level for DMA channel 0. 0: highest priority, 3: lowest priority, other values are reserved. This register along with cfg_priority_user register (0x8) configures the ingress arbiter priority levels. Values across both registers must have unique priority values.
cfg_priority_user	0x08	reserved	[31:4]	RO	28'h0	Reserved.
		port_0	[3:0]	RW	4'h1	Configured priority level for User_0 port. 0: highest priority, 3: lowest priority, ‘d4-‘d15: reserved. This register along with cfg_priority_dma register (0x4) configures the ingress arbiter priority levels. Values across both registers must have unique priority values.

Table 6. Packet Switch Ingress Arbiter Register Description.

Packet Switch Egress RX Demux Register Description¶

Register Name	Offset	Field	Width (bits)	Type	HW Reset Value	Description
scratch_reg	0x00	scratch	[31:0]	RW	32'h0	Scratch Register.
control_reg	0x04	reserved	[31:2]	RO	29'h0	Reserved.
		dma_1_drop_en	[1]	RW	1'h0	Enable drop threshold to be used for DMA CH_1.
		dma_0_drop_en	[0]	RW	1'h0	Enable drop threshold to be used for DMA CH_0.
dma_0_drop_threshold_reg	0x08	reserved	[31:16]	RO	16'h0	Reserved.
		drop_threshold	[15:0]	RW	16'd496	Drop threshold for DMA CH_0.
dma_1_drop_threshold_reg	0x0C	reserved	[31:16]	RO	16'h0	Reserved.
		drop_threshold	[15:0]	RW	16'd496	Drop threshold for DMA CH_1.

Table 7. Packet Switch Egress RX Demux Register Description.

Packet Switch Ingress RX Width Adjuster Register Description¶

Register Name	Offset	Field	Width (bits)	Type	HW Reset Value	Description
scratch_reg	0x00	scratch	[31:0]	RW	32'h0	Scratch Register.
control_reg	0x04	Reserved	[31:1]	RO	31'h0
		cfg_rx_pause_en	[0]	RW	1'h0	Enable RX pause.
cfg_threshold_reg	0x08	drop_threshold	[31:16]	RW	16'd1948	Configured threshold when packets are dropped.
		rx_pause_threshold	[15:0]	RW	16'd1024	Configured threshold when RX pause is asserted.

Table 8. Packet Switch Ingress RX Width Adjuster Register Description.

Packet Switch Egress RX Width Adjuster Register Description¶

Register Name	Offset	Field	Width (bits)	Type	HW Reset Value	Description
scratch_reg	0x00	scratch	[31:0]	RW	32'h0	Scratch Register.
control_reg	0x04	reserved	[31:1]	RO	31'h0	Reserved.
		drop_en	[0:0]	RW	1'h0	Enable drop threshold to be used for egress width adapter.
cfg_drop_threshold_reg	0x08	reserved	[31:16]	RO	16'h0	Reserved.
		drop_threshold	[15:0]	RW	16'd496	Drop threshold for egress width adapter.

Table 9. Packet Switch Egress RX Width Adjuster Register Description.

TCAM Key Register Map¶

Register Field	Register Offset	Register Bit	Key Field
Key_15	0x103C	[31:12]	reserved
Key_15	0x103C	[11:0]	rsvd[31:20]
Key_14	0x1038	[31:12]	rsvd[19:0]
Key_14	0x1038	[11:0]	flagField[15:4]
Key_13	0x1034	[31:28]	flagField[3:0]
Key_13	0x1034	[27:24]	messageType[3:0]
Key_13	0x1034	[23:16]	ip_protocol[7:0]
Key_13	0x1034	[15:0]	ethtype[15:0]
Key_12	0x1030	[31:16]	tci_vlana[15:0]
Key_12	0x1030	[15:0]	tci_vlanb[15:0]
Key_11	0x102C	[31:16]	l4_src_port[15:0]
Key_11	0x102C	[15:0]	l4_dst_port[15:0]
Key_10	0x1028	[31:0]	src_ip[127:96]
Key_9	0x1024	[31:0]	src_ip[95:64]
Key_8	0x1020	[31:0]	src_ip[63:32]
Key_7	0x101C	[31:0]	src_ip[31:0]
Key_6	0x1018	[31:0]	dst_ip[127:96]
Key_5	0x1014	[31:0]	dst_ip[95:64]
Key_4	0x1010	[31:0]	dst_ip[63:32]
Key_3	0x100C	[31:0]	dst_ip[31:0]
Key_2	0x1008	[31:0]	src_mac[47:16]
Key_1	0x1004	[31:16]	src_mac[15:0]
Key_1	0x1004	[15:0]	dst_mac[47:32]
Key_0	0x1000	[31:0]	dst_mac[31:0]

Table 10. TCAM key register map.

Interrupt Map¶

Below table provides the interrupts and their numbers from HW design and Linux OS with mapping. you can refer device tree socfpga_agilex5_ptp_2p10g.dtsi file of the design for more details.

Interrupt	F2H IRQ	Linux Interrupt
dipsw_pio	0
button_pio_irq	1
subsys_msgdma_p0_eth_tx_ch0_irq	17	32
subsys_msgdma_p0_eth_rx_ch0_irq	18	31
subsys_msgdma_p0_eth_tx_ch1_irq	19	30
subsys_msgdma_p0_eth_rx_ch1_irq	20	29
subsys_msgdma_p1_eth_tx_ch0_irq	21	28
subsys_msgdma_p1_eth_rx_ch0_irq	22	27
subsys_msgdma_p1_eth_tx_ch1_irq	23	26
subsys_msgdma_p1_eth_rx_ch1_irq	24	25

Table 11. Interrupt map.

Hardware Setup¶

Figure 10. Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES)

Set up the board default settings, as listed by the Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES) User Guide, "Default Settings" section:

Switch	Default Position
S22	OFF
S16 [1:4]	OFF/ON/ON/ON
S27 [1:4]	OFF/ON/ON/OFF
S23	OFF
S21 [1:4]	OFF/OFF/OFF/OFF
S24 [1:4]	ON/ON/ON/ON
S25 [1:4]	ON/OFF/OFF/OFF

Table 12. Factory Default Switch Settings

Connect the Type B USB cable from each development kit (J27 - Highlighted as 12 in Figure 11) to the host for JTAG access.

Connect the two Agilex™ 5 FPGA and SoC E-Series Premium Development Kits (ES) with a QSFP cable(DAC/AOC) via the QSFP+ cages - J12 and J13 (Highlighted as 16 in Figure 11).

Figure 11. Module Identification for Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES)

Follow instructions in "Installing the HPS Expansion Board (HPS-EB)" to install HPS Expansion Board in the Development Kit.

Connect the mini-USB port(J7 - Highlighted as 16 in Figure 11) from each of the HPS Expansion Board to your host machine. Both development kits with HPS Expansion Board are connected to the same host.

Figure 12 shows a high level connectivity diagram for both development kits.

Figure 12. High level hardware connectivity diagram

Configure the Serial Connection¶

The Embedded Linux OS on the Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES) can be accessed via a serial terminal such as Minicom or PuTTY. First, identify the serial connection IDs between your host and each development kit. On an Ubuntu host, list the most recently connected USB-to-Serial devices using:

admin@10.1.23.255:~$ dmesg | grep "ttyUSB*"
[1636111.470334] usb 1-1.2: FTDI USB Serial Device converter now attached to ttyUSB2
[1636111.470597] usb 1-1.2: FTDI USB Serial Device converter now attached to ttyUSB3
[1636114.478469] usb 1-11.2: FTDI USB Serial Device converter now attached to ttyUSB4
[1636114.478705] usb 1-11.2: FTDI USB Serial Device converter now attached to ttyUSB5

In this example, the four detected devices correspond to the serial connections for the Agilex™ 5 FPGA and SoC E-Series Premium Development Kits, as no other USB-to-Serial cables are connected to the host.

Start a serial session for each development kit using Minicom. Open separate terminal windows and launch a Minicom instance in each to monitor both kits concurrently.

Development Kit 1 terminal:

# Note: Device names may vary depending on your system. Adjust accordingly.
admin@10.1.23.255:~$ minicom -D /dev/ttyUSB3

Development Kit 2 terminal:

# Note: Device names may vary depending on your system. Adjust accordingly.
admin@10.1.23.255:~$ minicom -D /dev/ttyUSB5

Access the Minicom configuration screen using the following key combination:

Ctrl + A, then press Z for the Command Summary menu
SHIFT + O for the configuration menu

Configure each serial session with the following parameters:

Bps/Par/Bits: 115200 8N1
Hardware Flow Control: No
Software Flow Control: No

Your 'Serial port setup' screen should look like the following after adjusting the configuration parameters:

Welcome to minicom 2.7.1                                                                  

OPTI+--------------------------------------------------------------------#             
Comp| A -    Serial Device      : /dev/ttyUSB3                              |             
Port| B - Lockfile Location     : /var/lock                                 |             
    | C -   Callin Program      :                                           |             
Pres| D -  Callout Program      :                                           |             
    | E -    Bps/Par/Bits       : 115200 8N1                                |             
    | F - Hardware Flow Control : No                                        |             
    | G - Software Flow Control : No                                        |             
    |                                                                       |             
    |    Change which setting?                                              |             
    +--------------------------------------------------------------------#             
            | Screen and keyboard      |                                                  
            | Save setup as dfl        |                                                  
            | Save setup as..          |                                                  
            | Exit                     |                                                  
            +-----------------------#

Both terminal will remain inactive until the Agilex™ 5 device is configured.

User Flow¶

There are two ways to test the design based on use case.

User Flow 1: Testing with Pre-build Binaries.

User Flow 2: Testing Complete Flow.

User Flow	Description	Required for User Flow 1	Required for User Flow 2
Environment Setup	Tools Download and Installation	Yes	Yes
	Install the dependency packages for software compilation	No	Yes
	Package Download	Yes	Yes
Compilation	HW compilation	No	Yes
	SW compilation	No	Yes
Programming	Programming the SW binary	Yes	Yes
	SI5518A SyncE Clock generator Configuration	Yes	Yes
	Programming the HW binary	Yes	Yes
	Linux Boot	Yes	Yes
Testing	Initial System Configuration	Yes	Yes
	Run Ping test	Yes	Yes
	Run iPerf3 test	Yes	Yes
	Run Packet Generator test	Yes	Yes
	Run ptp4l test	Yes	Yes

Table 13. User Test Flows.

Environment Setup¶

Tools Download and Installation¶

Altera Quartus Prime Pro¶

Download the Quartus® Prime Pro Edition Software Version 25.3 from the FPGA Software Download Center. Follow the on-screen instructions to complete the installation process.

Refer to the Altera® FPGA Software Installation and Licensing for more information on the installation and licensing process.

Set up the Altera® Quartus® tools in the PATH environmental variable.

# Adjust QUARTUS_ROOTDIR target to reflect your Quartus installation path  
export QUARTUS_ROOTDIR=~/altera_pro/25.3/quartus/
export PATH=$QUARTUS_ROOTDIR/bin:$QUARTUS_ROOTDIR/linux64:$QUARTUS_ROOTDIR/../qsys/bin:$PATH

Install dependency packages for SW compilation¶

Arm GNU Toolchain 11.3.Rel1¶

Download the GCC ARM cross-compiler toolchain, add it to the PATH variable, to be used by the GHRD makefile to build the HPS Debug FSBL:

wget https://developer.arm.com/-/media/Files/downloads/gnu/11.3.rel1/binrel/\
arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu.tar.xz
tar xf gcc-arm-11.3.rel1-x86_64-aarch64-none-linux-gnu.tar.xz
rm -f gcc-11.3.rel1-x86_64-aarch64-none-linux-gnu.tar.xz
export PATH=`pwd`/gcc-arm-11.3.rel1-x86_64-aarch64-none-linux-gnu/bin:$PATH
export ARCH=arm64
export CROSS_COMPILE=aarch64-none-linux-gnu-

Yocto Build Prerequisites¶

Before building the Yocto-based Linux image, ensure the host system meets the Yocto system requirements.

The command to install the required packages and set the environment on Ubuntu 22.04-LTS is:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install openssh-server mc libgmp3-dev libmpc-dev gawk wget git diffstat unzip texinfo gcc \
build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \
python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint xterm python3-subunit mesa-common-dev zstd \
liblz4-tool git fakeroot build-essential ncurses-dev xz-utils libssl-dev bc flex libelf-dev bison xinetd \
tftpd tftp nfs-kernel-server libncurses5 libc6-i386 libstdc++6:i386 libgcc++1:i386 lib32z1 \
device-tree-compiler curl mtd-utils u-boot-tools net-tools swig -y
export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
export LC_NUMERIC="en_US.UTF-8"
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8

Bash as Default Command Interpreter¶

On Ubuntu 22.04, set Bash as the system default command interpreter:

sudo ln -sf /bin/bash /bin/sh

Package Download¶

Clone the GitHub repository to obtain the System Example Design source package.

git clone https://github.com/altera-fpga/agilex5-ed-ptp.git
cd agilex5-ed-ptp
git checkout SED-2X10GE_PTP-agilex5_dk_a5e065bb32aes1-Q25.3-Rel1.1
cd a5e065b-prem-devkit-exp-es
export TOP_FOLDER=`pwd`
mkdir bin

Directory Structure Used in This Example Design:

|--- a5e065b-prem-devkit-exp-es
  |   |--- src
  |   |   |--- hw
  |   |   |--- sw

Pre-built binaries are available under the GitHub repository releases. File descriptions are provided in the Binaries section.

Extract all files and copy them to $TOP_FOLDER/bin to run hardware tests on the development kit.

Compilation¶

The following steps outline the build process for both hardware (HW) and software (SW) components.

HW compilation¶

The src/hw/synth directory contains the Quartus project and a Makefile with the following build targets:

make synth - Runs synthesis stage of Altera® Quartus®
make compile - Runs the compile stage of Altera® Quartus®
make all - Runs a full Altera® Quartus® compilation flow

Run the following command to compile the project.

cd $TOP_FOLDER/src/hw/synth/
make all

Alternatively, launch Altera® Quartus® in GUI mode, open top.qpf, and run the compile operation. This generates top.sof at $TOP_FOLDER/src/hw/output_files/.

Build HPS and CORE RBF file¶

The configuration bitstream generated by Altera® Quartus® Prime includes the FPGA core, I/O, and the HPS First-Stage Bootloader (FSBL). After compiling the project, you must integrate your current U-Boot FSBL (u-boot-spl-dtb.hex) into the bitstream.

To embed the .hex file into the bitstream, run the following command:

cd $TOP_FOLDER
quartus_pfg -c -o hps=on -o hps_path=src/sw/artifacts/u-boot-spl-dtb.hex src/hw/synth/output_files/top.sof bin/top.rbf
cp src/hw/synth/output_files/top.sof bin/

The following files are generated:

$TOP_FOLDER/bin/top.hps.rbf - HPS First configuration bitstream, phase 1 (HPS and DDR)
$TOP_FOLDER/bin/top.core.rbf - HPS First configuration bitstream, phase 2 (FPGA fabric)

Build QSPI Image¶

The QSPI image will contain the FPGA configuration data and the HPS FSBL and it can be built using the following command:

cd $TOP_FOLDER 
quartus_pfg -c src/hw/synth/output_files/top.sof \
bin/top.jic \
-o hps_path=src/sw/artifacts/u-boot-spl-dtb.hex \
-o device=MT25QU128 \
-o flash_loader=A5ED065BB32AE6SR0 \
-o mode=ASX4 \
-o hps=1

The following files will be created:

$TOP_FOLDER/bin/top.hps.jic - Flash image for HPS First configuration bitstream, phase 1 (HPS and DDR)
$TOP_FOLDER/bin/top.core.rbf - HPS First configuration bitstream, phase 2 (FPGA fabric, discarded, as we already have it on the SD card)

SW Compilation¶

Build Yocto¶

Start the Yocto build process by executing the following command:

cd $TOP_FOLDER/src/sw/yocto
. agilex5_dk_a5e065bb32aes1-PTP_2P10G-build.sh
build_default

After a successful build, all required images are stored in the $TOP_FOLDER/src/sw/yocto/agilex5_dk_a5e065bb32aes1-gsrd-images directory. Build time varies depending on the host system's resource specifications. Upon successful compilation of the ${{ env_local.ETH_RATE }} system example design, the following files are generated:

$TOP_FOLDER/src/sw/yocto/agilex5_dk_a5e065bb32aes1-gsrd-images/u-boot-agilex5-socdk-gsrd-atf/u-boot-spl-dtb.hex
$TOP_FOLDER/src/sw/yocto/agilex5_dk_a5e065bb32aes1-gsrd-images/u-boot-agilex5-socdk-gsrd-atf/u-boot.itb
$TOP_FOLDER/src/sw/yocto/agilex5_dk_a5e065bb32aes1-gsrd-images/kernel_sed.itb
$TOP_FOLDER/src/sw/yocto/agilex5_dk_a5e065bb32aes1-gsrd-images/sdimage.tar.gz

Copy sdimage.tar.gz and kernel_sed.itb to the bin folder.

cp -rf $TOP_FOLDER/src/sw/yocto/agilex5_dk_a5e065bb32aes1-gsrd-images/sdimage.tar.gz $TOP_FOLDER/bin/sdimage.tar.gz
cp -rf $TOP_FOLDER/src/sw/yocto/agilex5_dk_a5e065bb32aes1-gsrd-images/kernel_sed.itb $TOP_FOLDER/bin/kernel_sed.itb

Yocto Update¶

If the hardware project is modified, the software must be updated to match the new bitstream. The HPS second-stage bootloader embeds a SHA signature of the FPGA bitstream during compilation. Any change to the bitstream alters the SHA, requiring a bootloader update.

To update the FPGA bitstream SHA signature in the HPS second-stage bootloader, follow these steps:

Replace $TOP_FOLDER/src/sw/yocto/meta-agilex5-sed/recipes-bsp/ghrd/files/agilex5_dk_a5e065bb32aes1_gsrd_ghrd_PTP_2P10G.core.rbf with the updated top.core.rbf
Update the recipe at $TOP_FOLDER/src/sw/yocto/meta-agilex5-sed/recipes-bsp/ghrd/hw-ref-design.bb using the following commands

cd $TOP_FOLDER
CORE_RBF=src/sw/yocto/meta-agilex5-sed/recipes-bsp/ghrd/files/agilex5_dk_a5e065bb32aes1_gsrd_ghrd_PTP_2P10G.core.rbf
rm -rf $CORE_RBF
cp -f bin/top.core.rbf $CORE_RBF
FILE=src/sw/yocto/meta-agilex5-sed/recipes-bsp/ghrd/hw-ref-design.bbappend
CORE_SHA=$(sha256sum $CORE_RBF | cut -f1 -d" ") 
OLD_SHA=".*sha256sum_PTP_2P10G.*"
NEW_SHA="sha256sum_PTP_2P10G = \"$CORE_SHA\"" 
sed -i "s/$OLD_SHA/$NEW_SHA/" "$FILE"

After completing the previous step, rebuild the design as described in Build Yocto.

Programming¶

If following User Flow 1, download the Prebuild Binaries. Ensure all steps under Hardware Setup are completed before proceeding.

Programming the SW binary¶

The SD card image file sdimage.tar.gz is provided in the as part of the Prebuild Binaries, you may refer to Release Content for more information.

Follow the instructions under "Write SD Card" in the HPS GSRD User Guide for the Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES) to create two bootable SD cards using the provided image file.

Insert the SD cards into each development kit.

SI5518A SyncE Clock generator Configuration¶

This Design needs 156.25 MHz on OUT2 of Si5518 for the ToD input for better performance. The default clock profile on Agilex™ 5 FPGA E-Series 065B Premium Development Kit outputs 125 MHz on OUT2. The clock profile has been regenerated using ClockBuilder Pro software for the desired output. Also, the clock profile is changed to default holdover mode, and on boot up, the boards need to be programmed as master or slave.

Download the Developement kit Installer packeage from Agilex™ 5 FPGA E-Series 065B Premium FPGA Development Kit Installer Package DK-A5E065BB32AES1.
Setup the Board Test System for Development on the host PC as mentioned in Section 4.1 of Agilex™ 5 FPGA E-Series 065B Premium Development Kit User Guide.
Open clock controller GUI fro SI5518A as mentioned in Section 4.3.2.2 of Agilex™ 5 FPGA E-Series 065B Premium Development Kit User Guide.

You can find the config files path $TOP_FOLDER/src/sw/clk_ic_config/SI5518A_clock_config.zip or you can download the clock configuration files from SI5518A_clock_config.zip and extract the files from unzip command.

$ unzip SI5518A_clock_config.zip
Archive:  SI5518A_clock_config.zip
   creating: SI5518A_clock_config/
inflating: SI5518A_clock_config/Si5518G-Bxxxxx-GM-v0-ISM72E1-Project_TOD156P25MHz.slabtimeproj
inflating: SI5518A_clock_config/Si5518G-Bxxxxx-GM-v0-ISM72E1_TOD156P25MHz-design_report.txt
inflating: SI5518A_clock_config/Si5518G-Bxxxxx-GM-v0-ISM72E1_TOD156P25MHz-user_config.boot.bin
inflating: SI5518A_clock_config/Si5518G-Bxxxxx-GM-v0-ISM72E1_TOD156P25MHzprod_fw_pps.boot.bin

Program the 156.25MHz clock profile using above extracted files by importing them into SI5518 clock controller GUI.
You can save the imported clock settings to flash if you want the board to load the user settings on power-up next time. To do so, follow these steps:
- To import user settings, click the Import button.
- Wait for the successful completion of importing, and press SW15 for 5 seconds. The board saves all the clock settings to flash. The LED D11 blinks once to notify you that it is in saving state. The saving only takes effect tat the next power cycling.
- If you want to restore the factory default clock settings, press SW14 for 5 seconds. The LED D11 blinks 5 times to notify you of its recovering state.

Programming the HW binary¶

Program the onboard Agilex™ 5 device¶

Using Quartus® Programmer Tool Version 25.3 GUI, configure the onboard A5ED065BB32AE6SR0 device with top.hps.rbf.

Alternatively, you can perform this operation via the command line. First, verify that all devices on the development kit are recognized and identify the JTAG cable number using the following command:

/home/user$ jtagconfig
1) Agilex 5E065B Premium DK [1-1.1]
  4BA06477   ARM_CORESIGHT_SOC_600
  0364F0DD   A5E(C065BB32AR0|D065BB32AR0)
  020D10DD   VTAP10

2) Agilex 5E065B Premium DK [1-11.1]
  4BA06477   ARM_CORESIGHT_SOC_600
  0364F0DD   A5E(C065BB32AR0|D065BB32AR0)
  020D10DD   VTAP10

From the jtagconfig output, two Agilex™ 5 FPGA and SoC E-Series Premium Development Kits are detected, with devices identified and assigned to cable 1 and cable 2.

Configure the development kits from your host using the following command:

cd $TOP_FOLDER/bin
# Update the -c parameter to match the JTAG cable numbers assigned to your development kits
# If the Agilex 5 FPGA is in position 1, update the parameter after @ to reflect the correct device index.
quartus_pgm -c 1 -m jtag -o "p;./bin/top.hps.rbf@2" && quartus_pgm -c 2 -m jtag -o "p;./bin/top.hps.rbf@2"

Linux Boot¶

On the HPS UART (Minicom connection), you’ll observe the HPS booting Linux from the SD card. Once booted, log in with username root and no password. The system is now ready for configuration.

If everything is functioning correctly, each Minicom terminal will display boot messages from the HPS running Linux.

agilex5dka5e065bb32aes1 login: root

WARNING: Poky is a reference Yocto Project distribution that should be used for
testing and development purposes only. It is recommended that you create your
own distribution for production use.

root@agilex5dka5e065bb32aes1:~# uname -a
Linux agilex5dka5e065bb32aes1 6.12.19-altera-2x10G-ptp-sed-Q25.3-R1.1 #1 SMP PREEMPT Wed Jan 28 10:13:27 UTC 2026 aarch64 GNU/Linux
root@agilex5dka5e065bb32aes1:~# cat /etc/os-release
ID=poky
NAME="Poky (Yocto Project Reference Distro)"
VERSION="5.0.5 (scarthgap)"
VERSION_ID=5.0.5
VERSION_CODENAME="scarthgap"
PRETTY_NAME="Poky (Yocto Project Reference Distro) 5.0.5 (scarthgap)"
CPE_NAME="cpe:/o:openembedded:poky:5.0.5"
root@agilex5dka5e065bb32aes1:~#

Repeat the same steps for the second Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES).

Clock configuration¶

Below commands will configure the boards on role as given below:

DUT as clock master: /home/root/scripts/si5518config.sh master
DUT as clock slave: /home/root/scripts/si5518config.sh slave
DUT as Boundary Clock: /home/root/scripts/si5518config.sh slave

Scenarios: 1. When link is UP and DUT as clock slave:

root@agilex5dka5e065bb32aes1:~# . ./scripts/si5518config.sh slave
[   65.672101] i2c 0-0059: i2c_si5518_set_holdover: pll: 2, state: 0
[   65.681605] i2c 0-0059: i2c_si5518_select_input: pll: 2, input: 0
[   65.689035] i2c 0-0059: input_status_store: input: 0
./si5518config.sh: line 58: echo: write error: Input/output error
./si5518config.sh: line 67: echo: write error: Input/output error
./si5518config.sh: line 67: echo: write error: Input/output error
./si5518config.sh: line 67: echo: write error: Input/output error
Input set to CDR and PLL is locked... Slave set successful!!

When link is DOWN and DUT as clock slave:

"CDR clock not available and PLL is in holdover... Slave set successful!!"
NOTE: once the link is UP the PLL will be set to locked and can be validated by reading the PLL state using command: “echo "0x2 0x1" > /sys/bus/i2c/devices/0-0059/pll_wait”. This will NOT RETURN ANY ERROR if the pll is locked. It may takes 5~10 seconds for the PLL A to be locked.

3) 10MHz input is available and DUT as master

root@agilex5dka5e065bb32aes1:~# . ./scripts/si5518config.sh master
"10MHz is connected and DSPLLA is locked... Master set successful!!"

10MHz input is not available and DUT as master. DUT is driven by TCXO
```
"10MHz is not connected and DSPLLA in holdover... Master set successful!!"
```
For this test please run below commands in respective development kit.

Execute in Development Kit 1:

/home/root/scripts/si5518config.sh master

Output:

root@agilex5dka5e065bb32aes1:~# /home/root/scripts/si5518config.sh master
Set master
[  492.972261] i2c 0-0059: i2c_si5518_set_holdover: pll: 2, state: 0
[  492.982345] i2c 0-0059: i2c_si5518_select_input: pll: 2, input: 5
[  492.990250] i2c 0-0059: input_status_store: input: 5
/home/root/scripts/si5518config.sh: line 13: echo: write error: Resource temporarily unavailable
[  493.002355] i2c 0-0059: input_status_store: input: 6
/home/root/scripts/si5518config.sh: line 15: echo: write error: Resource temporarily unavailable
10MHz input is invalid
10MHz is not connected and DSPLLA in holdover... Master set successful!!
root@agilex5dka5e065bb32aes1:~#

Execute in Development Kit 2:

/home/root/scripts/si5518config.sh slave

output:

root@agilex5dka5e065bb32aes1:~# /home/root/scripts/si5518config.sh slave
[  497.467281] i2c 0-0059: i2c_si5518_set_holdover: pll: 2, state: 0
[  497.477236] i2c 0-0059: i2c_si5518_select_input: pll: 2, input: 0
[  497.485005] i2c 0-0059: input_status_store: input: 0
CDR clock input is valid
/home/root/scripts/si5518config.sh: line 58: echo: write error: Input/output error
/home/root/scripts/si5518config.sh: line 67: echo: write error: Input/output error
/home/root/scripts/si5518config.sh: line 67: echo: write error: Input/output error
/home/root/scripts/si5518config.sh: line 67: echo: write error: Input/output error
Input set to CDR and PLL is locked... Slave set successful!!
root@agilex5dka5e065bb32aes1:~#

Ethernet Link Status¶

Check the network status on each Agilex™ 5 FPGA and SoC E-Series Premium Development Kit (ES) using the following command:

root@agilex5dka5e065bb32aes1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 62:c6:e6:70:a3:8e brd ff:ff:ff:ff:ff:ff
    inet 10.244.193.4/22 brd 10.244.195.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::60c6:e6ff:fe70:a38e/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
3: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
    link/void
4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
5: eth1: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 76:5f:25:84:cc:0f brd ff:ff:ff:ff:ff:ff
    inet 169.254.42.42/16 brd 169.254.255.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::745f:25ff:fe84:cc0f/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
6: eth2: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 4e:53:68:d4:d6:2f brd ff:ff:ff:ff:ff:ff
    inet 169.254.2.249/16 brd 169.254.255.255 scope global eth2
       valid_lft forever preferred_lft forever
    inet6 fe80::4c53:68ff:fed4:d62f/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
root@agilex5dka5e065bb32aes1:~#

Three Ethernet interfaces will be listed in the output of ip addr:

eth0 : HPS dedicated Ethernet interface (1Gbps)
eth1 : 10G Ethernet Port (10Gbps)
eth2 : 10G Ethernet Port (10Gbps)

Initial System Configuration¶

After booting on both development kits, the System Example design requires initialization of key components: DMA subsystem, User Logic (Packet Generator), Packet Switch, Egress QoS-TC, and Iperf. Two configuration methods are supported.

One-step configuration via script
Step-by-Step configuration of each interface.

Configuration Via Script¶

For manual configuration, skip this section.

The 2PortMCQ.sh script is included in the Yocto rootfs image at /home/root/scripts/. It accepts a single numeric argument specifying the target development kit. Usage:

source ./scripts/2PortMCQ.sh <development kit number>

Valid argument values are 1 or 2.

Run the following command on Development Kit 1:

source ./scripts/2PortMCQ.sh 1

Expected output:

root@agilex5dka5e065bb32aes1:~# source ./scripts/2PortMCQ.sh 1
Programming the Basic IP address...
Clearing old PacketSwitch rules Port - 0...
UIO device file found. Using /dev/uio2
Key Flush successful...
No Filters attached to eth1. Continuing...
Clearing old PacketSwitch rules Port - 1...
UIO device file found. Using /dev/uio2
Key Flush successful...
No Filters attached to eth2. Continuing...
Flushing old IPv4 and IPv6 addresses and routes
Setting DEVKIT to 1.
Running script for Devkit 1.
    link/ether 62:c6:e6:70:a3:8e brd ff:ff:ff:ff:ff:ff
    link/ether 76:5f:25:84:cc:0f brd ff:ff:ff:ff:ff:ff
    link/ether 4e:53:68:d4:d6:2f brd ff:ff:ff:ff:ff:ff
Programming the PacketSwitch Port - 0...
Programming the PacketSwitch Generic rule...
eth1 - 76:5f:25:84:cc:0f
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 0 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - Low priority rules...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 1 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 2 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - IPERF 530X to DMA0...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 3 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 4 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 5 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 6 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - IPERF 520X to DMA1...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 7 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 8 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 9 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 10 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - PTP Packets to DMA0...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 11 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 12 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 13 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 14 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch Port - 1...
Programming the PacketSwitch Generic rule...
eth2 - 4e:53:68:d4:d6:2f
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 0 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - Low priority rules...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 1 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 2 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - IPERF 530X to DMA0...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 3 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 4 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 5 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 6 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - IPERF 520X to DMA1...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 7 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 8 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 9 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 10 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - PTP Packets to DMA0...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 11 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 12 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 13 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 14 Success
Setting Result Register: 0. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - Port 0 User packets to User port...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 15 Success
Setting Result Register: 8. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the PacketSwitch - Port 1 User packets to User port...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 15 Success
Setting Result Register: 8. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the Packet Generator - Port 0
Tx traffic state set: Disabled
Fixed Gap set: Enabled
Packet length mode set: 1
Number of Idle Cycles set: 22
Pkt Checker set: Enabled
One Shot mode set: Disabled
Tx Packet Size set: 1024
Max Tx Packet Size set: 1024
Programming the Packet Generator - Port 1
Tx traffic state set: Disabled
Fixed Gap set: Enabled
Packet length mode set: 1
Number of Idle Cycles set: 22
Pkt Checker set: Enabled
One Shot mode set: Disabled
Tx Packet Size set: 1024
Max Tx Packet Size set: 1024
Programming the IPV6 rules - Port 0
Setting IPv6 local addresses
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 16 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 0 Key index: 17 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Programming the IPV6 rules - Port 1
Setting IPv6 local addresses
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 16 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
UIO device file found. Using /dev/uio2
Setting Entry: Success
Copying Keyfields: Port: 1 Key index: 17 Success
Setting Result Register: 1. Success
Setting Mask Register: Success
Setting Mgmt Cntrl Register: Success
Wait till operation is done: Key Insertion successful...
Traffic Class Egress QOS programming - Port - eth1
Create QDisc...
Create Filters - PTP packets to DMA0...
Create Filters - IPERF 530X packets to DMA0...
Create Filters - IPERF 520X packets to DMA1...
Create Filters - ICMP packets to DMA1...
Traffic Class Egress QOS programming - Port - eth2
Create QDisc...
Create Filters - PTP packets to DMA0...
Create Filters - IPERF 530X packets to DMA0...
Create Filters - IPERF 520X packets to DMA1...
Create Filters - ICMP packets to DMA2...
Configuration for Devkit 1 set
root@agilex5dka5e065bb32aes1:~#

Configure Development Kit 2:

source ./scripts/2PortMCQ.sh 2

Manual Configuration¶

Define Interrupt Core Handling¶

Set up SMP affinity for Ethernet interrupts to distribute handling across different CPUs.

Execute the following commands on both development kits:

echo -e "Programming the Basic IP address..."
echo "4" > /proc/irq/23/smp_affinity && echo "4" > /proc/irq/24/smp_affinity
echo "8" > /proc/irq/25/smp_affinity && echo "8" > /proc/irq/26/smp_affinity
echo "4" > /proc/irq/27/smp_affinity && echo "4" > /proc/irq/28/smp_affinity
echo "8" > /proc/irq/29/smp_affinity && echo "8" > /proc/irq/30/smp_affinity

Clear Old Configurations¶

Clear existing Packet Switch TCAM keys, traffic class (TC) configurations, and any pre-existing IPv6 settings.

Execute the following commands on both development kits:

echo -e "Clearing old PacketSwitch rules Port - 0..."
packetswitch --port 0 --flush-all-keys
echo -e "Clearing old TC rules Port - 0..."
tc filter del dev eth1 egress
tc qdisc del dev eth1 clsact
echo -e "Clearing old PacketSwitch rules Port - 1..."
packetswitch --port 1 --flush-all-keys
echo -e "Clearing old TC rules Port - 1..."
tc filter del dev eth2 egress
tc qdisc del dev eth2 clsact
echo -e "Flushing old IPv4 and IPv6 addresses and routes"
ip addres flush eth1 && ip route flush dev eth1
ip -6 addres flush eth1 && ip -6 route flush dev eth1
ip addres flush eth2 && ip route flush dev eth2
ip -6 addres flush eth2 && ip -6 route flush dev eth2

If no configuration is stored, the commands may return error messages. These errors are safe to ignore.

Set IPv4 Addresses¶

eth1 and eth2 are the Linux interface names assigned to GTS Ethernet Hard IP ports 8 and 9, respectively. Both interfaces must be in the UP state and have IP addresses assigned. Use the following commands to overwrite the IP addresses and bring the interfaces up.

Execute the following commands on development kit 1:

ip link set eth1 up && ip addr add 192.168.121.1 dev eth1 && ip route add 192.168.121.0/24 dev eth1 src 192.168.121.1
ip link set eth2 up && ip addr add 192.168.122.1 dev eth2 && ip route add 192.168.122.0/24 dev eth2 src 192.168.122.1

ip addr | grep ether

Execute the following commands on development kit 2:

ip link set eth1 up && ip addr add 192.168.121.2 dev eth1 && ip route add 192.168.121.0/24 dev eth1 src 192.168.121.2
ip link set eth2 up && ip addr add 192.168.122.2 dev eth2 && ip route add 192.168.122.0/24 dev eth2 src 192.168.122.2

ip addr | grep ether

Packet Switch Rules¶

Generic Traffic Rules¶

The following rules configure the Packet Switch to route ping requests to the lowest priority DMA (DMA-1) on both Ethernet interfaces.

echo -e "Programming the PacketSwitch Port - 0..."
echo -e "Programming the PacketSwitch Generic rule..."
packetswitch --port 0 --set-key --key-index 0 --dest-mac "eth1"  --result 0x1
echo -e "Programming the PacketSwitch - Low priority rules..."
packetswitch --port 0 --set-key --key-index 1 --ethtype 0x0806 --result 0x1
packetswitch --port 0 --set-key --key-index 2 --ethtype 0x0800 --protocol 0x01 --result 0x1


echo -e "Programming the PacketSwitch Port - 1..."
echo -e "Programming the PacketSwitch Generic rule..."
packetswitch --port 1 --set-key --key-index 0 --dest-mac "eth2"  --result 0x1
echo -e "Programming the PacketSwitch - Low priority rules..."
packetswitch --port 1 --set-key --key-index 1 --ethtype 0x0806 --result 0x1
packetswitch --port 1 --set-key --key-index 2 --ethtype 0x0800 --protocol 0x01 --result 0x1

The first rule matches packets with a destination MAC address equal to that of the Ethernet interface. The second rule filters Ethernet frames with Ethertype set to IPv4 and the IPv4 protocol field set to ICMP. The final rule filters frames with Ethertype set to ARP.

iPerf3 Synthetic Traffic Rules¶

iPerf3 is a tool for active measurement of maximum achievable bandwidth on IP networks. It can generate traffic between a client and server, with configurable parameters such as the network port used. The following rules assign a target DMA based on the network port of incoming Ethernet packets.

echo -e "Programming the PacketSwitch Port - 0..."
echo -e "Programming the PacketSwitch - IPERF 530X to DMA0..."
packetswitch --port 0 --set-key --key-index 3 --ethtype 0x0800 --dest-port 5301 --result 0x0
packetswitch --port 0 --set-key --key-index 4 --ethtype 0x0800 --dest-port 5302 --result 0x0
packetswitch --port 0 --set-key --key-index 5 --ethtype 0x0800 --src-port 5301 --result 0x0
packetswitch --port 0 --set-key --key-index 6 --ethtype 0x0800 --src-port 5302 --result 0x0
echo -e "Programming the PacketSwitch - IPERF 520X to DMA1..."
packetswitch --port 0 --set-key --key-index 7 --ethtype 0x0800 --dest-port 5201 --result 0x1
packetswitch --port 0 --set-key --key-index 8 --ethtype 0x0800 --dest-port 5202 --result 0x1
packetswitch --port 0 --set-key --key-index 9 --ethtype 0x0800 --src-port 5201 --result 0x1
packetswitch --port 0 --set-key --key-index 10 --ethtype 0x0800 --src-port 5202 --result 0x1
echo -e "Programming the PacketSwitch - PTP Packets to DMA0..."

echo -e "Programming the PacketSwitch Port - 1..."
echo -e "Programming the PacketSwitch - IPERF 530X to DMA0..."
packetswitch --port 1 --set-key --key-index 3 --ethtype 0x0800 --dest-port 5301 --result 0x0
packetswitch --port 1 --set-key --key-index 4 --ethtype 0x0800 --dest-port 5302 --result 0x0
packetswitch --port 1 --set-key --key-index 5 --ethtype 0x0800 --src-port 5301 --result 0x0
packetswitch --port 1 --set-key --key-index 6 --ethtype 0x0800 --src-port 5302 --result 0x0
echo -e "Programming the PacketSwitch - IPERF 520X to DMA1..."
packetswitch --port 1 --set-key --key-index 7 --ethtype 0x0800 --dest-port 5201 --result 0x1
packetswitch --port 1 --set-key --key-index 8 --ethtype 0x0800 --dest-port 5202 --result 0x1
packetswitch --port 1 --set-key --key-index 9 --ethtype 0x0800 --src-port 5201 --result 0x1
packetswitch --port 1 --set-key --key-index 10 --ethtype 0x0800 --src-port 5202 --result 0x1

The commands above configure the Packet Switch to route all traffic using port 540X to DMA-0, 530X to DMA-1.

PTP Traffic Rules¶

PTP Ethernet packets will be routed to the highest priority DMA (DMA-0) to keep the system time synchronized with the network time. Execute the following commands on both development kits to make both Ethernet interfaces prioritize the PTP traffic:

echo -e "Programming the PacketSwitch Port - 0..."
echo -e "Programming the PacketSwitch - PTP Packets to DMA0..."
packetswitch --port 0 --set-key --key-index 11 --dest-mac "01:80:C2:00:00:0E" --result 0x0
packetswitch --port 0 --set-key --key-index 12 --dest-mac "01:1B:19:00:00:00" --result 0x0
packetswitch --port 0 --set-key --key-index 13 --ethtype 0x88F7 --result 0x0
packetswitch --port 0 --set-key --key-index 14 --ethtype 0x88F8 --result 0x0

echo -e "Programming the PacketSwitch - PTP Packets to DMA0..."
packetswitch --port 1 --set-key --key-index 11 --dest-mac "01:80:C2:00:00:0E" --result 0x0
packetswitch --port 1 --set-key --key-index 12 --dest-mac "01:1B:19:00:00:00" --result 0x0
packetswitch --port 1 --set-key --key-index 13 --ethtype 0x88F7 --result 0x0
packetswitch --port 1 --set-key --key-index 14 --ethtype 0x88F8 --result 0x0

The commands above set rules to filter packets based on the destination address used for PTP broadcast messages and the protocol identifier encapsulated in the EtherType field of the Ethernet frame. The rules are applied to both Ethernet interfaces in this example, but there is an independent scheduler for each Ethernet interface, making it possible to have different rules for each one of them.

Packet Generator Traffic Rules¶

The following set of rules defines the routing to the client ports where the system example design packet generators are connected.

The packetgenerator commands set the source and destination MAC addresses to be used by each packet generator. Then, the packetswitch is used to filter incoming Ethernet frames coming from the opposite development kit and route them to one of the user ports (user ports are represented by '0x8'). There are two additional commands to program the packet generator module with traffic configuration parameters.

Execute the following commands on development kit 1:

echo -e "Programming the PacketSwitch - Port 0 User packets to User port..."
packetgenerator --device /dev/uio0 --dest-mac "12:34:56:78:0A:2" --src-mac "12:34:56:78:0A:1"
packetswitch --set-key --port 0 --key-index 15 --dest-mac "12:34:56:78:0A:1" --result 0x8
echo -e "Programming the PacketSwitch - Port 1 User packets to User port..."
packetgenerator --device /dev/uio1 --dest-mac "12:34:56:78:0A:4" --src-mac "12:34:56:78:0A:3"
packetswitch --set-key --port 1 --key-index 15 --dest-mac "12:34:56:78:0A:3" --result 0x8
echo -e "Programming the Packet Generator - Port 0"
packetgenerator --device /dev/uio0 --traffic false --fixed-gap true --pkt-len-mode 0x01 --num-idle-cycles 22 --packet-checker true --num-packets 0xFFFFFFFF --one-shot false --tx-pkt-size 1024 --tx-max-pkt-size 1024
echo -e "Programming the Packet Generator - Port 1"
packetgenerator --device /dev/uio1 --traffic false --fixed-gap true --pkt-len-mode 0x01 --num-idle-cycles 22 --packet-checker true --num-packets 0xFFFFFFFF --one-shot false --tx-pkt-size 1024 --tx-max-pkt-size 1024

Execute the following commands on development kit 2:

echo -e "Programming the PacketSwitch - Port 0 User packets to User port..."
packetgenerator --device /dev/uio0 --dest-mac "12:34:56:78:0A:1" --src-mac "12:34:56:78:0A:2"
packetswitch --set-key --port 0 --key-index 15 --dest-mac "12:34:56:78:0A:2" --result 0x8
echo -e "Programming the PacketSwitch - Port 1 User packets to User port..."
packetgenerator --device /dev/uio1 --dest-mac "12:34:56:78:0A:3" --src-mac "12:34:56:78:0A:4"
packetswitch --set-key --port 1 --key-index 15 --dest-mac "12:34:56:78:0A:4" --result 0x8
echo -e "Programming the Packet Generator - Port 0"
packetgenerator --device /dev/uio0 --traffic false --fixed-gap true --pkt-len-mode 0x01 --num-idle-cycles 22 --packet-checker true --num-packets 0xFFFFFFFF --one-shot false --tx-pkt-size 1024 --tx-max-pkt-size 1024
echo -e "Programming the Packet Generator - Port 1"
packetgenerator --device /dev/uio1 --traffic false --fixed-gap true --pkt-len-mode 0x01 --num-idle-cycles 22 --packet-checker true --num-packets 0xFFFFFFFF --one-shot false --tx-pkt-size 1024 --tx-max-pkt-size 1024

IPv6 Traffic Rules¶

The next commands set set IPv6 addresses and update the routing table for both ethernet interfaces. Then, the packetswitch rules define that that IPv6 ping requests needs to be channeled to the lowest-priority DMA (DMA-1) on both Ethernet interfaces.

Execute the following commands on development kit 1:

echo -e "Programming the IPV6 rules - Port 0"
echo -e "Setting IPv6 local addresses"
ip -6 addr add 2001:db8:abcd:0012::1/64 dev eth1 && ip link set dev eth1 up
sleep 2
ip -6 route add 2001:db8:abcd:0012::1/64 dev eth1 src 2001:db8:abcd:0012::1

packetswitch --port 0 --set-key --key-index 16 --ethtype 0x86DD --result 0x1
packetswitch --port 0 --set-key --key-index 17 --ethtype 0x86DD --protocol 0x3A  --result 0x1

echo -e "Programming the IPV6 rules - Port 1"
echo -e "Setting IPv6 local addresses"

ip -6 addr add 2001:db8:abcd:0013::1/64 dev eth2 && ip link set dev eth2 up
sleep 2
ip -6 route add 2001:db8:abcd:0013::1/64 dev eth2 src 2001:db8:abcd:0013::1

packetswitch --port 1 --set-key --key-index 16 --ethtype 0x86DD --result 0x1
packetswitch --port 1 --set-key --key-index 17 --ethtype 0x86DD --protocol 0x3A  --result 0x1

Execute the following commands on development kit 2:

echo -e "Programming the IPV6 rules - Port 0"
echo -e "Setting IPv6 local addresses"
ip -6 addr add 2001:db8:abcd:0012::2/64 dev eth1 && ip link set dev eth1 up
sleep 2
ip -6 route add 2001:db8:abcd:0012::2/64 dev eth1 src 2001:db8:abcd:0012::2

packetswitch --port 0 --set-key --key-index 16 --ethtype 0x86DD --result 0x1
packetswitch --port 0 --set-key --key-index 17 --ethtype 0x86DD --protocol 0x3A  --result 0x1

echo -e "Programming the IPV6 rules - Port 1"
echo -e "Setting IPv6 local addresses"

ip -6 addr add 2001:db8:abcd:0013::2/64 dev eth2 && ip link set dev eth2 up
sleep 2
ip -6 route add 2001:db8:abcd:0013::2/64 dev eth2 src 2001:db8:abcd:0013::2

packetswitch --port 1 --set-key --key-index 16 --ethtype 0x86DD --result 0x1
packetswitch --port 1 --set-key --key-index 17 --ethtype 0x86DD --protocol 0x3A  --result 0x1

Traffic Classes¶

Egress QoS is implemented using the Linux TC (Traffic Control) subsystem alongside the network stack.

The next steps define a qdisc-based TC configuration, which can be paired with filters to route egress packets to specific DMA paths. Packet routing is determined by the skb->priority field, which must be set according to test requirements. Execute the following commands on both development kits.

The tc filters define:

tc filters route traffic as follows:

PTP and iPerf3 on port 540X → DMA 0
iPerf3 on port 530X → DMA 1

echo -e "Traffic Class Egress QOS programming - Port - eth1"
echo -e "Create QDisc..."
tc qdisc add dev eth1 clsact
echo -e "Create Filters - PTP packets to DMA0..."
MAC1_ADDR="01:80:C2:00:00:0E"
MAC1_HEX=$(echo $MAC1_ADDR | sed 's/://g' | tr 'a-f' 'A-F')
MAC2_ADDR="01:1B:19:00:00:00"
MAC2_HEX=$(echo $MAC2_ADDR | sed 's/://g' | tr 'a-f' 'A-F')
tc filter add dev eth1 egress prio 0 u32 match ip dport 319 0xffff match ip protocol 17 0xff action skbedit priority 0
tc filter add dev eth1 egress prio 0 u32 match ip dport 320 0xffff match ip protocol 17 0xff action skbedit priority 0
tc filter add dev eth1 egress prio 0 u32 match u16 0x${MAC1_HEX:0:4} 0xFFFF at -14 match u32 0x${MAC1_HEX:4:8} 0xFFFFFFFF at -12 action skbedit priority 0
tc filter add dev eth1 egress prio 0 u32 match u16 0x${MAC2_HEX:0:4} 0xFFFF at -14 match u32 0x${MAC2_HEX:4:8} 0xFFFFFFFF at -12 action skbedit priority 0

echo -e "Create Filters - IPERF 530X packets to DMA0..."
tc filter add dev eth1 egress prio 0 u32 match ip dport 5301 0xffff match ip protocol 6 0xff action skbedit priority 0
tc filter add dev eth1 egress prio 0 u32 match ip sport 5301 0xffff match ip protocol 6 0xff action skbedit priority 0
echo -e "Create Filters - IPERF 520X packets to DMA1..."
tc filter add dev eth1 egress prio 0 u32 match ip dport 5201 0xffff match ip protocol 6 0xff action skbedit priority 1
tc filter add dev eth1 egress prio 0 u32 match ip sport 5201 0xffff match ip protocol 6 0xff action skbedit priority 1
echo -e "Create Filters - ICMP packets to DMA1..."
tc filter add dev eth1 egress prio 0 u32 match ip protocol 1 0xff action skbedit priority 1

echo -e "Traffic Class Egress QOS programming - Port - eth2"
echo -e "Create QDisc..."
tc qdisc add dev eth2 clsact
echo -e "Create Filters - PTP packets to DMA0..."
tc filter add dev eth2 egress prio 0 u32 match ip dport 319 0xffff match ip protocol 17 0xff action skbedit priority 0
tc filter add dev eth2 egress prio 0 u32 match ip dport 320 0xffff match ip protocol 17 0xff action skbedit priority 0
tc filter add dev eth2 egress prio 0 u32 match u16 0x${MAC1_HEX:0:4} 0xFFFF at -14 match u32 0x${MAC1_HEX:4:8} 0xFFFFFFFF at -12 action skbedit priority 0
tc filter add dev eth2 egress prio 0 u32 match u16 0x${MAC2_HEX:0:4} 0xFFFF at -14 match u32 0x${MAC2_HEX:4:8} 0xFFFFFFFF at -12 action skbedit priority 0

echo -e "Create Filters - IPERF 530X packets to DMA0..."
tc filter add dev eth2 egress prio 0 u32 match ip dport 5301 0xffff match ip protocol 6 0xff action skbedit priority 0
tc filter add dev eth2 egress prio 0 u32 match ip sport 5301 0xffff match ip protocol 6 0xff action skbedit priority 0
echo -e "Create Filters - IPERF 520X packets to DMA1..."
tc filter add dev eth2 egress prio 0 u32 match ip dport 5201 0xffff match ip protocol 6 0xff action skbedit priority 1
tc filter add dev eth2 egress prio 0 u32 match ip sport 5201 0xffff match ip protocol 6 0xff action skbedit priority 1
echo -e "Create Filters - ICMP packets to DMA2..."
tc filter add dev eth2 egress prio 0 u32 match ip protocol 1 0xff action skbedit priority 1
echo -e "Configuration for Devkit $DEVKIT set"

Ethernet Connectivity Test¶

After finishing the Initial System Configuration, verify the network connectivity between both development kits with the next steps.

Confirm both Ethernet interfaces are up in both development kits:

Execute the following commands on development kit 1:

root@agilex5dka5e065bb32aes1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 62:c6:e6:70:a3:8e brd ff:ff:ff:ff:ff:ff
    inet 10.244.193.4/22 metric 10 brd 10.244.195.255 scope global dynamic eth0
       valid_lft 14121sec preferred_lft 14121sec
    inet6 fe80::60c6:e6ff:fe70:a38e/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
3: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
    link/void
4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
5: eth1: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 76:5f:25:84:cc:0f brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.1/32 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 2001:db8:abcd:12::1/64 scope global
       valid_lft forever preferred_lft forever
6: eth2: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 4e:53:68:d4:d6:2f brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/32 scope global eth2
       valid_lft forever preferred_lft forever
    inet6 2001:db8:abcd:13::1/64 scope global
       valid_lft forever preferred_lft forever

Execute the following commands on development kit 2:

root@agilex5dka5e065bb32aes1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether c2:f1:56:21:c2:08 brd ff:ff:ff:ff:ff:ff
    inet 10.244.193.5/22 metric 10 brd 10.244.195.255 scope global dynamic eth0
       valid_lft 14102sec preferred_lft 14102sec
    inet6 fe80::c0f1:56ff:fe21:c208/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
3: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
    link/void
4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
5: eth1: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 62:a9:98:4a:12:e4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.2/32 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 2001:db8:abcd:12::2/64 scope global
       valid_lft forever preferred_lft forever
6: eth2: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 5a:9f:8b:a8:06:09 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.2/32 scope global eth2
       valid_lft forever preferred_lft forever
    inet6 2001:db8:abcd:13::2/64 scope global
       valid_lft forever preferred_lft forever

Verify connectivity between development kits using ping:

Execute the following commands on development kit 1:

root@agilex5dka5e065bb32aes1:~# ping -c 3 192.168.121.2
PING 192.168.121.2 (192.168.121.2): 56 data bytes
64 bytes from 192.168.121.2: seq=0 ttl=64 time=1.104 ms
64 bytes from 192.168.121.2: seq=1 ttl=64 time=0.271 ms
64 bytes from 192.168.121.2: seq=2 ttl=64 time=0.236 ms

--- 192.168.121.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.236/0.537/1.104 ms

Execute the following commands on development kit 2:

root@agilex5dka5e065bb32aes1:~# ping -c 3 192.168.121.1
PING 192.168.121.1 (192.168.121.1): 56 data bytes
64 bytes from 192.168.121.1: seq=0 ttl=64 time=0.551 ms
64 bytes from 192.168.121.1: seq=1 ttl=64 time=0.232 ms
64 bytes from 192.168.121.1: seq=2 ttl=64 time=0.251 ms

--- 192.168.121.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.232/0.344/0.551 ms
root@agilex5dka5e065bb32aes1:~#

Testing¶

Before running a test, ensure the hardware setup matches the Hardware Setup section, and you have executed the steps listed in Initial System Configuration.

Run Ping Test¶

The transcript below shows a ping test from development kit 1 serial session. 100,000 pings are sent to development kit 2. DMA-1 handles egress traffic, verified by polling its interrupt count. As shown, all ping requests were correctly filtered to DMA-1.

Execute the following commands on development kit 1:

root@agilex5dka5e065bb32aes1:~# ping -i 0.0001 -q -c 100000 -I eth1 192.168.121.2
PING 192.168.121.2 (192.168.121.2): 56 data bytes

--- 192.168.121.2 ping statistics ---
100000 packets transmitted, 100000 packets received, 0% packet loss
round-trip min/avg/max = 0.063/0.095/4.156 ms
root@agilex5dka5e065bb32aes1:~# cat /proc/interrupts | grep eth1
 23:         92          0        151          0     GICv3  49 Level     eth1
 24:          0          0          0          0     GICv3  50 Level     eth1
 25:         11          0          0     100009     GICv3  51 Level     eth1
 26:          0          0          0      99989     GICv3  52 Level     eth1
root@agilex5dka5e065bb32aes1:~#

Execute the following commands on development kit 2:

root@agilex5dka5e065bb32aes1:~# ping -i 0.0001 -q -c 100000 -I eth1 192.168.121.1
PING 192.168.121.1 (192.168.121.1): 56 data bytes

--- 192.168.121.1 ping statistics ---
100000 packets transmitted, 100000 packets received, 0% packet loss
round-trip min/avg/max = 0.064/0.098/4.172 ms
root@agilex5dka5e065bb32aes1:~# cat /proc/interrupts | grep eth1
 23:        111          0        149          0     GICv3  49 Level     eth1
 24:          0          0          0          0     GICv3  50 Level     eth1
 25:         11          0          0     200000     GICv3  51 Level     eth1
 26:          0          0          0     199987     GICv3  52 Level     eth1
root@agilex5dka5e065bb32aes1:~#

Run iPerf3 Test¶

The iPerf3 test uses development kit 2 as the server and development kit 1 as the client. Start the iPerf3 server on kit 2 with parallel instances listening on ports 5201 and 5301.

Execute the following commands on development kit 2:

iperf3 -D -s -B 192.168.121.2 -p 5201 > /var/log/iperf.eth1.1 2>&1 &
iperf3 -D -s -B 192.168.121.2 -p 5301 > /var/log/iperf.eth1.2 2>&1 &

iPerf traffic to DMA 0¶

The transcript below shows an iPerf3 test from development kit 1, targeting port 5301 on the server and using port 5302 locally. DMA-0 usage (highest priority) is confirmed by the interrupt count on its associated queue.

Execute the following commands on development kit 1:

root@agilex5dka5e065bb32aes1:~# iperf3 -M 1460 -c 192.168.121.2 -t 80000 -p 5301 --cport 5302
Connecting to host 192.168.121.2, port 5301
[  5] local 192.168.121.1 port 5302 connected to 192.168.121.2 port 5301
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  63.5 MBytes   532 Mbits/sec  479   25.5 KBytes
[  5]   1.00-2.00   sec  59.5 MBytes   499 Mbits/sec  393   63.6 KBytes
[  5]   2.00-3.00   sec  61.6 MBytes   517 Mbits/sec  433   36.8 KBytes
[  5]   3.00-4.00   sec  62.0 MBytes   520 Mbits/sec  534   36.8 KBytes
[  5]   4.00-5.00   sec  61.9 MBytes   519 Mbits/sec  386   28.3 KBytes
[  5]   5.00-6.00   sec  62.6 MBytes   526 Mbits/sec  482   55.1 KBytes
[  5]   6.00-7.00   sec  60.5 MBytes   508 Mbits/sec  433   36.8 KBytes
[  5]   7.00-8.00   sec  61.2 MBytes   514 Mbits/sec  430   25.5 KBytes
[  5]   8.00-9.00   sec  59.2 MBytes   497 Mbits/sec  470   19.8 KBytes
[  5]   9.00-10.00  sec  62.0 MBytes   520 Mbits/sec  452   38.2 KBytes
[  5]  10.00-11.00  sec  61.4 MBytes   515 Mbits/sec  438   25.5 KBytes
[  5]  11.00-12.00  sec  60.8 MBytes   509 Mbits/sec  488   28.3 KBytes
[  5]  12.00-13.00  sec  62.8 MBytes   527 Mbits/sec  471   29.7 KBytes
[  5]  13.00-14.00  sec  60.6 MBytes   508 Mbits/sec  414   32.5 KBytes
[  5]  14.00-15.00  sec  61.8 MBytes   518 Mbits/sec  454   31.1 KBytes
[  5]  15.00-16.00  sec  61.4 MBytes   515 Mbits/sec  435   28.3 KBytes
[  5]  16.00-17.00  sec  62.5 MBytes   524 Mbits/sec  405   36.8 KBytes
[  5]  17.00-18.00  sec  58.6 MBytes   492 Mbits/sec  434   21.2 KBytes
[  5]  18.00-19.00  sec  61.1 MBytes   513 Mbits/sec  471   29.7 KBytes
^C[  5]  19.00-19.18  sec  11.5 MBytes   540 Mbits/sec   72   35.4 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-19.18  sec  1.15 GBytes   515 Mbits/sec  8574            sender
[  5]   0.00-19.18  sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: interrupt - the client has terminated
root@agilex5dka5e065bb32aes1:~#

iPerf traffic to DMA 1¶

The transcript below shows an iPerf3 test from development kit 1, targeting port 5201 on the server and using port 5202 locally. DMA-1 usage is confirmed by the interrupt count on its associated queue.

root@agilex5dka5e065bb32aes1:~# iperf3 -M 1460 -c 192.168.121.2 -t 80000 -p 5201 --cport 5202
Connecting to host 192.168.121.2, port 5201
[  5] local 192.168.121.1 port 5202 connected to 192.168.121.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  66.8 MBytes   559 Mbits/sec  386   80.6 KBytes
[  5]   1.00-2.00   sec  66.5 MBytes   558 Mbits/sec  262   76.4 KBytes
[  5]   2.00-3.00   sec  64.4 MBytes   540 Mbits/sec  312    115 KBytes
[  5]   3.00-4.00   sec  66.8 MBytes   560 Mbits/sec  245   76.4 KBytes
[  5]   4.00-5.00   sec  66.2 MBytes   556 Mbits/sec  419    100 KBytes
[  5]   5.00-6.00   sec  65.5 MBytes   549 Mbits/sec  246   97.6 KBytes
[  5]   6.00-7.00   sec  66.2 MBytes   556 Mbits/sec  248    140 KBytes
[  5]   7.00-8.00   sec  66.8 MBytes   560 Mbits/sec  365   67.9 KBytes
[  5]   8.00-9.00   sec  66.0 MBytes   554 Mbits/sec  324   70.7 KBytes
[  5]   9.00-10.00  sec  65.1 MBytes   546 Mbits/sec  275    133 KBytes
[  5]  10.00-11.00  sec  65.9 MBytes   553 Mbits/sec  277   79.2 KBytes
[  5]  11.00-12.00  sec  66.0 MBytes   554 Mbits/sec  207    100 KBytes
[  5]  12.00-13.00  sec  65.9 MBytes   553 Mbits/sec  268   73.5 KBytes
[  5]  13.00-14.00  sec  66.5 MBytes   558 Mbits/sec  217   73.5 KBytes
[  5]  14.00-15.00  sec  66.1 MBytes   555 Mbits/sec  264   97.6 KBytes
[  5]  15.00-16.00  sec  65.1 MBytes   546 Mbits/sec  296    115 KBytes
[  5]  16.00-17.00  sec  65.4 MBytes   548 Mbits/sec  306   97.6 KBytes
[  5]  17.00-18.00  sec  65.2 MBytes   547 Mbits/sec  223   62.2 KBytes
^C[  5]  18.00-18.61  sec  39.4 MBytes   541 Mbits/sec   78    153 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-18.61  sec  1.20 GBytes   553 Mbits/sec  5218            sender
[  5]   0.00-18.61  sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: interrupt - the client has terminated
root@agilex5dka5e065bb32aes1:~# cat /proc/interrupts | grep eth1
 23:         92          0     431644          0     GICv3  49 Level     eth1
 24:          0          0      79375          0     GICv3  50 Level     eth1
 25:         11          0          0     589407     GICv3  51 Level     eth1
 26:          0          0          0     264472     GICv3  52 Level     eth1
root@agilex5dka5e065bb32aes1:~#

Run Packet Generator Test¶

System packet generators operate independently of the HPS and DMA engines. Their datapath traverses the Packet Switch and can saturate the 10GbE ports. No tc filters apply, as they are not HPS-connected and not software-controlled. Bandwidth allocation is managed solely by the Packet Switch arbiter.

Test execution involves configuring traffic parameters for the generators. The transcript below shows initial setup and traffic start. The --dump flag captures packet generator status registers, indicating TX bandwidth utilization of ~9.16 Gbps.

Execute the following commands on development kit 1:

root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --traffic true --fixed-gap true --pkt-len-mode 0x01 --num-idle-cycles 8 --packet-checker true  --one-shot false --tx-pkt-size 512 --tx-max-pkt-size 512
Tx traffic state set: Enabled
Dynamic Packet Mode set: Enabled
Fixed Gap set: Enabled
Packet length mode set: 1
Number of Idle Cycles set: 8
Pkt Checker set: Enabled
One Shot mode set: Disabled
Tx Packet Size set: 512
Max Tx Packet Size set: 512
root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --dump
Config Control: 0x8635
        Tx traffic: Enabled
        Packet Generation Mode: Continuous
        Soft Reset: Disabled
        Dynamic Mode: Enabled
        Pkt Checker: Enabled
        Counter Snapshot Status: Disabled
        Counter Clear Status: Disabled
        Internal Counter Clear Status: Disabled
        Fixed Gap: Enabled
        Packet Length Mode: Fixed
        Number of Idle Cycles: 8
Destination Mac Address: 12:34:56:78:0A:02
Source Mac Address: 12:34:56:78:0A:01
Number of Packets: 4294967295
Packet Size Config Control: 0x2000200
        Tx Packet Size: 512 Tx Max Packet Size: 512
Packet Generator Status: 0x1c
        SADB configuration status: Incomplete
        System Reset Sequence status: Incomplete
        HSSI SS tx_lanes_stable status: Asserted
        HSSI SS tx_pll_locked status: Asserted
        HSSI SS rx_pcs status: Asserted
Packet Checker Status: 0x0
        Data Mismatch status: Not seen
TX Start of Packet Count: 30221048
TX End of Packet Count: 30221247
TX Error Packet Count: 0
RX Start of Packet Count: 0
RX End of Packet Count: 0
RX Error Packet Count: 0
Pkt Checker Live Counter: 0
PKT TX Byte Count: 15473803208
PKT RX Byte Count: 0
PKT TX Num Ticks Count: 1934248083
PKT RX Num Ticks Count: 0
TX Bandwidth: 9166666496 bps
RX Bandwidth: 0 bps
root@agilex5dka5e065bb32aes1:~#

TX bandwidth utilization can be tuned by adjusting packet length and idle cycles. The transcript below modifies the number of idle cycles between packets in flight. The change is verified via a read of the packet generator status registers, which shows a maximum bandwidth of ~9.16 Gbps.

root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --num-idle-cycles 16 --tx-pkt-size 1024 --tx-max-pkt-size 1024
Number of Idle Cycles set: 16
Tx Packet Size set: 1024
Max Tx Packet Size set: 1024
root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --dump
Config Control: 0x10635
        Tx traffic: Enabled
        Packet Generation Mode: Continuous
        Soft Reset: Disabled
        Dynamic Mode: Enabled
        Pkt Checker: Enabled
        Counter Snapshot Status: Disabled
        Counter Clear Status: Disabled
        Internal Counter Clear Status: Disabled
        Fixed Gap: Enabled
        Packet Length Mode: Fixed
        Number of Idle Cycles: 16
Destination Mac Address: 12:34:56:78:0A:02
Source Mac Address: 12:34:56:78:0A:01
Number of Packets: 4294967295
Packet Size Config Control: 0x4000400
        Tx Packet Size: 1024 Tx Max Packet Size: 1024
Packet Generator Status: 0x1c
        SADB configuration status: Incomplete
        System Reset Sequence status: Incomplete
        HSSI SS tx_lanes_stable status: Asserted
        HSSI SS tx_pll_locked status: Asserted
        HSSI SS rx_pcs status: Asserted
Packet Checker Status: 0x0
        Data Mismatch status: Not seen
TX Start of Packet Count: 249008332
TX End of Packet Count: 249008430
TX Error Packet Count: 0
RX Start of Packet Count: 0
RX End of Packet Count: 0
RX Error Packet Count: 0
Pkt Checker Live Counter: 0
PKT TX Byte Count: 140296025664
PKT RX Byte Count: 0
PKT TX Num Ticks Count: 17537026176
PKT RX Num Ticks Count: 0
TX Bandwidth: 9166667392 bps
RX Bandwidth: 0 bps
root@agilex5dka5e065bb32aes1:~#

Enabling the packet generator on the second development kit starts the integrated packet checker and reports RX bandwidth. The transcript below shows the status change after activation.

Execute the following commands on development kit 2:

root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --num-idle-cycles 16 --tx-pkt-size 1024 --tx-max-pkt-size 1024
Number of Idle Cycles set: 16
Tx Packet Size set: 1024
Max Tx Packet Size set: 1024
root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --dump
Config Control: 0x10635
        Tx traffic: Enabled
        Packet Generation Mode: Continuous
        Soft Reset: Disabled
        Dynamic Mode: Enabled
        Pkt Checker: Enabled
        Counter Snapshot Status: Disabled
        Counter Clear Status: Disabled
        Internal Counter Clear Status: Disabled
        Fixed Gap: Enabled
        Packet Length Mode: Fixed
        Number of Idle Cycles: 16
Destination Mac Address: 12:34:56:78:0A:01
Source Mac Address: 12:34:56:78:0A:02
Number of Packets: 4294967295
Packet Size Config Control: 0x4000400
        Tx Packet Size: 1024 Tx Max Packet Size: 1024
Packet Generator Status: 0x1c
        SADB configuration status: Incomplete
        System Reset Sequence status: Incomplete
        HSSI SS tx_lanes_stable status: Asserted
        HSSI SS tx_pll_locked status: Asserted
        HSSI SS rx_pcs status: Asserted
Packet Checker Status: 0x0
        Data Mismatch status: Not seen
TX Start of Packet Count: 66484091
TX End of Packet Count: 66484164
TX Error Packet Count: 0
RX Start of Packet Count: 313858673
RX End of Packet Count: 313858741
RX Error Packet Count: 0
Pkt Checker Live Counter: 313858814
PKT TX Byte Count: 36689010400
PKT RX Byte Count: 286215113032
PKT TX Num Ticks Count: 4586143716
PKT RX Num Ticks Count: 40249032197
TX Bandwidth: 9166666496 bps
RX Bandwidth: 9166665088 bps
Number of words: 1
root@agilex5dka5e065bb32aes1:~#

RX bandwidth is reported to be ~9.166 Gbps.

To fully saturate an Ethernet port, run the following commands on both development kits to enable their respective packet generators:

packetgenerator --device /dev/uio0 --num-idle-cycles 16 --tx-pkt-size 1024 --tx-max-pkt-size 1024
packetgenerator --device /dev/uio0 --traffic 1

Both development kits are now transmitting and receiving Ethernet traffic on port 1. Run a status dump on either kit to report bandwidth utilization:

root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --num-idle-cycles 16 --tx-pkt-size 1024 --tx-max-pkt-size 1024
Number of Idle Cycles set: 16
Tx Packet Size set: 1024
Max Tx Packet Size set: 1024
root@agilex5dka5e065bb32aes1:~# packetgenerator --device /dev/uio0 --dump
Config Control: 0x10635
        Tx traffic: Enabled
        Packet Generation Mode: Continuous
        Soft Reset: Disabled
        Dynamic Mode: Enabled
        Pkt Checker: Enabled
        Counter Snapshot Status: Disabled
        Counter Clear Status: Disabled
        Internal Counter Clear Status: Disabled
        Fixed Gap: Enabled
        Packet Length Mode: Fixed
        Number of Idle Cycles: 16
Destination Mac Address: 12:34:56:78:0A:01
Source Mac Address: 12:34:56:78:0A:02
Number of Packets: 4294967295
Packet Size Config Control: 0x4000400
        Tx Packet Size: 1024 Tx Max Packet Size: 1024
Packet Generator Status: 0x1c
        SADB configuration status: Incomplete
        System Reset Sequence status: Incomplete
        HSSI SS tx_lanes_stable status: Asserted
        HSSI SS tx_pll_locked status: Asserted
        HSSI SS rx_pcs status: Asserted
Packet Checker Status: 0x0
        Data Mismatch status: Not seen
TX Start of Packet Count: 66484091
TX End of Packet Count: 66484164
TX Error Packet Count: 0
RX Start of Packet Count: 313858673
RX End of Packet Count: 313858741
RX Error Packet Count: 0
Pkt Checker Live Counter: 313858814
PKT TX Byte Count: 36689010400
PKT RX Byte Count: 286215113032
PKT TX Num Ticks Count: 4586143716
PKT RX Num Ticks Count: 40249032197
TX Bandwidth: 9166666496 bps
RX Bandwidth: 9166665088 bps
Number of words: 1

Both TX and RX channels are now active.

Run ptp4l Test¶

Development kit 1 acts as the network master; development kit 2 is the subordinate. Both are configured as ordinary clocks using ptp4l. Configuration files provided by the system example design are located at /root/cfg/.

root@agilex5dka5e065bb32aes1:~# ls /root/cfg/
boundary.cfg
master.cfg
slave.cfg

The transcript below configures development kit 1 as the network master.

Execute the following commands on development kit 1:

root@agilex5dka5e065bb32aes1:~# ptp4l -i eth1 -m -f /root/cfg/master.cfg
option slaveOnly is deprecated, please use clientOnly instead
option masterOnly is deprecated, please use serverOnly instead
ptp4l[1057.974]: selected /dev/ptp0 as PTP clock
ptp4l[1058.040]: port 1 (eth1): INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[1058.040]: port 0 (/var/run/ptp4l): INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[1058.040]: port 0 (/var/run/ptp4lro): INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[1058.513]: port 1 (eth1): LISTENING to MASTER on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
ptp4l[1058.514]: selected local clock 5e7ab9.fffe.7757c2 as best master
ptp4l[1058.514]: port 1 (eth1): assuming the grand master role

Development kit 2 loads slave.cfg to operate as the network slave.

Execute the following commands on development kit 2:

root@agilex5dka5e065bb32aes1:~# ptp4l -i eth1 -m -f /root/cfg/slave.cfg
option slaveOnly is deprecated, please use clientOnly instead
option masterOnly is deprecated, please use serverOnly instead
ptp4l[1067.085]: selected /dev/ptp0 as PTP clock
ptp4l[1067.128]: port 1 (eth1): INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[1067.128]: port 0 (/var/run/ptp4l): INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[1067.129]: port 0 (/var/run/ptp4lro): INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[1067.229]: port 1 (eth1): new foreign master 5e7ab9.fffe.7757c2-1
ptp4l[1067.479]: selected best master clock 5e7ab9.fffe.7757c2
ptp4l[1067.479]: port 1 (eth1): LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[1067.542]: master offset -2955285083 s0 freq      -0 path delay         0
ptp4l[1067.604]: master offset -2955285079 s0 freq      -0 path delay         0
ptp4l[1067.667]: master offset -2955285070 s0 freq      -0 path delay         8
ptp4l[1067.729]: master offset -2955285062 s0 freq      -0 path delay         8
ptp4l[1067.792]: master offset -2955285053 s0 freq      -0 path delay         7
ptp4l[1067.854]: master offset -2955285050 s0 freq      -0 path delay         7
ptp4l[1067.917]: master offset -2955285040 s0 freq      -0 path delay         7
ptp4l[1067.979]: master offset -2955285036 s0 freq      -0 path delay         7
ptp4l[1068.042]: master offset -2955285025 s0 freq      -0 path delay         8
ptp4l[1068.104]: master offset -2955285019 s0 freq      -0 path delay        10
ptp4l[1068.167]: master offset -2955285010 s0 freq      -0 path delay         9
ptp4l[1068.229]: master offset -2955284999 s0 freq      -0 path delay         6
ptp4l[1068.292]: master offset -2955284992 s0 freq      -0 path delay         8
ptp4l[1068.354]: master offset -2955284982 s0 freq      -0 path delay         6
ptp4l[1068.417]: master offset -2955284972 s0 freq      -0 path delay         6
ptp4l[1068.479]: master offset -2955284967 s0 freq      -0 path delay         6
ptp4l[1068.542]: master offset -2955284956 s0 freq      -0 path delay         8
ptp4l[1068.604]: master offset -2955284944 s0 freq      -0 path delay         6
ptp4l[1068.667]: master offset -2955284935 s0 freq      -0 path delay         6
ptp4l[1068.729]: master offset -2955284930 s0 freq      -0 path delay         6
ptp4l[1068.792]: master offset -2955284917 s0 freq      -0 path delay         7
ptp4l[1068.854]: master offset -2955284907 s0 freq      -0 path delay         5
ptp4l[1068.917]: master offset -2955284897 s0 freq      -0 path delay         5
ptp4l[1068.979]: master offset -2955284893 s0 freq      -0 path delay         5
ptp4l[1069.042]: master offset -2955284881 s0 freq      -0 path delay         7
ptp4l[1069.105]: master offset -2955284874 s0 freq      -0 path delay         9
ptp4l[1069.167]: master offset -2955284861 s0 freq      -0 path delay         5
ptp4l[1069.230]: master offset -2955284857 s0 freq      -0 path delay         5
ptp4l[1069.292]: master offset -2955284845 s0 freq      -0 path delay         7
ptp4l[1069.355]: master offset -2955284835 s0 freq      -0 path delay         7
ptp4l[1069.417]: master offset -2955284827 s0 freq      -0 path delay         9
ptp4l[1069.480]: master offset -2955284815 s0 freq      -0 path delay         7
ptp4l[1069.542]: master offset -2955284803 s0 freq      -0 path delay         6
ptp4l[1069.605]: master offset -2955284791 s0 freq      -0 path delay         5
ptp4l[1069.667]: master offset -2955284786 s0 freq      -0 path delay         5

<-- output truncated -->

ptp4l[3269.964]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.027]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.089]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.152]: master offset          0 s2 freq    +146 path delay         9
ptp4l[3270.214]: master offset          0 s2 freq    +146 path delay         9
ptp4l[3270.277]: master offset          1 s2 freq    +146 path delay         9
ptp4l[3270.339]: master offset          1 s2 freq    +146 path delay         9
ptp4l[3270.402]: master offset          1 s2 freq    +146 path delay         9
ptp4l[3270.464]: master offset          0 s2 freq    +146 path delay         9
ptp4l[3270.527]: master offset          0 s2 freq    +146 path delay         9
ptp4l[3270.589]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.652]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.714]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.777]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.839]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.902]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3270.964]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3271.027]: master offset         -1 s2 freq    +145 path delay         9
ptp4l[3271.089]: master offset         -1 s2 freq    +145 path delay        10

To verify that both systems use DMA-0 for PTP traffic, inspect /proc/interrupts and confirm that the highest-priority interrupts are triggered for PTP TX/RX handling.

Execute the following commands on development kit 1:

root@agilex5dka5e065bb32aes1:~# cat /proc/interrupts | grep eth1
 23:         23          0       4787          0     GICv3  49 Level     eth1
 24:          0          0        980          0     GICv3  50 Level     eth1
 25:          9          0          0          7     GICv3  51 Level     eth1
 26:          0          0          0         12     GICv3  52 Level     eth1
root@agilex5dka5e065bb32aes1:~#

Execute the following commands on development kit 2:

root@agilex5dka5e065bb32aes1:~# cat /proc/interrupts | grep eth1
 23:         34          0       1019          0     GICv3  49 Level     eth1
 24:          0          0       4746          0     GICv3  50 Level     eth1
 25:          9          0          0          8     GICv3  51 Level     eth1
 26:          0          0          0          7     GICv3  52 Level     eth1
root@agilex5dka5e065bb32aes1:~#

Debug¶

This section outlines common issues and solutions encountered during system bring-up.

Ethernet Interfaces are Missing¶

After login into the HPS, the Ethernet ports are not listed by Linux as shown below.

root@agilex5dka5e065bb32aes1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether fe:22:e4:3a:00:aa brd ff:ff:ff:ff:ff:ff
    inet 10.244.193.14/22 brd 10.244.195.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::fc22:e4ff:fe3a:aa/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
3: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
    link/void
    root@agilex5dka5e065bb32aes1:~#

A common cause of this error are listed below.

Ethernet Interfaces are DOWN¶

After login into the HPS, the Ethernet ports are not listed by Linux as shown below.

root@agilex5dka5e065bb32aes1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether fe:22:e4:3a:00:aa brd ff:ff:ff:ff:ff:ff
    inet 10.244.193.14/22 brd 10.244.195.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::fc22:e4ff:fe3a:aa/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
3: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
    link/void
4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
5: eth1: <NO-CARRIER,BROADCAST,MULTICAST,DYNAMIC,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 42:4b:9b:7a:4e:94 brd ff:ff:ff:ff:ff:ff
    valid_lft forever preferred_lft forever
6: eth2: <NO-CARRIER,BROADCAST,MULTICAST,DYNAMIC,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 36:00:e2:fb:03:0e brd ff:ff:ff:ff:ff:ff
    valid_lft forever preferred_lft forever
root@agilex5dka5e065bb32aes1:~#

A common cause of this error are listed below.

Suboptimal Link Quality¶

The System Example Design does not support auto-negotiation or link training, it uses fixed analog settings optimized for active optical cables (AOC). Validated with FS Q28-AO05 (5 m / 16 ft) 100G QSFP28 AOC and FS Q28-PC01 (1 m / 3 ft) 100G QSFP28 passive DAC. Longer DAC cables or different cable types (length, vendor, optical) may require manual tuning of the Ethernet interface analog settings.

Before debugging, ensure cables are properly connected to both development kits. Begin by assessing link health using the procedure in Reading the GTS Ethernet Hard IP Configuration and Status Registers with the HPS. If a port shows degraded status and a DAC cable is used, adjust analog settings as described in Enabling Transceiver Tool Kit for the GTS Ethernet Hard IP. Run BER and Eye Viewer tests; if results are suboptimal, follow the guidance in section 8.3.7 of the GTS Transceiver PHY User Guide Agilex™ 5 FPGAs and SoCs.

System Debug Tools¶

Enabling Transceiver Tool Kit for GTS Ethernet IP¶

The system example design supports the GTS Transceiver Toolkit for debugging potential link quality issues. Follow the next steps to enable the Transceiver Toolkit:

With Quartus® Prime Pro version 25.3, open the system example design project.
In 'Project Navigator' click on 'IP Components' Tab.
Double click on the entity 'GTS Ethernet Hard IP', the IP Parameter Editor will open the GTS Ethernet Hard IP instance.
In the 'IP' >> 'General Options' >> 'Configuration, Debug and Extension Options' tab, set to 'Enable' the 'Enable Debug endpoint for Ethernet toolkit' parameter. Refer to the screen shot below.
In the 'IP' >> 'General Options' >> 'Configuration, Debug and Extension Options' tab, set to 'Enable' the click on the 'Enable debug endpoint for transceiver toolkit' parameter.
Save and regenerate the IP.
Recompile the Altera Quartus Prime project.
Regenerate the software with the new generated 'core.rbf' file.

Refer to section '8. Debugging GTS Transceiver Links with Transceiver Toolkit' from the 'GTS Transceiver PHY User Guide Agilex™ 5 FPGAs and SoCs' for more information on link quality related issues and their resolution. Sections '8.3.5. Running BER Tests' and '8.3.6 Running Eye Viewer Tests' are essential to qualify the Ethernet link health.

Figure 13. Set 'Enable debug endpoint for transceiver toolkit' & 'Enable debug endpoint for transceiver toolkit'for the GTS Ethernet Hard IP.

Simulation¶

The Agilex™ 5 2x10GbE Precision Time Protocol System Example Design includes a suite of standalone UVM simulation tests for hardware verification. These tests validate the Quartus® project within a Universal Verification Methodology (UVM) environment, ensuring functional correctness.

The UVM suite provides a structured framework for simulating various operating conditions and use cases, enabling thorough validation of system behavior.

Simulation Environment Setup¶

The following third-party tools and associated verification IPs, along with valid licenses, are required to execute the UVM simulation test cases for the design:

Design Tool	Version
Synopsys VCS* Tool	V-2023.03-SP2-1
Altera® Quartus® Prime Pro Tool	25.3
Synopsys DesignWare VIP	W-2025.03C
Python	3.7.7
Perl	5.8.8
CMAKE	3.11.4
GCC	7.2.0

The system testbench instantiates two AXI Synopsys Verification IP (VIP) modules, requiring a separate license in addition to the Synopsys VCS simulation tool license.

Simulation Directory¶

Simulation source files and scripts are located at: $TOP_FOLDER/src/hw/verification/

UVM Test Use Cases¶

The design includes six UVM test cases to validate the following functionality:

Configuration and status register access
DMA Subsystem <-> Ethernet Subsystem data path
Packet Generator <-> Ethernet Subsystem data path
DMA and Packet generator data path test

The test cases are:

1. CSR access test

The test exercises full configuration and status register access for Ethernet Subsystem. After initial system configuration and Ethernet link bring-up, all registers are read and compared against expected default values. The test then performs read operations at target offsets to validate register accessibility.

Test Case Sequence: sm_ptp_hssi_csr_seq

2. Data path test - DMA base test

This test showcases the scenario where traffic is generated from all the channels of dma port port 0. This sequence enables prefetcher for all channels of DMA port 0. For each channel, the payload length for each eth packet is 90B.

Also, the packet switch is configured to route the packets to intended dma port. The key used is a unique combination of SA and DA of eth packet for each of the port channels

Test Case Sequence: sm_ptp_h2d0_90B_seq

3. Data path test - DMA traffic test

This test showcases the scenario where traffic is generated from all the channels of dma port port 0. This sequence enables prefetcher for all channels of DMA port 0. For each channel, the payload length for each eth packet is 90B.

Also, the packet switch is configured to route the packets to intended dma port. The key used is a unique combination of SA and DA of eth packet for each of the port channels

Test Case Sequence: sm_ptp_all_dma_ports_64B_traffic_seq

4. Packet generator data path test

This sequence showcases transactions from user client 0 and 1. Both user clients are configured to generate random number of ethernet packets.

Also, the packet switch are configured to route the packets to intended user port. The key used is a unique combination of SA and DA of eth packet for each of the ports

Test Case Sequence: sm_ptp_user1_user0_seq

5. DMA and Packet generator data path test

This test Showcases the scenario where traffic is generated from all the dma ports and user client ports simultaneously. This sequence enables prefetcher for all channels of both ports 1 and 2 of DMA. For each channel, the payload length for each eth packet is random. user client 0 and 1 are enabled to generate random number of packets

Also, the packet switch are configured to route the packets to intended dma and user port. The key used is a unique combination of SA and DA of eth packet for each of the ports

Test Case Sequence: sm_ptp_all_ports_traffic_seq

Configuring UVM Environment¶

Export the simulation folder path to the environment with the following command:

export ROOTDIR=$TOP_FOLDER/src/hw

Update setup.sh with values from your local environment to configure simulation variables. The script is located at: $TOP_FOLDER/src/hw/verification/setup.sh

The following parameter variables are required for simulation:

export ROOTDIR=$TOP_FOLDER/src/hw
export WORKDIR=$ROOTDIR
export QUARTUS_HOME=$QUARTUS_ROOTDIR
export QUARTUS_INSTALL_DIR=$QUARTUS_ROOTDIR
export DESIGNWARE_HOME=<synopsys vip location>
export VERDIR=$WORKDIR/verification
export DESIGN=src
export DESIGN_DIR=$ROOTDIR/$DESIGN/
export VCS_HOME=<Synopsys VCS simulation installation dir>
export UVM_HOME=$VCS_HOME/etc/uvm-1.2

QUARTUS_ROOTDIR and TOP_FOLDER must be defined as described in Environment Setup.

Test Flow¶

This section outlines the step-by-step procedures for simulating each of the test cases listed above.

Prerequisites¶

Navigate to the verification scripts directory:

cd $ROOTDIR/verification/scripts

Invoke Altera® Quartus®, Synopsys VCS* and Synopsys Verdi tool licenses.

Simulation Steps¶

Initial Compilation (One-time Setup)

Execute this command when compiling the DUT for the first time or after any IP changes:
```
make -f Makefile.mk cmplib
```
Note: This is a one-time operation required only during initial setup or when IP modifications occur.
Build DUT and Testbench

Compile and elaborate the Design Under Test (DUT) and testbench:
```
make -f Makefile.mk build
```

Execute Test Sequence

Run a specific test sequence using the following command:

make -f Makefile.mk run SEQNAME=<sequence_identifier>

EX:

```bash
make -f Makefile.mk run SEQNAME=sm_ptp_h2d0_90B_seq
```

Combined Build and Run (Alternative)

Steps 2 and 3 can be combined into a single command for efficiency:
```
make -f Makefile.mk build run SEQNAME=sm_ptp_h2d0_90B_seq
```

Waveform Generation¶

To enable waveform dumping, add the DUMP=1 option to the build and run commands.\

Method 1: Separate Build and Run Commands

make -f Makefile.mk build DUMP=1
make -f Makefile.mk run SEQNAME=sm_ptp_h2d0_90B_seq DUMP=1

Method 2: Combined Command

make -f Makefile.mk build run SEQNAME=sm_ptp_h2d0_90B_seq DUMP=1

Command Reference Summary:¶

Operation	Command
Initial compilation	`make -f Makefile.mk cmplib`
Build DUT/Testbench	`make -f Makefile.mk build`
Run Test sequence	`make -f Makefile.mk run SEQNAME=<sequence_identifier>`
Combined build/run	`make -f Makefile.mk build run SEQNAME=<sequence_identifier>`

Enable waveform dump Add DUMP=1 to any build/run command.

Replace <sequence_identifier> with the appropriate Test Case Sequence identifier for your specific test case.

Results¶

Simulation of each of the test cases listed above can be carried out from below steps.

cd $ROOTDIR/verification/scripts
Below is a one time run that needs to be given when compiling the DUT for the first time or if there is any change in the IP

make -f Makefile.mk cmplib
Run below make commands to compile and elaborate the DUT and TESTBENCH

make -f Makefile.mk build
Run below command to run a sequence

make -f Makefile.mk run SEQNAME=<sequence name>

Eg: make -f Makefile.mk run SEQNAME=sm_ptp_h2d0_90B_seq
Steps 3 and 4 can be combined and run in a single step

make -f Makefile.mk build run SEQNAME=sm_ptp_h2d0_90B_seq
Dumping a waveform Please add option DUMP=1 to steps 3 and 4 or step 5 to enable waveform dumping

Eg 1:

make -f Makefile.mk build DUMP=1

make -f Makefile.mk run SEQNAME=sm_ptp_h2d0_90B_seq DUMP=1

Eg 2:

make -f Makefile.mk build run SEQNAME=sm_ptp_h2d0_90B_seq DUMP=1

Output Directory Structure¶

The simulation framework stores test results in $ROOTDIR/verification/sim.

When the library compilation step (step 2) executes again, the system automatically renames the existing sim directory to sim.# where # represents an incremental number.

A fresh sim directory is then created for new results

The system saves logs and waveform files in $ROOTDIR/verification/sim/<sequence_identifier> directory

When running the same sequence multiple times, the system preserves previous results by renaming the existing sequence directory to $ROOTDIR/verification/sim/<sequence_identifier>.#.

A new $ROOTDIR/verification/sim/<sequence_identifier> directory is created for the current run.

This versioning system ensures that historical simulation data remains accessible while providing a clean workspace for new test executions.

Reference¶

Notices & Disclaimers¶

Altera^® Corporation technologies may require enabled hardware, software or service activation. No product or component can be absolutely secure. Performance varies by use, configuration and other factors. Your costs and results may vary. You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Altera or Intel products described herein. You agree to grant Altera Corporation a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document, with the sole exception that you may publish an unmodified copy. You may create software implementations based on this document and in compliance with the foregoing that are intended to execute on the Altera or Intel product(s) referenced in this document. No rights are granted to create modifications or derivatives of this document. The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Altera disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. You are responsible for safety of the overall system, including compliance with applicable safety-related requirements or standards. ^© Altera Corporation. Altera, the Altera logo, and other Altera marks are trademarks of Altera Corporation. Other names and brands may be claimed as the property of others.

OpenCL* and the OpenCL* logo are trademarks of Apple Inc. used by permission of the Khronos Group™.

Last update: February 3, 2026
Created: February 3, 2026