Published May 28, 2021 © TAPR-OHL

Network programmability kit for Ultra96

Network FPGA developer framework for Ultrascale+ - 6x SFP+ mezzanine, 1U enclosure, IP cores in Verilog managed with NETCONF/YANG interface

AdvancedFull instructions provided8 hours2,054

Things used in this project

Hardware components

Avnet Ultra96-V2

Lightside Instruments Spark-V2 6x SFP+ base board kit

Avnet 96Boards Click Mezzanine Starter Kit

MikroE GPS Click

GPS Active External Antenna MIKROE-3373

Avnet AES-ACC-U96-4APWR

1m (3ft) LC UPC to LC UPC Duplex OS2 Single Mode PVC (OFNR) 2.0mm Fibre Optic Patch Lead

SFP+ Optical Transceiver Modules

SFP Copper Ethernet Module

GPS clock extension board for Ultra96

Clock input extension board for Ultra96

Software apps and online services

Debian

yuma123

AMD Vivado Design Suite

Story

Introduction

The following walk-through will build Ultra96 Linux system with 6x SFP+ 1Gb Ethernet interfaces (eth0-eth5) with complete data-path processing in programmable logic (IP core instances in the FPGA). The SFP+ base-board and 1U rack enclosure used are documented in the spark project ver. 2.2 (OSHWA NO000005).

Network tester with a YANG/NETCONF management interface is provided as reference design. The YANG module is specified in draft-ietf-bmwg-network-tester-cfg. The code for the IP cores (Verilog/VHDL) and the corresponding NETCONF interface (C) is open-source (github).

Detailed design organization diagram is available here.

The functionality of the tester is demonstrated with a practical command line application executing RFC2544 compliant benchmark.

Assemble the components

Plug SFP 1Gb transciever modules (optical or copper). The walk-through will use eth0 and eth1 as source and destination interfaces for a RFC2544 test. You can use eht4 to connect the device to your wired network. Alternatively you can use wifi.

Step-by-step assembly of the kit

unboxing video:

network programmability kit unboxing

Install toolchains

It is assumed that you have Vivado and PetaLinux 2019.2 installed and the ultra96v2 board definition files link

You need to install the evaluation license for the AXI 1G/2.5G Ethernet Subsystem. Click here.

Build logic and software

You can use prebuild binaries from the latest release instead and skip to "Prepare SD card".

If you are using ultra96v1 you need to edit build.sh (board_ver=2 to board_ver=1)

git clone https://github.com/lightside-instruments/network-interconnect-tester-cores.git
cd network-interconnect-tester-cores
source /tools/Xilinx/Vivado/2019.2/settings64.sh
source /tools/petalinux-v2019.2/settings.sh
# edit the script if you are using ver. 1 of Ultra96
./build.sh
#... wait 6 hours
#INFO: Binary is ready.
#PetaLinux build script completed

Prepare SD card

Copy BOOT.BIN, image.ub to the boot partition and extract rootfs.tar.gz to the rootfs partition of the micro SD card. ( If this is the first time you create SD card for Ultra96 check this link)

The rootfs.tar.gz built is identical to the rootfs part of the ultra96v2_oob_2019_2 except the binary sds-lib package which is not part of the repository. You can use the Avnet relased rootfs.tar.gz if you need that package.

Power-up

Insert the SD card in the device and start the boot sequence by pushing the red button.

You should be able to connect to DHCP assigned IP address on interface eth4 - MAC 70:B3:D5:EC:20:04. If you want to assign static address instead you can edit the /etc/network/interfaces file on the SD card).

Wifi acces point identical to the default BSP image is also started by default.

SSH on port 22 is enabled by default.

user: root

pass: root

$ ssh root@192.168.0.21
root@192.168.0.21's password: 
root@spark:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:0A:35:00:22:01  
          inet addr:169.254.77.238  Bcast:169.254.255.255  Mask:255.255.0.0
          inet6 addr: fe80::20a:35ff:fe00:2201/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:89 errors:0 dropped:0 overruns:0 frame:0
          TX packets:333 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:40700 (39.7 KiB)  TX bytes:111126 (108.5 KiB)

eth1      Link encap:Ethernet  HWaddr 70:B3:D5:EC:20:01  
          inet addr:169.254.80.53  Bcast:169.254.255.255  Mask:255.255.0.0
          inet6 addr: fe80::72b3:d5ff:feec:2001/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:333 errors:0 dropped:0 overruns:0 frame:0
          TX packets:89 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:111216 (108.6 KiB)  TX bytes:40610 (39.6 KiB)

eth2      Link encap:Ethernet  HWaddr 70:B3:D5:EC:20:02  
          UP BROADCAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth3      Link encap:Ethernet  HWaddr 70:B3:D5:EC:20:03  
          UP BROADCAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth4      Link encap:Ethernet  HWaddr 70:B3:D5:EC:20:04  
          inet addr:192.168.0.21  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::72b3:d5ff:feec:2004/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:4403 errors:0 dropped:31 overruns:0 frame:0
          TX packets:535 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:384773 (375.7 KiB)  TX bytes:65155 (63.6 KiB)

eth5      Link encap:Ethernet  HWaddr 70:B3:D5:EC:20:05  
          UP BROADCAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:492 errors:0 dropped:0 overruns:0 frame:0
          TX packets:492 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:574474 (561.0 KiB)  TX bytes:574474 (561.0 KiB)

mon.p2p0  Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-30-00-00-00-00-00-00-00-00  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

p2p0      Link encap:Ethernet  HWaddr FA:F0:05:C3:2D:8C  
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::f8f0:5ff:fec3:2d8c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:108 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:25030 (24.4 KiB)

sit0      Link encap:IPv6-in-IPv4  
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

wlan0     Link encap:Ethernet  HWaddr F8:F0:05:C3:2D:8C  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:90 (90.0 B)

Disable the default wireless access point and proprietary connection management services

update-rc.d -f ultra96-ap-setup.sh remove
update-rc.d -f connman remove

Using wlan0 as management interface

If instead of eth4 you want to connect to an existing wifi AP with the wlan0 interface you can comment out "auto eth4" line in /etc/network/interfaces, edit /home/root/wpa_supplicant.conf with correct AP ESSID and password.

Add /home/root/wifi.sh to the scripts executed during each startup and reboot:

ln -s /home/root/wifi.sh /etc/init.d/wifi.sh
update-rc.d -f wifi.sh defaults
reboot

Install Debian

There are certain advantages in using chroot'ed Debian for package management of the user-space tools and libraries.

cd /usr/share
#Warning: Dirty work-around for outdated SSL certificates. You should update the certificates instead
git config --global http.sslVerify false
git clone https://salsa.debian.org/installer-team/debootstrap.git
git config --global http.sslVerify true

cd debootstrap
./debootstrap --arch=arm64 stable /srv/chroot/stable/ http://deb.debian.org/debian/
rsync -rav /dev/ /srv/chroot/stable/dev
mount --bind /proc /srv/chroot/stable/proc
mount --bind /sys /srv/chroot/stable/sys
chroot /srv/chroot/stable
# now you are running in Debian

NETCONF/YANG toolchain (libyuma, netconfd, yangcli)

# still running in Debian
su
cd
apt-get -y install git autoconf gcc libtool make gdb rsync man net-tools tcpdump
git clone https://github.com/lightside-instruments/yuma123.git
apt-get -y install libxml2-dev libssh2-1-dev libncurses5-dev zlib1g-dev libreadline-dev libssl-dev
cd yuma123
autoreconf -i -f
./configure CFLAGS="-g -O0"  CXXFLAGS="-g -O0" --prefix=/usr
make
make install
cd ..

cd yuma123/example-modules/ietf-traffic-generator
autoreconf -i -f
./configure CFLAGS="-g -O0"  CXXFLAGS="-g -O0" --prefix=/usr
make
make install
cd ../../../

git clone https://github.com/lightside-instruments/network-interconnect-tester-cores.git
cd network-interconnect-tester-cores/lib/sw/lsi/
autoreconf -i -f
./configure CFLAGS="-g -O0"  CXXFLAGS="-g -O0" --prefix=/usr
make
make install
cd ../../../../

rsync -rav /usr/lib/yuma/ /usr/lib/aarch64-linux-gnu/yuma

NETCONF/YANG python scripting tools for client side and RFC2544 test implementation

This step is optional if you intend to use external NETCONF client.

git clone https://github.com/lightside-instruments/litenc.git
apt-get -y install python python-lxml libxml2-utils python-paramiko
cd litenc
python setup.py install
cd tntapi
python setup.py install
cd ../..

cd yuma123/netconf/python
apt-get -y install python-all-dev
autoreconf -i -f
./configure CFLAGS="-g -O0"  CXXFLAGS="-g -O0" --prefix=/usr
make
make install
python setup.py install
cd ../../../

Start sshd

cd
apt-get install -y openssh-server
mv /etc/ssh/sshd_config /etc/ssh/sshd_config.original
echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config
echo 'PasswordAuthentication yes' >> /etc/ssh/sshd_config
echo 'Port 830' >> /etc/ssh/sshd_config
echo 'Subsystem netconf "/usr/sbin/netconf-subsystem"' >> /etc/ssh/sshd_config
/etc/init.d/ssh start

mkdir .ssh
ssh-keygen -t rsa -b 4096 -m PEM -f .ssh/id_rsa -N ""
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
ssh-keyscan -p 830 localhost >> .ssh/known_hosts
# ssh -p 830 localhost

#Add password for simple remote access
echo "root:root" | chpasswd

#Create empty startup configuration
mkdir .yuma
echo "<config/>" > .yuma/startup-cfg.xml

Start netconfd

/usr/sbin/netconfd --no-startup --module=ietf-interfaces --module=ietf-system --module=ietf-traffic-generator --module=ietf-traffic-analyzer --superuser=root

Connect with ssh (raw NETCONF)

root@spark:~# ssh localhost -p 830 -s netconf
<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"
  xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <capabilities>
    <capability>urn:ietf:params:netconf:base:1.0</capability>
    <capability>urn:ietf:params:netconf:base:1.1</capability>
    <capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability>
    <capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:validate:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:validate:1.1</capability>
    <capability>urn:ietf:params:netconf:capability:url:1.0?scheme=file</capability>
    <capability>urn:ietf:params:netconf:capability:xpath:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:notification:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:interleave:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:partial-lock:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:with-defaults:1.0?basic-mode=explicit&amp;also-supported=trim,report-all,report-all-tagged</capability>
    <capability>urn:ietf:params:xml:ns:yang:iana-crypt-hash?module=iana-crypt-hash&amp;revision=2014-08-06&amp;features=crypt-hash-md5,crypt-hash-sha-256,crypt-hash-sha-512</capability>
    <capability>urn:ietf:params:xml:ns:yang:iana-if-type?module=iana-if-type&amp;revision=2014-05-08</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&amp;revision=2013-07-15</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-interfaces?module=ietf-interfaces&amp;revision=2014-05-08&amp;features=arbitrary-names,pre-provisioning,if-mib</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-netconf-acm?module=ietf-netconf-acm&amp;revision=2018-02-14</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring?module=ietf-netconf-monitoring&amp;revision=2010-10-04</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-netconf-notifications?module=ietf-netconf-notifications&amp;revision=2012-02-06</capability>
    <capability>urn:ietf:params:xml:ns:netconf:partial-lock:1.0?module=ietf-netconf-partial-lock&amp;revision=2009-10-19</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults?module=ietf-netconf-with-defaults&amp;revision=2011-06-01</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-system?module=ietf-system&amp;revision=2014-08-06&amp;features=radius,authentication,local-users,radius-authentication,ntp,ntp-udp-port,timezone-name,dns-udp-tcp-port</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-traffic-analyzer?module=ietf-traffic-analyzer&amp;revision=2021-02-16&amp;features=egress-direction,ingress-direction,filter,idle-octets-counter,capture</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-traffic-generator?module=ietf-traffic-generator&amp;revision=2021-02-16&amp;features=egress-direction,ethernet,realtime-epoch</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-yang-library?module=ietf-yang-library&amp;revision=2016-06-21</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&amp;revision=2013-07-15</capability>
    <capability>http://yuma123.org/ns/interfaces-notifications?module=interfaces-notifications&amp;revision=2016-11-12</capability>
    <capability>urn:ietf:params:xml:ns:netmod:notification?module=nc-notifications&amp;revision=2008-07-14</capability>
    <capability>urn:ietf:params:xml:ns:netconf:notification:1.0?module=notifications&amp;revision=2008-07-14</capability>
    <capability>http://netconfcentral.org/ns/yuma-app-common?module=yuma-app-common&amp;revision=2012-08-16</capability>
    <capability>http://netconfcentral.org/ns/yuma-mysession?module=yuma-mysession&amp;revision=2010-05-10</capability>
    <capability>http://netconfcentral.org/ns/yuma-ncx?module=yuma-ncx&amp;revision=2012-01-13</capability>
    <capability>http://netconfcentral.org/ns/yuma-proc?module=yuma-proc&amp;revision=2012-10-10</capability>
    <capability>http://netconfcentral.org/ns/yuma-time-filter?module=yuma-time-filter&amp;revision=2012-11-15</capability>
    <capability>http://netconfcentral.org/ns/yuma-types?module=yuma-types&amp;revision=2012-06-01</capability>
    <capability>http://yuma123.org/ns/yuma123-mysession-cache?module=yuma123-mysession-cache&amp;revision=2018-11-12</capability>
    <capability>urn:ietf:params:xml:ns:netconf:base:1.0?module=ietf-netconf&amp;revision=2011-06-01&amp;features=writable-running,candidate,confirmed-commit,rollback-on-error,validate,startup,url,xpath</capability>
    <capability>http://yuma123.org/ns/yuma123-netconf-types?module=yuma123-netconf-types&amp;revision=2017-06-23</capability>
    <capability>http://yuma123.org/ns/yuma123-system?module=yuma123-system&amp;revision=2017-03-26</capability>
    <capability>urn:ietf:params:netconf:capability:yang-library:1.0?revision=2016-06-21&amp;module-set-id=a8b7aa9b4dece9a7d163d9ea60e4acfd3c6156f3</capability>
  </capabilities>
  <session-id>1</session-id>
</hello>]]>]]>

Connect with yangcli (over NETCONF)

root@spark:~# yangcli --server=localhost --user=root

  yangcli version 2.12-0
  libssh2 version 1.8.0

  Copyright (c) 2008-2012, Andy Bierman, All Rights Reserved.
  Copyright (c) 2013-2018, Vladimir Vassilev, All Rights Reserved.

  Type 'help' or 'help <command-name>' to get started
  Use the <tab> key for command and value completion
  Use the <enter> key to accept the default value in brackets

  These escape sequences are available when filling parameter values:

	?	help
	??	full help
	?s	skip current parameter
	?c	cancel current command

  These assignment statements are available when entering commands:

	$<varname> = <expr>	Local user variable assignment
	$$<varname> = <expr>	Global user variable assignment
	@<filespec> = <expr>	File assignment

yangcli: Starting NETCONF session for root on localhost

NETCONF session established for root on localhost

Client Session Id: 1
Server Session Id: 2

Server Protocol Capabilities
   base:1.0
   candidate:1.0
   confirmed-commit:1.0
   rollback-on-error:1.0
   validate:1.0
   url:1.0
   xpath:1.0
   notification:1.0
   interleave:1.0
   partial-lock:1.0
   with-defaults:1.0
   base:1.1
   validate:1.1
   confirmed-commit:1.1
   yang-library:1.0

Server Module Capabilities
   iana-crypt-hash@2014-08-06
      Features: 
         crypt-hash-md5 
         crypt-hash-sha-256 
         crypt-hash-sha-512 
   iana-if-type@2014-05-08
   ietf-inet-types@2013-07-15
   ietf-interfaces@2014-05-08
      Features: 
         arbitrary-names 
         pre-provisioning 
         if-mib 
   ietf-netconf-acm@2018-02-14
   ietf-netconf-monitoring@2010-10-04
   ietf-netconf-notifications@2012-02-06
   ietf-netconf-partial-lock@2009-10-19
   ietf-netconf-with-defaults@2011-06-01
   ietf-system@2014-08-06
      Features: 
         radius 
         authentication 
         local-users 
         radius-authentication 
         ntp 
         ntp-udp-port 
         timezone-name 
         dns-udp-tcp-port 
   ietf-traffic-analyzer@2021-02-16
      Features: 
         egress-direction 
         ingress-direction 
         filter 
         idle-octets-counter 
         capture 
   ietf-traffic-generator@2021-02-16
      Features: 
         egress-direction 
         ethernet 
         realtime-epoch 
   ietf-yang-library@2016-06-21
   ietf-yang-types@2013-07-15
   interfaces-notifications@2016-11-12
   nc-notifications@2008-07-14
   notifications@2008-07-14
   yuma-app-common@2012-08-16
   yuma-mysession@2010-05-10
   yuma-ncx@2012-01-13
   yuma-proc@2012-10-10
   yuma-time-filter@2012-11-15
   yuma-types@2012-06-01
   yuma123-mysession-cache@2018-11-12
   ietf-netconf@2011-06-01
      Features: 
         writable-running 
         candidate 
         confirmed-commit 
         rollback-on-error 
         validate 
         startup 
         url 
         xpath 
   yuma123-netconf-types@2017-06-23
   yuma123-system@2017-03-26

Server Enterprise Capabilities
   None

Protocol version set to: RFC 6241 (base:1.1)
Default target set to: <candidate>
Save operation mapped to: commit
Default with-defaults behavior: explicit
Additional with-defaults behavior: trim,report-all,report-all-tagged


Getting yang-library module set  ...
Current module-set-id: a8b7aa9b4dece9a7d163d9ea60e4acfd3c6156f3.


Checking Server Modules...

yangcli root@localhost>

Get state with yangcli

yangcli root@localhost> xget /

RPC Data Reply 2 for session 2:

rpc-reply {
  data {
    modules-state {
      module iana-crypt-hash 2014-08-06 {
        name iana-crypt-hash
        revision 2014-08-06
        namespace urn:ietf:params:xml:ns:yang:iana-crypt-hash
        conformance-type import
        feature crypt-hash-md5
        feature crypt-hash-sha-256
        feature crypt-hash-sha-512
      }
      module iana-if-type 2014-05-08 {
        name iana-if-type
        revision 2014-05-08
        namespace urn:ietf:params:xml:ns:yang:iana-if-type
        conformance-type implement
      }
      module ietf-inet-types 2013-07-15 {
        name ietf-inet-types
        revision 2013-07-15
        namespace urn:ietf:params:xml:ns:yang:ietf-inet-types
        conformance-type import
      }
      module ietf-interfaces 2014-05-08 {
        name ietf-interfaces
        revision 2014-05-08
        namespace urn:ietf:params:xml:ns:yang:ietf-interfaces
        conformance-type implement
        feature arbitrary-names
        feature pre-provisioning
        feature if-mib
      }
      module ietf-netconf-acm 2018-02-14 {
        name ietf-netconf-acm
        revision 2018-02-14
        namespace urn:ietf:params:xml:ns:yang:ietf-netconf-acm
        conformance-type implement
      }
      module ietf-netconf-monitoring 2010-10-04 {
        name ietf-netconf-monitoring
        revision 2010-10-04
        namespace urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring
        conformance-type implement
      }
      module ietf-netconf-notifications 2012-02-06 {
        name ietf-netconf-notifications
        revision 2012-02-06
        namespace urn:ietf:params:xml:ns:yang:ietf-netconf-notifications
        conformance-type implement
      }
      module ietf-netconf-partial-lock 2009-10-19 {
        name ietf-netconf-partial-lock
        revision 2009-10-19
        namespace urn:ietf:params:xml:ns:netconf:partial-lock:1.0
        conformance-type implement
      }
      module ietf-netconf-with-defaults 2011-06-01 {
        name ietf-netconf-with-defaults
        revision 2011-06-01
        namespace urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults
        conformance-type implement
      }
      module ietf-system 2014-08-06 {
        name ietf-system
        revision 2014-08-06
        namespace urn:ietf:params:xml:ns:yang:ietf-system
        conformance-type implement
        feature radius
        feature authentication
        feature local-users
        feature radius-authentication
        feature ntp
        feature ntp-udp-port
        feature timezone-name
        feature dns-udp-tcp-port
      }
      module ietf-traffic-analyzer 2021-02-16 {
        name ietf-traffic-analyzer
        revision 2021-02-16
        namespace urn:ietf:params:xml:ns:yang:ietf-traffic-analyzer
        conformance-type implement
        feature egress-direction
        feature ingress-direction
        feature filter
        feature idle-octets-counter
        feature capture
      }
      module ietf-traffic-generator 2021-02-16 {
        name ietf-traffic-generator
        revision 2021-02-16
        namespace urn:ietf:params:xml:ns:yang:ietf-traffic-generator
        conformance-type implement
        feature egress-direction
        feature ethernet
        feature realtime-epoch
      }
      module ietf-yang-library 2016-06-21 {
        name ietf-yang-library
        revision 2016-06-21
        namespace urn:ietf:params:xml:ns:yang:ietf-yang-library
        conformance-type implement
      }
      module ietf-yang-types 2013-07-15 {
        name ietf-yang-types
        revision 2013-07-15
        namespace urn:ietf:params:xml:ns:yang:ietf-yang-types
        conformance-type import
      }
      module interfaces-notifications 2016-11-12 {
        name interfaces-notifications
        revision 2016-11-12
        namespace http://yuma123.org/ns/interfaces-notifications
        conformance-type implement
      }
      module nc-notifications 2008-07-14 {
        name nc-notifications
        revision 2008-07-14
        namespace urn:ietf:params:xml:ns:netmod:notification
        conformance-type implement
      }
      module notifications 2008-07-14 {
        name notifications
        revision 2008-07-14
        namespace urn:ietf:params:xml:ns:netconf:notification:1.0
        conformance-type implement
      }
      module yuma-app-common 2012-08-16 {
        name yuma-app-common
        revision 2012-08-16
        namespace http://netconfcentral.org/ns/yuma-app-common
        conformance-type import
      }
      module yuma-mysession 2010-05-10 {
        name yuma-mysession
        revision 2010-05-10
        namespace http://netconfcentral.org/ns/yuma-mysession
        conformance-type implement
      }
      module yuma-ncx 2012-01-13 {
        name yuma-ncx
        revision 2012-01-13
        namespace http://netconfcentral.org/ns/yuma-ncx
        conformance-type implement
      }
      module yuma-proc 2012-10-10 {
        name yuma-proc
        revision 2012-10-10
        namespace http://netconfcentral.org/ns/yuma-proc
        conformance-type implement
      }
      module yuma-time-filter 2012-11-15 {
        name yuma-time-filter
        revision 2012-11-15
        namespace http://netconfcentral.org/ns/yuma-time-filter
        conformance-type implement
      }
      module yuma-types 2012-06-01 {
        name yuma-types
        revision 2012-06-01
        namespace http://netconfcentral.org/ns/yuma-types
        conformance-type import
      }
      module yuma123-mysession-cache 2018-11-12 {
        name yuma123-mysession-cache
        revision 2018-11-12
        namespace http://yuma123.org/ns/yuma123-mysession-cache
        conformance-type implement
      }
      module ietf-netconf 2011-06-01 {
        name ietf-netconf
        revision 2011-06-01
        namespace urn:ietf:params:xml:ns:netconf:base:1.0
        conformance-type implement
        feature writable-running
        feature candidate
        feature confirmed-commit
        feature rollback-on-error
        feature validate
        feature startup
        feature url
        feature xpath
      }
      module yuma123-netconf-types 2017-06-23 {
        name yuma123-netconf-types
        revision 2017-06-23
        namespace http://yuma123.org/ns/yuma123-netconf-types
        conformance-type import
      }
      module yuma123-system 2017-03-26 {
        name yuma123-system
        revision 2017-03-26
        namespace http://yuma123.org/ns/yuma123-system
        conformance-type implement
      }
      module-set-id a8b7aa9b4dece9a7d163d9ea60e4acfd3c6156f3
    }
    nacm {
      enable-nacm true
      read-default permit
      write-default deny
      exec-default permit
      enable-external-groups true
      denied-operations 0
      denied-data-writes 0
    }
    netconf {
      streams {
        stream NETCONF {
          name NETCONF
          description 'default NETCONF event stream'
          replaySupport true
          replayLogCreationTime 2021-05-11T21:52:13Z
        }
      }
    }
    netconf-state {
      capabilities {
        capability urn:ietf:params:netconf:base:1.0
        capability urn:ietf:params:netconf:base:1.1
        capability urn:ietf:params:netconf:capability:candidate:1.0
        capability urn:ietf:params:netconf:capability:confirmed-commit:1.0
        capability urn:ietf:params:netconf:capability:confirmed-commit:1.1
        capability urn:ietf:params:netconf:capability:rollback-on-error:1.0
        capability urn:ietf:params:netconf:capability:validate:1.0
        capability urn:ietf:params:netconf:capability:validate:1.1
        capability urn:ietf:params:netconf:capability:url:1.0?scheme=file
        capability urn:ietf:params:netconf:capability:xpath:1.0
        capability urn:ietf:params:netconf:capability:notification:1.0
        capability urn:ietf:params:netconf:capability:interleave:1.0
        capability urn:ietf:params:netconf:capability:partial-lock:1.0
        capability urn:ietf:params:netconf:capability:with-defaults:1.0?basic-mode=explicit&also-supported=trim,report-all,report-all-tagged
        capability urn:ietf:params:xml:ns:yang:iana-crypt-hash?module=iana-crypt-hash&revision=2014-08-06&features=crypt-hash-md5,crypt-hash-sha-256,crypt-hash-sha-512
        capability urn:ietf:params:xml:ns:yang:iana-if-type?module=iana-if-type&revision=2014-05-08
        capability urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&revision=2013-07-15
        capability urn:ietf:params:xml:ns:yang:ietf-interfaces?module=ietf-interfaces&revision=2014-05-08&features=arbitrary-names,pre-provisioning,if-mib
        capability urn:ietf:params:xml:ns:yang:ietf-netconf-acm?module=ietf-netconf-acm&revision=2018-02-14
        capability urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring?module=ietf-netconf-monitoring&revision=2010-10-04
        capability urn:ietf:params:xml:ns:yang:ietf-netconf-notifications?module=ietf-netconf-notifications&revision=2012-02-06
        capability urn:ietf:params:xml:ns:netconf:partial-lock:1.0?module=ietf-netconf-partial-lock&revision=2009-10-19
        capability urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults?module=ietf-netconf-with-defaults&revision=2011-06-01
        capability urn:ietf:params:xml:ns:yang:ietf-system?module=ietf-system&revision=2014-08-06&features=radius,authentication,local-users,radius-authentication,ntp,ntp-udp-port,timezone-name,dns-udp-tcp-port
        capability urn:ietf:params:xml:ns:yang:ietf-traffic-analyzer?module=ietf-traffic-analyzer&revision=2021-02-16&features=egress-direction,ingress-direction,filter,idle-octets-counter,capture
        capability urn:ietf:params:xml:ns:yang:ietf-traffic-generator?module=ietf-traffic-generator&revision=2021-02-16&features=egress-direction,ethernet,realtime-epoch
        capability urn:ietf:params:xml:ns:yang:ietf-yang-library?module=ietf-yang-library&revision=2016-06-21
        capability urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&revision=2013-07-15
        capability http://yuma123.org/ns/interfaces-notifications?module=interfaces-notifications&revision=2016-11-12
        capability urn:ietf:params:xml:ns:netmod:notification?module=nc-notifications&revision=2008-07-14
        capability urn:ietf:params:xml:ns:netconf:notification:1.0?module=notifications&revision=2008-07-14
        capability http://netconfcentral.org/ns/yuma-app-common?module=yuma-app-common&revision=2012-08-16
        capability http://netconfcentral.org/ns/yuma-mysession?module=yuma-mysession&revision=2010-05-10
        capability http://netconfcentral.org/ns/yuma-ncx?module=yuma-ncx&revision=2012-01-13
        capability http://netconfcentral.org/ns/yuma-proc?module=yuma-proc&revision=2012-10-10
        capability http://netconfcentral.org/ns/yuma-time-filter?module=yuma-time-filter&revision=2012-11-15
        capability http://netconfcentral.org/ns/yuma-types?module=yuma-types&revision=2012-06-01
        capability http://yuma123.org/ns/yuma123-mysession-cache?module=yuma123-mysession-cache&revision=2018-11-12
        capability urn:ietf:params:xml:ns:netconf:base:1.0?module=ietf-netconf&revision=2011-06-01&features=writable-running,candidate,confirmed-commit,rollback-on-error,validate,startup,url,xpath
        capability http://yuma123.org/ns/yuma123-netconf-types?module=yuma123-netconf-types&revision=2017-06-23
        capability http://yuma123.org/ns/yuma123-system?module=yuma123-system&revision=2017-03-26
        capability urn:ietf:params:netconf:capability:yang-library:1.0?revision=2016-06-21&module-set-id=a8b7aa9b4dece9a7d163d9ea60e4acfd3c6156f3
      }
      datastores {
        datastore candidate {
          name candidate
          last-modified 2021-05-11T21:52:13Z
        }
        datastore running {
          name running
          last-modified 2021-05-11T21:52:13Z
        }
      }
      schemas {
        schema iana-crypt-hash 2014-08-06 ncm:yang {
          identifier iana-crypt-hash
          version 2014-08-06
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:iana-crypt-hash
          location NETCONF
        }
        schema iana-if-type 2014-05-08 ncm:yang {
          identifier iana-if-type
          version 2014-05-08
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:iana-if-type
          location NETCONF
        }
        schema ietf-inet-types 2013-07-15 ncm:yang {
          identifier ietf-inet-types
          version 2013-07-15
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-inet-types
          location NETCONF
        }
        schema ietf-interfaces 2014-05-08 ncm:yang {
          identifier ietf-interfaces
          version 2014-05-08
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-interfaces
          location NETCONF
        }
        schema ietf-netconf-acm 2018-02-14 ncm:yang {
          identifier ietf-netconf-acm
          version 2018-02-14
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-netconf-acm
          location NETCONF
        }
        schema ietf-netconf-monitoring 2010-10-04 ncm:yang {
          identifier ietf-netconf-monitoring
          version 2010-10-04
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring
          location NETCONF
        }
        schema ietf-netconf-notifications 2012-02-06 ncm:yang {
          identifier ietf-netconf-notifications
          version 2012-02-06
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-netconf-notifications
          location NETCONF
        }
        schema ietf-netconf-partial-lock 2009-10-19 ncm:yang {
          identifier ietf-netconf-partial-lock
          version 2009-10-19
          format ncm:yang
          namespace urn:ietf:params:xml:ns:netconf:partial-lock:1.0
          location NETCONF
        }
        schema ietf-netconf-with-defaults 2011-06-01 ncm:yang {
          identifier ietf-netconf-with-defaults
          version 2011-06-01
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults
          location NETCONF
        }
        schema ietf-system 2014-08-06 ncm:yang {
          identifier ietf-system
          version 2014-08-06
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-system
          location NETCONF
        }
        schema ietf-traffic-analyzer 2021-02-16 ncm:yang {
          identifier ietf-traffic-analyzer
          version 2021-02-16
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-traffic-analyzer
          location NETCONF
        }
        schema ietf-traffic-generator 2021-02-16 ncm:yang {
          identifier ietf-traffic-generator
          version 2021-02-16
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-traffic-generator
          location NETCONF
        }
        schema ietf-yang-library 2016-06-21 ncm:yang {
          identifier ietf-yang-library
          version 2016-06-21
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-yang-library
          location NETCONF
        }
        schema ietf-yang-types 2013-07-15 ncm:yang {
          identifier ietf-yang-types
          version 2013-07-15
          format ncm:yang
          namespace urn:ietf:params:xml:ns:yang:ietf-yang-types
          location NETCONF
        }
        schema interfaces-notifications 2016-11-12 ncm:yang {
          identifier interfaces-notifications
          version 2016-11-12
          format ncm:yang
          namespace http://yuma123.org/ns/interfaces-notifications
          location NETCONF
        }
        schema nc-notifications 2008-07-14 ncm:yang {
          identifier nc-notifications
          version 2008-07-14
          format ncm:yang
          namespace urn:ietf:params:xml:ns:netmod:notification
          location NETCONF
        }
        schema notifications 2008-07-14 ncm:yang {
          identifier notifications
          version 2008-07-14
          format ncm:yang
          namespace urn:ietf:params:xml:ns:netconf:notification:1.0
          location NETCONF
        }
        schema yuma-app-common 2012-08-16 ncm:yang {
          identifier yuma-app-common
          version 2012-08-16
          format ncm:yang
          namespace http://netconfcentral.org/ns/yuma-app-common
          location NETCONF
        }
        schema yuma-mysession 2010-05-10 ncm:yang {
          identifier yuma-mysession
          version 2010-05-10
          format ncm:yang
          namespace http://netconfcentral.org/ns/yuma-mysession
          location NETCONF
        }
        schema yuma-ncx 2012-01-13 ncm:yang {
          identifier yuma-ncx
          version 2012-01-13
          format ncm:yang
          namespace http://netconfcentral.org/ns/yuma-ncx
          location NETCONF
        }
        schema yuma-proc 2012-10-10 ncm:yang {
          identifier yuma-proc
          version 2012-10-10
          format ncm:yang
          namespace http://netconfcentral.org/ns/yuma-proc
          location NETCONF
        }
        schema yuma-time-filter 2012-11-15 ncm:yang {
          identifier yuma-time-filter
          version 2012-11-15
          format ncm:yang
          namespace http://netconfcentral.org/ns/yuma-time-filter
          location NETCONF
        }
        schema yuma-types 2012-06-01 ncm:yang {
          identifier yuma-types
          version 2012-06-01
          format ncm:yang
          namespace http://netconfcentral.org/ns/yuma-types
          location NETCONF
        }
        schema yuma123-mysession-cache 2018-11-12 ncm:yang {
          identifier yuma123-mysession-cache
          version 2018-11-12
          format ncm:yang
          namespace http://yuma123.org/ns/yuma123-mysession-cache
          location NETCONF
        }
        schema ietf-netconf 2011-06-01 ncm:yang {
          identifier ietf-netconf
          version 2011-06-01
          format ncm:yang
          namespace urn:ietf:params:xml:ns:netconf:base:1.0
          location NETCONF
        }
        schema yuma123-netconf-types 2017-06-23 ncm:yang {
          identifier yuma123-netconf-types
          version 2017-06-23
          format ncm:yang
          namespace http://yuma123.org/ns/yuma123-netconf-types
          location NETCONF
        }
        schema yuma123-system 2017-03-26 ncm:yang {
          identifier yuma123-system
          version 2017-03-26
          format ncm:yang
          namespace http://yuma123.org/ns/yuma123-system
          location NETCONF
        }
      }
      sessions {
        session 2 {
          session-id 2
          transport ncm:netconf-ssh
          username root
          source-host 127.0.0.1
          login-time 2021-05-11T22:01:34Z
          in-rpcs 1
          in-bad-rpcs 0
          out-rpc-errors 0
          out-notifications 0
        }
      }
      statistics {
        netconf-start-time 2021-05-11T21:52:13Z
        in-bad-hellos 0
        in-sessions 2
        dropped-sessions 0
        in-rpcs 1
        in-bad-rpcs 0
        out-rpc-errors 0
        out-notifications 0
      }
    }
    proc {
      cpuinfo {
        cpu 0 {
          processor 0
        }
        cpu 1 {
          processor 1
        }
        cpu 2 {
          processor 2
        }
        cpu 3 {
          processor 3
        }
      }
      meminfo {
        MemTotal '2046020 kB'
        MemFree '353376 kB'
        Buffers '74308 kB'
        Cached '1340148 kB'
        SwapCached '0 kB'
        Active '486944 kB'
        Inactive '1043872 kB'
        Active_anon_ '118120 kB'
        Inactive_anon_ '4676 kB'
        Active_file_ '368824 kB'
        Inactive_file_ '1039196 kB'
        Unevictable '1896 kB'
        Mlocked '1896 kB'
        SwapTotal '0 kB'
        SwapFree '0 kB'
        Dirty '0 kB'
        Writeback '0 kB'
        AnonPages '118296 kB'
        Mapped '154556 kB'
        Shmem '4884 kB'
        Slab '133544 kB'
        SReclaimable '109544 kB'
        SUnreclaim '24000 kB'
        KernelStack '3712 kB'
        PageTables '3288 kB'
        NFS_Unstable '0 kB'
        Bounce '0 kB'
        WritebackTmp '0 kB'
        CommitLimit '1023008 kB'
        Committed_AS '559572 kB'
        VmallocTotal '263061440 kB'
        VmallocUsed '0 kB'
        VmallocChunk '0 kB'
        HugePages_Total 0
        HugePages_Free 0
        HugePages_Rsvd 0
        HugePages_Surp 0
        Hugepagesize '2048 kB'
      }
    }
    system-state {
      yuma {
        sysName localhost
        sysCurrentDateTime 2021-05-11T22:02:53Z
        sysBootDateTime 2021-05-11T21:52:13Z
        sysLogLevel info
        sysNetconfServerId 'netconfd 2.12-0'
        sysNetconfServerCLI {
          access-control enforcing
          default-style explicit
          delete-empty-npcontainers false
          eventlog-size 1000
          hello-timeout 600
          idle-timeout 3600
          indent 2
          log-level info
          max-burst 10
          feature-enable-default true
          module ietf-interfaces
          module ietf-system
          module ietf-traffic-generator
          module ietf-traffic-analyzer
          running-error stop
          no-startup 
          startup-error stop
          subdirs true
          superuser root
          system-sorted true
          target candidate
          warn-idlen 64
          warn-linelen 0
          with-startup false
          with-url true
          with-validate true
          max-sessions 1024
          with-nmda false
        }
        uname {
          sysname Linux
          release 4.19.0-xilinx-v2019.2
          version '#1 SMP Sat Mar 7 14:03:30 UTC 2020'
          machine aarch64
          nodename spark
        }
      }
      clock {
        current-datetime 2021-05-11T22:02:53.240665075Z
      }
    }
    interfaces-state {
      interface eth3 {
        name eth3
        oper-status down
        speed 1000000000
        statistics {
          in-octets 0
          in-unicast-pkts 0
          in-errors 0
          in-discards 0
          in-multicast-pkts 0
          out-octets 0
          out-unicast-pkts 0
          out-errors 0
          out-discards 0
        }
      }
      interface sit0 {
        name sit0
        oper-status down
        statistics {
          in-octets 0
          in-unicast-pkts 0
          in-errors 0
          in-discards 0
          in-multicast-pkts 0
          out-octets 0
          out-unicast-pkts 0
          out-errors 0
          out-discards 0
        }
      }
      interface lo {
        name lo
        oper-status unknown
        statistics {
          in-octets 200699
          in-unicast-pkts 1344
          in-errors 0
          in-discards 0
          in-multicast-pkts 0
          out-octets 200699
          out-unicast-pkts 1344
          out-errors 0
          out-discards 0
        }
      }
      interface eth1 {
        name eth1
        oper-status up
        speed 1000000000
        statistics {
          in-octets 691136
          in-unicast-pkts 1999
          in-errors 0
          in-discards 0
          in-multicast-pkts 0
          out-octets 235520
          out-unicast-pkts 422
          out-errors 0
          out-discards 0
        }
      }
      interface eth4 {
        name eth4
        oper-status up
        speed 1000000000
        statistics {
          in-octets 468909337
          in-unicast-pkts 346467
          in-errors 0
          in-discards 197
          in-multicast-pkts 0
          out-octets 6353981
          out-unicast-pkts 76840
          out-errors 0
          out-discards 0
        }
      }
      interface eth0 {
        name eth0
        oper-status up
        speed 1000000000
        statistics {
          in-octets 235610
          in-unicast-pkts 422
          in-errors 0
          in-discards 0
          in-multicast-pkts 0
          out-octets 691046
          out-unicast-pkts 1999
          out-errors 0
          out-discards 0
        }
      }
      interface eth2 {
        name eth2
        oper-status down
        speed 1000000000
        statistics {
          in-octets 0
          in-unicast-pkts 0
          in-errors 0
          in-discards 0
          in-multicast-pkts 0
          out-octets 0
          out-unicast-pkts 0
          out-errors 0
          out-discards 0
        }
      }
      interface eth5 {
        name eth5
        oper-status down
        speed 1000000000
        statistics {
          in-octets 0
          in-unicast-pkts 0
          in-errors 0
          in-discards 0
          in-multicast-pkts 0
          out-octets 0
          out-unicast-pkts 0
          out-errors 0
          out-discards 0
        }
      }
    }
  }
}

yangcli root@localhost>

RFC2544 testframe

The generator can output any frame but lets use a RFC2544 standard testframe. Use the helper tool to hexdump it:

$ traffic-generator-make-testframe --frame-size=64 --dst-mac-address="70:B3:D5:EC:20:01" --src-mac-address="70:B3:D5:EC:20:00" --src-ipv4-address="192.0.2.1" --ipv4-ttl=10 --src-ipv4-udp-port=49184 --dst-ipv4-address="192.0.2.2" --dst-ipv4-udp-port=7
70B3D5EC200170B3D5EC200008004500002E000000000A112CBCC0000201C0000202C0200007001A0000000102030405060708090A0B0C0D0E0F1011B84F9C36

Configure the traffic generator with yangcli

yangcli root@localhost> create /interfaces/interface[name='eth0'] -- type=ethernetCsmacd

RPC OK Reply 18 for session 1:

yangcli root@localhost> create /interfaces/interface[name='eth0']/traffic-generator -- frame-size=64 interframe-gap=67000000  testframe-type=dynamic frame-data=70B3D5EC200170B3D5EC200008004500002E000000000A112CBCC0000201C0000202C0200007001A0000000102030405060708090A0B0C0D0E0F1011B84F9C36

RPC OK Reply 19 for session 1:

yangcli root@localhost> commit

When /interfaces/interface/traffic-genenator is configured on any of the interfaces its GMII mux for the egress direction is multiplexing through the traffic from the corresponding generator core instead of the DMA traffic sent to the port from Linux. However ingress traffic is sent to both the analyzer module logic and to Linux through DMA so a instance of tcpdump can be started in addition.

tcpdump

root@spark:~/yuma123/example-modules/ietf-traffic-generator# tcpdump -i eth1 -XXX     
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
04:11:23.103105 IP 192.0.2.1.49184 > 192.0.2.2.echo: UDP, length 18
	0x0000:  70b3 d5ec 2001 70b3 d5ec 2000 0800 4500  p.....p.......E.
	0x0010:  002e 0000 0000 0a11 2cbc c000 0201 c000  ........,.......
	0x0020:  0202 c020 0007 001a 0000 0000 0000 0000  ................
	0x0030:  009d 0000 0000 1001 177f fb48            ...........H
04:11:23.639096 IP 192.0.2.1.49184 > 192.0.2.2.echo: UDP, length 18
	0x0000:  70b3 d5ec 2001 70b3 d5ec 2000 0800 4500  p.....p.......E.
	0x0010:  002e 0000 0000 0a11 2cbc c000 0201 c000  ........,.......
	0x0020:  0202 c020 0007 001a 0000 0000 0000 0000  ................
	0x0030:  009e 0000 0000 1001 3772 b348            ........7r.H
04:11:24.175097 IP 192.0.2.1.49184 > 192.0.2.2.echo: UDP, length 18
	0x0000:  70b3 d5ec 2001 70b3 d5ec 2000 0800 4500  p.....p.......E.
	0x0010:  002e 0000 0000 0a11 2cbc c000 0201 c000  ........,.......
	0x0020:  0202 c020 0007 001a 0000 0000 0000 0000  ................
	0x0030:  009f 0000 0000 1002 1bca a148            ...........H

Running RFC2544 as an open-source python implementation

Connect a Device Under Test (DUT) to the eth0 and eth1 interfaces. A loopback connection would be the simplest DUT to test if you do not have a bridge or a router. The RFC2544 test is implemented on top of a genearal traffic trial tool with a superset of the parameters required by any of the RFC2544 sec. 26 benchmarking tests (Throughput, Latency, Frame Loss rate, Back-to-Back frames):

root@spark:~/litenc/tntapi/example/ietf-network-interconnect-tester# python ./test-rfc2544-trial.py --config=../topology-localhost.xml --testframe-type=dynamic --test-time=5 --frame-size=1500 --interframe-gap=1000 --src-node=tester0 --src-node-interface=eth0 --dst-node=tester0 --dst-node-interface=eth1 --testframe-type=dynamic  --frame-data=`traffic-generator-make-testframe --frame-size=1500 --dst-mac-address="70:b3:d5:ec:20:01" --src-mac-address="70:b3:d5:ec:20:00"  --src-ipv4-address="192.0.2.1" --src-ipv4-udp-port=49184 --dst-ipv4-address="192.0.2.2" --dst-ipv4-udp-port=7`
...
5-config-tester0.xml - end
Transaction 5 started: 2021-05-12T14:43:35
Transaction 5 completed: 2021-05-12T14:43:35
Test time:                             5
Generated packets:                250000
Received  packets:                250000
Lost packets:                          0
Lost packets percent:           0.000000
Sequence errors:                       0
Sequence errors percent:        0.000000
Latency Min[nanoseconds]:            840
Latency Max[nanoseconds]:            856

Starting a complete test run with all 4 benchmarks and generation of RFC2544 report (text only) for 64 byte frames.

root@spark:~/litenc/tntapi/example/ietf-network-interconnect-tester# python ./rfc2544.py --config=../topology-localhost.xml --dst-node=tester0 --dst-node-interface=eth1 --src-node=tester0 --src-node-interface=eth0 --dst-mac-address="70:B3:D5:EC:20:01" --src-mac-address="70:B3:D5:EC:20:00" --dst-ipv4-address="192.0.2.2" --src-ipv4-udp-port=49184 --src-ipv4-address="192.0.2.1" --frame-size=64 --trial-time=120 --speed=1000000000 | grep "^#"

#===Throughput===
#1 1488095.238095 pps, 20 octets interframe gap, 100.00% ... 178571428 / 178571428
#Result: 1488095.238095 pps
#===Latency===
#Measurement style - bit forwarding
#1 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#2 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#3 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#4 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#5 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#6 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#7 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#8 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#9 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#10 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#11 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#12 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#13 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#14 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#15 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#16 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#17 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#18 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#19 896 ns (min=864 ns, max=896 ns) ... 178571428 / 178571428
#20 904 ns (min=872 ns, max=904 ns) ... 178571428 / 178571428
#Result: 900.800000 nanoseconds
#===Frame loss rate===
#1 100% rate, 0% loss, (100.000000% rate actual), 1488095.238095 pps (1488095.238095 pps actual), 20 octets interframe gap ... 178571428 / 178571428
#2 90% rate, 0% loss, (89.361702% rate actual), 1339285.714286 pps (1329787.234043 pps actual), 30 octets interframe gap ... 159574468 / 159574468
#===Back to back frames===
#1 2 back-to-back frames ... 120 / 120
#2 4 back-to-back frames ... 240 / 240
#3 8 back-to-back frames ... 480 / 480
#4 16 back-to-back frames ... 960 / 960
#5 32 back-to-back frames ... 1920 / 1920
#6 64 back-to-back frames ... 3840 / 3840
#7 128 back-to-back frames ... 7680 / 7680
#8 256 back-to-back frames ... 15360 / 15360
#9 512 back-to-back frames ... 30720 / 30720
#10 1024 back-to-back frames ... 61440 / 61440
#11 2048 back-to-back frames ... 122880 / 122880
#12 4096 back-to-back frames ... 245760 / 245760
#13 8192 back-to-back frames ... 491520 / 491520
#14 16384 back-to-back frames ... 983040 / 983040
#15 32768 back-to-back frames ... 1966080 / 1966080
#16 65536 back-to-back frames ... 3932160 / 3932160
#17 131072 back-to-back frames ... 7864320 / 7864320
#18 262144 back-to-back frames ... 15728641 / 15728641
#19 524288 back-to-back frames ... 31457282 / 31457282
#20 1048576 back-to-back frames ... 62914565 / 62914565
#21 1488095 back-to-back frames ... 89285707 / 89285707
#The back to back search is limited to bursts below 1 second.
#Result: >= 1488095

root@spark:~/litenc/tntapi/example/ietf-network-interconnect-tester#

Starting netconfd at boot

Create /etc/rc.local file in the Debian chroot directory ( /srv/chroot/stable/etc/rc.local in the petalinux rootfs):

/etc/init.d/ssh start
cd /root
ulimit -c unlimited
rm /tmp/ncxserver.sock
#--no-startup
export LD_LIBRARY_PATH=/usr/lib:/lib/aarch64-linux-gnu/
/usr/sbin/netconfd --startup=/root/.yuma/startup-cfg.xml --module=ietf-interfaces --module=ietf-system --module=ietf-traffic-generator --module=ietf-traffic-analyzer --superuser=root 1>/var/log/netconfd.log 2>/var/log/netconfd.stderr.log &
ulimit -c 0

In order to automate the start of netconfd after each reboot create /etc/init.d/netconfd script in the petalinux root:

#!/bin/sh -e

rsync -rav /dev/ /srv/chroot/stable/dev
mount --bind /proc /srv/chroot/stable/proc
mount --bind /sys /srv/chroot/stable/sys

chroot /srv/chroot/stable /etc/rc.local &

and install it for the default runlevels:

chmod ugo+x /srv/chroot/stable/etc/rc.local
chmod ugo+x /etc/init.d/netconfd
update-rc.d -f netconfd defaults

You can now reboot and still have the netconfd server running on port 830.

GPS support

apt-get -y install gpsd gpsd-clients
cd ~/yuma123/example-modules/geo-location
autoreconf -i -f
./configure CFLAGS="-g -O0"  CXXFLAGS="-g -O0" --prefix=/usr
make
make install

Start gpsd (add the line to /etc/rc.local for autostart).

gpsd -r -n -P /run/gpsd/gpsd.pid /dev/ttyS1

Add --module=geo-location to the netconfd parameter list (edit /etc/rc.local for autostart).

Restart.

Validate the PPS signal is present.

root@axi_eth_v1:~# ppscheck /dev/ttyS1
# Seconds  nanoSecs   Signals
1629236046 998480669 TIOCM_CD
1629236047 098461838
1629236047 998490765 TIOCM_CD
1629236048 098460593
...

Get coordinates:

root@spark:~# yangcli --server=localhost --user=root --display-mode=xml --run-command="xget /geo-location" --batch-mode
...
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <data>
    <geo-location xmlns="http://example.com/ns/geo-location">
      <latitude>59.9145666710000000</latitude>
      <longitude>10.7492606760000000</longitude>
    </geo-location>
  </data>
</rpc-reply>

At this point you can enable PPS synchronized realtime clock generation.

root@spark:~# rtclock-pps-sync

As long as the GPS lock lasts the timestamps will be synchronized to the PPS signal. You can measure latency accross multiple tester nodes e.g. run RFC2544 test with source and destination interfaces on different systems.

If you restart the system it will be back to standalone clock counting the uptime. To synchronize to PPS you need to do this again.

GPS time synchronization with both 1PPS and 10MHz

With the GPS Click extension board we provided a 1PPS signal and synchronized the realtime clock. However there still is a problem with frequency deviation of the real time clock and the Ultra96 local oscillator.

In terms of the current usecase the timestamps will be quite accurate (<24 ns jitter) but the traffic pattern will still progress with the speed of the frequency that the PLL generates from the Ultra96 internal oscillator.

There is a solution for that. The current version of the firmware supports a 10 MHz synchronization clock input. One option is to use external GNSS disciplined oscillator. If your GPSDO supports 3.3V biased squarewave signal over 50 ohm coaxial cable for both 1PPS and the 10MHz signal you can replace the GPS Click and the 96Boards Click Mezzanine with the Input Clock Extension Board for Ultra96.

You can alternatively use the GPS Clock Extension Board for Ultra96. It uses a LEA-M8T GNSS timing module from U-blox and is pin compatible with the GPS Click board with the addition of a 10MHz signal.

With this extension board you can add the --module=gnss-clock option to the netconfd parameters you will have YANG/NETCONF interface to the properties of the signal and will be able to force deviation of the frequency to test forward and back pressure conditions.

Here is an example where the default 10MHz frequency is replaced with 100 ppm higher frequency:

replace /gnss-clock/frequency --value=10001000
commit

Now your interfaces run with 100 ppm higher frequency at the high limit of the Ethernet standard.

SSH access control

You might want to use a different user for a netconfd superuser then root. To do that create a new user:

adduser user

Change the --superuser=root parameter value to --superuser=user in the command line starting netconfd (/etc/rc.local)

You can limit the SSH priviliges only to the NETCONF subsystem with adding the following section to the /etc/ssh/sshd_config file.

Match User user
       X11Forwarding no
       AllowTcpForwarding no
       PermitTTY no
       ForceCommand /usr/sbin/netconf-subsystem

Document Changelog

2021-06-14: Initial version.

2021-07-24: Added "GPS support" section.

2021-10-11: Added "GPS time synchronization with both 1PPS and 10MHz"

2021-11-03: Added SSH access control

Schematics

Code

Credits

lightside-instruments

7 projects • 7 followers

Lightside Instruments AS develops open-source hardware and software instruments for test automation.

Network programmability kit for Ultra96

Things used in this project

Hardware components

Software apps and online services

Story

Introduction

Assemble the components

Install toolchains

Build logic and software

Prepare SD card

Power-up

Disable the default wireless access point and proprietary connection management services

Using wlan0 as management interface

Install Debian

NETCONF/YANG toolchain (libyuma, netconfd, yangcli)

NETCONF/YANG python scripting tools for client side and RFC2544 test implementation

Start sshd

Start netconfd

Connect with ssh (raw NETCONF)

Connect with yangcli (over NETCONF)

Get state with yangcli

RFC2544 testframe

Configure the traffic generator with yangcli

tcpdump

Running RFC2544 as an open-source python implementation

Starting netconfd at boot

GPS support

GPS time synchronization with both 1PPS and 10MHz

SSH access control

Document Changelog

Custom parts and enclosures

1U box from 4 mm acrilyc - SVG for laser cutting

Schematics

Board project in KiCAD

Code

Cores (Verilog) and Vivado build scripts (Tcl)

NETCONF/YANG server for management (C)

Credits

lightside-instruments

Comments

Embed the widget on your own site

Network programmability kit for Ultra96

Network programmability kit for Ultra96

Things used in this project

Hardware components

Software apps and online services

Story

Introduction

Assemble the components

Install toolchains

Build logic and software

Prepare SD card

Power-up

Disable the default wireless access point and proprietary connection management services

Using wlan0 as management interface

Install Debian

NETCONF/YANG toolchain (libyuma, netconfd, yangcli)

NETCONF/YANG python scripting tools for client side and RFC2544 test implementation

Start sshd

Start netconfd

Connect with ssh (raw NETCONF)

Connect with yangcli (over NETCONF)

Get state with yangcli

RFC2544 testframe

Configure the traffic generator with yangcli

tcpdump

Running RFC2544 as an open-source python implementation

Starting netconfd at boot

GPS support

GPS time synchronization with both 1PPS and 10MHz

SSH access control

Document Changelog

Custom parts and enclosures

1U box from 4 mm acrilyc - SVG for laser cutting

Schematics

Board project in KiCAD

Code

Cores (Verilog) and Vivado build scripts (Tcl)

NETCONF/YANG server for management (C)

Credits

lightside-instruments

Comments

Related channels and tags