Software apps and online services | ||||||
_9nsOFQ7ama.png?auto=compress%2Cformat&w=48&h=48&fit=fill&bg=ffffff) |
| |||||
The project integrates several new features to the AdaCore Drivers library to provide an IoT Framework based on existing lwIP implementation ported for the embedded STM32 Ethernet family of devices, by means of adapting and integrating existing network protocol Ada libraries adapted for the new lwIP port now it's possible to have HTTP server and MQTT client, a basic and classic hello world example code is provided for them as a starting point for complex and reliable embedded IoT and Ethernet Gateway solutions based on 100% Ada.
About the ProjectWhen I first saw the contest I thought to do some IoT or even some advance application that relies on some networking protocol but soon I realize there was a lack for TCP stack for Ada Drivers Library, Ada-enet library provides UDP sockets but most of the well known protocols such HTTP needs TCP. So this project ends up as a development project to support TCP and Application Layer protocol such HTTP. Currently MQTT Client and HTTP Server offers basic functionality, supporting code is provided as well as basic demonstrations snippets to support more complex applications.
MilestonesI. Port the ipstack from AdaCore Spark2014 git to the embedded STM32 devices with Ethernet peripheral.
II. Adapt and port MQTT Client code from Dmitry A. Kazakov Simple Components Library.
III. Adapt and port the HTTP Server code from Simple Components, with underlying "socket" Server RAW lwIP support for use in other Client/Server implementations.
IV. Provide example code of MQTT Client and Hello World HTTP Server for SMT32.
Detailed Description1. IP stack initializationThe Initialization consist of bringing the Ethernet driver up and lwIP stack. The first one is done similar to the ada-enet library, except that ipstack don't have (yet) DHCP protocol, so we fix the IP address static in code which is done in Initialize function of AIP.OSAL package, since ipstack also need to have IP addresses and in order to set the values in one place, the ipstack takes values from Ifnet record. The function If_Init is in charge of ipstack initialization and some discussion about it follows.
function If_Init return Err_T is
Mac : LL_Address_Storage;
begin
Err := NOERR;
Name := "st";
NIF.Allocate_Netif (If_Id);
if If_Id = NIF.IF_NOID then
Err := ERR_MEM;
end if;
NIF.NIF_Set_Name (If_Id, Name);
-- Mac (1) := Ifnet.Mac (1);
for I in Integer range 1 .. 6 loop
Mac (Net.Uint8 (I)) := Ifnet.Mac (I);
Netif_MAC_Addr (Net.Uint8 (I)) := Ifnet.Mac (I);
end loop;
NIF.NIF_Set_Offload_Checksum (If_Id, NIF.IP_CS, True);
NIF.NIF_Set_Offload_Checksum (If_Id, NIF.ICMP_CS, True);
NIF.NIF_Set_Offload_Checksum (If_Id, NIF.UDP_CS, True);
NIF.NIF_Set_Offload_Checksum (If_Id, NIF.TCP_CS, True);
NIF.NIF_Set_LL_Address_Length (If_Id, LL_Address_Range'Last);
NIF.NIF_Set_LL_Address (If_Id, Mac);
NIF.NIF_Set_MTU (If_Id, Net.Uint16 (1500));
IP := IPaddrs.IP4 (Ifnet.Ip (1), Ifnet.Ip (2), Ifnet.Ip (3), Ifnet.Ip (4));
Mask := IPaddrs.IP4 (Ifnet.Netmask (1), Ifnet.Netmask (2), Ifnet.Netmask (3), Ifnet.Netmask (4));
Broadcast := IPaddrs.IP4 (192, 168, 2, 255);
Remote := IPaddrs.IP_ADDR_ANY;
-- make static ARP for tests to isolate the issue with ARP append buffer
declare
remote_mac : aliased AIP.Ethernet_Address;
remote_ip : aliased AIP.IPaddrs.IPaddr;
arp_err : AIP.Err_T;
begin
remote_mac := (16#f8#, 16#32#, 16#e4#, 16#88#, 16#57#, 16#0c#);
remote_ip := IPaddrs.IP4 (192, 168, 2, 5);
ARP.ARP_Update
(Nid => If_Id,
Eth_Address => remote_mac,
IP_Address => remote_ip,
Allocate => True,
Err => arp_err);
pragma Unreferenced (arp_err);
end;
-- define the callbacks
NIF.NIF_Set_Input_CB (If_Id, AIP.IP.IP_Input'Access);
NIF.NIF_Set_Output_CB (If_Id, AIP.ARP.ARP_Output'Access);
NIF.NIF_Set_Link_Output_CB (If_Id, Net.MinIf.LL_Output'Access);
NIF.If_Config (If_Id, IP, Mask, Broadcast, Remote, Err);
return Err;
end If_Init;
Notice that ipstack has prevision fo the use of Checksum Offload Checks, which we take advantage of, so you have to set the flags accordingly using the NIF_Set_Offload_Checksum procedure. Also notice that we leave the callbacks as the original code, which was calling C code but now will call only Ada code instead.
You can notice that we were setting an initial value on the ARP list, this was done to do the tests without ARP packet going on and off there, so you can replace your host IP address there or just comment the code. I am not an expert on networking but it makes me some noise the fact that ipstack sends an ARP query when it receives an IP packet instead of trying to update the ARP cache using the data from the received packet, really don't know how the standard should be, so suppose we have a server listening on port 7 (echo server), ipstack receive the SYN control packet and instead of replaying immediately with the SYN-ACK it put the SYN packet in the Unack Queue and sends an ARP Query for the Ip address received, then when ARP replay is received, it resumes the Three-Way handshake.
During the debugging tests I solve a lot of things with original Three-Way handshake so I decide to isolate ARP using that snippet.
So remember to change your desired IP address for STM32 there until DHCP is ready. (I will try to port DHCP from ada-enet one of these days :)
I started this project as I mentioned from ada-enet library so the "main" file remains called ping, which is something I will try to change later on, and so the Receiver and Demos packages, so the call to Initialize is actually done in Demos package, i.e. AIP.OSAL.Initialize.
During my initial tests I notice some problem with my STM32F769I-DISCO board and the ada-enet library, I notice that some packets were lost during ping, it was concerning to me since I was not wanting to continue working with an underlying problem, the issue was reported on git of ada-enet without a final definitive solution, later on I realize the problem might be related to some conflict between Ethernet Stack and LCD, when I reported the issue I disable the D-cache of STM32 which seems to alleviate the problem, after some tests I decide to disable the LCD Refresh procedure that is called periodically (which I used some way to do some debugging at the beginning), I mention this since you might notice some LCD function out there just in case you want to enable.
2. Raw lwIP and it's callbacksEverything will be a callback since ipstack is based on RAW lwIP so you better get used to before driving nuts :)
To start receiving Ethernet Packets that will feed the ipstack a low level drivers was adapted. Original code uses Linux based Tap Interface, the embedded STM32 device Ethernet driver should be use to replace it. The great example code of Ada-enet ecosystem was used as a starting point, but reducing it to only the code required to gather the Ethernet frames since the ipstack has ARP, ICMP, UDP and TCP. Basically the Net.Buffers package is used to gather Ethernet frames together with the Net.Interfaces package that servers to initialize the interface and provides the Send and Receive functions as well as other supporting packages.
Ada-enet uses a Receiver Task that loop forever delivering Ethernet frames as they arrive, that remains similar, so the call to Ifnet.Receive (Packet) will block until there is one frame available, after that we pass the buffer to a Low Level receive function that will copy the data into ipstack buffers, so this version don't provide Zero Copy at the moment.
To maintain ipstack original structure as much as possible can be, the package AIP.OSAL.Single wraps different functions and procedures, Process_Interface_Events is in charge of allocating the buffer for the ipstack, the reason we don't call directly the low level interface is because the parent package, that is AIP.OSAL, holds the Interface id variable, originally ipstack allow an array of this interfaces Id but the embedded only has one interface and it doesn't make sense to have the array, so we declare If_Id in the definition file.
Returning to Process_Interface_Events function, it calls the Low Level function LL_Input from Net.MinIf package under under src/dev folder to maintain a similar structure with the original code of Linux tap C driver.
LL_Input it does the job of copying the frame into the necessary pbufs. Original code do some filtering of received frames, first attempt was to try activating the native filtering of STM32 Ethernet driver, since it can filter by Mac, but for some reason that didn't work as expected and as time goes by very fast we did the filtering by hand inside this function, only passing packets that match the Mac address or Broadcast packets (needed for ARP).
Here is the LL_Input function code.
function LL_Input (Nid : AIP.NIF.Netif_Id;
Buf : in out Net.Buffers.Buffer_Type) return AIP.Buffers.Buffer_Id is
p : AIP.Buffers.Buffer_Id;
q : AIP.Buffers.Buffer_Id;
size : AIP.Buffers.Data_Length;
Ether : Net.Headers.Ether_Header_Access;
DstMac : Net.Ether_Addr;
LL_Add : AIP.LL_Address (1 .. 6);
LL_Add_Len : AIP.LL_Address_Range;
Bail_Out : Boolean := False;
bufLen : Standard.Integer;
srcAdd : System.Address;
dstAdd : System.Address;
begin
-- pragma Unreferenced (Nid);
Ether := Buf.Ethernet;
if Net.Headers.To_Host (Ether.Ether_Type) = Net.Protos.ETHERTYPE_IP then
-- If we got an IP packet verify the padding to alloc the correct size
-- since Ada LwIp implementation uses Buffers.Tlen for some calcs
declare
IpHdr : Net.Headers.IP_Header_Access;
begin
IpHdr := Buf.IP;
if Net.Headers.To_Host (IpHdr.Ip_Len) <= 46 then
-- We set the size to alloc according to packet size
size := Net.Headers.To_Host (IpHdr.Ip_Len) + 14;
else
size := Net.Buffers.Get_Data_Size (Buf, Net.Buffers.RAW_PACKET);
end if;
end;
else
size := Net.Buffers.Get_Data_Size (Buf, Net.Buffers.RAW_PACKET);
end if;
DstMac := Ether.Ether_Dhost;
AIP.NIF.Get_LL_Address (Nid, LL_Add, LL_Add_Len);
if (Uint_16 (size) < Uint_16 (LL_Add_Len)) then
return AIP.Buffers.NOBUF;
end if;
-- Compare is packet is for us
-- I know I can do this with Filter options of STM32 ...
-- but I couldn't make it work, see line 370 of net-interfaces-stm32.adb
for I in LL_Add'Range loop
if LL_Add (I) /= DstMac (Standard.Integer (I)) then
Bail_Out := True;
exit;
end if;
end loop;
-- Compare Broadcast if Packet Destination Addr match
if Bail_Out then
for I in 1 .. Standard.Integer (LL_Add_Len) loop
if DstMac (I) /= Bcast (I) then
return AIP.Buffers.NOBUF;
end if;
end loop;
end if;
AIP.Buffers.Buffer_Alloc (0, size, AIP.Buffers.LINK_BUF, p);
if p /= AIP.Buffers.NOBUF then
q := p;
srcAdd := Net.Buffers.Get_Data_Address (Buf);
loop
bufLen := Standard.Integer (AIP.Buffers.Buffer_Len (q));
dstAdd := AIP.Buffers.Buffer_Payload (q);
AIP.Conversions.Memcpy (dstAdd, srcAdd, bufLen);
q := AIP.Buffers.Buffer_Next (q);
exit when q = AIP.Buffers.NOBUF;
srcAdd := Net.Buffers.Get_Data_Address_Pos (Buf, Uint16 (bufLen));
end loop;
end if;
Net.Buffers.Release (Buf);
return p;
end LL_Input;
Important to notice here is the first part to determine the packet size that will be used to reserve the pbufs, it was noticed that the STM32 Ethernet driver has a check for minimum size of Ethernet frames, it does it for Tx and Rx as you will see in some tcpdump captures later on, for Tx is not concerning but for Rx it happens that it affects the ipstack if you reserve a size that includes the padding octects, which STM32 Ethernet drivers does for packets less than 60 bytes. I was trying to locate if Rx padding can be disable without success, so it seems that you can disable but it will affect Tx, so I never did the test and instead fix the size using the snippet before. The code uses some util function from ada-enet to extract the IP header length, which if less than 46 means padding will follow, this was only relevant for IP packets as ARP didn't bother about packet size, but TCP ipstack code will fail if wrong.
Receiver Task
The receiver endless task will dispatch packets depending on it's type, for now we have to discern from ARP and IP packets, here is the Receiver task.
task body Controller is
use type Net.Uint64;
use type Net.Ether_Addr;
Buf : AIP.Buffers.Buffer_Id;
Ethhr : System.Address;
Ftype : Net.Uint16;
Packet : Net.Buffers.Buffer_Type;
Err : AIP.Err_T;
begin
-- Wait until the Ethernet driver is ready.
Ada.Synchronous_Task_Control.Suspend_Until_True (Ready);
AIP.IO.Put_Line ("Receiver Task Starts");
-- Loop receiving packets and dispatching them.
loop
if Packet.Is_Null then
Net.Buffers.Allocate (Packet);
exit when Packet.Is_Null;
end if;
if not Packet.Is_Null then
-- Block until there is a packet to process
AIP.OSAL.Ifnet.Receive (Packet);
Buf := AIP.OSAL.Single.Process_Interface_Events (Packet);
if Buf /= AIP.Buffers.NOBUF then
Ethhr := AIP.Buffers.Buffer_Payload (Buf);
Ftype := AIP.EtherH.EtherH_Frame_Type (Ethhr);
case Ftype is
when AIP.EtherH.Ether_Type_IP =>
AIP.Buffers.Buffer_Header (Buf, AIP.S16_T (-14), Err);
if AIP.Any (Err) then
AIP.Buffers.Buffer_Blind_Free (Buf);
else
AIP.OSAL.Single.Process_Input (Buf);
end if;
when AIP.EtherH.Ether_Type_ARP =>
AIP.OSAL.Single.Process_Arp (Buf);
when others =>
AIP.Buffers.Buffer_Blind_Free (Buf);
end case;
end if;
end if;
end loop;
AIP.IO.Put_Line ("Problem with Net Stack Packet allocation");
end Controller;
One thing to notice here is that when an error is encountered it's necessary to Free the LwIP buffer. For other cases the Free will be done inside the calling procedures. Before moving to discuss how ipstack handle the packets, let's finish the low level output procedure.
The same package Net.MinIf that has the Low Level Input seen before has the Low Level Output procedure, and actually does the inverse, takes ipstack pbufs and copy it's data to a Net.Buffers for Ethernet Tx. The difficult part was to realize how to get track of next position to write in the Net.Buffers Packet which is done using the function Get_Data_Address_Pos, after all data have been copied, it only remains to set the Length and call the Send procedure of Net.Interfaces.STM32 package to deliver it.
The low level output procedure will be the called by ipstack using the AIP.NIF package Link_Output procedure which is done only in AIP.ARP package, the other Output procedure that will send IP packets is actually ARP_Ouput, because the Ethernet information is filled by info in ARP table.
IP Packets
The receiver dispatch IP packets by calling AIP.OSAL.Single.Process_Input (Buf) procedure, which indeed calls the configured callback IP_Input procedure in AIP.IP package.
IP_Input do some sanity checks on the IP Header itself, among them it checks the IP address is the configured, original ipstack has some (To Be Defined or TBD) functions to forward which does not make sense here with only one interface so it's always disable (Enable_Forwarding flag is False).
At the end it dispatch the packet by calling Dispatch_Upper, this procedure call the appropriately procedure for UDP, TCP or ICMP. The first thing you want to try is ICMP as you guess. AT this point if IP address is configured correctly it should work out of the Box.
TCP Packets
TCP packets are handle by TCP_Input procedure. It performs some sanity checks on TCP Header as well, but if packet is correct it try to finds the PCB to deliver it, this is done by PCBs.Find_PCB in AIP.PCBs package. PCB are named the Protocol Control Blocks and they are share between TCP and UDP, for sure you know better than me about this, but in short you will have one PCB for each "connection" you have, let's say you want to connect to a server, ipstack will reserve an available PCB from a PCB array for that connection during it's lifetime, so it uniquely identify it, perhaps that was an oversimplification in LwIP terms but someone that ones to works at upper layer don't have to know every detail of it.
If there is a PCB for the received TCP packet, the TCP_Input calls TCP_Receive to further process it, otherwise it might send a RST control segment to peer to inform about it.
TCP_Receive procedure handles the TCP packet by delivering it to the application when needed and changing TPCB State.
It might be boring at some point this discussion but even worse was to make it work (actually frustrating is more appropriately) since original ipstack seems to have some blackholes, for example after the ICMP test works I tried the Echo Server that was already in ipstack code, which "works" when tested with a simple python client that connects to port 7, send "Hello World" and disconnect, the console shows the received Echo message but tcpdump capture was concerning. See known issue for more details.
Here is a screenshot of such a first capture: Echo Three-Way wrong
As you can see the Server was not handling correctly the Three-Way handshaking, actually it was even worse since after the client disconnect by sending the FIN-ACK server sends an empty payload packet, in short it was needing some inspection.
We are not going to discuss all the lines of code of this function at this moment since it's very long, rather we will progress with the project documentation and come back to highlight certain changes and explanation that is done in this and other aip.tcp procedures and functions.
Important right now before moving on is to hightlight the way the TCP stack inform the App layer about reception of packets. It does by mean of TCP_Event procedure, which actually insert the Event and PCB from the stack and call (if defined by the Application layer) the assigned callback procedure, let's see how it works in the case of the simple echo server.
Echo Server
Echo server is really simple, all the code is located in a single package under src/services called raw_tcp_echo.adb. The server initialization is done in demos.adb by calling RAW_TCP_Echo.Init.
The Init function is shown below
procedure Init with
Refined_Global => (In_Out => ESP)
is
Pcb : AIP.PCBs.PCB_Id;
Err : AIP.Err_T;
begin
-- Initialize the application state pool, then setup to
-- accept TCP connections on the well known echo port 7.
Init_ES_Pool;
AIP.TCP.TCP_New (Pcb);
if Pcb = AIP.PCBs.NOPCB then
Err := AIP.ERR_MEM;
else
AIP.TCP.TCP_Bind
(PCB => Pcb,
Local_IP => AIP.IPaddrs.IP_ADDR_ANY,
Local_Port => 7,
Err => Err);
end if;
if Err = AIP.NOERR then
AIP.TCP.TCP_Listen (Pcb, Err);
pragma Assert (AIP.No (Err));
AIP.TCP.On_TCP_Accept
(Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Accept'Access));
end if;
end Init;
The function TCP_New (Pcb) creates the server Pcb, after that a bind and listen on that Pcb follows. The Accept callback is set for the Pcb using On_TCP_Accept procedure.
More interesting is what is done in the accept callback shown below.
-------------------------
-- ECHO_Process_Accept --
-------------------------
procedure ECHO_Process_Accept
(Ev : AIP.TCP.TCP_Event_T;
Pcb : AIP.PCBs.PCB_Id;
Err : out AIP.Err_T)
is
pragma Unreferenced (Ev);
Sid : ES_Id;
begin
ES_Alloc (Sid);
if Sid = NOES then
Err := AIP.ERR_MEM;
Raise_Exception
(Data_Error'Identity,
"Alloc Sid Failed!");
else
ESP (Sid).Kind := ES_ACCEPTED;
ESP (Sid).Pcb := Pcb;
ESP (Sid).Buf := AIP.Buffers.NOBUF;
-- AIP.IO.Put_Line ("New PCB Accept: Sid");
-- AIP.IO.Put_Line (Sid'Image);
AIP.TCP.TCP_Set_Udata (Pcb, ESP (Sid)'Address);
AIP.TCP.On_TCP_Sent
(Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Sent'Access));
AIP.TCP.On_TCP_Recv
(Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Recv'Access));
AIP.TCP.On_TCP_Abort
(Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Abort'Access));
AIP.TCP.On_TCP_Poll
(Pcb, RAW_TCP_Callbacks.To_CBID (ECHO_Process_Poll'Access), 500);
AIP.TCP.TCP_Accepted (Pcb);
Err := AIP.NOERR;
end if;
end ECHO_Process_Accept;
The creators of the ipstack were cleaver to create a record to hold the state of each connection, it's actually an array since server can have more than one client connected at a given time. The array type is Echo_State and consists of
type State_Kind is
(ES_FREE, ES_READY, ES_ACCEPTED, ES_RECEIVED, ES_CLOSING);
type Echo_State is record
Kind : State_Kind;
Pcb : AIP.PCBs.PCB_Id;
Buf : AIP.Buffers.Buffer_Id;
Err : AIP.Err_T;
end record;
The beauty of the stack is the way to recover the state when a callback is done, the state array is actually an array that lives inside this file, for the echo tcp demo it doesn't seems like a big deal but later on you will see how we take advantage of it. The ipstack IPCBs ins an array of IP_PCB, a record that holds information such remote/local ip and port, connection status and link type among others, but there is an item of the record called Udata which is actually a pointer to the application data, you guess, in out TCP echo that data is our Echo_State record. The way is assigned by the application is using the function TCP_Set_Udata shown in the accept callback after the Pcb has been reserved in our Echo_State array ESP, the demo allows to have 4 records on this array but it's a matter of resources to have more.
If the server accepts the connection, it set up the callbacks for Sent, Receive, Abort and Poll. The last one is important to notice, it's a callback that is executed periodically by TCP so you might ask, where is the timer?
TCP Timers
TCP has two functions called periodically by means of a Timer, the functions are configured in TCP_Init shown below.
--------------
-- TCP_Init --
--------------
procedure TCP_Init with
Refined_Global => (Output => (All_PCBs,
Boot_Time,
IPCBs,
TCP_Ticks,
TPCBs))
is
begin
-- Record boot time to serve as local secret for generation of ISN
Boot_Time := Time_Types.Now;
TCP_Ticks := 0;
-- Initialize all the PCBs, marking them unused, and initialize the list
-- of bound PCBs as empty.
IPCBs := TCP_IPCB_Array'(others => PCBs.IP_PCB_Initializer);
TPCBs := TCP_TPCB_Array'(others => TCP_PCB_Initializer);
All_PCBs := TCP_PCB_Heads'(others => PCBs.NOPCB);
-- Allocate and Set frequency of TCP timers
Timers.TID_Alloc (Timers.TIMER_EVT_TCPFASTTMR, TCP_Fast_Timer'Access);
Timers.TID_Alloc (Timers.TIMER_EVT_TCPSLOWTMR, TCP_Slow_Timer'Access);
Timers.Set_Interval (Timers.TIMER_EVT_TCPFASTTMR, TCP_Fast_Interval);
Timers.Set_Interval (Timers.TIMER_EVT_TCPSLOWTMR, TCP_Slow_Interval);
end TCP_Init;
I modify the original AIP.Timers since it was kind of limited in the sense that it was tedious and time consuming and more error prone to add a Timer that was not in a Fixed list of Timers, so I spend some time to make it flexible and so any application service that requieres a Timer might used it, it was actually that the reason I did it, since MQTT protocol needs a Timer (more on that later on). The function TID_Alloc allocates a new Timer by associating it with a callback. You might notice I still use the List of Fixed Timers, but indeed there is another function that uses Index values and so you reserve it without knowing which index of the Timer it will assigns to.
The Timer is not started until you assign an interval by means of Set_Interval.
One important thing is that the Timers runs in a separate task called Periodic of Os_Service package. This file is locate inside the demos/utils board since I didn't find a better place for it. Periodic dedicate itself to measure time and call Process_Timers every TIMER_PERIOD as shown below.
task body Periodic is
Prev_Clock, Clock : AIP.Time_Types.Time := AIP.Time_Types.Time'First;
timer_deadline : Ada.Real_Time.Time;
Poll_Freq : constant := 50;
begin
pragma Unreferenced (Prev_Clock, Poll_Freq);
Ada.Synchronous_Task_Control.Suspend_Until_True (Ready);
Clock := AIP.Time_Types.Now;
timer_deadline := Ada.Real_Time.Clock;
loop
delay until timer_deadline;
Clock := AIP.Time_Types.Now;
AIP.OSAL.Single.Process_Timers (Clock);
timer_deadline := timer_deadline + TIMER_PERIOD;
end loop;
end Periodic;
The Timers works very good but callbacks should not take too long to return or it will start to fail, in the future a better approach of having the callbacks fired should be investigated.
Echo Server callbacks
The callbacks has no mean to know the actual state of the given connection unless once inside it by some way it finds which Echo State Pool corresponds the call, so this was the reason we assign Application data to the IPCB as shown before, so to retrieve it the callback does:
Es : Echo_State;
for Es'Address use AIP.TCP.TCP_Udata (Pcb);
This way the execution of code is ruled by the current state i.e., the more obvious usage is during reception, for example if reception callback is called without any buffer in the Event parameter (TCP_Event_T) and there are no more buffers to process, we might call Close on the Pcb but if there is pending data to process, the server sends it and delegates the Close to the Polling callback since it already set the state as ES_CLOSING. The code is shown below for clarity.
-----------------------
-- ECHO_Process_Recv --
-----------------------
procedure ECHO_Process_Recv
(Ev : AIP.TCP.TCP_Event_T;
Pcb : AIP.PCBs.PCB_Id;
Err : out AIP.Err_T)
is
Es : Echo_State;
for Es'Address use AIP.TCP.TCP_Udata (Pcb);
begin
if Ev.Buf = AIP.Buffers.NOBUF then
-- Remote host closed connection. Process what is left to be
-- sent or close on our side.
Es.Kind := ES_CLOSING;
if Es.Buf /= AIP.Buffers.NOBUF then
Echo_Send (Pcb, Es);
else
Echo_Close (Pcb, Es);
end if;
else
-- Signal TCP layer that we can accept more data
case Es.Kind is
when ES_ACCEPTED =>
Es.Kind := ES_RECEIVED;
Es.Buf := Ev.Buf;
AIP.Buffers.Buffer_Ref (Ev.Buf);
Echo_Send (Pcb, Es);
when ES_RECEIVED =>
-- Read some more data
if Es.Buf = AIP.Buffers.NOBUF then
AIP.Buffers.Buffer_Ref (Ev.Buf);
Es.Buf := Ev.Buf;
Echo_Send (Pcb, Es);
else
AIP.Buffers.Buffer_Chain (Es.Buf, Ev.Buf);
end if;
when others =>
-- Remote side closing twice (ES_CLOSING), or inconsistent
-- state. Trash.
AIP.TCP.TCP_Recved (Pcb, AIP.Buffers.Buffer_Tlen (Ev.Buf));
Es.Buf := AIP.Buffers.NOBUF;
end case;
end if;
Err := AIP.NOERR;
end ECHO_Process_Recv;
As you can see, Echo server is simple, it just call Echo_Send on data received by actually reusing the Pcb since data it's not even changed. That will not be the case of other servers you and I might have in mind.
One point here that I will take the opportunity to highlight. If you see the tcpdump capture of the TAP demo of ipstack that works, you will notice something odd.
ipstack TCP Echo tcpdump capture
The Server don't replay the received Data with an Ack, as far as I know that is not correct but it might do the Job. After I was having issues with HTTP server I decide to get's my hands on this issue too.
The original stack calls the procedure AIP.TCP.TCP_Recved (Pcb, Plen); to adjust the receive window once the application has consumed the data but leverages the Ack send to the stack state machine, amd for some reason it was missing.
Among several modifications I did, I added sort of Ack Now inside the function taking care to remove the Buffer from the Unack_Queue after that. Actually when looking at this I spend some time reading the C implementation of lwIP mqtt where there is such Ack Now function, I personally think there might be a better or different way to correct the problem but this one works so far as you can see in the following capture of the embedded STM32 ipstack Echo server.
Here is the TCP_Recved snippet
----------------
-- TCP_Recved --
----------------
procedure TCP_Recved
(PCB : PCBs.PCB_Id;
Len : AIP.U16_T)
with
Refined_Global => (In_Out => TPCBs)
is
Seg : Buffers.Buffer_Id;
Thdr : System.Address;
begin
-- Open receive window now that application has consumed the data
TPCBs (PCB).RCV_WND := TPCBs (PCB).RCV_WND + AIP.M32_T (Len);
if Len > 0 then
Buffers.Buffer_Alloc
(Size => TCPH.TCP_Header_Size / 8,
Offset => Inet.HLEN_To (Inet.IP_LAYER),
Kind => Buffers.SPLIT_BUF,
Buf => Seg);
Thdr := Buffers.Buffer_Payload (Seg);
TCPH.Set_TCPH_Data_Offset (Thdr, 5);
-- No options present
TCPH.Set_TCPH_Seq_Num (Thdr, TPCBs (PCB).SND_NXT);
TCPH.Set_TCPH_Urg (Thdr, 0);
TCPH.Set_TCPH_Psh (Thdr, 0);
TCPH.Set_TCPH_Syn (Thdr, 0);
TCPH.Set_TCPH_Fin (Thdr, 0);
TCPH.Set_TCPH_Rst (Thdr, 0); -- Force Reset since we are embedded
TCPH.Set_TCPH_Ack (Thdr, 1);
Buffers.Set_Packet_Info (Seg, Thdr);
TCP_Send_Segment
(Tbuf => Seg,
IPCB => IPCBs (PCB),
TPCB => TPCBs (PCB));
Buffers.Remove_Packet
(Buffers.Transport, TPCBs (PCB).Unack_Queue, Seg);
Buffers.Buffer_Blind_Free (Seg);
end if;
end TCP_Recved;
The procedure TCP_Output can also be used to Ack Now but if it also Flush the Send_Queue, anyway this might be a discussion on github with other folks that know this better than me.
When the echo server App close the connection it also release it's Echo State Pool (ESP) by calling ES_Release procedure, which basically marks the connection as ES_FREE state and null address the Application Data.
So now the Echo_Send first call TCP_Recved and after that it writes directly using TCP_Write.
Three-Way Handshaking
Soon as I started working on the mqtt client, I realize the tcp stack was failing to act as a client, but I will discuss those modifications in a moment. Original code of server seems to implement the active close but not the passive close or at least I did't find a way to make it work out of the box, so start looking at what a passive close should look like from server side. It happens that once the FIN is received the server should go to Close_Wait State and send it's corresponding FIN, after that it transition to Last_Ack.
Here is a very helpful diagram of how this.
TCPIP_State_Transition_Diagram.pdf
I added the Last_Ack State check to the TCP_Receive since when this Ack is received it only concerns the TCP stack actions, and so it don't need to be inform application about it. The action consist of transition to Close state and release/reset the Pcb for later reuse, the snippet is
when Last_Ack =>
-- Passive Three Way Last Ack processing
Set_State (PCB, Closed);
TCP_Free (PCB);
Err := AIP.NOERR;
It works because I also did a modification in TCP_Close, I will bring it here the few snippets out there that plays on this.
First all packets with data are passed to the application using this on aip.tcp.adb
if New_Data_Len > 0 then
Deliver_Segment (PCB, Seg.Buf);
end if;
But for some reason I can really understand (that makes me want to rewrite the code) the New_Data_Lenvariable counts theFIN bit, that is, the FIN if received add one "1" to this variable so it enter there for processing, but in fact nothing is done unless the packet has some data, since inside Deliver_Segment procedure the application callback is done if the segment len is not zero, and then nothing else happens. SO is in fact the next piece of code of TCP_Receive that informs our application of the FIN received, that is (the relevant part right now)
if TCPH.TCPH_Fin (Seg.Thdr) = 1 then
case TPCBs (PCB).State is
when Closed | Listen | Syn_Sent =>
null;
when Established | Syn_Received =>
-- Notify connection closed: deliver 0 bytes of data.
-- First transition to Close_Wait, as the application
-- may decide to call Close from within the TCP_Event
-- callback.
Set_State (PCB, Close_Wait);
TCP_Event
(Ev => TCP_Event_T'(Kind => TCP_EVENT_RECV,
Len => 0,
Buf => Buffers.NOBUF,
Addr => IPaddrs.IP_ADDR_ANY,
Port => PCBs.NOPORT,
Err => AIP.NOERR),
PCB => PCB,
Cbid => TPCBs (PCB).Callbacks (TCP_EVENT_RECV),
Err => Err);
The first thing is to change it's state as the diagram told us to do, next call application layer. And now the TCP_Close that follows (relevant part only shown)
when Close_Wait =>
-- Transition to LAST_ACK after sending FIN
TCP_Fin (PCB, Err);
if AIP.No (Err) then
Set_State (PCB, Last_Ack);
-- Purge any remaining Buffer from Unack Queue
TCP_Purge (PCB);
Flush := False;
end if;
The modification I include is a function that Purge PCB queues here, and thou just waiting the Last_Ack to Free the PCB as seen before. The reason of this TCP_Purge is because original stack calls TCP_Fin which sends the FIN but also includes the the packet in the Unack_Queue. This was giving me problems of retransmissions, but it has the possibility that if Ack is not received it will not send again the FIN, so really this need further revisions. One problem I faced is debugging this things, sometimes you need a real fast printf that don't affect execution time and semihosting on ST-Link is all but that.
3. Working on top of ipstack (raw lwIP)Make it smart
The echo server by itself it's not fun nor very useful alone but it motivate me to continue working on this. Since contest time frame was really short (specially for me as I started in mid July) and the fact that I am really new to Ada world, I was not willing to reinvent the wheel, so I start looking at existing code for Application Layer protocols, specially I was looking for HTTP which is nice and really necessary to have. I found a few stacks out there but none made for the bare metal embedded world or even that I was able to compile using the Ada ARM version so I spend some time figuring out which one would be feasible to port. Then I found the work of Dmitry A. Kazakov, which really took my attention because it provides not only HTTP but other network protocols such MQTT and MOdbus TCP. I have to say at first it was really overwhelming work to read and understand for me but perseverance sometimes pays. It's work it's called Simple Components and you can find it here
The HTTP at first was too much work for someone with too little experience in Ada, it uses gnat.Sockets and there were a lot of code to port, then I realize the MQTT client was simpler in the sense that it was less code to port so I start looking at it closer. I also notice that there is a raw lwIP C implementation of MQTT which serve me as a guide when in doubt.
MQTT ClientI did that introduction since I consider this my First attempt to port SImple Components, I actually make it work but hold to see my second attempt while doing HTTP. So keep in mind this because in the future if the project evolves MQTT might change too.
Simple Components implements an MQTT Pier Client that is simple in the sense that it's a minimal implementation, so no brokering on this version. But the author also implements the MQTT Server that supports brokering for the enthusiast :)
The first thing as explained in the docs is that all calls should be done in the context of the connection server, since I have no gnat.Sockets I end up interpreting this as to keep the Connection State for all the calls in my raw lwIP server, but there is a problem, "it's all about callbacks". Did you remember the User Data of the App layer explained in the Echo Server? Well We took advantage of it to implement the State, but moreover this time the MQTT code was on different package and MQTT_Pier type that holds the connection state was really obscure thing to port at first.
The original mqtt client package is named GNAT.Sockets.MQTT, so I start to import those files in my project and remove the GNAT.Sockets from them, so they call mqtt.adb and mqtt.ads in my project. This is mostly the MQTT stack by itself which hopefully as you will see don't have too many modifications. I implement the Client as a child package, that is Mqtt.Client package (mqtt-client.adb). It was nice to start with client since the porting was not that hard as http but the ipstack actually hasn't any client, so at the end it took me some effort but the nice part is that client is there, not as reusable as I would want it but that's another effort.
MQTT protocol handles the connection and disconnection, which is ontop of the TCP layer 4 of OSI model. So the first thing is to connect with a mqtt broker (server). For this I pick up mosquitto for my tests, it was handy since I develop under Linux and I can verify correct functioning of mqtt pier.
First attempt was to pass the MQTT_Pier type as the Udata of the TCP stack, and then kicks in kicks get out my head until I realize there are several things when working with pointers under Ada which are a little different from C/C++ world. The short story is that MQTT_Pier is a non limited type which make me almost impossible to assign as a System.Address for later retrieval. Please don't laugh if you find an easier or better way just let me know so.
So I end up giving up trying that approach and instead keep my original array of state pool, this time I call them MSP for MQTT State Pool.
Before going further let's examine MQTT_Pier type closer.
type MQTT_Pier
(Max_Subscribe_Topics : Positive
) is new Connection with
record
State : MQTT_State := MQTT_Header;
Count : Stream_Element_Count := 0;
Length : Stream_Element_Count := 0;
Max_Size : Stream_Element_Count := 0;
Header : Stream_Element := 0;
Version : Stream_Element;
Flags : Stream_Element;
Keep_Alive : Duration;
QoS : QoS_Level;
Packet_ID : Packet_Identifier;
List_Length : Natural := 0;
-- Secondary : Output_Buffer_Ptr;
List : Unbounded_Ptr_Array;
Data : Stream_Element_Array_Ptr :=
new Stream_Element_Array (1 .. Max_Message_Size);
You might wonder about connection, well here it is
type Connection is abstract new Object.Entity with
record
State_Pool : Mqtt_Conn_State_Access;
App_Cbs : Mqtt_Client_Cbks;
Sid : MS_Id;
Client_Address : AIP.IPaddrs.IPaddr;
Err : AIP.Err_T;
end record;
Original version has several fields on Connection record that did't apply here like socket and because I was afraid of big jumps here if you know what I mean so I end up reducing it as above, you will see later on the http port how I keep most of them except of course the socket.
The State_Pool is an access I use to attach the lwIP client with mqtt code, which is needed as you will see in a moment. Sometimes is difficult to write about code without boring or even worse make the reader lost so I will pick up the example code, which the Top level code of Mqtt so I can go down to highlight the internal mechanism of how Simple Components is connected with lwIP.
The example code serves to proof the mqtt client and as a guide for advance client code. It's shown below and you can find the source under the demos folder.
procedure Test_1 (Client_Ip : in AIP.IPaddrs.IPaddr)
is
Client_Err : AIP.Err_T;
begin
Set (Test_Client_Array (0), new Test_Client (Max_Subscribe_Topics => 20));
declare
Client : Test_Client renames Test_Client (Ptr (Test_Client_Array (0)).all);
PId : Packet_Identification (Qos => At_Most_Once);
begin
Client.Set_Client_Ip (Client_Ip);
Client.Set_Client_Cbs (Test_Connect'Access, Test_Receive'Access);
Client.Connect (Client_Err);
if Client_Err = AIP.NOERR then
while Client.Is_Connected = False loop
delay 0.01;
end loop;
Send_Connect (Client, "TestMQTTclient");
while Client.Is_Connected = False loop
delay 0.01;
end loop;
Send_Ping (Client);
delay 1.0;
Send_Publish (Client,
"makewithAda/ipstack/test",
"bonjour Ada world!",
PId);
delay 2.0;
Send_Disconnect (Client);
else
AIP.IO.Put_Line ("STM32 Connect Error");
end if;
end;
end Test_1;
You specify the IP address of broker, i.e. Test_MQTT_Clients.Test_1 (Client_Ip); where Client_Ip is defined as
Client_Ip : constant AIP.IPaddrs.IPaddr := AIP.IPaddrs.IP4 (192, 168, 2, 5);
The first part allows to have several clients active at a given time, so you basically call new on a Test_Client and save it's reference in a Handle array. The example only uses one client so later you declare a Test_Client type that renames the first entry of the Handle array.
First thing is to apply assign the IP address of broker, perhaps the name is not the more appropriate here, anyway after that you define two callbacks one for the connection and another for the reception, both refers to TCP events, the first one might do nothing as it actually does, but the second one is very important since it was the way around I came to feed the data of TCP connection into the right MQTT Pier. I was some how lazy here since the code of reception is not correct in the sense that it pick up the first not null handle client to feed the data in. So another check should be performed, but it's so easy that let's do it right now.
First let's check the callback.
procedure Test_Receive
(Pcb : AIP.PCBs.PCB_Id;
DATA : Stream_Element_Array;
Err : out AIP.Err_T)
is
begin
pragma Unreferenced (Pcb);
for Hitem in Test_Client_Array'Range loop
if Test_Client_Array (Hitem) /= Null_Handle then
declare
Client : Test_Client renames Test_Client (Ptr (Test_Client_Array (Hitem)).all);
Pointer : Stream_Element_Offset := DATA'First;
begin
Client.Received (DATA, Pointer);
end;
end if;
end loop;
Err := AIP.NOERR;
end Test_Receive;
Notice how we pass Pcb as parameter. We need to compare this with the Client Pcb, since that is private type one option is to write a little function that does for us. So let's go to mqtt.adb and add such a function we will call Does_Mqtt_Pier_Match as follows.
function Does_Mqtt_Pier_Match
(Pier : in out MQTT_Pier;
Pcb : AIP.PCBs.PCB_Id) return Boolean
is
begin
return Pier.State_Pool.Pcb = Pcb;
end Does_Mqtt_Pier_Match;
Now we can rewrite the callback as follows:
procedure Test_Receive
(Pcb : AIP.PCBs.PCB_Id;
DATA : Stream_Element_Array;
Err : out AIP.Err_T)
is
begin
for Hitem in Test_Client_Array'Range loop
if Test_Client_Array (Hitem) /= Null_Handle then
declare
Client : Test_Client renames Test_Client (Ptr (Test_Client_Array (Hitem)).all);
Pointer : Stream_Element_Offset := DATA'First;
begin
if Client.Does_Mqtt_Pier_Match (Pcb) then
Client.Received (DATA, Pointer);
end if;
end;
end if;
end loop;
Err := AIP.NOERR;
end Test_Receive;
You start getting the idea, so now let continue with the demo code. After callbacks are set we do a Connect, which indeed calls Mqtt_Client_Connect with the parameters of Client. Here is the code.
--------------------------
-- Mqtt_Client_Connect ---
--------------------------
procedure Mqtt_Client_Connect
(Pier : in out MQTT_Pier;
Port : Natural := MQTT_Port;
Err : out AIP.Err_T)
is
Pcb : AIP.PCBs.PCB_Id;
Sid : MS_Id;
begin
-- check if client already is allocated or sort of
if Pier.State_Pool /= null then
Err := AIP.ERR_MEM;
return;
end if;
AIP.TCP.TCP_New (Pcb);
if Pcb = AIP.PCBs.NOPCB then
Err := AIP.ERR_MEM;
return;
end if;
AIP.TCP.TCP_Bind
(PCB => Pcb,
Local_IP => AIP.IPaddrs.IP_ADDR_ANY,
Local_Port => 0,
Err => Err);
if Err /= AIP.NOERR then
goto Tcp_Error;
end if;
AIP.TCP.TCP_Connect
(PCB => Pcb,
Addr => Pier.Client_Address,
Port => Net.Uint16 (Port),
Cb => RAW_TCP_Callbacks.To_CBID (MQTT_Process_Connect'Access),
Err => Err);
if Err /= AIP.NOERR then
goto Tcp_Error;
end if;
MS_Alloc (Sid);
if Sid = NOMS then
-- AIP.TCP.TCP_Free (Pcb); -- I think original stack miss this need
Err := AIP.ERR_MEM;
goto Tcp_Error;
else
Pier.Sid := Sid;
Pier.State_Pool := MSP (Sid)'Access;
Pier.State_Pool.Kind := TCP_CONNECTING;
Pier.State_Pool.Pcb := Pcb;
Pier.State_Pool.Buf := AIP.Buffers.NOBUF;
MSP (Sid).App_Client_Cbs := Pier.App_Cbs;
AIP.TCP.TCP_Set_Udata (Pcb, MSP (Sid)'Address);
-- Start cyclic timer for the corresponding client
Timers.Timer_Alloc (Pier.State_Pool.Tmr_Id, MQTT_Timer'Access);
Timers.Set_Interval (Pier.State_Pool.Tmr_Id, MQTT_CYCLIC_TIMER_INTERVAL * MQTT_Tick_Interval);
Pier.State_Pool.Cyclic_Tick := 0;
end if;
-- set error callback
AIP.TCP.On_TCP_Abort
(Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Abort'Access));
return;
<<Tcp_Error>>
declare
tcpErr : AIP.Err_T;
begin
pragma Unreferenced (tcpErr);
AIP.TCP.TCP_Close (Pcb, tcpErr);
end;
end Mqtt_Client_Connect;
First part check if client is in use, as actually after the connection gets thru the pointer is set to the assigned MSP variable of Pool. We still use the TCP_Set_Udata to save the address of it for recovering at the callbacks.
Original code from Dmitry don't has the Timers, not sure but at least the MQTT Pier doesn't have it, but after looking at the lwIP implementation I decide it was necessary so the Timer is allocated and set with initial interval, some sanity check and the abort callback. The Connect callback is set in the same TCP_Connect procedure and it's code is as follow.
--------------------------
-- MQTT_Process_Connect --
--------------------------
procedure MQTT_Process_Connect
(Ev : AIP.TCP.TCP_Event_T;
Pcb : AIP.PCBs.PCB_Id;
Err : out AIP.Err_T)
is
Ms : Mqtt_Conn_State;
for Ms'Address use AIP.TCP.TCP_Udata (Pcb);
Data : Stream_Element_Array (0 .. -1);
begin
pragma Unreferenced (Ev);
-- AIP.IO.Put_Line ("New PCB Accept: Sid");
AIP.TCP.On_TCP_Sent
(Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Sent'Access));
AIP.TCP.On_TCP_Recv
(Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Recv'Access));
AIP.TCP.On_TCP_Poll
(Pcb, RAW_TCP_Callbacks.To_CBID (MQTT_Process_Poll'Access), 2 * 500);
-- Enter MQTT connect state
Ms.Kind := TCP_CONNECTED;
-- Reset the Timer Tick for client
Ms.Cyclic_Tick := 0;
-- Call the User Procedure if available
RAW_MQTT_Dispatcher.MQTT_Event
(PCB => Pcb,
DATA => Data,
Cbid => Ms.App_Client_Cbs.App_Connect_Cb,
Err => Err);
end MQTT_Process_Connect;
You should be already familiar with the callbacks here except that we are calling the higher layer TCP connect callback, remember the one that does nothing. Let's move into reception to close that loop. The Reception callback is not very different from echo server, but instead of forward the Pcb to an Echo_Send, we need to dispatch the received data to upper layer, so here is the dispatching function.
------------------------
-- Mqtt_Dispatch_Recv --
------------------------
procedure Mqtt_Dispatch_Recv
(Pcb : AIP.PCBs.PCB_Id;
Ms : in out Mqtt_Conn_State)
is
Buf : AIP.Buffers.Buffer_Id;
srcAdd : System.Address;
Plen : AIP.U16_T;
TPlen : AIP.U16_T;
-- Err : AIP.Err_T := AIP.NOERR;
Item_Size : Stream_Element_Offset;
Err : AIP.Err_T;
begin
if Ms.Buf = AIP.Buffers.NOBUF then
return;
end if;
TPlen := AIP.Buffers.Buffer_Tlen (Ms.Buf);
declare
Data : Stream_Element_Array (1 .. Stream_Element_Offset (TPlen));
Pointer : Stream_Element_Offset := Data'First;
begin
loop
Buf := Ms.Buf;
Plen := AIP.Buffers.Buffer_Len (Buf);
Item_Size := Stream_Element_Offset (Plen);
declare
type SEA_Pointer is
access all Stream_Element_Array (1 .. Item_Size);
srcPtr : SEA_Pointer;
function As_SEA_Pointer is
new Ada.Unchecked_Conversion (System.Address, SEA_Pointer);
Data_Ptr : SEA_Pointer;
begin
srcAdd := AIP.Buffers.Buffer_Payload (Buf);
srcPtr := As_SEA_Pointer (srcAdd);
Data_Ptr := As_SEA_Pointer (Data (Pointer)'Address);
Data_Ptr.all (1 .. Item_Size) := srcPtr.all (1 .. Item_Size);
-- Grab reference to the following Buf, if any
Ms.Buf := AIP.Buffers.Buffer_Next (Buf);
if Ms.Buf /= AIP.Buffers.NOBUF then
AIP.Buffers.Buffer_Ref (Ms.Buf);
end if;
-- Deallocate the processed buffer
AIP.Buffers.Buffer_Blind_Free (Buf);
exit when Ms.Buf = AIP.Buffers.NOBUF;
-- Pointer "points" to Next element to copy
Pointer := Pointer + Item_Size + 1;
if Pointer > Data'Last then
Raise_Exception
(Layout_Error'Identity,
"Invalid pointer"
);
end if;
end;
end loop;
-- Signal TCP layer that we can accept more data
AIP.TCP.TCP_Recved (Pcb, TPlen);
pragma Warnings (Off, """Err"" modified by call, *");
RAW_MQTT_Dispatcher.MQTT_Event
(PCB => Pcb,
DATA => Data,
Cbid => Ms.App_Client_Cbs.App_Receive_Cb,
Err => Err);
pragma Warnings (On, """Err"" modified by call, *");
end;
end Mqtt_Dispatch_Recv;
This was the part of the project that I learn (one way good or bad?) how to write to a Stream_Element_Array. At the end of the loop Data ends up with the received bytes, so the callback is done.
Mqtt Client Send
The Mqtt stack reacts to events, the "main" code initiates a TCP connect, then it follows the Mqtt Connect Command which obviously need to send a TCP segment, so we create the Send procedure that will be used all over the mqtt package when there is an Stream_Element_Array of data to send. The code lives in the mqtt.adb and is shown below.
procedure Send
(Pier : in out MQTT_Pier;
Data : Stream_Element_Array
) is
Pointer : Stream_Element_Offset := Data'First;
size : AIP.Buffers.Data_Length;
BufLen : AIP.Buffers.Buffer_Length;
p : AIP.Buffers.Buffer_Id;
dstAdd : System.Address;
q : AIP.Buffers.Buffer_Id;
Item_Size : Stream_Element_Offset;
begin
size := AIP.U16_T (Data'Last);
AIP.Buffers.Buffer_Alloc (0, size, AIP.Buffers.LINK_BUF, p);
if p /= AIP.Buffers.NOBUF then
q := p;
loop
BufLen := AIP.Buffers.Buffer_Len (q);
Item_Size := Stream_Element_Offset (BufLen);
declare
type SEA_Pointer is
access all Stream_Element_Array (1 .. Item_Size);
dstPtr : SEA_Pointer;
function As_SEA_Pointer is
new Ada.Unchecked_Conversion (System.Address, SEA_Pointer);
Data_Ptr : SEA_Pointer;
begin
dstAdd := AIP.Buffers.Buffer_Payload (q);
dstPtr := As_SEA_Pointer (dstAdd);
Data_Ptr := As_SEA_Pointer (Data (Pointer)'Address);
-- copy the actual data using the streams access pointers
dstPtr.all (1 .. Item_Size) := Data_Ptr.all (1 .. Item_Size);
end;
q := AIP.Buffers.Buffer_Next (q);
-- check if there is one more buffer to fill
exit when q = AIP.Buffers.NOBUF;
-- Pointer "points" to Next element to copy
Pointer := Pointer + Item_Size + 1;
if Pointer > Data'Last then
Raise_Exception
(Layout_Error'Identity,
"Invalid pointer"
);
end if;
end loop;
declare
Ms : Mqtt_Conn_State;
for Ms'Address use Clients.MSP (Pier.Sid)'Address;
begin
Ms.Buf := p;
Clients.Mqtt_Send (Ms.Pcb, Ms);
end;
else
Pier.Err := ERR_MEM;
end if;
end Send;
Somehow is the inverse operation of reception, this time we need to reserve lwIP buffers to hold the data.
Mqtt Timer
Last but not least the mqtt timer helps with Keep alive functionality, for more info go here. The keepalive is implemented for each connected client and the code is below.
-----------------------------------------------
-- MQTT_Timer -- CycleTick Timer Callback --
-----------------------------------------------
procedure MQTT_Timer (Id : Integer) with
Refined_Global => (In_Out => (MSP))
is
Sid : MS_Id := NOMS;
begin
null;
for J in MSP'Range loop
if MSP (J).Tmr_Id = Id then
Sid := J;
exit;
end if;
end loop;
if Sid = NOMS then
return; -- this should not happen
end if;
-- Sid Points to the Client Connection State Data that MQTT_Timer belongs
case MSP (Sid).Kind is
when MS_CONNECTING =>
MSP (Sid).Cyclic_Tick := MSP (Sid).Cyclic_Tick + 1;
if MSP (Sid).Cyclic_Tick * MQTT_CYCLIC_TIMER_INTERVAL >= MQTT_CONNECT_TIMOUT then
-- Disconnect TCP
Mqtt_Close (MSP (Sid).Pcb, MSP (Sid));
Timers.Timer_Stop (Id);
end if;
when MS_CONNECTED =>
-- keep_alive > 0 means keep alive functionality shall be used
if MSP (Sid).Keep_Alive > 0 then
MSP (Sid).Server_Watchdog := MSP (Sid).Server_Watchdog + 1;
-- If reception from server has been idle for 1.5*keep_alive time,
-- server is considered unresponsive
if MSP (Sid).Server_Watchdog * MQTT_CYCLIC_TIMER_INTERVAL >=
MSP (Sid).Keep_Alive + MSP (Sid).Keep_Alive / 2
then
Mqtt_Close (MSP (Sid).Pcb, MSP (Sid));
Timers.Timer_Stop (Id);
end if;
if MSP (Sid).Cyclic_Tick * MQTT_CYCLIC_TIMER_INTERVAL >=
MSP (Sid).Keep_Alive
then
-- Sending keep-alive message to server
declare
Ps : MQTT_Pier_Ptr;
for Ps'Address use AIP.TCP.TCP_Udata (MSP (Sid).Pcb);
begin
Send_Ping (Ps.all);
MSP (Sid).Cyclic_Tick := 0;
end;
else
MSP (Sid).Cyclic_Tick := MSP (Sid).Cyclic_Tick + 1;
end if;
end if;
when TCP_CONNECTING =>
MSP (Sid).Cyclic_Tick := MSP (Sid).Cyclic_Tick + 1;
if MSP (Sid).Cyclic_Tick * MQTT_CYCLIC_TIMER_INTERVAL >= TCP_CONNECT_TIMOUT then
-- Disconnect TCP
Mqtt_Close (MSP (Sid).Pcb, MSP (Sid));
Timers.Timer_Stop (Id);
end if;
when others =>
-- Timer should not be running in this state - perhaps TCP_ABORT
Timers.Timer_Stop (Id);
end case;
end MQTT_Timer;
Notice that it might close and unresponsive connection and also send the Ping Commands when needed by keep-alive.
Demo Time
A short video of the Test demo code follows.
HTTP ServerWith more knowledge and previous experience I toward to HTTP server porting. The HTTP require the connection state machine and it's http state machine which heavily depend on gnat.Sockets.Server which fortunately is a package under the Simple Components source. This Server package has a task worker that handle all incoming connections using gnat.sockets, so a big part of the code doesn't apply here to the lwIP scheme, but some supporting code was needed in its place, so what I did was to copy all of it into a rename package called RAW_LwIp_Server (raw_lwip_server.adb) and star commenting code like if I wasn't sure what I was doing, yes very crazy moment just desperate to get rid of all compiler errors and warnings.
At the end I used like 60 % of the code but the commented original code is still there since I haven't tested all the HTTP code, so basically I have a working Hello World.htm generated in code. It was my desire to be able to load.htm from the SD card but actually I had a hard time to make the SD card file system work on my STM32F769I_DISCO.
Let's discuss how it works and have a final demo (yes you can go ahead to watch the demo first! ;)
It's difficult and boring to step over each detail here, so similar to mqtt explanation, I will start from the Demo Code and try to give as much details as possible.
Hello World
The demo consists of the classic hello world HTTP GET. The package Test_HTTP_Servers contains all the code, from the "main" file we only start the task which is suspended waiting for a flag. The task called worker is shown below.
-- ------------------------------
-- Start the HTTP Server Loop.
-- ------------------------------
procedure Start is
begin
Ada.Synchronous_Task_Control.Set_True (HTTP_Ready);
end Start;
task body Worker is
timer_deadline : Ada.Real_Time.Time;
begin
-- Wait until started
Ada.Synchronous_Task_Control.Suspend_Until_True (HTTP_Ready);
AIP.IO.Put_Line ("HTTP Server Task Starting");
declare
Factory : aliased Test_HTTP_Factory
(Request_Length => 200,
Input_Size => 1024,
Output_Size => 1024,
Max_Connections => 100
);
Server : Connections_Server (Factory'Access, Port_Type (80));
begin
timer_deadline := Ada.Real_Time.Clock;
loop
Server.Poll_Connections;
delay until timer_deadline;
timer_deadline := timer_deadline + HTTP_Polling;
end loop;
end;
end Worker;
Factory and Server types were the reasons to have the original Server ported, they provide an easy and clean way to create the Server. The Connections_Factory is redefined as:
type Connections_Factory is
new Ada.Finalization.Limited_Controlled with
record
Port : Natural;
Socket_SP : Socket_State_Access_Array;
end record;
Where similar to the State Pool seen before, Socket_State_Access_Array is an array of those Pools.
type Socket_State is record
Kind : State_Kind;
Pcb : AIP.PCBs.PCB_Id;
Buf : AIP.Buffers.Buffer_Id;
Err : AIP.Err_T;
Server_Cbs : Socket_Server_Cbks;
Client : Connection_Ptr;
end record;
type Socket_State_Access is access all Socket_State;
type Socket_State_Array is array (Valid_SS_Id) of aliased Socket_State;
type Socket_State_Access_Array is array (Valid_SS_Id) of Socket_State_Access;
Instead of having an assignment of State_Pool pointers, here we have a one to one relationship of the pointer array with the serve pool array. This is independent of HTTP code, so other server can use the same Factory primitive right now.
Connections_Server has been reduce to:
type Connections_Server
(Factory : access Connections_Factory'Class;
Port : Port_Type
) is new Ada.Finalization.Limited_Controlled with
record
Clients : Natural := 0;
Servers : Natural := 0;
Postponed_Count : Natural := 0;
Postponed : Connection_Ptr;
Unblock_Send : Boolean := False;
end record;
The trick here is that when you instantiate a Connections_Server it will call Initialize, as my knowledge holds this is because it's a type of Ada.Finalization.Limited_Controlled, so an Initialize procedure will override the default one, which is the case of:
overriding
procedure Initialize (Listener : in out Connections_Server) is
Err : AIP.Err_T;
begin
-- pragma Unreferenced (Err, Listener);
Listener.Factory.Port := Integer (Listener.Port);
API_Process.Initialize (Listener'Access, Listener.Factory, Err);
if Err /= AIP.NOERR then
AIP.IO.Put_Line ("Error Init Server");
end if;
end Initialize;
In order to not mess up the original Server file, I created a child package with the underlying already well known ipstack server, called API_Process. The structure is very similar to a mix of the Echo_Server seen before and the MQTT_Client since we use similar dispatch procedure for receiving data, but there are some novel changes here of course, let's discuss them. (Notice this is general Server that can be used for other protocol servers.)
Initialize
Let's examine the Initialize code shown below.
----------
-- Init --
----------
procedure Initialize
(Listener : access Connections_Server'Class;
Factory : access Connections_Factory'Class;
Err : out AIP.Err_T)
is
Pcb : AIP.PCBs.PCB_Id;
Port : Integer;
begin
-- pragma Unreferenced (Pcb, Port, Err);
-- pragma Unreferenced (Listener);
Init_SS_Pool;
lwIp_Listener := Listener.all'Unchecked_Access;
lwIp_Factory := Factory.all'Unchecked_Access;
Port := Integer (lwIp_Factory.Port);
AIP.TCP.TCP_New (Pcb);
if Pcb = AIP.PCBs.NOPCB then
Err := AIP.ERR_MEM;
else
AIP.TCP.TCP_Bind
(PCB => Pcb,
Local_IP => AIP.IPaddrs.IP_ADDR_ANY,
Local_Port => AIP.U16_T (Port),
Err => Err);
end if;
if Err = AIP.NOERR then
AIP.TCP.TCP_Listen (Pcb, Err);
pragma Assert (AIP.No (Err));
-- Factory Socket State Pool is linked
for Sid in lwIp_Factory.Socket_SP'Range loop
lwIp_Factory.Socket_SP (Sid) := SSP (Sid)'Access;
end loop;
AIP.TCP.On_TCP_Accept
(Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Accept'Access));
end if;
end Initialize;
The problem once again was how to link the lwIP state pool with the application layer. The answer after a huge fight against my ignorance in Ada language was to have a pointer to the Connections_Server and Factory, right now the code only allows one server since there are only one instance of each, so further adjustment is needed to have two servers.
The access types are declared as:
lwIp_Listener : Connections_Server_Ptr;
lwIp_Factory : access Connections_Factory'Class;
And you can see how the assignment is done, after the server is bonded and listening, the state pool is assigned as well, this time indexed by the Sid. Finally the Accept callback is configured. The Accept callback has new stuff to discover.
procedure SOCKET_Process_Accept
(Ev : AIP.TCP.TCP_Event_T;
Pcb : AIP.PCBs.PCB_Id;
Err : out AIP.Err_T)
is
-- Factory : constant access Connections_Factory'Class := ;
Sid : SS_Id;
Factory : access Connections_Factory'Class renames lwIp_Factory;
begin
Err := AIP.NOERR;
SS_Alloc (Sid);
if Sid = NOSS then
Err := AIP.ERR_MEM;
else
Factory.Socket_SP (Sid).Kind := SS_ACCEPTED;
Factory.Socket_SP (Sid).Pcb := Pcb;
Factory.Socket_SP (Sid).Buf := AIP.Buffers.NOBUF;
AIP.TCP.TCP_Set_Udata (Pcb, SSP (Sid)'Address);
AIP.TCP.On_TCP_Sent
(Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Sent'Access));
AIP.TCP.On_TCP_Recv
(Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Recv'Access));
AIP.TCP.On_TCP_Abort
(Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Abort'Access));
AIP.TCP.On_TCP_Poll
(Pcb, RAW_TCP_Callbacks.To_CBID (SOCKET_Process_Poll'Access), 500);
AIP.TCP.TCP_Accepted (Pcb);
declare
Data : Ada.Streams.Stream_Element_Array (1 .. 1);
begin
-- This is our way around to associate the Server Connection
-- Pool to a given Pcb and thou fill specific client callbacks
-- at the upper layer.
Data (Data'First) := Ada.Streams.Stream_Element (Sid);
RAW_SOCKET_Dispatcher.SOCKET_Event
(Client => Factory.Socket_SP (Sid).Client,
PCB => Pcb,
DATA => Data, -- Not really used in upper layer let's see...
Cbid => RAW_SOCKET_Callbacks.To_CBID (Do_Create'Access),
Err => Err);
if Factory.Socket_SP (Sid).Client = null then
Err := AIP.ERR_MEM;
Socket_Close (Pcb, SSP (Sid));
else
declare
This : Connection'Class renames Factory.Socket_SP (Sid).Client.all;
begin
This.Sid := Sid;
This.Client := False;
This.Connect_No := 0;
This.Client_Address := Ev.Addr;
This.Client_Port := Ev.Port;
This.Err := AIP.NOERR;
Clear (This);
This.Listener := lwIp_Listener.all'Unchecked_Access;
if This.Transport = null then -- Ready
This.Session := Session_Connected;
Connected (This);
else
This.Session := Session_Handshaking;
end if;
Err := This.Err;
end;
end if;
end;
end if;
end SOCKET_Process_Accept;
If Socket State allocation goes well we set the callbacks and accept the connection as before, also we modify the State Pool, but notice we use the access pointer of our factory from now on. Thereafter comes a callback to the parent Server package, this callback is a replacement of the underlying overriding behavior of gnat.Sockets that original code used to take advantage of, so the callback is actually calling the Create procedure in RAW_lwip_server shown below.
procedure Do_Create
(Client : in out Connection_Ptr;
Pcb : AIP.PCBs.PCB_Id;
DATA : Stream_Element_Array;
Err : out AIP.Err_T)
is
begin
Client := Create
(Factory => API_Process.lwIp_Factory, -- API_Process.Server'Access,
Pcb => Pcb,
Data => DATA,
Err => Err);
end Do_Create;
function Create
(Factory : access Connections_Factory;
Pcb : AIP.PCBs.PCB_Id;
Data : Stream_Element_Array;
Err : out AIP.Err_T
) return Connection_Ptr is
begin
pragma Unreferenced (Factory, Pcb, Data, Err);
-- Default Implementation returns null.
return null;
end Create;
As you can see by default it does nothing, but the overriding Create procedure in Test_HTTP_Servers executes in place of:
overriding
function Create
(Factory : access Test_HTTP_Factory;
Pcb : AIP.PCBs.PCB_Id;
Data : Stream_Element_Array;
Err : out AIP.Err_T
) return Connection_Ptr is
Result : Connection_Ptr;
begin
pragma Unreferenced (Pcb, Data);
-- if Get_Clients_Count (Listener.all) < Factory.Max_Connections then
Result :=
new Test_HTTP_Client
(Request_Length => Factory.Request_Length,
Input_Size => Factory.Input_Size,
Output_Size => Factory.Output_Size
);
-- Receive_Body_Tracing (Test_Client (Result.all), True);
-- Receive_Header_Tracing (Test_Client (Result.all), True);
Err := AIP.NOERR;
return Result;
end Create;
It's under this Create function that HTTP Clients are created and final bonding to the Pcb is done when returns to Accept callback of lwIP. Notice that we check the Client for null value, in such a case it fails to create it we proceed to close the connection, otherwise an initialization is done where Ip and Port number are set from the Event received.
The receive dispatch procedure is very similar, will only show the callback in this case. Notice that it's fixed to always call a Do_Receive.
RAW_SOCKET_Dispatcher.SOCKET_Event
(Client => Ss.Client,
PCB => Pcb,
DATA => Data,
Cbid => RAW_SOCKET_Callbacks.To_CBID (Do_Receive'Access),
Err => Err);
Do_Receive and the Received are similar to Do_Create, this time the Received procedure is overwriting in Connections_State_Machine.HTTP_Servers package, since this has the ability to dispatch the request accordingly. The Test Server implements higher level of abstraction, for example the hello world demo is responded using Do_Get procedure, here is for reference:
overriding
procedure Do_Get (Client : in out Test_HTTP_Client)
is
Status : Status_Line renames Get_Status_Line (Client);
begin
case Status.Kind is
when None =>
null;
when File =>
if Status.File = "hello.htm" then
Send_Status_Line (Client, 200, "OK"); -- Response status line
Send_Date (Client); -- Date header line
Send_Server (Client); -- Server name
Send_Connection (Client, False);
Send_Content_Type (Client, "text/html"); -- Content type
Accumulate_Body (Client, "<html><body>"); -- Begin content construction
Accumulate_Body (Client, "<p>Hello world!</p>");
Accumulate_Body (Client, "</body></html>");
Send_Body (Client);
-- Send_Body_Now (Client); -- Evaluate total length, send length
end if;
when URI =>
null;
end case;
end Do_Get;
We are not ready for the demo yet. Remember the HTTP task created? Original code uses gnat.Sockets to pool network descriptors like a select will do on sockets, ready for read, ready for write and so.. it was very cleaver how it works when I finally figure it out but actually does not apply here, but since we need to adapt our code, it happens that the HTTP state machine has a funny way to work. Each HTTP client has a type that is very long record defined at HTTP_Server but important to notice is that is a new type of type State_Machine which in fact is new type of Connection, the one that we mess up during MQTT client. But now we are using Connection type as original code (which means MQTT Client can be improved and bring broker alive one of these days...).
I was going to highlight that Connection type has a Read and Written Buffer which the HTTP (and others under Simple Components) server read and write to send data over the network, and with some cleaver logic application informs the socket task (Worker) when there is Data to send, in the Simple Components documentation this is referred as to Unblock. Application calls like before example code, such as Send_Status_Line (Client, 200, "OK"); write to the Written buffer and then Unblock the "socket" for sending it.
I was going to give up when I realize it but then I thought ipstack server can also has a task that uses "almost" similar mechanism to unblock the "Send". So I present to you the Pool_Connections procedure that mimic such original code or sort of it.
procedure Poll_Connections (Listener : in out Connections_Server) is
fab : access Connections_Factory'Class;
Block : Boolean;
begin
while Listener.Unblock_Send loop
Listener.Unblock_Send := False;
fab := Listener.Factory.all'Unchecked_Access;
for Sid in fab.Socket_SP'Range loop
if fab.Socket_SP (Sid).Kind = SS_RECEIVED then
declare
Client : Connection_Ptr renames fab.Socket_SP (Sid).Client;
begin
if not Client.all.Written.Send_Blocked then
-- Time to Process this Client Buffer
Write (Client.all, Block);
if not Block then
Listener.Unblock_Send := True;
else
Client.all.Written.Send_Blocked := True;
if Client.all.Data_Sent then
Data_Sent (Listener, Client);
end if;
end if;
end if;
end;
end if;
end loop;
end loop;
end Poll_Connections;
I setup this procedure to be call from Worker task every few milliseconds, haven't play a lot with it but it works well for 50 or 20 ms. Here you can see a tcpdump capture of the hello.htm GET request (please notice the ARP cache was outdated during this capture).
Demo Video of HTTP Hello World
Bogus BonusI did my best to do something extra during the deadline extension of three days, so I end up porting the DNS code from ada-enet to the ipstack. The example is included and working capture of google lookup is show here.
The last dns code is already on github (below the link)
The code uses a similar approach to ada-enet, it defines an array of Query, but instead of the UDP socket from ada-enet, I uses a connection object as shown below. This approach uses a private Array of Dns queries access to points to the Application Array defined in Dns_List package.
type UdpClient is abstract new Object.Entity with
record
Udp_State_Pool : Dns_Conn_State_Access;
Sid : DS_Id;
Client_Address : AIP.IPaddrs.IPaddr;
Err : AIP.Err_T;
end record;
protected type Request is
procedure Set_Result (Addr : in AIP.IPaddrs.IPaddr;
Time : in Net.Uint32);
procedure Set_Status (State : in Status_Type);
function Get_IP return AIP.IPaddrs.IPaddr;
function Get_Status return Status_Type;
function Get_TTL return Net.Uint32;
private
Status : Status_Type := NOQUERY;
Ip : AIP.IPaddrs.IPaddr := 0;
Ttl : Net.Uint32;
end Request;
type Query is new UdpClient with record
Name : String (1 .. DNS_NAME_MAX_LENGTH);
Name_Len : Natural := 0;
Deadline : Ada.Real_Time.Time;
Xid : Net.Uint16;
Result : Request;
end record;
The function DNS.Init has to be call before any Resolve, the initialization pass the Array of Query to the package DNS in order to set the access pointers correctly.
procedure Init (Query_Array : access Dns_Query_Array)
is
begin
for item in Dns_Queries'Range loop
Dns_Queries (item) := Query_Array (item)'Access;
end loop;
Init_Done := True;
end Init;
After that a Resolve can be done very similar to original ada-enet. The example code shows a simple loop to wait until DNS server respond. Below is the code.
Dns_List.Queries (1).Resolve ("www.google.com");
-- This won't get thru since original code accept non-authoritative only
-- Dns_List.Queries (1).Resolve ("x83vdeb2.localdeb.net");
declare
Timeout : Natural := 0;
begin
timer_deadline := Ada.Real_Time.Clock;
loop
timer_deadline := timer_deadline + timer_delay;
delay until timer_deadline;
if Dns_List.Queries (1).Get_Status = NOERROR then
if Dns_List.Queries (1).Get_Name'Length > 0 then
AIP.IO.Put_Line (Dns_List.Queries (1).Get_Name);
AIP.IO.Put_Line (To_String (Dns_List.Queries (1).Get_Ip));
exit;
end if;
elsif Timeout > 10 then
AIP.IO.Put_Line ("DNS Lookup Timeout");
exit;
else
Timeout := Timeout + 1;
end if;
end loop;
end;
I intentionally left a comment in the code above about the local query, this is because I setup dnsmasq on my development machine to work on the interface I have connected the discovery board. I found that there is a check performed by ada-enet code and Flags received, in this case the Flags value was 0x8580, and the code checks for 0x8100, this is because the domain is local the NDS Answer as an Authoritative Server, setting that bit which makes it 0x85xx in this case.
I haven't study the code to know if it can parse or not the AnswerThe important oart is that there is one less thing to work on for a more complete ipstack (it remains DHCP and NTP which ada-enet provides).
Source code and build instructions can be found on github project so the idea is to call for contributors and testers.
Known IssuesThere should be a section on git project to talk about this, but let's mention here for project completeness.
As I have commented, the lwIP was modified, I did my best with a lot of pressure to get results in doing so, but more tests are needed and possible some rework.
One issue I have notice is about some packet from HTTP server after some time during operation and several queries that seems to break state machine or client state. The tcpdump.zip file contains some captures that contains unexpected segments, for example capture FIN_ACK_RETRANS shows how several HTTP GET queries were successfully handle by server but then unexpected FIN-ACK re transmission appears.
Similar unexpected segment from server was shown in another capture, this time a RST-ACK segment during what seems to be a normal GET HTTP query.
Also the captures show coincidental or not with the unexpected segments how the Server send some ICMP segments, I notice it and comment the part of the code that executes those Pings, original ipstack IP_Input procedure while dispatching packets, if it doesn't find the protocol the packet belongs it send's an ICMP_Reject packet (See Dispatch_Upper procedure inside IP_Input). That might not be the case here but also a couple of packets from my Linux machine came in just before with MAC address of broadcast, perhaps the ARP engine is failing here.
The reason I add a permanent entry to the STM ARP cache was because I notice it fails to subsequent Server responses when ARP was in he middle, it's necessary more tests here since at the end I also correct a problem with pbufs been allocated and not free which might have to do with ARP since it uses a lot the Unack_Queue when working this way.
It happen that buffers are allocated using TCP_New, but somehow during development I break the mechanism of TCP Queues, actually I found an issue that makes me go back to test the ipstack on my Linux host to confirm if the issue was there, and it actually has a bug, I suspect is related to the buffer allocation and queues since I correct it for the stm32 ipstack, it happen that if you take a buffer out of some Queue you better do something with the buffer after that. For example TCP_Output get buffers from the Send_Queue and after sending the segments it place the buffer in the Unack_Queue so the logic that follows is that after they are taken out from Unack_Queue they should be Free, otherwise they will live in memory, so the problem will not be notice until TCP_New fails which will brake the state machine. Which was actually sort of the issue I notice during ARP queries.
You can easily confirm the ipstack bug, it took me some iteration to make it work under linux since the low level driver does something with OS calls to configure the interface, so I manually tweak the stuff to make it work. Just briefly, there is a config file of ipstack called aip-config.ads, the number of data buffer statically allocated is Data_Buffer_Num which is 32, just do the test an lower it to 5 for example in order to test it quickly, then using the echo server try to do five "echo tests", that is, connect, send some data and disconnect, for sure it will end up with an exception and program ends abruptly. I found the issue since the embedded was doing the same, then after a long debugging session I notice that the Unack_Queue state when I new connection starts was holding an always increasing buffer_id number, which should not happen since disconnect should free the memory.
I mentioned somewhere that Callbacks from Timer and in general callbacks from the TCP stack should not take too much before returning in, the MQTT_DELAY_EXAMPLE pcap capture shows the effect of a delay, you can notice the FIN-ACK from server that waits for the Last_ack, in this case from our MQTT client is re-transmitted, but can notice that the ACK is then transmitted, after debugging the issue I notice it was the semihosting printing on the disconnect callback. As you know semihosting here is really bad since ST-Link has not good support for it.
Final thoughtsThis project address several problems or better said, several missing components of Ada Library, and left a good path continue development since it's 100% Ada code that can be improved and extended but also it's innovative in the way that apart from solving those lacking areas it merges different projects in a cleaver way, which due to it's nature was difficult to test and debug in the Time Frame of the Contest, so I am satisfy with results achieved.
I couldn't discuss all the details here about the implementation but try to cover important topics, so I left aside some aspect about the porting to stm32, for example there is a folder called build under sources that has some code to handle the network packets, it happens that this was generated using xmlada which is not available for ARM version, so the trick was to include the already generated files. HTTP has some package dependency, which is Ada.Calendar that it's not available, I did my best to fill the skeleton functions but those actually do not process anything right now. I share the code in github since I believe in collaboration and free software, after all I got this opportunity in part from existing free code out there, so this will be my return and of course my willing is to have a chance in the contest to win with this project.
This project was a very good challenge to me since I have to learn Ada as quickly as possible, and not been very use to lwIP it was almost new stuff everyday to learn before starting getting some result, but the deadline pressure sometimes give you what is needed to do it, so I enjoy doing it and I appreciate the people interacted with me, specially thanks to Fabien that share me the ipstack code (it's some obscure path on git far away from google sight) and Stephane for the git issue support.
But also specially thanks to my wife for the support on many overnight working hours on this project.
Comments