This guide is written for embedded developers who work on connected products. That includes microcontroller based systems that operate in bare metal or RTOS mode, as well as microprocessor based systems which run embedded Linux.
All fundamental concepts are explained in details, so the guide should be understood by a reader with no prior networking knowledge.
A reader is expected to be familiar with microcontroller C programming - for that matter, I recommend reading my bare metal programming guide. I will be using Ethernet-enabled Nucleo-H743ZI board throughout this guide. Examples for other architectures are summarized in a table below - this list will expand with time. Regardless, for the best experience I recommend Nucleo-H743ZI to get the most from this guide: buy it on Mouser.
When any two devices communicate, they exchange discrete pieces of data called frames. Frames can be sent over the wire (like Ethernet) or over the air (like WiFi or Cellular). Frames differ in size, and typically range from couple of dozen bytes to a 1.5Kb. Each frame consists of a sequence of protocol headers followed by user data:
The purpose of the headers is as follows:
MAC (Media Access Control) header is only 3 fields: destination MAC
address, source MAC addresses, and an upper level protocol. MAC addresses are
6-byte unique addresses of the network cards, e.g. 42:ef:15:c8:29:a1.
Protocol is usually 0x800, which means that the next header is IP. MAC header
handles addressing in the local network (LAN).
IP (Internet Protocol) header has many fields, but the most important are:
destination IP address, source IP address, and upper level protocol. IP
addresses are 4-bytes, e.g. 209.85.202.102, and they identify a machine
on the Internet, so their purpose is similar to phone numbers. The upper level
protocol is usually 6 (TCP) or 17 (UDP). IP header handles global addressing.
TCP or UDP header has many fields, but the most important are destination port and source ports. On one device, there can be many network applications, for example, many open tabs in a browser. Port number identifies an application.
Application protocol depends on the target application. For example, there are servers on the Internet that can tell an accurate current time. If you want to send data to those servers, the application protocol must SNTP (Simple Network Time Protocol). If you want to talk to a web server, the protocol must be HTTP. There are other protocols, like DNS, MQTT, etc, each having their own headers, followed by the application data.
Install Wireshark tool to observe network frames. It helps to identify issues quickly, and looks like this:
The structure of a frame described above, makes it possible to accurately deliver a frame to the correct device and application over the Internet. When a frame arrives to a device, a software that handles that frame (a network stack), is organised in four layers.
Layer 1: Driver layer, only reads and writes frames from/to network hardware
Layer 2: TCP/IP stack, parses protocol headers and handles IP and TCP/UDP
Layer 3: Network Library, parses application protocols like DNS, MQTT, HTTP
Layer 4: Application - Web dashboard, smart sensor, etc
Let's provide an example. In order to show your this guide on the Github, your browser first needs to find out the IP address of the Github's machine. For that, it should make a DNS (Domain Name System) request to one of the DNS servers. Here's how your browser and the network stack work for that case:
1. Your browser (an application) asks from the lower layer (library), "what
IP address github.com has?". The lower layer (layer 3, a library layer - in
this case, it is a C library) provides an API function gethostbyname()
that returns an IP address for a given host name. So everything said below,
essentially describes how gethostbyname() works.
2. The library layer gets the name github.com and creates a properly
formatted, binary DNS request: struct dns_request. Then it calls an API
function sendto() provided by the TCP/IP stack layer (layer 2), to send that
request over UDP to the DNS server. The IP of the DNS server is known to the library
from the workstation settings. The UDP port is also known - port 53, a standard
port for DNS.
3. The TCP/IP stack's sendto() function receives a chunk of data to send.
it contains DNS request, but sendto() does not know that and does not care
about that. All it knows is that this is the piece of user data that needs to
be delievered over UDP to a certain IP address (IP address of a DNS server) on
port 53. Hence TCP/IP stack prepends UDP, IP, and MAC headers
to the user data to form a frame. Then it calls API function send_frame()
provided by the driver layer, layer 1.
4. A driver's send_frame() function transmits a frame over the wire or
over the air, the frame travels to the destination DNS server. A chain of
Internet routers pass that frame from one to another, until a frame finally hits
DNS server's network card.
5. A network card on the DNS server gets a frame and generates a hardware
interrupt, invoking interrupt handler. It is part of a driver - layer 1. It
calls a function recv_frame() that reads a frame from the card, and passes
it up by calling ethernet_input() function provided by the TCP/IP stack
6. TCP/IP stack parses the frame, and finds out that it is for the UDP port
53, which is a DNS port number. TCP/IP stack finds an application that listens
on UDP port 53, which is a DNS server application, and wakes up its recv()
call. So, DNS server application that is blocked on a recv() call, receives
a chunk of data - which is a DNS request. A library routine parses that request
by extracting a host name, and passes that parsed DNS request to the application.
7. A DNS server application receives DNS request: "someone wants an
IP address for github.com". Then the application layer looks at its
configuration, figures out "Oh, it's me who is responsible for the github.com
domain, and this is the IP address I should respond with". The application
extracts an IP address from the configuration, and calls a library function
"get this IP, wrap into a DNS response, and send back". And the response
travels all the way back in the reverse order.
The communication between layers are done via a function calls. So, each layer has its own API, which upper and lower levels can call. They are not standardized, so each implementation provides their own set of functions. However, on OSes like Windows/Mac/Linux/UNIX, a driver and TCP/IP layers are implemented in kernel, and TCP/IP layer provides a standard API to the userland which is called a "BSD socket API":
This is done becase kernel code does not implement application level protocols
like MQTT, HTTP, etc, - so it let's user application to implement them in
userland. So, a library layer and an application layer reside in userland.
Some library level routines are provided in C standard library, like
DNS resolution function gethostbyname(), but that DNS library functions
are probably the only ones that are provided by OS. For other protocols,
many libraries exist that provide HTTP, MQTT, Websocket, SSH, API. Some
applications don't use any external libraries: they use BSD socket API
directly and implement library layer manually. Usually that is done when
application decides to use some custom protocol.
Embedded systems very often use TCP/IP stacks that provide the same BSD API as mainstream OSes do. For example, lwIP (LightWeight IP) TCP/IP stack, Keil's MDK TCP/IP stack, Zephyr RTOS TCP/IP stack - all provide BSD socket API. Thus let's review the most important BSD API stack functions:
socket(protocol)- creates a connection descriptor and assigns an integer ID for it, a "socket"bind(sock, addr)- assigns a local IP:PORT for a listening socketaccept(sock, addr)- creates a new socket, assigns local IP:PORT and remote IP:PORT (incoming)connect(sock, addr)- assigns local IP:PORT and remote IP:PORT for a socket (outgoing)send(sock, buf, len)- sends datarecv(sock, buf, len)- receives dataclose(sock)- closes a socket
Some implementations do not implement BSD socket API, and there are perfectly good reasons for that. Examples for such implementation is lwIP raw API, and Mongoose Library.
Let me demonstrate the two approaches (using socket and non-socket API) on a simple TCP echo server example. TCP echo server is a simple application that listens on a TCP port, receives data from clients that connect to that port, and writes (echoes) that data back to the client. That means, this application does not use any application protocol on top of TCP, thus it does not need a library layer. Let's see how this application would look like written with a BSD socket API. First, a TCP listener should bind to a port, and for every connected client, spawn a new thread that would handle it. A thread function that sends/receives data, looks something like this:
void handle_new_connection(int sock) {
char buf[20];
for (;;) {
ssize_t len = recv(sock, buf, sizeof(buf), 0); // Receive data from remote
if (len <= 0) break; // Error! Exit the loop
send(sock, buf, len); // Success. Echo data back
}
close(sock); // Close socket, stop thread
}Note that recv() function blocks until it receives some data from the client.
Then, send() also blocks until is sends requested data back to the client.
That means that this code cannot run in a bare metal implementation, because
recv() would block the whole firmware. For this to work, an RTOS is required.
A TCP/IP stack should run in a separate RTOS task, and both send() and
recv() functions are implemented using an RTOS queue API, providing a
blocking way to pass data from one task to another. Overall, this is how an
embedded receive path looks like with socket API:
The send() part would work in the reverse direction. Note that this approach
requires TCP/IP stack implement data buffering for each socket, because
an application consumes received data not immediately, but after some time,
when RTOS queue delivers data. Note that using non-blocking sockets and
select()/poll() changes things that instead of many application tasks,
there is only one application task, but the mechanism stays the same.
Therefore this approach with socket API has the following major characteristics:
- It uses queues for exchanging data between TCP/IP stack and application tasks, which consumes both RAM and time
- TCP/IP stack buffers received and sent data for each socket. Note that the app/library layer may also buffer data - for example, buffering a full HTTP request before it can be processed. So the same data goes through two buffering "zones" - TCP/IP stack, and library/app
That means, socket API implementation takes extra time for data to be processed, and takes extra RAM for double-buffering in the TCP/IP stack.
Now let's see how the same approach works without BSD socket API. Several implementations, including lwIP and Mongoose Library, provide callback API to the TCP/IP stack. Here is how TCP echo server would look like written using Mongoose API:
// This callback function is called for various network events, MG_EV_*
void event_handler(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
if (ev == MG_EV_READ) {
// MG_EV_READ means that new data got buffered in the c->recv buffer
mg_send(c, c->recv.buf, c->recv.len); // Send back the data we received
c->recv.len = 0; // Discard received data
}
}In this case, all functions are non-blocking, that means that data exchange between TCP/IP stack and an app can be implemented via direct function calls. This is how receive path looks like:
As you can see, in this case TCP/IP stack provides a callback API which a library or application layer can use to receive data directly. No need to send it over a queue. A library/app layer can buffer data, and that's the only place where buffering takes place. This approach wins for memory usage and performance. A firmware developer should use a proprietary callback API instead of BSD socket API.
lwIP TCP/IP stack, for example, provides both socket and non-socket (raw) API, and raw API is more efficient in terms of RAM and performance. However developers rarely use raw API, because it is not trivial to understand and use compared to the socket API. The API of the Mongoose Library shown above is designed to be simple and easy to understand. API design can make things very easy or very difficult, so it is important to have a good API.
Now let's make our hands dirty and implement a working network stack on a microcontroller board. I will be using Mongoose Library for all examples further on, for the following reasons:
- Mongoose is very easy to integrate: just by copying two files, mongoose.c and mongoose.h
- Mongoose has a built-in drivers, TCP/IP stack, HTTP/MQTT/Websocket library, and TLS 1.3 all in one, so it does not need any other software to create a network-enabled application
- Mongoose provides a simple, polished callback API designed specifically for embedded developers
The diagram below shows Mongoose architecture. As you can see, Mongoose can use external TCP/IP stack and TLS libraries, as well as built-in ones. In the following example, we are going to use only a built-in functionality, so we won't need any other software.
All source code in this guide is MIT licensed, however Mongoose is licensed under a dual GPLv2/commercial license. I will be using a Nucleo board from ST Microelectronics, and there are several choices for the development environment:
- Use Cube IDE provided by ST: install Cube
- Use Keil from ARM: install Keil
- Use make + GCC compiler, no IDE: follow this guide
Here, I am going to use Cube IDE. In the templates, however, both Keil and make examples are provided, too. So, in order to proceed, install Cube IDE on your workstation, and plug in Nucleo board to your workstation.
Note: this and the following sections has a Youtube helper video recorded: https://www.youtube.com/watch?v=lKYM4b8TZts
The first step would be to create a minimal, skeleton firmware that does nothing but logs messages to the serial console. Once we've done that, we'll add networking functionality on top of it. The table below summarises peripherals for various boards:
| Board | UART, TX, RX | Ethernet | LED |
|---|---|---|---|
| STM32H747I-DISCO | USART1, A9, A10 | A1, A2, A7, C1, C4, C5, G12, G11, G13 | I12, I13, I14 |
| STM32H573I-DK | USART1, A9, A10 | A1, A2, A7, C1, C4, C5, G12, G11, G13 | I8, I9, F1 |
| Nucleo-H743ZI | USART3, D8, D9 | A1, A2, A7, C1, C4, C5, B13, G11, G13 | B0, E1, B14 |
| Nucleo-H723ZG | USART3, D8, D9 | A1, A2, A7, C1, C4, C5, B13, G11, G13 | B0, E1, B14 |
| Nucleo-H563ZI | USART3, D8, D9 | A1, A2, A7, C1, C4, C5, B15, G11, G13 | B0, F4, G4 |
| Nucleo-F746ZG | USART3, D8, D9 | A1, A2, A7, C1, C4, C5, B13, G11, G13 | B0, B7, B14 |
| Nucleo-F756ZG | USART3, D8, D9 | A1, A2, A7, C1, C4, C5, B13, G11, G13 | B0, B7, B14 |
| Nucleo-F429ZI | USART3, D8, D9 | A1, A2, A7, C1, C4, C5, B13, G11, G13 | B0, B7, B14 |
Step 1. Start Cube IDE. Choose File / New / STM32 project
Step 2. In the "part number" field, type the microcontroller name,
for example "H743ZI". That should narrow down
the MCU/MPU list selection in the bottom right corner to a single row.
Click on the row at the bottom right, then click on the Next button
Step 3. In the project name field, type any name, click Finish.
Answer "yes" if a pop-up dialog appears
Step 4. A configuration window appears. Click on Clock configuration tab.
Find a field with a system clock value. Type the maximum value, hit enter,
answer "yes" on auto-configuration question, wait until configured
Step 5. Switch to the Pinout tab, Connectivity, then enable the UART controller
and pins (see table above), choose "Asynchronous mode"
Step 6. Click on Connectivity / ETH, Choose Mode / RMII, verify that the
configured pins are like in the table above - if not, change pins
Step 7. Lookup the LED GPIO from the peripherals table, and configure it
for output. Click on the corresponding pin, select "GPIO output"
Step 8. Click Ctrl+S to save the configuration. This generates the code
and opens main.c file
Step 9. Navigate to the main() function and add some logging to the
while loop. Make sure to insert your code between the "USER CODE" comments,
because CubeIDE will preserve it during code regeneration:
/* USER CODE BEGIN WHILE */
while (1)
{
printf("Tick: %lu\r\n", HAL_GetTick());
HAL_Delay(500);Step 10. Redirect printf() to the UART. Note the UART global variable
generated by Cube at the beginning of main.c - typically it is
UART_HandleTypeDef huart3;. Copy it, open syscalls.c, find function
_write() and modify it the following way :
#include "main.h"
__attribute__((weak)) int _write(int file, char *ptr, int len) {
if (file == 1 || file == 2) {
extern UART_HandleTypeDef huart3;
HAL_UART_Transmit(&huart3, (unsigned char *) ptr, len, 999);
}
return len;
}Step 11. Click on "Run" button to flash this firmware to the board.
Step 12. Attach a serial monitor tool (e.g. putty on Windows, or
cu -l COMPORT -s 115200 on Mac/Linux) and observe UART logs:
Tick: 90358
Tick: 90860
...
Our skeleton firmware is ready!
Now it's time to implement a functional TCP/IP stack. We'll use Mongoose Library for that. To integrate it, we need to copy two files into our source tree.
Step 1. Open https://github.com/cesanta/mongoose in your browser, click on "mongoose.h". Click on "Raw" button, and copy file contents into clipboard.
In the CubeIDE, right click on Core/Inc, choose New/File in the menu, type
"mongoose.h", paste the file content and save.
Step 2. Repeat for "mongoose.c". On Github, copy mongoose.c contents
to the clipboard. In the CubeIDE, right click on Core/Src, choose New/File
in the menu, type "mongoose.c", paste the file content and save.
Step 3. Right click on Core/Inc, choose New/File in the menu, type
"mongoose_custom.h", and paste the following contents:
#pragma once
// See https://mongoose.ws/documentation/#build-options
#define MG_ARCH MG_ARCH_NEWLIB
#define MG_ENABLE_TCPIP 1 // Enables built-in TCP/IP stack
#define MG_ENABLE_CUSTOM_MILLIS 1 // We must implement mg_millis()
#define MG_ENABLE_DRIVER_STM32H 1 // On STM32Fxx series, use MG_ENABLE_DRIVER_STM32FStep 4. Implement Layer 1 (driver), 2 (TCP/IP stack) and 3 (library) in
our code. Open main.c. Add #include "mongoose.h" at the top:
/* USER CODE BEGIN Includes */
#include "mongoose.h"
/* USER CODE END Includes */Step 5. Before main(), define function mg_millis() that returns
an uptime in milliseconds. It will be used by Mongoose Library for the time
keeping:
/* USER CODE BEGIN 0 */
uint64_t mg_millis(void) {
return HAL_GetTick();
}
/* USER CODE END 0 */Step 6. Navigate to main() function and change the code around while
loop this way:
/* USER CODE BEGIN WHILE */
struct mg_mgr mgr;
mg_mgr_init(&mgr);
mg_log_set(MG_LL_DEBUG);
// On STM32Fxx, use _stm32f suffix instead of _stm32h
struct mg_tcpip_driver_stm32h_data driver_data = {.mdc_cr = 4};
struct mg_tcpip_if mif = {.mac = {2, 3, 4, 5, 6, 7},
// Uncomment below for static configuration:
// .ip = mg_htonl(MG_U32(192, 168, 0, 223)),
// .mask = mg_htonl(MG_U32(255, 255, 255, 0)),
// .gw = mg_htonl(MG_U32(192, 168, 0, 1)),
.driver = &mg_tcpip_driver_stm32h,
.driver_data = &driver_data};
NVIC_EnableIRQ(ETH_IRQn);
mg_tcpip_init(&mgr, &mif);
while (1) {
mg_mgr_poll(&mgr, 0);
/* USER CODE END WHILE */Step 7. Connect your board to the Ethernet. Flash firmware. In the serial log, you should see something like this:
bb8 3 mongoose.c:14914:mg_tcpip_driv Link is 100M full-duplex
bbd 1 mongoose.c:4676:onstatechange Link up
bc2 3 mongoose.c:4776:tx_dhcp_discov DHCP discover sent. Our MAC: 02:03:04:05:06:07
c0e 3 mongoose.c:4755:tx_dhcp_reques DHCP req sent
c13 2 mongoose.c:4882:rx_dhcp_client Lease: 86400 sec (86403)
c19 2 mongoose.c:4671:onstatechange READY, IP: 192.168.2.76
c1e 2 mongoose.c:4672:onstatechange GW: 192.168.2.1
c24 2 mongoose.c:4673:onstatechange MAC: 02:03:04:05:06:07
If you don't, and see DHCP requests message like this:
130b0 3 mongoose.c:4776:tx_dhcp_discov DHCP discover sent. Our MAC: 02:03:04:05:06:07
13498 3 mongoose.c:4776:tx_dhcp_discov DHCP discover sent. Our MAC: 02:03:04:05:06:07
...
The most common cause for this is you have your Ethernet pins wrong. Click
on the .ioc file, go to the Ethernet configuration, and double-check the
Ethernet pins against the table above.
Step 8. Open terminal/command prompt, and run a ping command against
the IP address of your board:
$ ping 192.168.2.76
PING 192.168.2.76 (192.168.2.76): 56 data bytes
64 bytes from 192.168.2.76: icmp_seq=0 ttl=64 time=9.515 ms
64 bytes from 192.168.2.76: icmp_seq=1 ttl=64 time=1.012 msNow, we have a functional network stack running on our board. Layers 1,2,3 are implemented. It's time to create an application - a simple web server, hence implement layer 4.
Let's add a very simple web server that responds "ok" to any HTTP request.
Step 1. After the mg_tcpip_init() call, add this line that creates HTTP listener
with fn event handler function:
mg_http_listen(&mgr, "http://0.0.0.0:80", fn, NULL);Step 2. Before the mg_millis() function, add the fn event handler function:
static void fn(struct mg_connection *c, int ev, void *ev_data) {
if (ev == MG_EV_HTTP_MSG) {
struct mg_http_message *hm = ev_data; // Parsed HTTP request
mg_http_reply(c, 200, "", "ok\r\n");
}
}That's it! Flash the firmware. Open your browser, type board's IP address and see the "ok" message.
Note that the mg_http_reply()
function is very versatile: it cat create formatted output, like printf
on steroids. See mg_snprintf()
for the supported format specifiers: most of them are standard printf, but
there are two non-standard: %m and %M that accept custom formatting
function - and this way, Mongoose's printf can print virtually anything.
For example, JSON strings. That said, with the aid of mg_http_reply(),
we can generate HTTP responses of arbitrary complexity.
So, how the whole flow works? Here is how. When a browser connects,
an Ethernet IRQ handler (layer 1) kicks in. It is defined by Mongoose, and activated by
the #define MG_ENABLE_DRIVER_STM32H 1 line in the mongoose_custom.h: ETH_IRQHandler. Other environments, like CubeIDE, implement ETH_IRQHandler
and activate it when you select "Enable Ethernet interrupt" in the Ethernet
configuration. To avoid clash with Cube, we did not activate Ethernet interrupt.
IRQ handler reads frame from the DMA, copies that frame to the Mongoose's receive queue, and exits. That receive queue is special, it is a thread-safe single-producer-single-consumer non-blocking queue, so an IRQ handler, being executed in any context, can safely write to it.
The mg_poll() function in the infinite while() loop constantly
verifies, whether we receive any data in the receive queue. When it detects
a frame in the receive queue, it extracts that frame, passes it on to the
mg_tcp_rx() function - which is an etry point to the layer 2 TCP/IP stack.
That mg_tcp_rx() function parses headers, starting from Ethernet header,
and when it detects that a received frame belongs to one of the Mongoose
TCP or UDP connections, it copies frame payload to the connection's c->recv
buffer and calls MG_EV_READ event.
At this point, processing leaves layer 2 and enters layer 3 - a library layer.
Mongoose's HTTP event handlers catches MG_EV_READ, parses received data,
and when it detects that the full HTTP message is buffered, it sends the
MG_EV_HTTP_MSG with parsed HTTP message to the application - layer 4.
And this is where our event handler function fn() gets called. Our code is
simple - we catch MG_EV_HTTP_MSG event, and use Mongoose's API function
mg_http_reply() to craft a simple HTTP response:
HTTP/1.1 200 OK
Content-Length: 4
ok
This response goes to Mongoose's c->send output buffer, and mg_mgr_poll()
drains that data to the browser, splitting the response by frames
in layer 2, then passing to the layer 1. An Ethernet driver's output function mg_tcpip_driver_stm32h_tx() sends those frames back to the browser.
This is how Mongoose Library works.
Other implementations, like Zephyr, Amazon FreeRTOS-TCP, Azure, lwIP, work in a similar way. They implement BSD socket layer so it is a bit more complicated cause it includes an extra socket layer, but the principle is the same.
Using mg_http_reply() function is nice, but it's very good for creating
custom responses. It is not suitable for serving files. And the standard way
to build a web UI is to split it into two parts:
- a static part, which consists of directory with
index.html, CSS, JavaScript and image files, - a dynamic part, which serves REST API
So instead of using mg_http_reply() and responding with "ok" to any request,
let's create a directory with index.html file and serve that directory.
Mongoose has API function mg_http_serve_dir() for that. Let's change the
event handler code to use that function:
static void fn(struct mg_connection *c, int ev, void *ev_data) {
if (ev == MG_EV_HTTP_MSG) {
struct mg_http_message *hm = ev_data; // Parsed HTTP request
struct mg_http_serve_opts opts = {.root_dir = "/web_root"};
mg_http_serve_dir(c, hm, &opts);
}
}Build it and get build error "undefined reference to 'mkdir'". This is because
mg_http_serve_dir() function tries to use a default POSIX filesystem to
read files from directory /web_root, and our firmware does not have support
for the POSIX filesystem.
What are the possibilities here? First, we can implement POSIX filesystem,
by using an internal or external flash memory. Then we can copy our web_root
directory there, and our code will start to work. This is the hard way.
The easy way is to use a so-called embedded filesystem, by transforming all files in the web directory into C arrays, and compiling them into the firmware binary. This way, all UI files are simply hardcoded into the firmware binary, and there is no need to implement a "real" filesystem:
Step 1. Tell mg_http_serve_dir() to use packed filesystem:
struct mg_http_serve_opts opts = {.root_dir = "/web_root", .fs = &mg_fs_packed};Step 2. Enable packed filesystem, and disable POSIX filesystem in mongoose_custom.h:
#define MG_ENABLE_PACKED_FS 1
#define MG_ENABLE_POSIX_FS 0Step 3. Create a new file Core/Src/packed_fs.c. Go to https://mongoose.ws/ui-pack/,
review UI files. Copy/paste the contents of generated packed_fs.c, save.
Build the firmware - and now it should build with no errors.
Let's review what that UI packer does. As you can see, it has 3 files, which
implement a very simple Web UI with LED control. The index.html file
loads main.js file, which defines a button click handler. When a button
gets clicked, it makes a request to the api/led/toggle URL, and when
than request completes, it makes another request to api/led/get URL,
and sets the status span element to the result of the request.
The tool has a preview window, and if any of the files are changed,
it automatically refreshes preview and regenerates packed_fs.c. The packed_fs.c
is a simple C file, which contains three C arrays, representing three files
we have, and two helper functions mg_unlist() and mg_unpack(), used by
Mongoose:
- the
mg_unlist()function allows to scan the whole "filesystem" and get names of every file, - the
mg_unpack()function returns file contents, size, and modification time for a given file.
Mongoose provides a command line utility pack.c
to generate packed_fs.c automatically during the build. The example of that
is a Makefile
for device dashboard example in Mongoose repository, which not only packs,
but also compresses files to minimise their size. But here, we'll use the
web tool because it is visual and makes it easy to understand the flow.
The static HTML is extremely simple. There 3 files: index.html, style.css
and main.js. The index.html references, or loads, the other two:
index.html:
<!DOCTYPE html>
<html>
<head>
<link href="style.css" rel="stylesheet" />
</head>
<body>
<div class="main">
<h1>My Device</h1>
<span>LED status:</span>
<span id="status">0</span>
<button id="btn">Toggle LED</button>
</div>
<script src="main.js"></script>
</body>
</html>style.css:
.main { margin: 1em; }
#status { display: inline-block; width: 2em; }The Javascript code in the main.js file installs an event handler on button
click, so when a user clicks on a button, JS code makes HTTP requests -
I'll comment down below how it all works together:
main.js:
var getStatus = ev => fetch('api/led/get')
.then(r => r.json())
.then(r => { document.getElementById('status').innerHTML = r; });
var toggle = ev => fetch('api/led/toggle')
.then(r => getStatus());
document.getElementById('btn').onclick = toggle;Now, let's flash the firmware. Go to the IP address in the browser - and now we see the Web UI with a button! Click on the button, and see that nothing happens! The LED does not turn on and off. Open developer tools and see that on every click, a browser makes "toggle" and "get" requests which return 404 error - not found. Let's implement those API calls.
Change the event handler in the following way:
static void fn(struct mg_connection *c, int ev, void *ev_data) {
if (ev == MG_EV_HTTP_MSG) {
struct mg_http_message *hm = (struct mg_http_message *) ev_data;
if (mg_http_match_uri(hm, "/api/led/get")) {
mg_http_reply(c, 200, "", "%d\n", HAL_GPIO_ReadPin(GPIOB, GPIO_PIN_0));
} else if (mg_http_match_uri(hm, "/api/led/toggle")) {
HAL_GPIO_TogglePin(GPIOB, GPIO_PIN_0); // Can be different on your board
mg_http_reply(c, 200, "", "true\n");
} else {
struct mg_http_serve_opts opts = {.root_dir = "/web_root", .fs = &mg_fs_packed};
mg_http_serve_dir(c, hm, &opts);
}
}
}Note the mg_http_match_uri() checks. There, we are making different responses
to different URLs. On /api/led/get URL request, we're responding with
LED status, and on /api/led/toggle request, we're toggling the pin and
responding with true.
Build and flash this firmware. Refresh the page in the browser. Click on the button - and now, LED toggle works! If we open developer tools in the browser, we can see the sequence of the network requests made by the browser.
Below is the diagram of the interaction between the browser and the device, with explanations of every step:
This is the flow for the Web UI of any complexity. Now, it is just a matter of creating a professional UI interface using any suitable JS/CSS framework, and extending the event handler function with the API calls that that UI invokes. That's all it takes.
Let me show you how to repeat everything we did in Cube - in the make + GCC
environment in one minute. Navigate to https://mongoose.ws/demo/?clear=1
This simple web tool creates a make project completely in your browser.
Choose the board, the "simple project". You can download the project to your
workstation and build manually. But we'll build in a browser - click on Build
button. That zips the projects and sends it to mongoose.ws site, which has
ARM GCC pre-installed. It simply runs make, creates firmware binary, and
sends that binary back to your browser. Now you can download that binary,
or flash it directly from your browser.
The "simple" project repeats what we've already done in Cube, with one important difference - it also implements TLS. In other words, it can serve both HTTP and HTTPS. Note that the binary size is less than 60 Kb! We will cover TLS later, as it needs a separate discussion.
Now, let's click on "Start Over" button and build "Web UI Dashboard" project.
It follows absolutely the same flow as "simple" project, just the Web UI is
significantly more versatile, built with Preact JS framework and Tailwind CSS
framework. The event handler function moved into a separate file, net.c,
and supports many API calls required by Web UI - to show dashboard stats,
settings, and firmware update. By the way, the firmware update is completely
functional - but I won't cover it here, as it is a big topic on itself.
I won't cover the process of static UI creation in React, as there are tons
of very good tutorials on that. But if you want me to cover that, join our
Discord server and let me know.
What I'll do is to move that UI into the Cube project of ours.
Step 1. Copy net.c, net.h, packed_fs.c into the Cube project
Step 2. Add the following include "net.h" at the top of the main.c file
Step 3. Comment out mg_http_listen(...) call, add web_init() call
Step 4. Open net.h, modify HTTP_URL port 8000 to port 80
Rebuild, reflash, refresh your browser. We have a functional versatile Web UI device dashboard reference running!
I am Sergey Lyubka, an engineer and entrepreneur. I hold a MSc in Physics from Kyiv State University, Ukraine. I am a director and a co-founder at Cesanta - a technology company based in Dublin, Ireland. Cesanta develops embedded solutions:
- https://mongoose.ws - an open source HTTP/MQTT/Websocket network library
- https://vcon.io - a remote firmware update / serial monitor framework
You are welcome to register for my free webinar on embedded network programming