Device Synchronization

This page describes how the device can update its configuration and firmware via MQTT. The process is called Device Synchronization.

Device Synchronization is initiated by the device.

Synchronization Process

The synchronization process is as follows:

The device requests a sync by publishing its current firmware names and configuration ETag to the [topic prefix]/[DevEUI]/req/sync topic after a reset and during Daily Status.
The backend should respond within 5 seconds by publishing to [topic prefix]/[DevEUI]/down/sync with the changed firmware names and/or changed configuration ETag.
If a firmware name is not null in the response, the device downloads it using one of two methods, depending on the form of the app/boot value (see Synchronization Response):
- Firmware name — the device requests the firmware chunk-by-chunk by publishing to [topic prefix]/[DevEUI]/req/fw. The backend should then respond within 10 seconds by publishing the requested chunk to [topic prefix]/[DevEUI]/down/fw.
- HTTP(S) URL — the device downloads the firmware directly from that URL as a single streamed request (see Firmware Download via HTTP(S)), without using req/fw/down/fw at all. Supported since firmware v0.15.0.
If the configuration ETag is not null in the response, the device will request the changed configuration by publishing to [topic prefix]/[DevEUI]/req/config. The backend should then respond within 5 seconds by publishing the configuration to [topic prefix]/[DevEUI]/down/config.

Avoid device restart loops

In the current version of the device firmware, an ETAG with an invalid config is requested again. The server should limit the retransmission to a maximum of 3 times, to avoid infinite device restart loops.

Future plans:
We plan a fix where the firmware will blacklist the ETag of an invalid configuration, so that the device will not request it again.

Configuration ETags

To identify configuration changes, the backend needs to assign a random value each time the configuration changes, called ETag.

Synchronization Messages

All device requests have the following payload message structure:

i (string): The device EUI.
n (uint64): The uplink frame counter.
q (string): The query type, as written in the topic. Example: sync.
d: The request data. Structure depends on the query type.

All backend responses have the following payload message structure:

n (uint64): The downlink frame number.
r (uint64): Value of the n field from request the downlink is responding to.
d: The response data. Structure depends on the request type.

↑ Synchronization Request

Topic: [topic prefix]/[DevEUI]/req/sync

The device publishes its current synchronization state to this topic.

Expected response: Synchronization Response.

The message is published:

After reset.
During Daily Status.

Message data contents:

app (string): Name of the installed application firmware.
boot (string): Name of the installed secure firmware.
etag (string): ETag of the installed configuration, as assigned by the backend.

Payload example

Topic: LOB/70b3d5e050010233/req/sync
{
  "i": "70b3d5e050010233",
  "n": 123,
  "q": "sync",
  "d": {
    "app": "app-nrf91-origin v0.1.1 TZ3 (Aug 16 2024 16:25:20)",
    "boot": "app-nrf91-secure v0.1.8 TZ3 (Aug 12 2024 17:58:42)",
    "etag": "92da513367c4d781598c85586bb2997954ced4e1"
  }
}

↓ Synchronization Response

Topic: [topic prefix]/[DevEUI]/down/sync

The backend publishes possibly changed firmware and/or configuration ETag to this topic.

Message data contents:

app (string|null): The to-be-installed application firmware, or null. Either the firmware name (downloaded chunk-by-chunk via Firmware Chunk Request/Firmware Chunk Response) or an absolute http:///https:// URL (downloaded directly, see Firmware Download via HTTP(S)).
boot (string|null): The to-be-installed secure firmware, or null. Same format as app.
etag (string|null): ETag of the to-be-installed configuration, or null.

Payload example

Topic: LOB/70b3d5e050020901/down/sync
{
  "n": 7890,
  "r": 123,
  "d": {
    "app": null,
    "boot": null,
    "etag": "12345678901234567890"
  }
}

↑ Config Request

Topic: [topic prefix]/[DevEUI]/req/config

The device publishes to this topic to request a configuration update.

Expected response: Config Response.

Message data contents:

etag (string): ETag of the requested configuration, as assigned by the backend.

Payload example

Topic: LOB/70b3d5e050010233/req/config
{
  "i": "70b3d5e050010233",
  "n": 124,
  "q": "config",
  "d": {
    "etag": "12345678901234567890"
  }
}

↓ Config Response

Topic: [topic prefix]/[DevEUI]/down/config

The backend publishes the configuration to this topic, as response to the Config Request.

Only the provided config parameters are changed on the device, all other parameters keep unchanged.

Message data contents:

etag (string): ETag of the new configuration.
config (object): The new configuration.

Payload example

Topic: LOB/70b3d5e050010233/down/config
{
  "n": 7891,
  "r": 124,
  "d": {
    "etag": "12345678901234567890",
    "config": {
      "listenCron": "H H 10 * * *",
      "cmodeDursec": 21,
      "smodeDurSec": 0
    }
  }
}

↑ Firmware Chunk Request

Topic: [topic prefix]/[DevEUI]/req/fw

The device publishes to this topic to request a firmware chunk.

Expected response: Firmware Chunk Response.

Message data contents:

f (string): Name of the firmware that the device is downloading.
c (uint64): Chunk number.

Payload example

Topic: LOB/70b3d5e050010233/req/fw
{
  "i": "70b3d5e050010233",
  "n": 125,
  "q": "fw",
  "d": {
    "f": "app-nrf91-origin+0.3.8+hw4.hex",
    "c": 0
  }
}

↓ Firmware Chunk Response

Topic: [topic prefix]/[DevEUI]/down/fw

The backend publishes a firmware chunk to this topic, as response to the Firmware Chunk Request.

Each chunk should be at maximum 512 bytes long.

Message data contents:

c (uint64): Chunk ID.
t (uint64): Total number of chunks.
a (uint64): Memory address to write the chunk to.
d (string): Hex encoded chunk.

Payload example

Topic: LOB/70b3d5e050010233/down/fw
{
  "n": 7892,
  "r": 125,
  "d": {
    "c": 0,
    "t": 553,
    "a": 364032,
    "d": "d3e9002326f090f80022002326f0d2fc0028eed0002c36db3443e9d000261f4ce7e7332d06ddb5f5806f25d041ec100bbde8f081a3f213434ff0ff37df403842f4d013a3d3e9002326f06ef80022002326f0b0fc0028ccd0002c334609dd142d06d0c5f134050122aa4013449e4200d9013423ea0706bce702460b4626f054f841ec100bd4e700264ff00044b1e700bf9c7500883ce4377effff0f000000f03f70b555ec104bc5f30a522b46064672bb25f00043234329d03b4b20462946002226f0e8f9394b04460d469e4252dbc1f30a520b46363a4cf250318e4223dc164440f2fe7296421edc002e34dc16f1350f38db6ff31e5336360022204643ea06552b4b294626f0c6f941ec100b70bd40f2ff718a42dfd122462046294626f004f841ec100b70bd002d17a1d1e9000107db15a3d3e9002326f0adf941ec100b70bd13a1d1e9000110a3d3e9002326f0a2f941ec100bf3e76ff31e5343ea065545ec104b70bd002d0ca1d1e9000102da0ca1d1e9000108a3d3e9002326f08bf941ec100b70bdaff300809c7500883ce4377e9c7500883ce437fe59f3f8c21f6ea50159f3f8c21f6ea58100005043b03cffff0000903c000000002de9f04fc76999b08346229e09910e92139355ec104b8ded040b7fb9102027f075fd0246cbf81c0020b9a74bef21a74827f03efec0e901770760c760dbf81c30196859b15a680123"
  }
}

Firmware Download via HTTP(S)

As an alternative to the chunk-by-chunk transfer described above, the backend can respond to the Synchronization Request with an absolute http:// or https:// URL in the app/boot field instead of a firmware name (supported since firmware v0.15.0). The device then downloads the firmware directly from that URL as a single streamed HTTP(S) GET request, without publishing any req/fw/down/fw messages.

Requirements for the served file:

It must be an Intel HEX file (.hex), the same format used for regular firmware releases.
The server must respond with a 2xx status. Up to 5 redirects (3xx with a Location header) are followed.
Content-Length or chunked transfer encoding are both supported; the device streams and flashes the body as it arrives, without buffering the whole file.

note

The device only has a single TLS context available for the secure firmware. If the download URL uses https:// and the sync itself was received over mqtts, the device closes the MQTTS connection before starting the HTTPS download, then reconnects and resubscribes once the download finishes. Plain http:// URLs do not require this, since no TLS context is needed for the download itself.

This is primarily intended for backends that already host firmware files on an HTTP(S) file server (e.g. object storage) and want to avoid re-implementing chunked transfer.

Using http:// does not weaken update security

Serving the file over plain http:// instead of https:// does not reduce the security of the firmware update itself. Every firmware image is cryptographically signed by Lobaro and verified by MCUboot before it is executed — if the downloaded file is altered in transit (e.g. by a man-in-the-middle) or otherwise not intact and validly signed, the device rejects it and does not run it, regardless of whether it was fetched via http://, https://, or MQTT chunk transfer. https:// mainly adds confidentiality (hiding which firmware version/URL is being fetched) and requires the additional TLS handshake/context switch described above; it is not required to guarantee that only genuine Lobaro firmware is installed.

Example communication flow of a configuration update

The device sends its current configuration ETag in a [topic prefix/[DevEUI]/req/sync uplink.

To trigger a configuration update, respond with a different ETag. The device will then request the new configuration via [topic prefix/[DevEUI]/req/config, to which you must respond with the complete new configuration.

Device publishes sync request to LOB/70b3d5e050010233/req/sync:

{
  "i": "70b3d5e050010233",
  "n": 123,
  "q": "sync",
  "d": {
    "app": "app-nrf91-origin v0.1.1 TZ3 (Aug 16 2024 16:25:20)",
    "boot": "app-nrf91-secure v0.1.8 TZ3 (Aug 12 2024 17:58:42)",
    "etag": "92da513367c4d781598c85586bb2997954ced4e1"
  }
}

Backend responds with a changed ETag by publishing to LOB/70b3d5e050010233/down/sync within 5 seconds:

{
  "n": 7890,
  "r": 123,
  "d": {
    "app": null,
    "boot": null,
    "etag": "12345678901234567890"
  }
}

Device requests new configuration by publishing the desired ETag to LOB/70b3d5e050010233/req/config:

{
  "i": "70b3d5e050010233",
  "n": 124,
  "q": "config",
  "d": {
    "etag": "12345678901234567890"
  }
}

Backend responds with the new configuration by publishing to LOB/70b3d5e050010233/down/config within 5 seconds:

{
  "n": 7891,
  "r": 124,
  "d": {
    "etag": "12345678901234567890",
    "config": {
      "listenCron": "H H 10 * * *",
      "cmodeDursec": 21,
      "smodeDurSec": 0
    }
  }
}

Example communication flow of a firmware update

💡 Firmware update tool available on request

Lobaro provides a tool that handles the MQTT-based firmware update process on your own broker. Contact sales@lobaro.de for more information.

Device publishes sync request to LOB/70b3d5e050010233/req/sync:

{
  "i": "70b3d5e050010233",
  "n": 123,
  "q": "sync",
  "d": {
    "app": "app-nrf91-origin v0.1.1 TZ3 (Aug 16 2024 16:25:20)",
    "boot": "app-nrf91-secure v0.1.8 TZ3 (Aug 12 2024 17:58:42)",
    "etag": "92da513367c4d781598c85586bb2997954ced4e1"
  }
}

Backend initiates firmware update by publishing to LOB/70b3d5e050010233/down/sync within 5 seconds:

{
  "n": 7890,
  "r": 123,
  "d": {
    "app": "app-nrf91-origin+0.3.8+hw4.hex",
    "boot": null,
    "etag": null
  }
}

Device starts requesting firmware chunks by publishing to LOB/70b3d5e050010233/req/fw:

{
  "i": "70b3d5e050010233",
  "n": 124,
  "q": "fw",
  "d": {
    "f": "app-nrf91-origin+0.3.8+hw4.hex",
    "c": 0
  }
}

Backend responds to each chunk request with firmware chunk by publishing to LOB/70b3d5e050010233/down/fw within 10 seconds:

{
  "n": 7891,
  "r": 124,
  "d": {
    "c": 0,
    "t": 553,
    "a": 364032,
    "d": "d3e9002326f090f80022002326f0d2fc0028eed0002c36db3443e9d000261f4ce7e7332d06ddb5f5806f25d041ec100bbde8f081a3f213434ff0ff37df403842f4d013a3d3e9002326f06ef80022002326f0b0fc0028ccd0002c334609dd142d06d0c5f134050122aa4013449e4200d9013423ea0706bce702460b4626f054f841ec100bd4e700264ff00044b1e700bf9c7500883ce4377effff0f000000f03f70b555ec104bc5f30a522b46064672bb25f00043234329d03b4b20462946002226f0e8f9394b04460d469e4252dbc1f30a520b46363a4cf250318e4223dc164440f2fe7296421edc002e34dc16f1350f38db6ff31e5336360022204643ea06552b4b294626f0c6f941ec100b70bd40f2ff718a42dfd122462046294626f004f841ec100b70bd002d17a1d1e9000107db15a3d3e9002326f0adf941ec100b70bd13a1d1e9000110a3d3e9002326f0a2f941ec100bf3e76ff31e5343ea065545ec104b70bd002d0ca1d1e9000102da0ca1d1e9000108a3d3e9002326f08bf941ec100b70bdaff300809c7500883ce4377e9c7500883ce437fe59f3f8c21f6ea50159f3f8c21f6ea58100005043b03cffff0000903c000000002de9f04fc76999b08346229e09910e92139355ec104b8ded040b7fb9102027f075fd0246cbf81c0020b9a74bef21a74827f03efec0e901770760c760dbf81c30196859b15a680123"
  }
}

Example communication flow of a firmware update via HTTP(S)

Device publishes sync request to LOB/70b3d5e050010233/req/sync:

{
  "i": "70b3d5e050010233",
  "n": 123,
  "q": "sync",
  "d": {
    "app": "app-nrf91-origin v0.1.1 TZ3 (Aug 16 2024 16:25:20)",
    "boot": "app-nrf91-secure v0.1.8 TZ3 (Aug 12 2024 17:58:42)",
    "etag": "92da513367c4d781598c85586bb2997954ced4e1"
  }
}

Backend initiates the firmware update by publishing an HTTP(S) URL instead of a firmware name to LOB/70b3d5e050010233/down/sync within 5 seconds:

{
  "n": 7890,
  "r": 123,
  "d": {
    "app": "https://files.example.com/app-nrf91-origin+0.3.8+hw4.hex",
    "boot": null,
    "etag": null
  }
}

The device downloads the file directly from the given URL as a single streamed HTTP(S) GET request, flashing it as it arrives. No req/fw/down/fw messages are published for this update.

Synchronization Process​

Configuration ETags​

Synchronization Messages​

↑ Synchronization Request​

↓ Synchronization Response​

↑ Config Request​

↓ Config Response​

↑ Firmware Chunk Request​

↓ Firmware Chunk Response​

Firmware Download via HTTP(S)​

Example communication flow of a configuration update​

Example communication flow of a firmware update​

Example communication flow of a firmware update via HTTP(S)​

Synchronization Process

Configuration ETags

Synchronization Messages

↑ Synchronization Request

↓ Synchronization Response

↑ Config Request

↓ Config Response

↑ Firmware Chunk Request

↓ Firmware Chunk Response

Firmware Download via HTTP(S)

Example communication flow of a configuration update

Example communication flow of a firmware update

Example communication flow of a firmware update via HTTP(S)