ArbyBot Development Log #4 - AMP Candidate 3: CBOR
6,164 words Estimated 31 min read
Introduction
In this post, I propose the third design candidate for the Arby Message Protocol (AMP). AMP defines how the microcontroller units (MCUs) aboard the ArbyBot platform communicate. AMP builds on top of the ArbyBot CAN (ABC) protocol proposed in my earlier blog post.
The goal of AMP Candidate 3 is to move away from the globally-defined key-value system proposed in AMP design candidates 1 & 2. The new design should adopt a more flexible method which allows the transference of more complex objects, nested data structures, and arbitrary-length values. The motivation for this is to be able to include timestamps and metadata for events, as well as arrays for multi-sensor readings, and not require devices to have a dictionary of all defined key/values.
AMP Candidate 3 assumes the ArbyBot platform will utilise CAN-FD as the base layer of the transport, due to its higher throughput and larger packet size.
The finalised design of AMP will be documented in a design report after research & development has concluded. The AMP design candidates and other developer logs document my design progress and are provided for educational purposes.
Acronyms
Definitions employed are presented (Table 1).
| Acronym | Definition |
|---|---|
| ABC | ArbyBot CAN |
| AMP | Arby Message Protocol |
| API | Application Programming Interface |
| CAN | Controller Area Network |
| CAN-FD | CAN Flexible Data Rate |
| CBOR | Concise Binary Object Representation (RFC 8949) |
| CRC | Cyclic Redundancy Check |
| CRC-32 | 32-bit CRC |
| HDLC | High-Level Data Link Control |
| GNU | “GNU’s Not Unix!” |
| GZIP | GNU Zip |
| ID | Identifier |
| ISO | International Organization for Standardization |
| JSON | JavaScript Object Notation (RFC 8259) |
| MCU | Microcontroller Unit |
| PPS | Pulse Per Second |
| RAM | Random Access Memory |
| RFC | Request For Comments |
| RPC | Remote Procedure Call |
| UTC | Coordinated Universal Time |
| UTF-8 | Unicode Transformation Format (8-bit) |
| UUID | Universally Unique Identifier |
Table 1. Acronym Definitions
RFC Register
Referenced RFCs are listed (Table 2).
| RFC | Title |
|---|---|
| RFC 1952 | GZIP file format specification version 4.3 |
| RFC 5424 | The Syslog Protocol |
| RFC 8259 | The JavaScript Object Notation (JSON) Data Interchange Format |
| RFC 8949 | Concise Binary Object Representation (CBOR) |
Table 2. Referenced RFCs.
AMP Design Candidate 3
The third iteration of the Arby Message Protocol (AMP) replaces the AMP Frame Segment, an array of AMP Frames, with the AMP Message Body. The AMP Message Body is encoded as a Concise Binary Object Representation (CBOR) payload (RFC 8949).
CBOR allows the representation of the following data types:
- Unsigned integers
- Negative integers
- Byte strings (binary data)
- Text strings (UTF-8)
- Arrays (ordered lists)
- Maps (key-value pairs)
- Semantic tags (for dates, UUIDs, or custom extensions)
- Special and floating-point values (booleans, null, floats)
CBOR implementations exist for both CPython & MicroPython environments (although verification of exact RFC compatibility remains future work). Utilising CBOR fulfills the desire for flexible support of variable-length data types, and complex and nested data types. It shifts the work of defining the exact layout of the encoded binary data to an existing standard, allowing AMP to simply define the schema of the data (the keys, values, and types), but not how that schema is encoded. CBOR is an alternative to JavaScript Object Notation (JSON) encoding, but for resource-constrained transports, such as CAN.
The AMP Message Body is preceded by the Body Length field and, for messages spanning multiple frames, followed by the CRC-32.
The magic word was dropped from the AMP Message to save two bytes.
The Body Length field was increased from uint16 (2 bytes) to uint24 (3 bytes).
This increases the maximum Body Length from 65,535 to 16,777,215 bytes, allowing larger binary blobs to be carried within the CBOR body.
The CRC-32 at the end of the payload provides an additional integrity check for messages spanning multiple frames, once reassembled. It is omitted from single-frame messages because CAN-FD already includes a CRC check for each frame. When forwarding outside the CAN network, the gateway verifies and removes the CRC-32, when present, before forwarding the CBOR body.
The separate checks are therefore:
- CAN-FD CRC: detects transmission corruption within a frame.
- AMP CRC-32: checks the reassembled message across multiple frames.
- Payload validation: checks that the body contains valid CBOR and follows the AMP Message schema including required fields, types, and permitted values.
AMP also introduces an addressing scheme, in addition to that defined by ABC.
The ABC sender_id and target_id support 127 individual device addresses.
The ABC address scheme applies only to a single CAN bus.
AMP defines a sender and a destination which comprise machine IDs where available or UUIDv4s, which can be used to address devices throughout the AMP network.
The ABC gateway extracts the AMP message from the ABC packets and forwards it through the appropriate interface, preserving the AMP sender and destination.
For a destination outside the CAN bus, the ABC target identifies the gateway, while the AMP destination identifies the final recipient.
The ABC gateway will hold the mapping of which devices it has communication with and through which interface.
ABC devices on the CAN network can still utilise the ABC address scheme to ignore messages addressed to other devices, while still accepting broadcasts.
As in AMP Candidate 2, AMP Messages longer than 64 bytes must be fragmented into multiple CAN-FD frames for transmission.
Where necessary, the final ABC packet is zero-padded to the next supported CAN-FD payload length. Padding follows the CBOR body or, when present, the CRC-32. It is excluded from both Body Length and the CRC calculation. CAN-FD supported lengths are as follows: 0–8, 12, 16, 20, 24, 32, 48, or 64 bytes
AMP Candidate 3 is experimental. Compatibility between revisions is not guaranteed.
To support flexibility, compatibility with extensions, and rapid prototyping, all schemas permit additional, undocumented keys alongside the documented keys unless otherwise stated. Implementations should ignore keys they do not recognise. Future updates may reserve keys already used by individual implementations. In such cases, the definitions in the final AMP design report take precedence.
The Message Structure
The modified AMP Message is presented (Table 3).
| Position | Type | Required | Size | Name | Description |
|---|---|---|---|---|---|
| 0 | uint24 |
True | 3 bytes | Body Length | The length of the remainder of the message in bytes, including the CRC-32 when present. |
| 3 | CBOR Blob | True | Body Length - 4 bytes when CRC-32 is present, otherwise Body Length. | AMP Message Body | The message content encoded in CBOR. |
| 3 + Body Length - 4 | uint32 |
True if the message spans multiple CAN-FD frames. | 4 bytes | CRC-32 | A 32-bit Cyclic Redundancy Check (CRC-32/ISO-HDLC, RFC 1952) for the preceding bytes of the message. Only provided if the message exceeds one ABC Packet. |
Table 3. The AMP Message (AMP Design Candidate 3).
AMP Message Fragmentation, Transmission & Validation
The process of message fragmentation, transmission, and validation is presented in a flowchart (Figure 1). The sender encodes an event or RPC message as CBOR. The message is prepended with a 3-byte Body Length. Messages fitting within 64 bytes including the three-byte header omit the CRC-32. Larger messages include the CRC-32 and are fragmented into ABC packets. The ABC packets are transmitted through the CAN bus. The receiver reassembles the message, checks its length and any CRC-32, then decodes CBOR, and validates the schema before processing it. For messages requiring multiple frames, Body Length also includes the four-byte CRC-32 before the checksum is calculated.
flowchart LR
accTitle: AMP message fragmentation, transmission and validation
accDescr: AMP message fragmentation, transmission and validation
subgraph sender["Sending device"]
direction TB
message["Event or RPC message"] --> encode["Encode the body as CBOR"]
encode --> size{"CBOR body + 3-byte header<br/>at most 64 bytes?"}
size -->|Yes| single["Set Body Length to CBOR size<br/>Omit CRC-32"]
size -->|No| multiple["Set Body Length to CBOR size + 4<br/>Append CRC-32 of header and body"]
single --> one["Send one ABC packet<br/>in one CAN-FD frame"]
multiple --> many["Fragment into ABC packets<br/>Send multiple CAN-FD frames"]
end
subgraph receiver["Receiving device"]
direction TB
receive["Receive ABC packets<br/>CAN-FD checks each frame's CRC"]
receive --> assemble["Reassemble if fragmented<br/>Check Body Length and exclude padding"]
assemble --> fragmented{"Message spans multiple frames?"}
fragmented -->|Yes| crc["Verify CRC-32"]
fragmented -->|No| decode["Decode the CBOR body"]
crc --> decode
decode --> schema["Validate the AMP Message schema"]
schema --> process["Process the event or RPC message"]
end
sender -->|"ABC packets carried in CAN-FD frames"| receiver
Figure 1. Encoding and validation of an AMP Message (AMP Design Candidate 3).
The receiving device only processes a message after the applicable checks have passed. Incomplete messages, invalid lengths, failed CRC checks, and invalid payloads are rejected.
Single Frame Example
An example AMP Message is presented (Figure 2) with a 54-byte CBOR body. Positions are bit offsets, with each row representing 16 bytes. The presented AMP Message, including the body length, consists of 57 bytes and therefore fits within one CAN-FD frame.
---
title: AMP Message — Single Frame
config:
packet:
bitsPerRow: 128
bitWidth: 8
---
packet
accTitle: Final iteration AMP message without a CRC
accDescr: A 57-byte AMP message consists of a three-byte Body Length field with value 54, followed by a 54-byte CBOR body. The message fits within one CAN-FD frame and omits the CRC-32.
0-23: "Body Length = 54"
24-455: "CBOR Body — 54 bytes"
456-511: "CAN-FD zero-padding"
Figure 2. An AMP Message fitting within one CAN-FD frame (AMP Design Candidate 3).
Multi-frame Example
An example of a fragmented AMP Message is presented with a 65-byte CBOR body. The 65-byte CBOR body and three-byte header exceed the single-frame limit, so the message requires a CRC-32 and two CAN-FD frames. Frame 1 is presented, holding the Body Length and the first 61 bytes of the CBOR Body (Figure 3).
---
title: AMP Message (Frame 1).
config:
packet:
bitsPerRow: 128
bitWidth: 8
---
packet
accTitle: Frame 1 of a fragmented AMP Log Message
accDescr: Frame 1 of a fragmented AMP Log Message
0-23: "Body Length = 69"
24-511: "CBOR Body — 65 bytes (first fragment)"
Figure 3. Frame 1 of a fragmented AMP Message.
Frame 2 is presented, holding the remaining 4 bytes of the CBOR Body and the 4-byte CRC-32 (Figure 4). As the second frame only contains 8 bytes, no additional zero-padding is required.
---
title: AMP Message (Frame 2).
config:
packet:
bitsPerRow: 64
bitWidth: 16
---
packet
accTitle: Frame 2 of a fragmented AMP Log Message
accDescr: Frame 2 of a fragmented AMP Log Message
0-31: "CBOR Body — 65 bytes (second fragment)"
32-63: "CRC-32 — 4 bytes"
Figure 4. Frame 2 of a fragmented AMP Message.
AMP Event Messages
The AMP Message Body carries events, RPC requests, and RPC responses. Event messages allow devices to broadcast information such as sensor readings, PPS synchronisation, and heartbeats.
The event message schema is presented (Table 4). All schema keys are required unless explicitly declared as optional.
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
t |
String | True | Message Type | Identifies the message as an event. Set to event. |
e |
String | True | Event Key | Identifies the event being reported. |
p |
Map | True | Payload | The data associated with the event. Its structure is determined by the Event Key. |
s |
Bytes | True | Sender ID | Identifies the original sender, usually Machine ID or a UUIDv4. |
b |
Bytes (8 or 16) | False | Boot ID | Identifies the current boot session. Generated once per boot as either eight random bytes or a UUIDv4 encoded as sixteen bytes. |
tms |
Integer or null |
True | Timestamp (milliseconds) | Timestamp (§2.1.4.1) made at the point the event was recorded. |
Table 4. The Event Message Schema (AMP Design Candidate 3).
Event messages with an invalid Sender ID must be discarded. A Sender ID is invalid if it is not a CBOR byte string. Invalid Boot IDs must be treated as absent. A Boot ID is invalid if it is not a CBOR byte string containing exactly eight bytes or sixteen bytes. Functions that depend on the Boot ID must follow their defined behaviour for an absent Boot ID.
Reserved events are presented (Table 5). Definition of additional events is future work.
| Event Key | Description |
|---|---|
hb |
Heartbeat |
pps |
PPS synchronisation. |
log |
Log |
upd |
Variable updates |
Table 5. Reserved Events (AMP Design Candidate 3).
The examples below use CBOR diagnostic notation, where h'...' represents a byte string written in hexadecimal.
Sizes are calculated using the preferred encodings in RFC 8949, with definite-length maps and arrays and no CBOR tags.
AMP message sizes include the 3-byte header and the CRC-32 when required, but exclude CAN-FD padding and frame overhead.
Timestamp Variable Convention
Timestamps are referred to as the length of time in a given unit since a given epoch. Absolute timestamps use the standard Unix epoch (1970-01-01 00:00:00 UTC). Any exceptions must be explicitly declared, and not overload a reserved key name. The three reserved absolute timestamp keys are presented (Table 6).
| Key | Name | Value Type | Unit of Time | Epoch |
|---|---|---|---|---|
ts |
Time in Seconds | Integer or null |
Seconds | 1970-01-01 00:00:00 UTC |
tms |
Time in Milliseconds | Integer or null |
Milliseconds | 1970-01-01 00:00:00 UTC |
tns |
Time in Nanoseconds | Integer or null |
Nanoseconds | 1970-01-01 00:00:00 UTC |
Table 6. Reserved Absolute Timestamp Keys.
Timestamp is null when valid absolute time is unavailable, including when time lock has not been acquired or clock uncertainty exceeds the permitted limit.
AMP uses CBOR’s preferred encoding, which represents each integer in the smallest available size, so smaller values use fewer bytes.
Encoding the timestamp unit in the key (ts, tms, or tns) uses fewer CBOR bytes than storing it in a separate field such as tu, where 0, 1, and 2 represent seconds, milliseconds, and nanoseconds.
The three-letter variable names add only 1 byte extra for storing tms or tns, whereas storing tu: 1 will add 4 bytes to the payload.
AMP does not impose an upper bound on timestamp values.
This design assumes implementations support unsigned integers from 0 to 2⁶⁴ − 1, inclusive.
The MicroPython environments for STM32 and RP2 (Pico/Pico 2) support uint64.
Larger values require CBOR positive-bignum support (tag 2) in both the encoder and decoder.
The maximum dates representable by each timestamp using uint64 are presented (Table 7).
| Key | Type | Unit | Limit reached |
|---|---|---|---|
ts |
uint64 |
Seconds | Year 584,554,051,223 |
tms |
uint64 |
Milliseconds | Year 584,556,019 |
tns |
uint64 |
Nanoseconds | 21 July 2554 |
Table 7. Estimated Timestamp Limits.
Given the protocol’s educational purpose, support through the year 2554 is sufficient.
To save data where many timestamps must be used within a single message, relative timestamps may be used. Each relative timestamp is an offset in nanoseconds or milliseconds from a shared absolute timestamp. The shared absolute timestamp must be included in the same message.
The reserved relative time keys are presented (Table 8).
| Key | Description | Data Type |
|---|---|---|
ms |
Milliseconds since Timestamp | Integer |
ns |
Nanoseconds since Timestamp | Integer |
Table 8. Relative Timestamp Keys.
If the shared absolute timestamp is null, the offsets still indicate the time between readings, but their absolute times are unknown.
The Heartbeat Event
The heartbeat event is used to announce that each device is alive.
It also communicates some important health metrics, including the time lock, PPS lock and RPC availability.
Future metrics and settings may be communicated.
Heartbeat event p (payload) object schema is presented (Table 9).
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
u |
Integer | True | Uptime in seconds | Seconds since startup. |
tl |
Boolean | False | Time Lock | Indicates whether the system has established the current Unix time from a received timestamp, regardless of PPS synchronisation. |
pl |
Boolean | False | PPS Lock | Indicates whether the system is currently synchronised to the PPS signal. |
rpc |
Boolean | False | RPC Availability | Indicates whether the AMP-RPC service is accepting requests. |
Table 9. The Heartbeat Data Object (AMP Design Candidate 3)
An example Heartbeat Event with a fictional sender ID, and payload parameters is shown (Code Snippet 1) in CBOR diagnostic notation.
1 | |
Code Snippet 1. An example Heartbeat Message (AMP Design Candidate 3).
The h'...' notation represents a CBOR byte string containing the hexadecimal bytes of the device ID.
Calculated using the encoding rules in RFC 8949, the example above encodes to 72 bytes of CBOR data.
The key and value encodings are shown separately in hexadecimal, with their sizes in bytes (Table 10).
The map rows count only their headers; the entries within each map are counted separately.
| Component | Key encoding (size) | Value encoding (size) | Total |
|---|---|---|---|
| Outer map header | — | a6 (1 byte) |
1 byte |
t: “event” |
61 74 (2 bytes) |
65 65 76 65 6e 74 (6 bytes) |
8 bytes |
e: “hb” |
61 65 (2 bytes) |
62 68 62 (3 bytes) |
5 bytes |
s: h’dedc106eabee4a79’ |
61 73 (2 bytes) |
48 de dc 10 6e ab ee 4a 79 (9 bytes) |
11 bytes |
b: h’8a32d17f4906bc25’ |
61 62 (2 bytes) |
48 8a 32 d1 7f 49 06 bc 25 (9 bytes) |
11 bytes |
tms: 1791578232000 |
63 74 6d 73 (4 bytes) |
1b 00 00 01 a1 22 62 54 c0 (9 bytes) |
13 bytes |
p key and inner map header |
61 70 (2 bytes) |
a4 (1 byte) |
3 bytes |
u: 8640000 |
61 75 (2 bytes) |
1a 00 83 d6 00 (5 bytes) |
7 bytes |
tl: true |
62 74 6c (3 bytes) |
f5 (1 byte) |
4 bytes |
pl: true |
62 70 6c (3 bytes) |
f5 (1 byte) |
4 bytes |
rpc: true |
63 72 70 63 (4 bytes) |
f5 (1 byte) |
5 bytes |
| Total | 26 bytes | 46 bytes | 72 bytes |
Table 10. Heartbeat message size estimate (AMP Design Candidate 3).
The 3-byte AMP header and 4-byte CRC-32 bring the total AMP message size to 79 bytes. The message fits within two CAN-FD frames, with one padding byte bringing the total frame payload to 80 bytes.
The encoded sizes depend on the values being transmitted.
For example, a smaller uptime (u) may require fewer bytes.
The PPS Synchronisation Event
The design of the PPS Synchronisation Event structure which allows peripheral devices to acquire time-lock is presented (Table 11).
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
ts |
Integer | True | Timestamp | Timestamp (§2.1.4.1) corresponding to the preceding PPS pulse. |
c |
Integer | True | Counter | Number of PPS pulses emitted since the sending device last restarted. |
Table 11. PPS Synchronisation Structure (AMP Design Candidate 3).
An example PPS Synchronisation Event is presented (Code Snippet 2), which occupies 66 bytes.
1 | |
Code Snippet 2. An example PPS Synchronisation Event.
The PPS pulse provides the timing reference, while the event identifies the Unix second associated with that pulse (Figure 5). The peripheral records the pulse when it arrives and associates the timestamp with that recorded edge when the event is received.
sequenceDiagram
accTitle: Associating a PPS pulse with Unix time
accDescr: A time source emits its first PPS pulse since restarting, and a peripheral records the edge. A later AMP PPS event supplies the Unix second associated with that edge and a pulse counter of one. The event's arrival time is not the PPS timing reference.
participant source as Time source / PPS emitter
participant peripheral as Peripheral device
source->>peripheral: First physical PPS edge since restart, marking second T
peripheral->>peripheral: Record the edge using the local timer
source->>peripheral: AMP event: e = pps, p.ts = T, p.c = 1
peripheral->>peripheral: Associate Unix second T with the recorded edge
source->>peripheral: Next physical PPS edge, one second later
Figure 5. PPS timing and the associated timestamp event (AMP Design Candidate 3).
This example assumes the event arrives before the next PPS pulse. A delayed event must not be associated with a newer pulse; handling delayed or missing PPS events requires a separate timing rule.
The Update Event
The design of the Update Event structure (Table 12) supports sending single or batched readings from one or more sensors, with individual timestamps. The notation is used to describe nested values within a Map:
<variable_key>represents the variable’s key, e.g.m1_rpostranslating to Motor 1’s rotational position.[i]identifies a sample within the array.
Two methods exist for defining timestamps (§2.1.4.1): absolute and relative. Each Update message must use one method, even if time lock is unavailable.
For the absolute method, each sample includes either a nanosecond (tns) or millisecond (tms) timestamp.
For the relative method, the payload includes one shared absolute timestamp, in nanoseconds (tns) or milliseconds (tms).
Each sample includes an offset from that reference, using nanoseconds (ns) or milliseconds (ms), respectively.
The reference timestamp and all offsets must use the same units.
When valid absolute time is unavailable, absolute timestamps must be null.
Relative offsets may still be supplied: use the earliest sample in the message as the shared reference, giving it an offset of zero.
All other offsets indicate the elapsed time since that sample.
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
tns |
Integer or null |
Required if using the relative timestamp scheme at nanosecond resolution. | Shared Timestamp (Nanoseconds) | Shared absolute timestamp. Used for relative, nanosecond offsets. |
tms |
Integer or null |
Required if using the relative timestamp scheme at millisecond resolution. | Shared Timestamp (Milliseconds) | Shared absolute timestamp. Used for relative millisecond offsets. |
| Variable Key | Array | True | Samples | An array of samples for the specified variable. |
<variable_key>[i].v |
Defined by Variable Key, or null |
True | Value | The variable’s value. Set to null when unavailable, for example due to a sensor error. |
<variable_key>[i].tns |
Integer or null |
Required if using the absolute timestamp scheme at nanosecond resolution. | Absolute Timestamp (Nanoseconds) | Absolute sample time in nanoseconds. |
<variable_key>[i].tms |
Integer or null |
Required if using the absolute timestamp scheme at millisecond resolution. | Absolute Timestamp (Milliseconds) | Absolute sample time in milliseconds. |
<variable_key>[i].ns |
Integer | Required if using the relative timestamp scheme at nanosecond resolution. | Relative Timestamp (Nanoseconds) | Non-negative offset from the shared timestamp, in nanoseconds. |
<variable_key>[i].ms |
Integer | Required if using the relative timestamp scheme at millisecond resolution. | Relative Timestamp (Milliseconds) | Non-negative offset from the shared timestamp, in milliseconds. |
Table 12. Update Event Structure.
An example Update Event for a fictional sensor (identified as m1_rpos) is presented (Code Snippet 3), which occupies 80 bytes.
1 | |
Code Snippet 3. An example Update Event.
The Log Event
The Log Event allows for remote monitoring and debugging of devices attached to the CAN bus. Sending, receiving, or failing to deliver a logging event must not generate another network logging event. The log events can be loosely structured but must include at least a human-readable message and a level.
The Log Event uses the severity levels documented in the Syslog Protocol (RFC 5424). The severity levels are presented (Table 13).
| Value | Level | Meaning |
|---|---|---|
0 |
Emergency | The system cannot operate. |
1 |
Alert | Immediate intervention is needed. |
2 |
Critical | A serious failure has occurred. |
3 |
Error | An operation has failed. |
4 |
Warning | A potential problem needs attention. |
5 |
Notice | A significant event during normal operation. |
6 |
Informational | Routine operational information. |
7 |
Debug | Details intended for troubleshooting. |
Table 13. Syslog Protocol Logging Severity Levels.
The Log Event structure is presented (Table 14). Additional parameters not documented can be included.
One timestamp (either tms or tns) must be present in the Log Event Message Payload.
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
lvl |
Integer (0-7) | True | Severity Level | The logging severity level indicates the urgency or priority of the problem being reported. |
msg |
string | True | Log Message | The human-readable log message. |
tms |
Integer or null |
True if tns not present. |
Timestamp (Milliseconds) | Timestamp (§2.1.4.1) at which the log was recorded, in milliseconds. |
tns |
Integer or null |
True if tms not present. |
Timestamp (Nanoseconds) | Timestamp (§2.1.4.1) at which the log was recorded, in nanoseconds. |
Table 14. The Log Event Structure.
An example Log Event with a severity level 3 and a fictional message, sender ID and timestamp is presented (Code Snippet 4), which occupies 90 bytes.
1 | |
Code Snippet 4. An example Log Event.
AMP-RPC Messages
The AMP-RPC Message is an AMP Message with a structure for facilitating request-and-response. Inspiration is taken heavily from JSON-RPC 2.0. Each request identifies a method and any arguments, much like a function call, together with a caller-generated Interaction ID. The responses will re-use the same Interaction ID so that the response can be matched to the original request. A success response indicates completion or acceptance for execution, as defined by the method.
The request-and-response exchange sequence is presented (Figure 6). The response carries the same Interaction ID as the request, regardless of outcome.
sequenceDiagram
accTitle: Matching an AMP RPC response to its request
accDescr: A caller sends an RPC request containing an Interaction ID, method and arguments. The receiving device validates the request and attempts the operation. It returns either a success response or an error response with the same Interaction ID, allowing the caller to match the response to the pending request.
participant requester as Requesting Device
participant responder as Responding Device
requester->>requester: Create request message <br>with Interaction ID
requester->>responder: AMP RPC Request Message
responder->>responder: Validate request and<br>attempt the operation
alt Operation succeeds or is accepted for execution
responder-->>requester: AMP RPC Response Message with `r` payload
else Operation fails
responder-->>responder: Rollback and clean up<br>where atomicity is required
responder-->>requester: AMP RPC Response Message with `err` payload
end
requester->>requester: Match responding Interaction<br>ID to the pending request
Figure 6. AMP RPC request and response, assuming a readable Interaction ID (AMP Design Candidate 3).
Provisionally, one request should receive one response. Future work could expand or modify the proposed protocol to include a mechanism to send multiple responses, including recoverable errors, for a given request until completion.
Future work could also include a simplistic mechanism for enabling the delay and scheduling of a request. This mechanism would allow the precise execution of the remote procedure, instead of simply when it is received. Additionally, if a request can be scheduled, it should also be cancellable. Cancellation of requests could also be expanded to those that are in progress, if possible.
A further expansion of AMP-RPC could include a real task scheduling system, enabling recurrent processes to occur without individual requests being sent. This would require an Application Programming Interface (API) that provides methods for creating, modifying, enabling, disabling, and executing tasks. It would also require persistent storage so that reboots do not lose track of scheduled tasks.
AMP-RPC Duplicate Request Protection Mechanism
To avoid executing a request twice, each recipient keeps a small cache of accepted requests, identified by Sender ID and Interaction ID. Requests are added before execution begins. A matching request is discarded without executing it again or extending its cache lifetime.
The cache holds up to 255 requests per sender, each for no more than 24 hours from its first receipt. During a boot session, age is measured using local elapsed time rather than the sender’s timestamp. Senders must not reuse an Interaction ID while its earlier request may still be cached. Protection ends when a record reaches 24 hours old or is removed to make room for a newer request.
If a response is lost, the caller may not know whether the request succeeded. During this stage of development, retrying with the same Interaction ID produces no response while the request remains cached. Retrying with a new Interaction ID may execute the operation again. In future, a mechanism could be implemented to see the status of a previously sent request.
Duplicate Request Protection Data Budget Estimate
A preliminary data budget for a basic request register is presented (Table 15).
For eight devices communicating with the recipient, using 16-byte sender and interaction IDs and a four-byte receipt-time record, the raw storage requirement is approximately 41 kB.
The estimate assumes receipt times are stored as uint32 uptime values in seconds.
Future implementations may store additional data.
Container and other implementation overhead are excluded and may be significant, particularly in MicroPython.
| Data Type | Size | Quantity | Total |
|---|---|---|---|
| Sender ID | 16 bytes | 8 | 128 bytes |
| Interaction ID | 16 bytes | 255 per each sender: 2040 | 32,640 bytes |
| Receipt time (seconds) | 4 bytes | 255 per each sender: 2040 | 8,160 bytes |
| Total | 40,928 bytes |
Table 15. Duplicate request protection register RAM allocation estimate.
Duplicate Request Protection Restart Mitigation
Devices may preserve the cache in persistent storage. Otherwise, restarting loses the request history. Recovery of persisted records is left to the implementation. AMP Candidate 3 does not guarantee duplicate protection across restarts.
A grace period of up to five seconds for startup and time synchronisation is permitted. A device that can communicate, but is not yet ready to execute requests should reject them with an error.
When request history has been lost and valid time is available, reject requests whose timestamps are at or before the estimated restart time. If either time lock or the request timestamp is unavailable, do not reject a request solely because its age cannot be established. Normal validation and any remaining duplicate checks still apply. This fallback is best-effort and does not compensate for differences between device clocks.
The AMP-RPC Request Message Schema
The AMP-RPC Request Message schema is presented (Table 16).
A request and a response are expected to use time in milliseconds when time is known.
The timestamp is null when time is not known.
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
t |
"req" |
True | Message Type | Indicates the message is an AMP RPC Request. |
id |
Bytes | True | Interaction ID | Identifies the request so that a response can be matched to it (Duplicate Request Protection). |
m |
String | True | Method Key | The identifier of the function to call on the remote host. |
p |
Unknown | False | Payload | The argument, or arguments provided to the facilitator of the request |
s |
Bytes | True | Sender ID | Identifies the original sender, usually Machine ID or a UUIDv4. |
d |
Bytes | True | Destination ID | Identifies the destination, usually Machine ID or a UUIDv4. |
b |
Bytes (8 or 16) | False | Boot ID | Identifies the current boot session. Generated once per boot as either eight random bytes or a UUIDv4 encoded as sixteen bytes. |
tms |
Integer or null |
True | Timestamp | Timestamp (§2.1.4.1) at which the message was sent. |
Table 16. The Request Message Structure.
RPC requests with a missing or invalid Sender ID, Destination ID or Interaction ID must be discarded. These IDs are invalid if they are not CBOR byte strings.
AMP-RPC Request Methods
RPC methods are presented (Table 17). Additional methods will be defined as part of future work.
| Method Key | Method Name | Description |
|---|---|---|
get |
Get | Get a variable from a remote system. |
set |
Set | Set a variable on a remote system. |
restart |
Restart | Start a reboot sequence on the remote system. |
halt |
Emergency Halt | Instruct the remote system to halt operations, stop all actuators and end its current task. |
Table 17. Request Message Methods.
The AMP-RPC Get Request Message
The Get request is used to retrieve a variable from a remote system. The payload of the get request is a String or an array of Strings where each String is a variable key. The response payload is the value of the variable, or a Map of each variable key and value as key/value pairs of the Map.
The AMP-RPC Set Request Message
The Set request is used to set a variable on a remote system. The payload of the set request is a Map of key/values where each key is a variable key, and each value is the new value.
All supplied values must pass validation before any changes are applied. Values must be accepted unchanged; otherwise, the request fails and an error is returned. A successful Set response contains null.
Set updates must be atomic: either all requested values are applied or none are. Atomicity applies to updating the requested values, not to the completion of the resulting physical actions. If validation or application of any value fails, all requested values must remain unchanged or be restored to their previous values, and an error must be returned.
The AMP-RPC Restart Request Message
The Restart request is used to start a reboot sequence on the remote system.
The response message is sent when the request has been accepted and scheduled, with a null payload.
The responder should attempt to transmit the acknowledgement before rebooting, but should not wait in case of a busy communication channel. If the device reboots immediately, the acknowledgement may not be sent. The requester should observe the Heartbeat for the occurrence of the reboot event, in this case.
The AMP-RPC Halt Request Message
The Halt request is used to instruct the remote system to halt operations, stop all actuators and end its current task. The payload of the halt is a boolean for the halt style: abrupt (true) or graceful (false).
Expected actions for a graceful halt are:
- Slow down actuators to a gradual stop.
- Straighten wheels to a neutral position.
- Stop active non-essential sensor readings but continue to send buffered data.
- Stop active non-essential radio communications at the nearest opportunity.
Expected actions for an immediate halt are:
- Lock motors in position, regardless of speed.
- Do not straighten wheels, hold in place.
- Stop active non-essential sensor readings and discard buffered data.
- Immediately stop all radio communications.
An analogy would be coming to a graceful stop in a car and applying the parking brake, versus an emergency stop in the middle of the road to avoid hitting an obstacle. Non-essential sensors are to be defined, but would refer to sonar, lidar, etc. or anything that would be noisy to the CAN bus. Essential sensors would include odometry, position, and velocity.
The response message is sent when the request has been accepted and scheduled, with a null payload.
The AMP-RPC Response Message Schema
The schema of the AMP-RPC Response Message is presented (Table 18).
The type of response is determined by the presence of the Result (r) or Error (err) key.
Exactly one of these keys must be present.
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
t |
res |
True | Message Type | Indicates the message is an AMP RPC Response. |
id |
Bytes | True | Interaction ID | The request’s Interaction ID, echoed unchanged. |
r |
Unknown or null |
True if err is absent. |
Result | The result body, structure determined by the method and arguments in the associated request. Set to null if void and not an error response. |
err |
Map | True if r is absent. |
Error | The object representation of error incurred during the facilitation of the associated request. |
s |
Bytes | True | Sender ID | Identifies the sender of the message, usually Machine ID or a UUIDv4. |
d |
Bytes | True | Destination ID | Identifies the destination, which must be the sender of the request, usually Machine ID or a UUIDv4. |
b |
Bytes (8 or 16) | False | Boot ID | Identifies the current boot session. Generated once per boot as either eight random bytes or a UUIDv4 encoded as sixteen bytes. |
tms |
Integer or null |
True | Timestamp | Timestamp (§2.1.4.1) at which the request was completed, accepted for execution, or failed, as defined by the method. |
Table 18. The AMP-RPC Response Message Structure.
RPC messages with a missing or invalid Sender ID, Destination ID or Interaction ID must be discarded. These IDs are invalid if they are not CBOR byte strings.
A successful response to the AMP-RPC Get Request returns the requested value(s) directly, using the same Interaction ID and reversing the sender and destination. An example Get Response is presented (Code Snippet 5).
1 | |
Code Snippet 5. An example Get Success Response, occupying 52 bytes.
The schema of the error map is presented (Table 19).
| Key | Type | Required | Name | Description |
|---|---|---|---|---|
c |
Integer | True | Error Code | The code used to describe the error. |
m |
String | True | Error Message | The message used to describe the error. |
Table 19. The AMP-RPC Response Error Map Structure.
The suggested AMP-RPC Error Codes are presented (Table 20). Future work may alter the codes, or add new codes.
| Code | Error | Description |
|---|---|---|
| 0 | Invalid request | The decoded message does not follow the RPC Request schema. |
| 1 | Method not found | The requested method is unsupported. |
| 10 | Invalid parameters | The payload has the wrong structure, types or combination of arguments. |
| 11 | Unknown variable | A requested variable key is not recognised. |
| 12 | Read-only variable | A set attempts to modify a read-only variable. |
| 13 | Value out of range | A supplied value violates the variable’s permitted limits. |
| 20 | Not ready | The device has not reached the state needed to perform the operation. |
| 21 | Busy | The device temporarily lacks capacity to accept the operation. |
| 22 | Invalid state | The operation cannot be performed in the device’s current state. |
| 23 | Time unavailable | The operation is rejected because the scheduled timing cannot be guaranteed. |
| 30 | Operation failed | The operation was valid but could not be completed, for example because a peripheral failed. |
| 31 | Internal error | An unexpected implementation failure prevented completion. |
| 40 | Permission denied | The requester is not permitted to invoke the operation. |
Table 20. Suggested AMP-RPC Error Codes.
Additional error codes and error messages may be reserved in future work.
Presented is an example error response for a Get request of an unknown key, where $test is the unknown key (Code Snippet 6).
1 | |
Code Snippet 6. An example Error Response, occupying 82 bytes.
Conclusion
To summarise, so far I have drafted three different AMP design candidates. The first candidate looked at cramming single updates and commands into 8-byte CAN 2.0B frames. The second candidate grouped those updates and commands into messages spanning multiple packets, with a CRC-32 to detect corruption. The third candidate replaced the fixed binary layouts with CBOR, allowing more flexible message structures, field names, and value types. Batching of separate AMP Frames was dropped in favour of prioritising single-packet efficiency, and single-message extensibility.
Through these iterations, I have designed a protocol for requesting data, updating device settings, and receiving events and logs. This design is still a candidate at this stage, and certain parts of the design have been postponed for future work, mainly defining additional methods, error codes, and event types.
AMP Candidate 3 allows routine communication, including Heartbeats and PPS Synchronisation, to fit within a single CAN-FD frame, given 8-byte sender IDs. Heartbeats can be kept to one frame with the absence of optional metrics and/or the absence of the boot ID. PPS events can be kept to one frame with the absence of the optional boot ID. Get and Set requests can also fit within a single frame, depending on the size and number of supplied parameters. The Get response example (Code Snippet 5) includes a 64-bit timestamp and fits within one CAN-FD frame for a single-variable request. Keeping these messages within a single frame reduces the overhead of fragmenting and reassembling routine communication.
As an alternative to CBOR, Google’s Protocol Buffers (Protobuf) could reduce encoding overhead and provide schema-defined message types.
Small-footprint libraries such as nanopb already support resource-constrained MCUs.
Although Protobuf supports compatible schema evolution, managing shared schemas across devices would introduce additional coordination during active development and research.
AMP v1 will therefore continue with the flexible CBOR design presented here. A future AMP v2 could adopt Protobuf and use ABC data protocol value 3.
The next work will include implementation of a CAN transport stack which can run on both MicroPython and CPython and interface with the underlying CAN drivers.
The ABC gateway will parse the arbitration IDs, group fragments by sender, target, protocol, and message counter, and reassemble messages when required.
Messages using protocol 0 will pass to the AMP gateway for length checks, CRC-32 verification when required, CBOR decoding, and schema validation.
The proposed reception path is presented (Figure 7).
---
config:
sequence:
actorMargin: 40
width: 150
messageMargin: 28
diagramMarginX: 12
mirrorActors: false
wrap: false
---
sequenceDiagram
accTitle: Receiving and validating an AMP message
accDescr: The CAN driver delivers frames to the ABC gateway, which groups fragments and checks their sequence until the message is complete or reception fails. Invalid or incomplete messages are discarded. For protocol zero, the reassembled payload and fragmentation metadata pass to the AMP gateway. It checks Body Length, excludes padding, and verifies CRC-32 only for fragmented messages, stopping on failure. It then decodes CBOR and validates the schema. Only valid AMP messages reach the application.
participant can as CAN driver
participant abc as ABC gateway
participant amp as AMP gateway
participant app as Application
loop Receive CAN-FD frames
can->>abc: CAN-FD Frame
abc->>abc: Parse Arbitration ID
abc->>abc: Wait for remaining frames<br/>if message is incomplete
end
alt Complete and valid ABC message
abc->>abc: Reassemble payload<br/>if fragmented
abc->>amp: AMP Message Blob
amp->>amp: Check 3-byte Body Length<br/>and exclude CAN-FD padding
opt Fragmented
amp->>amp: If length is valid: Verify CRC-32
end
amp->>amp: If previous checks pass:<br/>Decode CBOR and validate schema
alt All AMP checks pass
amp->>app: Decoded AMP Message
else Invalid length, CRC-32, CBOR or schema
amp->>amp: Discard message
end
else Invalid sequence <br>or reception timeout <br>or message too large
abc->>abc: Discard message
end
Figure 7. Receiving, validating and delivering an AMP Message to the application (AMP Design Candidate 3).
The AMP sender and destination are read from the CBOR body, independently of the local ABC addresses.
The diagram follows protocol 0, with one frame for a single-frame message or multiple frames for a fragmented message.
Reception stops at the first failed check; incomplete or invalid messages are discarded before delivery to the application.
Proposed changes to AMP include:
- Introducing protocol version identification, either within each message or through the transport protocol identifier.
- Extending AMP-RPC to support:
- Execution at a specified absolute time or after a delay measured from request acceptance.
- Creation, inspection, and cancellation of scheduled and recurring tasks.
- Cancellation of an operation already in progress, where supported by the method.
- Multiple responses per request, distinguishing acknowledgement, intermediate outcomes, and final completion.
- Rules for whether an intermediate error permits continued execution and a later successful result.
These changes are outside the scope of Candidate 3 and may require incompatible protocol changes. A robust task scheduler, cancellation mechanism, and supporting communication protocol introduce significant complexity.
While exploring these features for this design milestone, the number of edge cases grew quickly, particularly when devices had different time-lock states or lost synchronisation during an operation. Defining predictable behaviour for each condition would substantially expand the scope of this candidate, so these features are deferred to future work.