How to effectively build a byte array from calculated parts?

Question

I need to build a byte array that represents commands to a device. It may look something like this:

let cmds = [
    0x01, // cmd 1
    0x02, // cmd 2
    0x03, 0xaa, 0xbb, // cmd 3
    0x04, // cmd 4
    0x05, 0xaa, // cmd 5
];

Some commands take parameters, some don't. Some parameters require calculations. Each command is fixed in size, so it's known at compile time how big the array needs to be.

It'd be nice to construct it like this, where I abstract groups of bytes into commands:

let cmds = [
    cmd1(),
    cmd2(),
    cmd3(0, true, [3, 4]),
    cmd4(),
    cmd5(0xaa)
];

I haven't found any way to do this with functions or macros. I am in no_std, so I am not using collections.

How to achieve something resembling this in Rust?

You want to store functions in an array? You can use trait objects. — Ibraheem Ahmed, Jan 26 '21 at 18:57
The [restruct](https://docs.rs/restruct_derive/0.1.1/restruct_derive/) crate might be of help if you can run on nightly. — user2722968, Jan 26 '21 at 21:15

kmdreko · Accepted Answer · 2021-01-26T21:32:46.127

You can have each command function return an array or Vec of bytes:

fn cmd1() -> [u8; 1] { [0x01] }
fn cmd2() -> [u8; 1] { [0x02] }
fn cmd3(_a: u8, _b: bool, _c: [u8; 2]) -> [u8; 3] { [0x03, 0xaa, 0xbb] }
fn cmd4() -> [u8; 1] { [0x04] }
fn cmd5(a: u8) -> Vec<u8> { vec![0x05, a] }

And then build your commands like so:

let cmds = [
    &cmd1() as &[u8],
    &cmd2(),
    &cmd3(0, true, [3, 4]),
    &cmd4(),
    &cmd5(0xaa),
];

This builds an array of slices of bytes. To get the full stream of bytes, use flatten:

println!("{:?}", cmds);
println!("{:?}", cmds.iter().copied().flatten().collect::<Vec<_>>());

[[1], [2], [3, 170, 187], [4], [5, 170]]
[1, 2, 3, 170, 187, 4, 5, 170]

You can make this more elaborate by returning some types that implement a Command trait and collecting them into an array of trait objects, but I'll leave that up to OP.

Edit: Here's a macro that can build the array directly, using the arrayvec crate:

use arrayvec::ArrayVec;

fn cmd1() -> [u8; 1] { [0x01] }
fn cmd2() -> [u8; 1] { [0x02] }
fn cmd3(_a: u8, _b: bool, _c: [u8; 2]) -> [u8; 3] { [0x03, 0xaa, 0xbb] }
fn cmd4() -> [u8; 1] { [0x04] }
fn cmd5(a: u8) -> [u8; 2] { [0x05, a] }

macro_rules! combine {
    ($($cmd:expr),+ $(,)?) => {
        {
            let mut vec = ArrayVec::new();
            $(vec.try_extend_from_slice(&$cmd).unwrap();)*
            vec.into_inner().unwrap()
        }
    }
}

fn main() {
    let cmds: [u8; 8] = combine![
        cmd1(),
        cmd2(),
        cmd3(0, true, [3, 4]),
        cmd4(),
        cmd5(0xaa),
    ];
    
    println!("{:?}", cmds);
}

If you're worried about performance, this example compiles the array into a single instruction:

movabs  rax, -6195540508320529919 // equal to [0x01‬, 0x02, 0x03, 0xAA, 0xBB, 0x04, 0x05, 0xAA]

See it on the playground. Its limited to types that are Copy. The length of the array must be supplied. It will panic at runtime if the array size doesn't match the combined size of the results.

Thanks, but with flatten and Vec, I'll need either std or an allocator, right? Don't have these available to me. — puritii, Jan 26 '21 at 19:11
You don't have to use `Vec`, and `flatten` doesn't use any allocation itself — kmdreko, Jan 26 '21 at 19:16
After `flatten`, the data will need to be collected and I don't believe it's possible to collect into an array. Don't think this solution works because of `no_std`. If I had `std` available to me, I'd simply append the commands to the back of a `Vec`. — puritii, Jan 26 '21 at 19:46
You *could* collect into something like an [`ArrayVec`](https://docs.rs/arrayvec/0.5.2/arrayvec/struct.ArrayVec.html). Though I suspect a macro-based implementation would be more ergonomic than what I've suggested. — kmdreko, Jan 26 '21 at 19:51
The macro approach is what I'm trying to find, but there is no way for a macro to return a part of an array, e.g. 3 bytes. For C preprocessor, this would be trivial, but not in Rust. — puritii, Jan 26 '21 at 19:53

score 0 · Answer 2 · answered Jan 27 '21 at 08:58

You can do it with no external dependencies if you do it as a macro:

macro_rules! cmd_array {
    (@ [ $($acc:tt)* ]) => { [ $($acc)* ] };
    (@ [ $($acc:tt)* ] cmd1(), $($tail:tt)*) => { cmd_array!{@ [ $($acc)* 0x01, ] $($tail)* } };
    (@ [ $($acc:tt)* ] cmd2(), $($tail:tt)*) => { cmd_array!{@ [ $($acc)* 0x02, ] $($tail)* } };
    (@ [ $($acc:tt)* ] cmd3 ($a:expr, $b:expr, $c:expr), $($tail:tt)*) => { cmd_array!{@ [ $($acc)* 0x03, 0xaa, 0xbb, ] $($tail)* } };
    (@ [ $($acc:tt)* ] cmd4(), $($tail:tt)*) => { cmd_array!{@ [ $($acc)* 0x04, ] $($tail)* } };
    (@ [ $($acc:tt)* ] cmd5 ($a:expr), $($tail:tt)*) => { cmd_array!{@ [ $($acc)* 0x05, $a, ] $($tail)* } };
    ($($tail:tt)*) => {
        cmd_array!(@ [] $($tail)*)
    }
}

fn main() {
    let cmds: [u8; 8] = cmd_array![
        cmd1(),
        cmd2(),
        cmd3(0, true, [3, 4]),
        cmd4(),
        cmd5(0xaa),
    ];
    
    println!("{:?}", cmds);
}

This macro is built using an incremental TT muncher to parse the commands, combined with push-down accumulation to build the final array.

How to effectively build a byte array from calculated parts?

2 Answers2