All Articles

FFI with Dart and Rust

I was working on porting https://medium.com/flutter/flutter-on-raspberry-pi-mostly-from-scratch-2824c5e7dcb1 the flutter framework [<flutter.dev>] to the raspberry pi by compiling from source and after a successful run I began wondering how I would write native extensions for apps that I had ported. After looking at projects like flutter-rs https://github.com/flutter-rs/flutter-rs I decided to try out writing a native library to interface with dart programs. Here’s what I found out.

What was I building?

Picana https://github.com/kituyiharry/picana is a dart library using a native rust shared library to communicate with CANBushttps://en.wikipedia.org/wiki/CAN_bus interfaces on linux. Its definitely more of a case study on using dart:ffi with native libraries than a production grade piece of code. I opted to use rust for this to avoid wrangling around with C/C++ toolchains and build systems and focus a lot more on learning the intricasies of interacting with the Dart VM.

The Project is able to:

  1. Bind to native CAN interfaces (including Virtual Ones)
  2. Write to the interfaces from Dart
  3. Read CANframes from the interfaces
  4. Read CANDumps

What I learnt

Using bindgen to glue C header files

I wanted to interact asynchronously to the Dart VM and conveniently a lot of helper methods to work with Ports are available in the Dart C Api. However to get these definitions from Rust I need a way to get the header definitions available. I found out about bindgen[https://rust-lang.github.io/rust-bindgen/] and how it automatically generates Rust FFI bindings to C and C++ libraries.

To achieve this I had to make a *-sys package. A Rust library project with a build.rs file that tells bindgen :

  1. Where the headers are.
  2. How to configure for binding

Here’s an example(dart_sys/lib/build.rs):

use std::env;
use std::path::PathBuf;

use bindgen::EnumVariation;

//We create a build.rs file in our crate's root. Cargo will pick up on the existence of this file, then compile and execute it before the rest of the crate is built. This can be used to generate code at compile time. And of course in our case, we will be generating Rust FFI bindings to bzip2 at compile time. The resulting bindings will be written to $OUT_DIR/bindings.rs where $OUT_DIR is chosen by cargo and is something like

fn main() {
    let mut bindings_builder = bindgen::Builder::default();

    println!("cargo:rerun-if-env-changed=BINDGEN_DART_SDK_PATH");

    println!("Ensure that BINDGEN_DART_SDK_PATH is set to get dart headers!");

    let dartsdk_path = if let Ok(path) = env::var("BINDGEN_DART_SDK_PATH") {
        PathBuf::from(path)
    } else {
        panic!("BINDGEN_DART_SDK_PATH not found in env");
    };

    bindings_builder = bindings_builder
        .header(format!("{}/include/dart_api.h", dartsdk_path.display()))
        .header(format!(
            "{}/include/dart_native_api.h",
            dartsdk_path.display()
        ))
        .header(format!(
            "{}/include/dart_tools_api.h",
            dartsdk_path.display()
        ));

    let bindings = bindings_builder
        .generate_inline_functions(true)
        .default_enum_style(EnumVariation::Rust {
            non_exhaustive: false,
        })
        .use_core()
        .clang_arg("-std=c++14")
        // required for macOS LLVM 8 to pick up C++ headers:
        .clang_args(&["-x", "c++"])
        .generate()
        .expect("Unable to generate bindings");

    // Where the bindings are generated!
    bindings
        .write_to_file("./src/bindings.rs")
        .expect("Couldn't write bindings!");

    //println!("cargo:rustc-link-search=native={}", out_dir);
    //println!("cargo:rustc-link-lib=static=hello");
}

This takes out a lot of pain in bringing the C-Header definitions to the rust-side. Just make sure you include!(concat!(bindings.rs)) in the main lib file.

lazy_static for static initialization

I had to create an object that lives for as long as the library is loaded so as to handle internal state. The crate description https://docs.rs/lazy_static/1.4.0/lazy_static/ puts it simply:

A macro for declaring lazily evaluated statics Using this macro, it is. possible to have statics that require code to be executed at runtime in. order to be initialized. This includes anything requiring heap allocations,. like vectors or hash maps, as well as anything that requires function calls. to be computed .

Kinda like this:

    ...
    lazy_static! {
        /// Creates a global static reference lazily
        static ref PICANA: Arc<RwLock<super::core::Picana>> =
            Arc::new(RwLock::new(super::core::Picana::new()));
    }
    ...

I found this pattern a little handy especially for code organization.

C is Lingua Franca

Despite using Rust, good knowledge of C and how data looks like in memory is a huge plus. I not only had to interact with some Dart Types but also had to send complex types across as well. One of my objects was a FrameResource which is just a CANFrame.

    ...
    // A resource to share across FFI boundaries!
    /// A Bulkier resource carrying information of a CANFrame specifically from a candump!
    #[no_mangle]
    #[repr(C)]
    pub struct FrameResource {
        /// Timestamp with microseconds
        t_usec: u64,
        /// ID of the frame
        id: u32,
        /// Device name eg can0, can1
        device: *const c_char,
        /// Data Section (8 bytes)
        data: *const c_uchar,
        /// Whether it is a remote Frame
        remote: bool,
        /// Whether an Error Code
        error: bool,
        /// Whether the frame is extended
        extended: bool,
        /// Associated error code?
        error_code: u32,
    }
    ...

A lot of these types (specifically prefixed by a c_) come from the libc crate https://docs.rs/libc/0.2.72/libc/. When building objects for this, Its wise to be cautious of a few things:

Stack allocations

If a function returns an object value allocated on the stack, its likely you’ll need to std::mem::forget that object lest it be freed when the function returns. If you need that object on the other side of the FFI bridge its best to std::boxed::Box it and convert it into a raw pointer.

{
   let exitframe = ...
   ...
   Box::into_raw(Box::new(exitframe))
}

A little snippet (from the code):

Rust’s owned boxes (Box<T>) use non-nullable pointers as handles which point to the contained object. However, they should not be manually created because they are managed by internal allocators. References can safely be assumed to be non-nullable pointers directly to the type. However, breaking the borrow checking or mutability rules is not guaranteed to be safe, so prefer using raw pointers(*) if that’s needed because the compiler can’t make as many assumptions about them

Unsafe with *-sys packages

Recall we created a package to interact with the the Dart C API, calling these functions requires the use of unsafe blocks. Its good practice to use unsafe sparingly and using it alot leads to muddying up your code. I created a mod with Macros to resolve this but using macros can make you lose a lot of flexibility with linting!

e.g To asynchronously send a value to a port, Dart provides a Dart_PostCObject which takes a Dart_Port and a valid Dart type. As such I created a macro like so:

//If you’re invoking unsafe blocks through an interface not marked as unsafe, it is the callee’s, not the caller’s, responsibility
//to make sure that every possible call is safe, since the whole point of unsafe is “static analysis can’t prove this is safe”,
//so I doubt static analysis can meaningfully capture this sort of unsafe hygine, nor do I think it is the compiler’s responsibility to do so.
// ________|
// |
//Why to not use unsafe block in macros! (https://internals.rust-lang.org/t/explicitly-marking-unsafe-macro-expressions/9425/3)
#[inline(always)]
macro_rules! send {
    ($x:expr, $y:expr) => {
        $crate::sys::Dart_PostCObject($x, &mut $y);
    };
}

As the note says, static analysis is lost and such everywhere i use the macro has to be in an unsafe block:

   unsafe {
      send!(port_id, dart_c_double!(-12213321.2331));
   }

Dart types across the bridge

To create and send Dart types to and from Native types requires different semantics due to differences in representations.

Fore example here’s what it takes to create a Dart String from C string on the native side to pass to the Dart VM.

///Creates a list of length size to hold elements
impl Value<DartString> {
    pub unsafe fn from_c_string(
        value: *const std::os::raw::c_char,
    ) -> Result<Self, exception::VmError> {
        let handle = Dart_NewStringFromCString(value);
        check_if_error!(handle, {
            std::mem::forget(handle);
            Ok(Value {
                raw: handle,
                _marker: PhantomData,
            })
        })
    }

    pub unsafe fn size(&self) -> Result<isize, exception::VmError> {
        let mut string_size: isize = -1;
        let handle = Dart_StringStorageSize(self.raw, &mut string_size);
        check_if_null!(handle, { Ok(string_size) })
    }

    pub unsafe fn from_str(slice: &str) -> Result<Self, exception::VmError> {
        let string = CString::new(slice).unwrap().into_raw();
        let handle = Dart_NewStringFromCString(string);
        check_if_error!(handle, {
            std::mem::forget(handle);
            Ok(Value {
                raw: handle,
                _marker: PhantomData,
            })
        })
    }
}

This can then be tranmitted asynchronously to the Dart interface with the aforementioned send!(...) macro.

Here how to create a Rust type from Dart

Pointer<LiteFrame> createFrame(int id, List<int> data, [bool remote = false, bool error = false]) {
        Pointer<Uint8> p = allocate();                                                                   ```
        //final data = [99, 101, 102, 103, 104, 105, 106, 107];
        for (var i = 0, len = data.length; i < len; ++i) {
                p[i] = data[i];
        }
        final liteframe = allocate<LiteFrame>();
        liteframe.ref.id = id;
        liteframe.ref.data = p;
        liteframe.ref.remote = remote ? 1 : 0;
        liteframe.ref.error = error ? 1 : 0;
        return liteframe;
}

Its best to remember that the order of items should be similar to its struct declaration.

---[To Be Continued]---

Published Jul 17, 2020

From Embedded systems to the web