Bridging Rust and Swift: Building a GPU Tile Engine with wgpu

Astropad Workbench is built with Swift for the app layer and Rust for the LIQUID engine core. This hybrid approach gives you Swift's excellent Apple ecosystem integration and Rust's performance for compute-heavy paths like GPU tile diffing and video encoding.

This post covers the practical mechanics of calling Rust from Swift.

Why Rust for the GPU Engine?

Swift has Metal. Why not just use Metal directly?

1. Cross-platform: wgpu runs on Metal (macOS/iOS), Vulkan (Linux/Android), and DX12 (Windows). Rust engine works everywhere.
2. Memory safety: Rust's ownership model prevents GPU resource leaks and data races that are common in Metal code.
3. Ecosystem: LZ4 (lz4_flex), zstd, and other compression libraries are excellent in Rust.
4. Astropad does it: Their entire liquid-next workspace is Rust, compiled to a static library and called from Swift via FFI.

Architecture

Swift App Layer (SwiftUI, AppKit)
    ↓ C-ABI FFI
Rust Static Library (.a)
    ├── liquid-tile-engine/  (wgpu compute, tile diff, compression)
    ├── liquid-bridge/       (safe FFI wrapper)
    └── shaders/             (wgsl compute shaders)

Step 1: Create the Rust Workspace

mkdir -p rust && cd rust
cargo init --lib liquid-tile-engine
cargo init --lib liquid-bridge

rust/Cargo.toml (workspace root):

[workspace]
members = ["liquid-tile-engine", "liquid-bridge"]
resolver = "2"

rust/liquid-tile-engine/Cargo.toml:

[package]
name = "liquid-tile-engine"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["staticlib"]  # Produces .a for Swift linking

[dependencies]
wgpu = "24.0"
lz4_flex = "0.11"
bytemuck = { version = "1.14", features = ["derive"] }

[build-dependencies]
cbindgen = "0.27"

Step 2: Define the FFI Interface

rust/liquid-tile-engine/src/ffi.rs:

use std::slice;

/// Opaque handle to the tile engine
pub struct TileEngine {
    tile_size: u32,
    width: u32,
    height: u32,
    previous_frame: Option<Vec<u8>>,
}

#[repr(C)]
pub struct ChangedTile {
    pub tile_x: u16,
    pub tile_y: u16,
    pub compressed_data: *const u8,
    pub compressed_size: u32,
}

#[repr(C)]
pub struct TileResult {
    pub tiles: *mut ChangedTile,
    pub count: u32,
    pub codec_mode: u8,  // 0=tiles, 1=h26x, 2=idle
}

/// Create a new tile engine
#[no_mangle]
pub extern "C" fn liquid_tile_engine_create(
    width: u32,
    height: u32,
    tile_size: u32,
) -> *mut TileEngine {
    let engine = Box::new(TileEngine {
        tile_size,
        width,
        height,
        previous_frame: None,
    });
    Box::into_raw(engine)
}

/// Process a frame and return changed tiles
#[no_mangle]
pub extern "C" fn liquid_tile_engine_process(
    engine: *mut TileEngine,
    pixel_data: *const u8,
    pixel_data_len: usize,
) -> TileResult {
    let engine = unsafe { &mut *engine };
    let frame = unsafe { slice::from_raw_parts(pixel_data, pixel_data_len) };
    
    // Compare with previous frame, find changed tiles, compress them
    // ... (implementation details)
    
    TileResult {
        tiles: std::ptr::null_mut(),
        count: 0,
        codec_mode: 0,
    }
}

/// Free a tile result
#[no_mangle]
pub extern "C" fn liquid_tile_result_free(result: TileResult) {
    if !result.tiles.is_null() {
        unsafe {
            let tiles = Vec::from_raw_parts(result.tiles, result.count as usize, result.count as usize);
            for tile in tiles {
                if !tile.compressed_data.is_null() {
                    let _ = Vec::from_raw_parts(
                        tile.compressed_data as *mut u8,
                        tile.compressed_size as usize,
                        tile.compressed_size as usize,
                    );
                }
            }
        }
    }
}

/// Destroy the tile engine
#[no_mangle]
pub extern "C" fn liquid_tile_engine_destroy(engine: *mut TileEngine) {
    if !engine.is_null() {
        unsafe { let _ = Box::from_raw(engine); }
    }
}

Step 3: Generate C Header with cbindgen

rust/liquid-tile-engine/cbindgen.toml:

language = "C"
header = "// Auto-generated by cbindgen. Do not edit."
include_guard = "LIQUID_TILE_ENGINE_H"

[export]
include = ["ChangedTile", "TileResult"]

rust/liquid-tile-engine/build.rs:

fn main() {
    let crate_dir = std::env::var("CARGO_MANIFEST_DIR").unwrap();
    cbindgen::Builder::new()
        .with_crate(crate_dir)
        .with_language(cbindgen::Language::C)
        .generate()
        .expect("Unable to generate bindings")
        .write_to_file("include/liquid_tile_engine.h");
}

Build:

cd rust/liquid-tile-engine
cargo build --release --target aarch64-apple-darwin

This produces target/aarch64-apple-darwin/release/libliquid_tile_engine.a and include/liquid_tile_engine.h.

Step 4: Import into Swift

Add to Package.swift:

.systemLibrary(
    name: "CLiquidTileEngine",
    path: "rust/liquid-tile-engine/include",
    pkgConfig: nil,
    providers: []
)

Or link directly:

.executableTarget(
    name: "LiquidHost",
    dependencies: ["LiquidCore"],
    linkerSettings: [
        .unsafeFlags([
            "-L", "rust/target/aarch64-apple-darwin/release",
            "-l", "liquid_tile_engine",
        ]),
    ]
)

Create a bridging header or a Swift wrapper:

import CLiquidTileEngine  // or use the generated .h directly

class TileEngineWrapper {
    private var engine: OpaquePointer?
    
    init(width: Int, height: Int, tileSize: Int = 64) {
        engine = liquid_tile_engine_create(
            UInt32(width), UInt32(height), UInt32(tileSize)
        )
    }
    
    func processFrame(pixelData: UnsafeRawPointer, length: Int) -> [TileData] {
        let result = liquid_tile_engine_process(engine, pixelData, length)
        defer { liquid_tile_result_free(result) }
        
        var tiles: [TileData] = []
        for i in 0..<Int(result.count) {
            let tile = result.tiles[i]
            let data = Data(bytes: tile.compressed_data, count: Int(tile.compressed_size))
            tiles.append(TileData(x: tile.tile_x, y: tile.tile_y, data: data))
        }
        return tiles
    }
    
    deinit {
        if let engine = engine {
            liquid_tile_engine_destroy(engine)
        }
    }
}

Step 5: Build Script

Scripts/build-rust.sh:

#!/bin/bash
cd rust/liquid-tile-engine
cargo build --release --target aarch64-apple-darwin
echo "Static library: $(ls -la target/aarch64-apple-darwin/release/libliquid_tile_engine.a)"
echo "Header: $(ls -la include/liquid_tile_engine.h)"

What Astropad's Binary Reveals

Astropad's FFI layer is in liquid_ffi/include/StateSync/StateSync.swift — a Swift file that bridges to the Rust state sync system. Their bridge passes complex state objects (capturable content, client view, clipboard, input, virtual display config) between Swift and Rust.

Key Astropad internal crates that use this pattern:

• liquid_statesync_core::ffi — State sync FFI with Swift
• liquid_codec — Video encoding called from Swift
• liquid_net — Networking called from Swift via LiquidRuntime.swift

Their build process uses GitHub Actions with a Cargo workspace at AstroDeps/liquid-next/.

Tips for a Clean FFI Boundary

1. Keep the FFI surface small — expose 5-10 functions, not 50. Complex logic stays in Rust.
2. Use opaque pointers — Swift doesn't need to know Rust struct layouts.
3. Allocate/free on the same side — if Rust allocates memory, Rust must free it.
4. No panics across FFI — use catch_unwind at every FFI boundary.
5. Thread safety — mark your Rust types as Send + Sync if they'll be called from multiple Swift threads.

---

Part 7 of the "Building a Remote Desktop from Scratch" series. Based on reverse engineering analysis of Astropad Workbench 1.1.0.