I’m really enjoying using no_std Rust for embedded programming on the Lilygo T-deck and I want to share what I’ve learned so far. There are many excellent tutorials and docs on Embedded Rust in general and the ESP32 in particular, so I’m going to cover things that are specific to the T-deck or that I’ve found to be under documented. Today let’s start by looking at the T-deck’s signature feature, the keyboard.

Keyboard

In some devices the keys and buttons are directly connected to pins on the MCU. This is not the case with the T-Deck. Since there are so many keys, and because it’s such a central part of the T-Deck experience, the keyboard is controlled by a secondary ESP32 chip connected to the main S3 chip by an I2C bus. To get keyboard events we just need to read from a specific address on the bus. The basic code looks like this:

let LILYGO_KB_I2C_ADDRESS: u8 = 0x55;

let mut i2c = I2c::new(
    peripherals.I2C0,
    Config::default()
        .with_frequency(Rate::from_khz(100))
        .with_timeout(BusTimeout::Disabled),
)
.unwrap()
.with_sda(peripherals.GPIO18)
.with_scl(peripherals.GPIO8);

info!("looping over the keyboard");
loop {
    let mut data = [0u8; 1];
    let kb_res = i2c.read(LILYGO_KB_I2C_ADDRESS, &mut data);
    match kb_res {
        Ok(_) => {
            if data[0] != 0x00 {
                info!("key {:?}", String::from_utf8_lossy(&data));
                delay.delay_millis(100);
            }
        }
        Err(e) => {
            info!("kb_res = {e}");
            delay.delay_millis(1000);
        }
    }
}

The firmware for the keyboard is minimal and buggy. It sends out ascii characters, but you can’t detect the shift or symbol keys. They’ve released an updated keyboard firmware but it seems cumbersome to install. I haven’t tried it out yet. Here’s another keyboard replacement firmware that looks interesting too. I’ve also not tried it.

Full reading keyboard example.

Trackball

The trackball is one of the T-Deck’s most unique features. It’s very odd, though. Real physical mice and trackballs use rotary encoders to give you changes along the x and y axis. On the T-Deck instead you get a pin to monitor for each of the four cardinal directions. It’s more like polling buttons. The standard code looks like this.

let tdeck_trackball_click = Input::new(
    peripherals.GPIO0,
    InputConfig::default().with_pull(Pull::Up),
);
let tdeck_trackball_right = Input::new(
    peripherals.GPIO2,
    InputConfig::default().with_pull(Pull::Up),
);
let tdeck_trackball_left = Input::new(
    peripherals.GPIO1,
    InputConfig::default().with_pull(Pull::Up),
);
let tdeck_trackball_up = Input::new(
    peripherals.GPIO3,
    InputConfig::default().with_pull(Pull::Up),
);
let tdeck_trackball_down = Input::new(
    peripherals.GPIO15,
    InputConfig::default().with_pull(Pull::Up),
);
let mut last_click_high = false;
let mut last_right_high = false;
let mut last_left_high = false;
let mut last_up_high = false;
let mut last_down_high = false;

info!("running");
loop {
    info!("button pressed is {} ", 
        tdeck_track_click.is_low());
    if tdeck_trackball_click.is_high() != last_click_high {
        info!("trackball click changed ");
        last_click_high = tdeck_trackball_click.is_high();
    }
    if tdeck_trackball_right.is_high() != last_right_high {
        info!("trackball right changed ");
        last_right_high = tdeck_trackball_right.is_high();
    }
    if tdeck_trackball_left.is_high() != last_left_high {
        info!("trackball left changed ");
        last_left_high = tdeck_trackball_left.is_high();
    }
    if tdeck_trackball_up.is_high() != last_up_high {
        info!("trackball up changed ");
        last_up_high = tdeck_trackball_up.is_high();
    }
    if tdeck_trackball_down.is_high() != last_down_high {
        info!("trackball down changed ");
        last_down_high = tdeck_trackball_down.is_high();
    }
    // wait one msec
    let delay_start = Instant::now();
    while delay_start.elapsed() < Duration::from_millis(1) {}
}

This code hooks up the four pins as inputs and then reads them every millisecond. This works but is a little clunky. The inputs are noisy and there is no suggested sampling rate. For use in a real application I would wrap this up into an async task with Embassy and use a channel to emit higher level mouse and scroll events. This is exactly what I do in my bigger projects.

Full trackball example.

Display

The display itself is easy to set up (if verbose). It works like any other SPI screen. Use the ST7789 driver from the mipidsi package like this:

use esp_hal::spi::master::{Config as SpiConfig, Spi};
...
use mipidsi::{models::ST7789, Builder};


...
// set TFT CS to high
let mut tft_cs = Output::new(peripherals.GPIO12, High, OutputConfig::default());
tft_cs.set_high();
let tft_miso = Input::new(
    peripherals.GPIO38,
    InputConfig::default().with_pull(Pull::Up),
);
let tft_sck = peripherals.GPIO40;
let tft_mosi = peripherals.GPIO41;
let tft_dc = Output::new(peripherals.GPIO11, Low, 
    OutputConfig::default());
let mut tft_enable = Output::new(peripherals.GPIO42, High, 
    OutputConfig::default());
tft_enable.set_high();

info!("creating spi device");
let spi = Spi::new(
    peripherals.SPI2,
    SpiConfig::default().with_frequency(Rate::from_mhz(40))
)
.unwrap()
.with_sck(tft_sck)
.with_miso(tft_miso)
.with_mosi(tft_mosi);
let mut buffer = [0u8; 512];

info!("setting up the display");
let spi_delay = Delay::new();
let spi_device = ExclusiveDevice::new(spi, tft_cs, spi_delay).unwrap();
let di = SpiInterface::new(spi_device, tft_dc, &mut buffer);
info!("building");
let mut display = Builder::new(ST7789, di)
    .display_size(240, 320)
    .invert_colors(ColorInversion::Inverted)
    .color_order(ColorOrder::Rgb)
    .orientation(Orientation::new().rotate(Rotation::Deg90))
    .init(&mut delay)
    .unwrap();

From here use the standard embedded graphics APIs to draw.

Full example code here.

Note that the display is mounted sideways so _the height and width are swapped_ to 240 x 320.

Also note that I set the SPI rate to 40mhz which seems to be the max speed of the t-deck’s screen. I know that sounds fast but remember that SPI is a serial bus. Refreshing an entire 16bit 320x240 screen takes is _153.6 kilobytes per frame_ so base case, with no overhead, you can get about 30fps. In practice, it’s more like 20 and still has tearing. These devices don’t have any way to do accelerated drawing like lines or textures. (DMA might be a possibility) so be careful to draw as little as possible. I typically refresh my GUI every 50 msec (40fps) but only draw things that have changed. For example, here’s a simple block breaking game that only redraws the ball on most frames.

Here is the code to a little brick breaking game to show off fast graphics.

Touchscreen

The touchscreen is more interesting. The T-Deck has a real capacitive touch screen with up to (I think) five simultaneous touches. There’s an existing driver api for the GT911 chip that makes it very easy to use. Setup an i2c bus, init the driver, and loop to read touch points. Here’s a simple example:

use gt911::Gt911Blocking;

...
let mut i2c = I2c::new(
    peripherals.I2C0,
    Config::default()
        .with_frequency(Rate::from_khz(100))
        .with_timeout(BusTimeout::Disabled),
)
.unwrap()
.with_sda(peripherals.GPIO18)
.with_scl(peripherals.GPIO8);

let touch = Gt911Blocking::default();
touch.init(&mut i2c).unwrap();
loop {
    if let Ok(points) = touch.get_multi_touch(&mut i2c) {
        // stack allocated Vec containing 0-5 points
        info!("{:?}", points)
    }
}

See the full example code for more details.

That's all for this week. Next time we'll look at WiFi and HTTP requests. Have a great weekend!