Building TychoTTY: A Terminal Emulator from Scratch in Rust

I spend most of my time in a terminal, both at work and at home. I've used them all — Alacritty, Kitty, Wezterm, iTerm2 — and none of them felt like mine. So I built one.

TychoTTY is named after Tycho Brahe, the 16th century astronomer, and my English Bulldog who shares his name. It's a cross-platform terminal emulator written in Rust with built-in multiplexing, vim-modal keybindings, an embedded browser, and an AI-augmented shell.

The Problem

My daily workflow involves:

• Multiple terminal panes (tmux)
• Vim keybindings for everything
• Switching between a Linux workstation (NVIDIA RTX 3060) and a macOS laptop (M4 Pro)
• Frequently asking AI assistants questions alongside running shell commands

I wanted all of this in one tool, with zero mouse usage.

The Stack

After evaluating options, I landed on:

• Rust — Cross-platform, fast, and I already knew it from other projects
• GTK4 — Mature toolkit that works on both Linux and macOS, with natural WebKit integration on Linux
• Cairo/Pango — Terminal cell rendering with full font and color support
• portable-pty — Cross-platform PTY management
• vte — Alacritty's ANSI escape sequence parser
• llama-cpp-2 — Embedded llama.cpp for local LLM inference (CUDA on Linux, Metal on macOS)
• webkitgtk-6.0 — Full browser engine embedded as a pane type via minimal FFI

Architecture

The whole thing is about 1,900 lines across 10 source files:

src/
├── main.rs        — GTK4 app, UI, action processing (470 lines)
├── grid.rs        — Terminal grid, VTE parser, scrollback (364 lines)
├── input.rs       — Vim-modal input handler (285 lines)
├── layout.rs      — Tree-based pane layout engine (214 lines)
├── config.rs      — TOML configuration (133 lines)
├── renderer.rs    — Cairo/Pango terminal rendering (109 lines)
├── browser.rs     — WebKitGTK 6.0 FFI bindings (107 lines)
├── classifier.rs  — Embedded llama.cpp intent classifier (152 lines)
├── ai.rs          — AI agent dispatcher (64 lines)
└── pty.rs         — Cross-platform PTY management (34 lines)

Terminal Grid & VTE Parsing

The terminal grid implements the vte::Perform trait to handle ANSI escape sequences — cursor movement, SGR colors (16 ANSI, 256-color, and 24-bit truecolor), scroll regions, erase operations, and more. It maintains a 10,000-line scrollback buffer and supports /pattern search across the full history.

Each pane gets its own grid, VTE parser, and PTY. The PTY reader runs on a background thread and pushes data to the main thread via async channels, where it's parsed and triggers a redraw.

Pane Layout Engine

The layout engine uses a binary tree structure. Each node is either a leaf (a terminal or browser pane) or a split (horizontal or vertical, with a ratio). Splitting a pane replaces the leaf with a split node containing the original pane and a new one. Closing a pane collapses the split back to its sibling.

This gives you arbitrary nesting — split vertically, then split one of those horizontally, and so on. Each pane runs its own independent shell.

Vim-Modal Input

TychoTTY starts in Insert mode (keystrokes go to the shell) and supports five modes:

• Insert — All keys go to the active pane's PTY
• Normal — hjkl to navigate panes, Ctrl-W v/s to split, gt/gT for tabs, gg/G for scrollback
• Command — :q, :split, :vsplit, :tabnew, :tabclose
• Search — /pattern to search scrollback
• Visual — Text selection with y to yank

The status bar at the bottom shows the current mode, window/pane count, and AI status.

The AI Shell Wrapper

This is the part I'm most excited about. When you press Enter, TychoTTY intercepts your input and runs it through a local LLM to classify it as either a shell command or an AI query.

The classifier uses a Qwen 2.5 0.5B model (Q4 quantized, 380MB) loaded directly into the process via llama.cpp. On Linux it uses CUDA, on macOS it uses Metal. The prompt is simple: given the input, output SHELL or AI. With greedy sampling and only 5 output tokens, classification is fast.

There's a 150ms timeout with fail-open semantics — if the model doesn't respond in time, the input is treated as a shell command. You never notice the latency.

If the classifier says AI, the input is routed to a configurable agent. I use my own agent (Brain) at home and kiro-cli at work. The agent runs as a subprocess and its response is displayed inline in the terminal.

[ai]
enabled = true
agent = "brain"
agent_command = "brain"
model_path = "~/.config/tychotty/models/classifier.gguf"

Cross-Platform

The build works on both Linux and macOS:

# Linux (with CUDA)
CUDA_PATH=/opt/cuda cargo build --release

# macOS (Metal is automatic)
cargo build --release

The Cargo.toml uses platform-specific dependencies:

[target.'cfg(target_os = "linux")'.dependencies]
llama-cpp-2 = { version = "0.1", features = ["cuda"] }

[target.'cfg(target_os = "macos")'.dependencies]
llama-cpp-2 = { version = "0.1", features = ["metal"] }

The browser pane uses webkitgtk-6.0 on Linux (detected at build time via build.rs and pkg-config). On macOS, the browser functionality would use WKWebView — that's still on the roadmap.

What's Next

This is a v0.1. Things I want to add:

• PTY resize propagation when panes change size
• Mouse support for text selection
• True OSC 52 clipboard integration
• Browser pane fully wired into the layout with :browse command
• Configurable keybindings
• Sixel/Kitty image protocol support
• A proper terminfo entry

The code is intentionally minimal — under 2,000 lines for a functional terminal emulator with multiplexing, a browser, and AI integration. Rust made this possible by providing the right building blocks (the VTE and PTY crates especially) and the confidence that if it compiles, the memory management is sound.

Tycho (the dog) approves.