Engineering a Local-First Future: The ThoughtStream Case Study

Modern software has a “Cloud-First” problem. We’ve become so accustomed to the convenience of centralized APIs that we’ve accepted a fundamental trade-off: our applications only work when the internet does, and our data is only as private as the service provider allows.

ThoughtStream is my attempt to invert this model. It is a personal journaling application built on Local-First principles, where the user’s device is the primary source of truth, and the cloud is relegated to its rightful place as an asynchronous synchronization layer.

The Source of Truth Inversion

In a traditional architecture, the client is a “thin” UI that waits for a REST API to confirm every action. In ThoughtStream, the UI talks exclusively to a local SQLite database.

This architectural decision has three immediate benefits:

1. Zero Latency: Every interaction is instantaneous. There are no loading spinners because the data is already on the disk.
2. Offline-Native: The app doesn’t “support” offline mode; it is offline by default.
3. Resilience: The system remains fully functional even if the sync server is down or unreachable.

Solving Distributed Conflicts with HLCs

The biggest challenge in local-first systems is Conflict Resolution. When two devices (e.g., your phone and your laptop) edit the same note while offline, how do you merge those changes without a central clock?

Relying on system time is dangerous because hardware clocks drift. Instead, ThoughtStream implements Hybrid Logical Clocks (HLC).

HLCs combine the best of physical wall-clock time and logical counters. Every write in ThoughtStream is timestamped with a unique HLC string (e.g., 2026-04-24T16:00:00.000Z-0001-device_id). This allows the system to achieve a Total Ordering of events across all devices. When a sync occurs, the system deterministically chooses the update with the “greater” HLC, ensuring every device eventually arrives at the exact same state without needing a central coordinator.

The Delta Sync Protocol

Moving data between a local SQLite database and a remote PostgreSQL server requires a robust sync protocol. ThoughtStream uses a custom Delta Sync mechanism:

1. Change Tracking: Every table in the local database includes a modified_at HLC column.
2. The Push Phase: The client identifies all records modified since the last successful sync and pushes these “deltas” to the server.
3. The Authoritative Sequence: The server acts as an authoritative sequencer. It validates incoming HLCs and commits them to a global change log, assigning each a monotonic Sequence ID.
4. The Pull Phase: The client requests all changes from the server that occurred after its last known Sequence ID.
5. Local Application: The client applies these remote changes to its local state, using the HLC comparison to handle any last-minute conflicts.

Technical Stack

• Frontend: Flutter was chosen for its excellent SQLite (sqflite) support and cross-platform consistency. The reactive nature of Riverpod ensures the UI stays in sync with the local database state.
• Backend: A lightweight FastAPI (Python) service manages the authoritative PostgreSQL database. It doesn’t perform complex business logic; its primary job is to be a reliable “traffic controller” for incoming deltas.
• Storage: PostgreSQL on the server and SQLite on the client. By keeping the schemas nearly identical, the mapping layer remains thin and performant.

Why Local-First Matters

ThoughtStream isn’t just about making a faster note-taking app. It’s about Data Sovereignty. By making the software local-first and self-hostable via Docker, the user regains control over their digital thoughts.

In an era where “the cloud” often means “someone else’s computer,” building local-first software is a return to the original promise of personal computing: tools that work for you, on your hardware, under your control.

Project Link: dhiraj-ydv/thought_stream-localfirst

No resources added for this note.