Decentralized Collaboration

We at Skiff believe the future of collaboration is decentralized

How do existing collaboration products work?

Most collaboration products today rely on central servers to maintain document state and merge simultaneous changes from multiple collaborators. This technology - known as operational transforms (OT) - stems from a seminal paper "High-latency, low-bandwidth windowing in the Jupiter collaboration system" published in 1995. In the paper, David Nichols and other authors (many from Xerox PARC) describe a novel, forward-looking collaborative platform wherein users, distributed around the world, can share software and collaborate on widgets, audio/video, and documents. At a time when very few US households had internet access (by 1997, around 20% of US households did), the Jupiter paper foresaw a renaissance in distributed work. Technology has changed quite a bit since then.Although performant and reasonably straightforward to implement, operational transforms are limited by centralization. OT requires every client maintain an open connection to the server and, on making changes to a document, send over document operations (such as the insertion or removal of a character). As the server applies and composes operations from multiple clients, it controls document state and re-synchronizes each client. Today, operational transforms remain the basis of many collaborative apps, including Google Docs and others.

Operational transforms and privacy

Operational transforms are also fundamentally anti-private. By mandating that a central server maintain document state and merge changes from all live collaborators, operational transforms expose every keystroke, word, and sentence to the technology provider and possibly network provider. While encryption in transit (https) may increase privacy of network transmission, the OT model still shares users' documents and writing with the provider itself. As a result, maintaining end-to-end encryption required new models for collaboration and simultaneous editing.

CRDTs, decentralization, and the future

More recent papers have suggested new, decentralized approaches. Unlike operational transforms, conflict-free replicated data types (CRDTs) allow multiple peers to maintain simultaneous, synchronized copies of a data structure and merge conflicting changes. As outlined in this excellent blog post on the history of OTs and CRDTs, CRDTs evolved from expensive and impractical to pragmatic and logical.While early implementations were slow and required enormous on-disk storage (largely because CRDTs maintain some records of deletions, known as tombstones), performant implementations of CRDTs (including Automerge and YJS) are used in production collaboration, communication, and design applications. More recent research (such as the paper "Near Real-Time Peer-to-Peer Shared Editing on Extensible Data Types," which evolved into YJS) has presented scalability and performance improvements feasible for use in the web.

Conclusion

Operational transforms introduced us to real-time collaboration on the web. Yet, even as implementations have evolved in speed and complexity, the core technology was imagined (in 1995) over a decade before web-based collaborative editors were widely used (Google Docs was first released in 2006). Today, CRDTs offer improved performance and transformational privacy, enabling full decentralization and real-time synchronization.At Skiff, CRDTs fit closely into our view of how technology should be built - private, performant, and beautifully designed for the future.