go-hx — The language-neutral core of the hx changeset engine — the changeset contract and the content-addressed store shared by every hx frontend.

go-hx is the language-neutral core of the hx changeset engine, shared by the hx CLI and every hx frontend (hx-go, hx-json, hx-text, …) so that no frontend depends on another’s binary. hx stores code as content-addressed contextual changes, not textual diffs: a diff records which bytes changed, not what the author meant, so a rename, an extracted function and an unrelated fix all arrive as the same undifferentiated soup. Here the ground truth is a log of intent-bearing changes over a content-addressed substrate, and trees and text are projections of the store rather than the master copy.

• Source: gomatic/go-hx
• API reference: pkg.go.dev/github.com/gomatic/go-hx
• Framework docs: docs.hx

Install

go get github.com/gomatic/go-hx

The packages

changeset owns the contract a frontend emits and the store ingests — the lossless file segmentation, the canonical content-addressing, the classified ops, and the name bindings. It also owns what “well-formed” means: ChangeSet.Validate holds every producer to the same rules, not only those that go through the CLI.

store owns the engine that content-addresses those definitions, records changes, and materializes any revision back to byte-identical source. It never parses source; all language knowledge lives in a frontend.

The root package holds only the sentinel error vocabulary both layers speak, as constants matchable with errors.Is across the module boundary.

The object model

Two kinds of object, both content-addressed, sharing one keyspace and told apart by a domain tag.

A definition is one addressable unit supplied by a frontend: a structural Fingerprint (identity with renamable names abstracted away, which is what lets the store detect moves and deduplicate without understanding the language) plus the literal Source text that drives byte-exact materialization.

A change records a parent, a message, the lossless segmentation of every touched file, the classified ops, and the resulting name bindings. A file is an ordered list of segments — verbatim raw spans and addressable def segments — whose concatenation reproduces the file’s bytes exactly:

def := changeset.Definition{Fingerprint: "fp1", Source: "func A() {}\n"}

cs := changeset.ChangeSet{
	Message: "genesis",
	Files: []changeset.File{{
		Path: "a.go",
		Segments: []changeset.Segment{
			{Kind: changeset.SegmentRaw, Raw: "package a\n\n"},
			{Kind: changeset.SegmentDef, Def: def},
		},
	}},
	Ops:      []changeset.Op{{Kind: changeset.OpAdd, Name: "a.A"}},
	Bindings: map[changeset.Name]changeset.Hash{"a.A": def.Address()},
}

Committing and reading back

The engine is built over two injected seams — Blobs (a content-addressed object store) and Refs (the append-only ref log) — so it is fully testable without touching disk. OSStore satisfies both against an .hx/ directory; MemoryStore satisfies both in memory:

backing, err := store.Init(store.Dir(".hx"))
engine := store.New(backing, backing)

addr, err := engine.Commit(cs)          // returns the change's content address
head, err := engine.Head()              // the single tip, or ErrDivergentHistory
entries, err := engine.Log()            // newest first, with each change's Bindings
tree, err := engine.Materialize(addr)   // path -> byte-exact contents

A malformed change-set is refused before anything is stored, with a sentinel the caller can match:

_, err := engine.Commit(bad)
if errors.Is(err, hx.ErrUnknownSegmentKind) { /* … */ }

What the addressing guarantees

The store’s entire claim is content = identity. These are the properties that make it true rather than merely intended; each is enforced, and each has a regression test behind it.

An object’s address is the SHA-256 of exactly its stored bytes — not of the value that produced it, and not of a decoder’s interpretation of it. Every read recomputes the address and refuses a mismatch before decoding, so a hand edit, a partial write or a corrupted sync is caught before anything parses it. Objects are filed under the address derived from their own bytes, so an object cannot be stored under an address it does not hash to.

The encoding is a bijection on accepted objects: one content, one byte string, one address. Both directions matter, and both were once broken. Every field is length-prefixed and every variable-length section is framed with its element count, so a section boundary cannot move without changing the bytes — without that framing, a change-set of one file plus one operation and a change-set of three files produced the identical byte stream and therefore the identical address, and committing the second silently destroyed the first. And a decoded object must re-encode to the bytes it came from, so a producer cannot reorder map entries or repeat a key to give one change several valid addresses, each verifying under its own bytes and indistinguishable to a reader.

The encoding is binary-safe. Fields carry arbitrary bytes, including sequences that are not valid UTF-8, because materialization is byte-exact and a version-control system that silently rewrites a byte is not one.

The layout is versioned. A store written by a version this package cannot read says so, instead of failing object by object as though it had been tampered with — and initializing over one is refused rather than silently relabelling it.

File paths are constrained. A materialized tree is keyed by the paths a change-set carries, and writing that tree to disk is the obvious thing to do with it, so a traversal path would be an arbitrary file write handed to every consumer. Those are rejected at the contract, along with two paths that name one location by different spellings. The rules are lexical: they cannot see a symlink, so a consumer writing a tree should also confine the write, for example with an os.Root.

History is a ref log, not a pointer

There is no HEAD file. The ref log holds write-once entries named by the change address they record, carrying no contents and no device, host or user identity. It records tip candidates rather than one entry per change — a parent’s entry is dropped once its child is recorded — so the log stays proportional to the tips while history lives in the objects, walked by parent pointer. Keeping an entry per change instead made every commit re-read the whole history, which cost O(N²).

The tips are derived: the recorded addresses that no other recorded address descends from. Deriving rather than storing them is what makes a re-introduced entry harmless — a sync that brings back a superseded one is filtered out, not mistaken for a second head.

That shape is what makes concurrency and sync safe. A single mutable pointer is a last-writer-wins cell — two commits that both read it before either wrote produced two objects and one surviving pointer, so a change that had been committed, and reported committed, was simply unreachable. With an append-only log both writers create their own entry, both survive, and the result is a divergence that is reported rather than resolved:

tips, err := engine.Tips()   // every tip, ascending
head, err := engine.Head()   // ErrDivergentHistory when len(tips) > 1

// Commit derives its parent from the single tip, so name one to continue from
// after a divergence — the engine will not choose for you.
addr, err := engine.CommitOnto(tips[0], cs)

More than one tip is an explicit state to merge — never corruption, never silent loss. It is also why two stores that committed different changes offline merge by naive file union, which is what a sync engine does, with nothing to conflict.

Verifying a store

objects, err := engine.Verify()

Verify reads every object the ref log reaches and confirms each both against its address and as renderable — an object can hash correctly and still be unreadable, such as a segment kind this version does not know, so a check that stopped at the address would call a store sound that cannot be checked out. Reachable is the honest scope: an object no recorded change refers to is unreferenced garbage rather than history. It is two operations that happen to be the same walk: the integrity check that verification-on-read cannot be, since that answers one object at a time and only for objects someone asked for; and the operation that makes a store usable offline, because reading every object is what pulls back the contents of a file a sync engine has evicted to a placeholder.