polycratia

A symlink is not code: refusing a tree entry before reading its bytes

· 9 min read

A tool that verifies quoted code against a repository has to read bytes at a commit and compare them with the block printed in the prose. Not every path at a commit holds bytes that are code. A symlink entry holds a target path. A submodule entry names a commit that belongs to another repository, one the clone may not even have. Read either and compare it with a quoted block, and you get a confident verdict about the wrong bytes.

I hit this while building srcproof, a lockfile and gate for prose that cites code (the code in this post is from github.com/polycratia/srcproof). The design is deliberately unforgiving: every repository link in an article must name a commit as 40 lowercase hex characters, and every fenced block anchored by a permalink with a line range gets a record in srcproof.lock, written as quote ARTICLE SHA256 REPO COMMIT PATH START-END. On check, those lines are sliced out of the blob at that commit and compared line by line, after only the normalizations the lock itself names.

That comparison is total. If it runs at all, it produces a verdict. There is no "probably" in a byte-for-byte loop. So every piece of correctness in the checker lives before the comparison, in the decision about whether these bytes are the kind of thing the question was even asked about.

The failure mode is a pass, not an error#

You could frame reading a symlink as a bug because it yields nonsense. That undersells it. Git stores a symlink as a blob whose content is the target path: one line, no trailing newline. Suppose an article quotes a single line from a repository, and the path recorded in the lock happens to be a symlink whose target is a path that also appears in the quoted block. The slice is one line, the blob is one line, and they are equal. The checker then reports that the block is this file at this commit, byte for byte.

That is not a crash. It is not a misleading error message either. It is a certification, issued by the one component whose entire value is that a pass means something. Low probability is not the same as acceptable, because the class of outcome is wrong.

A submodule entry fails in the other direction, and it lies about the remedy. The entry's object id is a commit in another repository. Asked for that object, the outer clone will normally not have it, and a plain "object absent" answer maps onto ErrMissingHistory, which check reports as cannot decide and exits 2 with a message telling the reader to fetch the commit. Fetching into this clone will never make a submodule pointer quotable as a file. The verdict is unfollowable advice. And in the rarer case where the clone does hold that object, the read succeeds and the comparison runs: a code block against the bytes of a commit object.

Refuse on the mode, with the listing that found the path#

So the read refuses the entry on its tree mode, before the content is fetched:

go
    clean, err := cleanPath(path)
    if err != nil {
        return nil, err
    }
    r.mu.Lock()
    defer r.mu.Unlock()
    if err := r.requireCommit(commit); err != nil {
        return nil, err
    }
    entry, found, err := r.entry(commit, clean)
    if err != nil {
        return nil, err
    }
    if !found {
        return nil, fmt.Errorf("%s is not in commit %s: %w", clean, commit, ErrMissingPath)
    }
    switch {
    case entry.mode == modeSymlink:
        return nil, &NotQuotableError{Commit: commit, Path: clean, Mode: entry.mode, Kind: "symlink"}
    case entry.mode == modeSubmodule:
        return nil, &NotQuotableError{Commit: commit, Path: clean, Mode: entry.mode, Kind: "submodule"}
    case entry.typ != "blob":
        return nil, &NotQuotableError{Commit: commit, Path: clean, Mode: entry.mode, Kind: entry.typ}
    }

The refusal is free. entry runs ls-tree --full-tree -z on the commit with the path after --, and accepts a record only when its name equals the cleaned path exactly. The mode arrives in that same record, next to the object id and the type. The command that establishes the path exists in this commit is the command that establishes and it is a 120000 entry. Nothing extra has to be fetched to pay for this guard, which makes it the cheapest kind of correctness there is: a fact you already have in hand, checked before you act on its neighbour.

The third case matters too. An entry whose type is not blob is refused with its git type as the kind. Usually that means a directory, because somebody pinned a tree link and the quote record kept the directory path. A directory would otherwise be "read" into something that is not the file anybody meant.

The order of the whole function is a precedence rule, not decoration. The commit must match the 40-lowercase-hex pattern or the call fails with ErrObjectName before git is involved at all. The path goes through cleanPath, which refuses an empty path, a leading slash, any form of .., and a zero or newline byte. Only then does requireCommit ask whether the commit is present. When two things are wrong at once, the caller is told about the outer one: a symlink at a path in a commit the clone does not have reports missing history, because the mode is not knowable without the commit. You cannot refuse an entry you have not seen.

Three refusals, three actions#

All three not-quotable cases unwrap to one sentinel, ErrNotQuotable. It would be simpler to fold them further and give the caller one "cannot read this" error. That simplification destroys the only thing the error is for. Each case asks the caller to do something different:

Error What is true What the caller must do
ErrMissingPath the commit is here, the path is not in it fix the path, or pin again
ErrMissingHistory the question has not been answered fetch the commit, or name a clone that has it
ErrNotQuotable the path is here and is not code quote something that is a regular file

The quote checker consumes exactly that distinction:

go
    blob, err := clone.Blob(record.SHA, record.Path)
    switch {
    case errors.Is(err, gitrepo.ErrMissingHistory):
        result.Reason = ReasonUndecided
        result.Message = fmt.Sprintf("cannot decide: %v", err)
        return result, nil
    case errors.Is(err, gitrepo.ErrMissingPath), errors.Is(err, gitrepo.ErrNotQuotable):
        result.Reason = ReasonDrift
        result.Message = err.Error()
        return result, nil
    case err != nil:
        return Result{}, fmt.Errorf("%s:%d: %w", block.File, block.Fence, err)
    }

Missing history is undecided, and the other two are drift. The wording is not where the difference lives, because the two reasons travel all the way out to the process exit code. check counts drifted and undecided results separately, and an undecided count returns 2 before the generic failure returns 1. CI reads that number. An absent commit must not be able to turn a failing check into a passing one, and an unquotable entry must not be able to look like a transient fetch problem that somebody retries into a green build.

I have written this taxonomy badly before, in reconciliation code. "Not found" and "not yet known" collapsed into one error, and the retry logic treated a permanent answer as a temporary one for as long as the operator's patience lasted. The shape of the fix is the same every time: name the states by the action they demand, not by the layer that noticed them.

The same rule, stated twice on purpose#

Listing the files of a commit applies the identical filter, so a walk over a commit never asks for bytes that the read would refuse:

go
        fields := strings.Fields(info)
        if len(fields) != 3 || fields[1] != "blob" {
            continue
        }
        if fields[0] == modeSymlink || fields[0] == modeSubmodule {
            continue
        }
        names = append(names, name)

And the symbol table builder, which reads every .go file of a pinned commit to decide whether the names in code spans exist, still tolerates a refusal on each individual read:

go
        data, err := tree.Blob(commit, name)
        switch {
        case errors.Is(err, gitrepo.ErrMissingPath), errors.Is(err, gitrepo.ErrNotQuotable):
            continue
        case err != nil:
            return nil, err
        }

That second check looks redundant against the first. It is not, because the builder takes a Tree interface, not a concrete clone. It does not get to assume that the implementation which produced the file list is the one that filters symlinks out of it. A symlink named handler.go is a perfectly ordinary thing for some other tree implementation to hand over, and the builder's answer, skip the file and do not fail the article, is the right one at that boundary.

What I would change#

The Kind field carries two vocabularies in one string. For a symlink and a submodule it is a word for a reader; for anything else it is a git object type, so a pinned directory is reported as tree. A reader who pinned a folder by accident is being answered in plumbing terms. I would separate the reader-facing word from the git type and keep both in the error.

I would also give the mode its own type rather than comparing the raw six-character field against string constants. It works because ls-tree prints a fixed-width mode, but a typed mode makes the set of cases visible at the point where the switch claims to cover them.

What I would keep without hesitation is the position of the guard. Refusing on metadata you already hold, before fetching the thing you would then have to interpret, is the same discipline as validating a payment instruction before you move money: a wrong refusal costs a message, and a wrong acceptance costs something you cannot take back. Here the irreversible thing is not funds but a verdict a reader builds on. When the comparison at the centre of a tool is total, its honesty is made entirely out of the comparisons it declines to run, and out of telling each caller which of the three different problems they actually have...

react

$ new-project --brief

or email hey@polycratia.com