Reserving a length prefix and its body in one bounds check
A serialiser that writes a count and then writes the payload it counted has two failure points, and the second one is the dangerous one: the count has landed in the buffer, the payload has not, and whatever reads that buffer back trusts a length nothing backs. You don't close that gap with a better error path after the fact. You close it by reserving the prefix and the body in a single bounds check, then refusing the whole field before a single byte of it goes down.
I have written length-prefixed framing for card-rail payment messages, for on-chain payloads, and for service-to-service protocols that were never supposed to meet hostile input and eventually did. The partial-write bug has the same shape every time, and it does not look wrong in review.
The second failure point is the one that escapes#
Here is the shape I am arguing against. It is not how bytespan does it; it is how almost everything else does it.
// Two calls, two bounds checks, two chances to refuse.
bool write_blob(std::span<std::byte> out, std::size_t& at,
std::span<const std::byte> payload) {
if (!write_u16(out, at, static_cast<std::uint16_t>(payload.size()))) return false;
return write_bytes(out, at, payload); // the failure that matters
}If out has room for the count but not for the body, this returns false and the buffer now ends with a count and nothing behind it. The failure gets observed on the writing side. The damage is on the reading side, later, quite possibly in another process. A reader that trusts the count reads past the end of the record and into whatever follows it: the next frame, the next record, uninitialised memory. The caller that logged the write error has no idea it produced a readable lie.
There is a second defect in those four lines, and it is the cast. A payload of 65536 bytes becomes a count of 0. Truncating a count is worse than refusing, because the result is well-formed: nothing downstream can tell it apart from an empty field. A payload too large to be counted in its prefix has to fail the writer.
One check, then two writes that cannot refuse#
In bytespan both prefix widths go through one private template, and length_prefixed_u16 and length_prefixed_u32 are the two instantiations:
template <typename T>
bool length_prefixed(std::span<const std::byte> payload, endian order) noexcept {
if (!ok()) return false;
const std::size_t size = payload.size();
if constexpr (sizeof(std::size_t) > sizeof(T)) {
if (size > static_cast<std::size_t>(std::numeric_limits<T>::max())) {
poison({offset_, reason::too_long, size,
static_cast<std::size_t>(std::numeric_limits<T>::max())});
return false;
}
}
// Prefix and body are reserved in one check, so a refusal cannot leave
// a lone count behind. Subtracting from the room left, rather than
// adding the two, keeps the sum from wrapping on a 32-bit size_t.
const std::size_t room = out_.size() - offset_;
if (size > room || sizeof(T) > room - size) {
poison({offset_, reason::no_room, detail::saturating_add(sizeof(T), size), room});
return false;
}
put(static_cast<T>(size), order);
return bytes(payload);
}The order is the design. First the poisoned-writer guard, so a field after an earlier refusal is a no-op instead of a write into a buffer whose offset nobody trusts any more. Then the counting question, decided entirely on the payload size and the prefix type: a payload larger than std::numeric_limits<T>::max() is reason::too_long, with the payload size as wanted and the prefix maximum as available. That test sits inside if constexpr, so it disappears where std::size_t and T have the same width and the comparison could not fail. A 32-bit size_t with a u32 prefix has nothing to check.
Only then the room question, and you ask it once for both pieces. After it passes, the two writes are not failure points any more. They go through the same private helper every other write uses:
bool take(std::size_t n) noexcept {
if (!ok()) return false;
if (n > out_.size() - offset_) {
poison({offset_, reason::no_room, n, out_.size() - offset_});
return false;
}
offset_ += n;
return true;
}Both take calls are bound to succeed, because the room for sizeof(T) + size was established a line earlier. That is why put has its return value ignored there, the one place in the writer where ignoring it is defensible. What you get out of it is the property worth having: written() either moves past the whole field or does not move at all. There is no state in which a count exists for a reader to trust without its body.
The arithmetic is part of the guarantee#
The natural way to write that bounds check is sizeof(T) + size > room, and on a 32-bit size_t it is a vulnerability. A payload size near the top of the type plus two bytes of prefix wraps to something small, the comparison passes, and the writer goes ahead and places a count it cannot back. Subtract instead, size > room || sizeof(T) > room - size, and every term stays bounded by room; the second comparison is only reached once the first has established that room - size does not underflow. Same question, no sum, nothing to wrap.
The refusal record carries the same hazard for a different audience. wanted is meant to name what the field needed, which is prefix width plus payload, so computing it means adding exactly the two numbers the check just refused to add. detail::saturating_add folds them together and clamps instead of wrapping; a diagnostic that lies about the size it refused is worse than no diagnostic at all. The reader side is symmetric: a header claiming 65535 bytes of payload in a 4-byte input reports wanted 65537 and available 4, at the offset of the prefix rather than somewhere inside the field.
And the record is the first refusal. poison keeps only what it is given first:
void poison(refusal record) noexcept {
if (!refusal_) refusal_ = record;
}Once a writer is poisoned, every later call fails as a consequence. Letting those consequences overwrite the record would bury the one failure that explains the output.
When the length is not known until the body is written#
Reserving both halves requires knowing the size of the body before you write it, and sometimes you do not: the body gets assembled by nested calls that are themselves writers of fields. For that there is the slot API. open_u16 and open_u32 reserve the prefix, zero it, and hand back a length_slot, and close fills it in with what was written since.
const auto slot = w.open_u16();
if (!slot) return false; // the one call you do have to check
// ... write the body through w ...
w.close(*slot, bytespan::endian::big);This path writes the prefix first, so the invariant has to be weaker, and it is made explicit rather than left to luck. open zeroes the reserved bytes, because the caller's buffer need not have arrived zeroed and an unclosed field has to read as zero length instead of as whatever was sitting there. close computes body = offset_ - (slot.offset + slot.width), refuses a body too large for its width as too_long at the field's own offset, and leaves the zeros alone when it does. A slot closed out of order, closed twice, or opened on another buffer is bad_slot, recorded at the offset the writer had reached: such a slot's own offset may name a position in a buffer that has nothing to do with this one.
A varint written whole#
The same rule applies where the field has no prefix at all. varint_u64 emits unsigned LEB128, seven bits per byte, low group first, and a half-written one is not merely short: the last byte it placed has its continuation bit set, pointing the reader at whatever happens to come next. Since the reader accepts only the canonical encoding and judges each byte as it reads it, a truncated varint followed by unrelated bytes can decode cleanly to a value the writer never meant.
So the writer builds the groups into a std::array of detail::varint_u64_max_bytes, ten, the most a 64-bit value can need, and hands them to bytes in one call. One bound, one write, all or nothing. The property, as a test:
void a_varint_that_does_not_fit_writes_nothing() {
std::array<std::byte, 2> buffer{};
writer w{buffer};
CHECK(!w.varint_u64(0xFFFFFFFFFFFFFFFFull));
CHECK(!w.ok());
CHECK(w.written() == 0);
}What I would do differently#
Two things. The first is that length_prefixed takes a std::span<const std::byte>, which means the body has to exist contiguously in memory before the field can be written. That is what makes the single bounds check possible, but it splits the API: a body you can measure takes the strong path that refuses before writing, and a body you cannot takes the slot path that leaves zeros behind. I spent a while trying to collapse those into one call and concluded the split is honest, it falls exactly where the knowledge falls, but it is still two failure stories for one concept, and a caller has to know which one they are in.
The second is smaller and bothers me more: put(static_cast<T>(size), order); with a discarded bool is correct for a reason that lives three lines above it. Correct code that reads like a missing check is a maintenance hazard, and I would rather the discard were loud than rely on the comment surviving the next edit...
A count is a promise about the bytes that follow it. Under a failing write, the only way to keep that promise is to decide you can keep it before you make it.