Binary Messaging
Owners: Jay and Merritt · Project: Binary Messaging · Task: FMV-888
This is a review proposal, not a deployed protocol. The sizes and field capacities
come from our agreed design. Positions, code values, padding and channel behavior
below are proposed choices for us to review. No code deployment or database
migration is performed by this document.
Agreed budget
| Record |
Metadata |
Content |
Total application bytes |
| Send |
8 |
32 |
40 |
| Receipt body |
4 |
None |
4 |
WebSocket/TLS framing, authentication and initial catalog/context establishment
are outside these application-record sizes and must be measured separately.
Models write/read plain text. Software performs encoding, decoding and lookups.
Proposed send layout
Each letter is one bit. A divider separates two four-bit fields. Byte numbers
start at one; the leftmost bit in each byte is the most significant bit.
| Byte |
Bit layout |
Meaning |
| 1 |
FFFF │ TTTT |
Relay family and message type |
| 2 |
RRRRRRRR |
Recipient crew index |
| 3 |
FFFFFFFF |
Request flags |
| 4 |
AAAA │ SSSS |
Account index and server index |
| 5 |
PPPP │ DDDD |
Data prefix and database index |
| 6 |
TTTTTTTT |
Table locator |
| 7 |
MMMMMMMM |
Message ID, most significant eight bits |
| 8 |
MMMMMMMM |
Message ID, least significant eight bits |
| 9–40 |
32 complete bytes |
Direct message text, with padding if shorter |
The main new placement proposal is account/server at byte four. The record
locator stays contiguous at bytes five through eight. The two-byte message ID
uses most-significant byte first.
| Field |
Available values |
| Family, type, account, server, prefix, database |
16 each |
| Recipient, table |
256 each |
| Message ID |
65,536 within its established namespace |
Sender comes from the authenticated caller/session. Account and server explicitly
state the intended destination and must match the recipient's current canonical
binding. The numeric recipient maps to the exact native session; it is not a name
or an authorization grant.
Proposed initial catalogs and flags
The database holds the meanings and immutable catalog versions. The following
numbers are proposals, written in binary only. They are not existing allocations.
| Catalog |
Proposed value |
Meaning |
| Family |
0001 |
Crew relay |
| Type |
0000 |
Send instruction |
| Other family/type values |
Unassigned |
Reject until explicitly defined |
The message type already identifies send. Do not repeat an operation field.
Recipient/account/server/prefix/database/table indexes are allocated from actual
central records, not hard-coded person-specific values in the codec. These are
references to canonical identities and routes, not a competing roster.
| Request flags |
Meaning |
A0000000 |
A is the explicit approval boolean; all other bits reserved |
A true approval bit does not replace current authorization. Unknown families,
types or nonzero reserved bits fail explicitly. Unsupported existing operations
can retain their explicitly negotiated legacy path during migration.
Proposed text rule
Use UTF-8, limited to 32 encoded bytes, with no embedded zero byte. A shorter
message is followed by zero bytes until the content area is exactly 32 bytes.
A full 32-byte message needs no terminator. Decode until the first zero byte, or
the end of the area; require all bytes after the first zero to be zero.
Reject invalid UTF-8, an empty send instruction and overlong input before dispatch.
Do not truncate, split a Unicode sequence or silently send multiple frames.
Detailed instructions remain in the task record; the short message can point to
that task. This is direct language, not a cached vocabulary of instructions.
Example readable content, comfortably within the limit:
Continue FMV-888; report.
Proposed receipt body
| Byte |
Bit layout |
Meaning |
| 1 |
MMMMMMMM |
Message ID, most significant eight bits |
| 2 |
MMMMMMMM |
Message ID, least significant eight bits |
| 3 |
E │ CCCCCCC |
Error boolean and response code |
| 4 |
AH000000 |
Acknowledgement, history verification, six reserved bits |
E = 0 means success-class status; E = 1 means error/indeterminate class.
The seven code bits allow 128 codes in each class. The database supplies readable
meanings. Gateway status is represented here rather than echoed separately.
Proposed initial code meanings:
| E |
Code bits |
Meaning |
0 |
0000000 |
Native input accepted |
0 |
0000001 |
Saved, awaiting dispatch |
1 |
0000000 |
Invalid record |
1 |
0000001 |
Authorization denied |
1 |
0000010 |
Destination binding unavailable or changed |
1 |
0000011 |
Namespace exhausted |
1 |
0000100 |
Message identity conflicts with saved request |
1 |
0000101 |
Delivery unconfirmed; do not replay |
A is true only with an exact native acknowledgement. H is true only after the
same input is independently observed in retained recipient conversation history.
H being false can mean verification has not happened. Neither flag means the
assigned work is complete. Followup and completion remain reserved until we
choose their exact meanings and evidence requirements.
The same message ID identifies request, receipt and audit lookup. There is no
second request ID, acknowledgement ID or separately repeated idempotency token
on the compact wire. An actual conversational answer is separate from this
receipt; its binary payload format is outside this first proposal.
Namespace and return routing
Four receipt bytes cannot identify database/table context on their own. Propose
one authenticated logical channel bound at setup to a fixed catalog version and
prefix/database/table namespace. Requests on that channel must carry matching
namespace indexes. Receipts inherit that context. Different namespaces use
separate channels; do not ambiguously interleave them behind an unlabelled
four-byte receipt. The physical transport/multiplexing choice remains to review.
The channel establishes sender identity, protocol version and namespace once.
No repeated account/server echo is needed in the receipt. Its reporting peer
must still be authenticated and checked against the saved request route.
A message number is a lookup key, never proof of reporting authority.
Do not rebind a channel while earlier receipts can still arrive. Reconnection
must recover the same namespace/catalog context or explicitly reject it. Namespace
lifetime and reuse rules are part of correctness, not merely storage cleanup.
Database and identity lifecycle
Use the central SQLite authority. Persist numeric catalogs, canonical-binding
references, namespace counters, messages and event evidence. Actual table names
and migrations are implementation choices; avoid a second task or roster source.
Allocate a two-byte index transactionally within its namespace and save the
immutable sender, route, content and digest before forwarding. Retain the same
identity across retries and reconnects. Reject an identity reused for different
content or bindings. Keep full audit evidence and status transitions queryable.
When the index range is exhausted, return namespace-exhausted. Do not wrap or
recycle IDs silently. A new namespace requires an explicit safe context change.
Database allocation/control exchanges count toward measured total traffic; they
are not concealed by the 40-byte record claim.
Where the native daemon API requires a UUID, persist a stable internal UUID
mapping for this record. That API identifier stays inside the adapter/database;
it does not become a second compact wire field. Preserve native history
correlation and existing duplicate-dispatch fences.
Daemon path in both directions
| Direction |
Processing |
| Outgoing |
Readable instruction → validate and save → encode 40 bytes → forward → decode → native recipient input |
| Returning |
Native evidence → save → encode four-byte receipt → forward → save and decode → compact readable result for sender |
Extend the existing communication daemon/relay/connector path. The receiving
agent never performs bit arithmetic. The sender never decodes a binary dump.
Native APIs may keep their required structured text interfaces.
An acknowledgement lost after native dispatch must yield an unconfirmed outcome,
not a fresh send. Persist dispatch intent before the native write. A restart or
repeated record must not run the instruction twice. Detailed proof remains
available by message coordinate without another model turn.
The coordinator completion/stop notification work, FMV-898, is a separate
integration requirement. A four-byte transport receipt must not be confused with
verified task completion or an automatic model acknowledgement conversation.
Proposed implementation sequence
- Review the proposed placements, codes, flags, text rule and namespace channel.
- Implement catalogs and durable identity allocation with migration/rollback.
- Implement and test the fixed codec without changing live transport.
- Add daemon adapters and explicit legacy/binary negotiation.
- Prove one bounded round trip through sender, relay and recipient.
- Qualify wrong-route rejection, lost acknowledgements, duplicate delivery,
restart recovery, size boundaries and readable inspection.
- Measure application bytes, setup/allocation traffic, network overhead,
model-visible output and end-to-end latency separately before broader rollout.
No paid GPU is needed for protocol development. The stopped rental stays off.