API reference for golded-ftn

All dataclass constructors are keyword-only. Values are frozen and slotted. The tables below give every constructor field, its type and its default. Required means the constructor has no default. Core annotations describe the contract; they do not validate every value at runtime. Format writers check encoded lengths, numeric ranges, representable fields and date restrictions.

Strings are Unicode. Source bytes stay separate. Dates are datetime values; core does not infer timezones or convert timestamps. Unknown metadata is None. An empty string, 0, (), None and UNSET have different meanings.

MessageControlLines and OutgoingMessage convert their collection fields to tuples during construction. Other models require callers to pass their declared types; freezing a model does not recursively freeze incorrectly typed inputs.

FtnAddress

FtnAddress(*, zone: int, net: int, node: int,
           point: int | None = None, domain: str | None = None)
FtnAddress.from_string(value: str) -> FtnAddress
FtnAddress.try_from_string(value: str) -> FtnAddress | None
str(address) -> str

The parsers accept zone:net/node, optional .point and optional @domain, with surrounding whitespace stripped. Numeric parts contain ASCII digits only. Domains begin with an ASCII letter or digit and then allow letters, digits, periods, underscores and hyphens. Parsing accepts arbitrary non-negative integer sizes; it does not enforce a format's 16-bit address limits. Direct construction does not enforce this grammar.

from_string raises ValueError for invalid syntax. try_from_string returns None. str emits the full address and includes .0 when point is explicitly zero. It does not normalize domain case or remove an explicit point.

ControlLine

ControlLine(*, name: str, value: str, raw: str)

raw includes any trailing nulls on the loaded line. It is decoded text, not original source bytes. For newly supplied writer controls, use raw=""; concrete serializers use structured names and values and validate them.

MessageControlLines

MessageControlLines(*, kludges=(), msgid=None, reply=None, charset=None,
                    seen_by=(), path=(), tearline=None, origin=None,
                    origin_address=None)

kludges, seen_by and path are converted to tuples. Direct construction does not reconcile convenience fields with the contents of kludges.

parse_message

parse_message(text: str) -> MessageControlLines

Splits CRLF, CR and LF without changing the supplied string. Parsing ignores empty lines and quoted lines beginning with optional whitespace followed by > or one to six ASCII letters/digits followed by >. It ignores trailing nulls when interpreting a line but retains them in ControlLine.raw.

A kludge must start at column zero with SOH (\x01) and use NAME: value. Names begin with an ASCII letter and then allow ASCII letters, digits and hyphens. Names are uppercased; first MSGID, REPLY and charset values win. Valid unknown kludges and repetitions remain in kludges. There is no conflict validation here: a writer may reject conflicting declarations.

Plain SEEN-BY: and PATH: lines are case-sensitive and have their surrounding value whitespace stripped. The current parser keeps SOH \x01PATH: in kludges but does not add it to path. Space-separated SOH controls such as \x01INTL 2:1/2 2:1/3 are not parsed by this core helper; concrete readers may extend its syntax. Routing strings are never expanded or deduplicated.

Origin recognition requires a leading whitespace character followed by * Origin:. origin keeps the first recognized origin text; a later recognized origin line can supply origin_address if earlier lines had no valid address. Ordinary body lines are not returned separately by this function.

extract_msgid

extract_msgid(text: str) -> str | None

Returns parse_message(text).msgid. It uses exactly the same quoting and syntax rules, including first-value precedence and an empty string as a valid result. It does not generate an ID or validate uniqueness.

from golded_ftn import FtnAddress, extract_msgid, parse_message

address = FtnAddress.from_string(" 2:236/77.0@fidonet ")
assert str(address) == "2:236/77.0@fidonet"
assert FtnAddress.try_from_string("not an address") is None
text = "> \x01MSGID: quoted\r\x01msgid: actual\r\x01X-CUSTOM: value\0"
controls = parse_message(text)
assert extract_msgid(text) == "actual"
assert controls.kludges[-1].name == "X-CUSTOM"
assert controls.kludges[-1].raw.endswith("\0")
assert parse_message("SEEN-BY: 1/2 3\nPATH: 1/4").path == ("1/4",)

DecodeErrors

DecodeErrors = Literal["strict", "replace", "ignore"]

Type alias for to_utf8's error policy. strict raises on malformed input; replace inserts replacement characters; ignore discards undecodable bytes. Neither lossy mode retains the original bytes for you.

detect_charset

detect_charset(raw_body: bytes, fallback: str = "CP850") -> str

Finds the first case-insensitive SOH CHRS: or CHARSET: declaration anywhere in the byte string. The token ends at whitespace, null or another SOH. Detection does not apply the text parser's quoted-line rules. A recognized FTN alias is resolved; an absent or unknown declaration uses fallback. This is declaration lookup, not statistical encoding detection, and it does not reject conflicting later declarations.

The fallback is always resolved and validated through Python's codec registry, even when the message has a recognized declaration. Invalid fallbacks raise LookupError. Supported declaration aliases, matched case-insensitively:

The historical Cyrillic aliases and KOI8-U/CP1125 declaration support above were added in 1.2.2. CP850 remains the default, including IBMPC. RFC Content-Type does not participate in body-charset detection.

Fallbacks additionally accept names understood by Python's codec registry. The returned spelling is an alias target or the validated fallback spelling, not necessarily the codec registry's canonical name.

to_utf8

to_utf8(value: bytes, charset: str = "CP850", *,
        errors: DecodeErrors = "strict") -> str

Removes trailing null bytes, then decodes with charset into Python Unicode. Despite its name, the result is a str, not UTF-8 bytes. Interior nulls remain. The charset is passed directly to Python's codec registry: resolve FTN aliases with detect_charset first. Unknown codec names raise LookupError; malformed bytes raise UnicodeDecodeError in strict mode. An unsupported error policy raises ValueError.

parse_body

parse_body(text: str) -> str

Removes trailing null characters and normalizes CRLF and CR to LF. It preserves interior nulls, controls, routing and ordinary text. It does not decode, extract metadata or repair mojibake.

read_null_padded_field

read_null_padded_field(raw: bytes, offset: int, length: int) -> bytes

Takes the requested byte slice and returns its bytes before the first null. Negative offset or length raises ValueError. A slice beyond the buffer is short or empty; this helper does not enforce binary record bounds. Validate record size before interpreting fixed fields.

synthetic_id

synthetic_id(from_name: str, to_name: str, subject: str,
             date: str | None, body: str) -> str

Returns hash:sha256: plus 64 lowercase hexadecimal digits. The digest covers the exact UTF-8 encoding of JSON [from_name, to_name, subject, date, body], with ensure_ascii=False and separators (",", ":"). The complete body is included. Names, text and date are not normalized. None and "" dates differ. Consumers choose their date string format.

This is a content-derived ID, separate from source record identity and revision. It does not prove that two source records are the same. Concrete writers do not automatically generate MSGID, and a synthetic ID must not be supplied as an external MSGID.

from golded_ftn import (
    detect_charset,
    parse_body,
    read_null_padded_field,
    synthetic_id,
    to_utf8,
)

raw = b"\x01CHRS: IBMPC 2\rBruger m\x9bde\0\0"
charset = detect_charset(raw)
assert charset == "CP850"
assert parse_body(to_utf8(raw, charset)).endswith("Bruger møde")
assert read_null_padded_field(b"xxAlice\0padding", 2, 13) == b"Alice"
assert read_null_padded_field(b"x", 8, 2) == b""
assert parse_body("a\0b\r\n\0") == "a\0b\n"
assert to_utf8(b"\xff", "UTF-8", errors="replace") == "\ufffd"
identity = synthetic_id("Alice", "Bob", "Ping", None, "body")
assert identity.startswith("hash:sha256:") and len(identity) == 76
assert identity != synthetic_id("Alice", "Bob", "Ping", "", "body")

MojibakeRepairResult

MojibakeRepairResult(*, text: str, changed: bool, confidence: float)

The score is not a probability. Returned scores lie between 0 and 1; direct construction does not validate that range.

repair_mojibake

repair_mojibake(text: str, declared_charset: str | None = None,
               prefer_quoted_lines: bool = True) -> MojibakeRepairResult

Attempts RFC 2047 decoding and strict encoding round trips per line. It scores damage markers, plausible European letters and a small word list. Default visible encodings are CP850, CP437, CP865, ISO-8859-1 and Windows-1252; intended encodings are UTF-8, ISO-8859-1, Windows-1252 and CP850. A valid first token of declared_charset, including an FTN alias, adds a candidate. Unknown declarations fall back to the default candidates.

prefer_quoted_lines=True lowers the score threshold for matching quote lines; it does not mean only quoted lines are examined. Candidates that cannot encode and decode strictly are discarded. Malformed RFC 2047 words remain unchanged when decoding fails. Version 1.2.2 safeguards allow Latin-1 fallback only for MIME words declared ASCII/US-ASCII, while rejecting output that introduces C0 controls (except tab), DEL, C1 controls or replacement characters. Repair always returns a new result and never modifies source data.

Version 1.2.2 repair safeguards preserve graphic frames, block art, already-correct Danish/German words and symbol-only noise. Isolated ┼/▀ inside words can still represent damaged letters, and strict UTF-8 recovery may repair graphic-looking bytes such as m├©de; these are contextual heuristics. Supported damaged words can be repaired beside correct words. Recognizable uuencode data lines and PGP armour regions are excluded. Pass complete text to retain PGP context; separate per-line calls cannot retain that state.

Literal degree signs are protected after an ASCII digit or before uppercase C/F followed by a non-letter or end of line. Spaces and tabs are allowed between the digit/sign/unit; the context never crosses a line break. Thus m°de at 20 °C becomes møde at 20 °C, and SÃ¥dan at 10° becomes Sådan at 10°. Other degree-shaped damage, such as °l, remains repairable.

CRLF and CR are normalized to LF even when changed is false. confidence is the mean of line scores, including zero scores for unchanged lines, capped at 1. This is an opt-in heuristic tuned to European-language fixtures, not a general detector. Protected regions also undergo LF normalization, so output is not a byte-preserving copy or a signature-verification input. Readers and writers do not invoke it automatically.

Cyrillic charset support means that bytes labelled CP866, KOI8-R, CP1251, KOI8-U or CP1125 can be decoded with the selected codec. It does not establish that repair_mojibake can recover Russian or Ukrainian text already decoded with the wrong codec. The scorer is tuned to Western European text; Cyrillic repair has not been evaluated against a representative corpus. Such an evaluation must check damaged text with known correct originals, unchanged clean text, mixed scripts and graphic/payload preservation before claiming support.

from golded_ftn import repair_mojibake

result = repair_mojibake("Bruger m°de")
assert result.text == "Bruger møde" and result.changed
assert 0 < result.confidence <= 1
plain = repair_mojibake("a\r\nb")
assert plain.text == "a\nb" and not plain.changed
assert plain.confidence == 0

MessageProvenance

MessageProvenance(*, source_type: str, source_path=None,
                  source_id=None, source_offset=None)

Provenance records where a value was read. It is separate from external MSGID and does not itself provide a writer revision or permission to mutate a base.

MessageSource

MessageSource(*, source_type: str, path: str, code: str, name: str,
              sort_order: int = 0, meta_key: str | None = None)

Core does not discover paths or interpret meta_key.

ParsedArea

ParsedArea(*, code: str, name: str, source_type: str,
           sort_order: int = 0, meta_key: str | None = None)

Area metadata has no path field. Use MessageSource for a located source.

ParsedMessage

ParsedMessage(*, msgno: int, from_name: str, to_name: str, subject: str,
              body_text: str, attributes_raw: int, posted_at=None,
              external_id=None, from_address=None, to_address=None,
              reply_to_msgno=None, reply1st_msgno=None, reply_next_msgno=None,
              area_code=None, area_name=None, area_sort_order=None,
              area_meta_key=None, control_lines=None, provenance=None)

This decoded view does not contain all raw headers, unknown binary metadata or the complete Squish reply list. Writers update from raw records under their lock; rebuilding an existing record from ParsedMessage can lose metadata.

ReaderIssue

ReaderIssue(*, source_type: str, source_path: str,
            action: Literal["recovered", "skipped", "stopped"],
            code: str, detail: str, source_id=None, source_offset=None)

recovered describes an accepted deviation. skipped excludes a record. stopped means traversal is incomplete. Several issues may describe one record; an earlier recovery does not prevent a later skip. Core does not enforce issue codes or sanitize a directly supplied detail string.

ReaderOptions

ReaderOptions(*, fallback_charset: str = "CP850", archive_mode: bool = False,
              on_issue: Callable[[ReaderIssue], None] | None = None)

archive_mode=True without on_issue raises ValueError at construction. The constructor does not validate the charset. Readers validate it when used. Strict mode is the default. Supplying a callback alone does not enable recovery. Filesystem and callback exceptions propagate; archive mode does not suppress them. Each format defines its permitted recovery steps. Archive reads do not authorize editing a damaged base.

from golded_ftn import ReaderIssue, ReaderOptions

issues: list[ReaderIssue] = []
options = ReaderOptions(archive_mode=True, on_issue=issues.append)
assert options.fallback_charset == "CP850"
assert options.on_issue is not None
options.on_issue(
    ReaderIssue(
        source_type="msg",
        source_path="area/1.MSG",
        source_id="1",
        source_offset=190,
        action="skipped",
        code="example",
        detail="Record cannot be decoded.",
    )
)
assert issues[0].action == "skipped"

OutgoingMessage

OutgoingMessage(*, from_name: str, to_name: str, subject: str, body_text: str,
                external_id=None, from_address=None, to_address=None,
                posted_at=None, attributes_raw=None, control_lines=(),
                provenance=None, reply_to_msgno=None, reply1st_msgno=None,
                reply_next_msgno=None, reply_list=(), routing_seen_by=(),
                routing_path=())

Collection fields are converted to tuples. Core does not generate MSGID, route messages, detect duplicates or validate that replies exist. Concrete formats reject unrepresentable fields and contradictory metadata. Numeric reply links are independent of external MSGID and REPLY controls.

WriterOptions

WriterOptions(*, target_charset: str = "CP850", lock_timeout: float = 5.0,
              concurrent: bool = False)

Negative, infinite or NaN lock_timeout raises ValueError at construction. Zero requests an immediate lock attempt. The constructor does not validate the codec; concrete writers do so during serialization. Encoding is strict, with no silent truncation or replacement characters. Conflicting charset metadata is rejected. Pure attribute changes preserve existing text bytes.

concurrent=True is a request, not permission or a safety guarantee. Current MSG, JAM, Squish and Hudson writers reject it. GoldED must remain closed during offline editing. Live support requires format, platform and build-specific competing writer, reader and refresh tests. Windows starts with offline use.

Unset

Unset()

Marker type used in patch annotations. It has no fields and is not a dataclass. Use the exported singleton UNSET rather than creating another Unset(); concrete writers can use identity comparisons with that singleton.

UNSET

UNSET: Unset

The singleton default for every patch field. It means leave the stored field untouched. It is different from None, which explicitly requests clearing.

MessagePatch

MessagePatch(*, from_name=UNSET, to_name=UNSET, subject=UNSET,
             body_text=UNSET, external_id=UNSET, from_address=UNSET,
             to_address=UNSET, posted_at=UNSET, attributes_raw=UNSET,
             control_lines=UNSET, provenance=UNSET, reply_to_msgno=UNSET,
             reply1st_msgno=UNSET, reply_next_msgno=UNSET, reply_list=UNSET,
             routing_seen_by=UNSET, routing_path=UNSET)

Every field defaults to UNSET. Each accepts None as a clear request; the concrete writer rejects clearing fields it cannot represent. Required names and body generally need a string, including "" when allowed, rather than None. Core does not validate or convert patch collections.

Names have the same meaning as in OutgoingMessage. An omitted field preserves existing raw metadata, including reply structures the decoded model cannot fully display. Explicit control/routing replacements follow the concrete format's representation rules. Deleting a message does not rewrite other messages' reply links, and message numbers are not renumbered.

MessageIdentity

MessageIdentity(*, format: str, base: str, msgno: int,
                board: int | None = None)

Use identities returned by sessions. Core does not canonicalize base paths or validate board/number ranges. External MSGID is not part of this identity.

RevisionToken

RevisionToken(*, identity: MessageIdentity, location: tuple[int, ...], digest: str)

A session issues this token from a consistent read and checks it again under the operation lock. Global base counters are not message revisions. Changes to other messages do not invalidate a token. Treat the entire token as opaque; core does not validate a manually constructed digest or location. It is an optimistic concurrency check, not authentication or a permanent message ID.

WriteResult

WriteResult(*, identity: MessageIdentity, revision: RevisionToken)

Use the returned revision for the next update/delete. An earlier revision can be stale after content or attribute changes.

SessionMessage

SessionMessage(*, message: ParsedMessage, identity: MessageIdentity,
               revision: RevisionToken)

Core does not enforce that manually constructed triples refer to the same record. Obtain them through the concrete session.

from golded_ftn import UNSET, MessageIdentity, MessagePatch, OutgoingMessage

message = OutgoingMessage(
    from_name="Alice",
    to_name="Bob",
    subject="Ping",
    body_text="Hello",
    reply_to_msgno=7,
    routing_seen_by=("1/2 3",),
)
patch = MessagePatch(subject="Edited", external_id=None)
assert patch.subject == "Edited"
assert patch.external_id is None
assert patch.body_text is UNSET
record = MessageIdentity(format="hudson", base="area", board=2, msgno=7)
assert record.board == 2 and message.external_id is None

These are structural typing protocols. A matching concrete object can satisfy them without inheriting from the protocol. They contain method signatures, not concrete behavior. They are not marked runtime_checkable; do not use them with isinstance or instantiate them. Paths accept str | os.PathLike[str]. Concrete APIs may expose extra format-specific arguments.

MessageBaseReader

read(self, path: str | PathLike[str],
     options: ReaderOptions | None = None) -> Iterable[ParsedMessage]

Read an area through a concrete reader. None selects default reader options. The iterable may be lazy; errors can appear during iteration. A reader's plain ParsedMessage does not include a session revision for editing.

MessageSourceCatalog

sources(self, path: str | PathLike[str],
        options: ReaderOptions | None = None) -> Iterable[MessageSource]

Discover configured message sources through a concrete catalog. Ordering and path interpretation belong to that catalog. Core has no discovery implementation.

MessageSourceLocator

find(self, path: str | PathLike[str]) -> str | None

Find a source path through a concrete locator. None means no match. Search rules and the meaning of the input path belong to the concrete locator.

MessageWriter

write(self, path: str | PathLike[str], messages: Iterable[OutgoingMessage],
      options: WriterOptions | None = None) -> int

Batch-writing convenience contract returning the number written. Core does not provide a batch implementation or promise a transaction around the iterable. The MSG convenience writer commits each successful message separately; check other formats' APIs before assuming they implement this legacy method.

MessageBaseWriter

create(self, path: str | PathLike[str]) -> None
open(self, path: str | PathLike[str], options: WriterOptions | None = None)
    -> AbstractContextManager[MessageWriterSession]

create initializes an empty base and rejects existing base files. It does not open a session. open opens an existing base, with default options when omitted. Use the returned context manager to scope the session. Hudson's concrete open requires an explicit board; whole-base creation remains independent of board. Required initialization/lastread files are format-specific.

MessageWriterSession

read(self, msgno: int) -> SessionMessage
append(self, message: OutgoingMessage) -> WriteResult
update(self, identity: MessageIdentity, patch: MessagePatch,
       expected_revision: RevisionToken) -> WriteResult
delete(self, identity: MessageIdentity,
       expected_revision: RevisionToken) -> MessageIdentity

Each operation takes the format lock, reloads control data and holds the lock through validation, writing and flush. read supplies a consistent decoded view and revision. append allocates a format-specific identity; update preserves identity and returns the new revision. delete returns the deleted identity. A missing or changed update/delete target raises ConflictError.

One operation is the commit unit. Earlier successful operations survive a later failure. Ordinary write failures trigger rollback under the lock. Failed rollback makes the session unusable and reports RollbackError; reopening a session is not proof that the base is healthy. Concrete writers reject damaged bases and do not use archive recovery for editing.

Callers must not access base files directly while writer operations run. A Python writer lock alone does not establish safe coexistence with GoldED. Rollback has no guarantee for process termination or power loss, and there is no automatic detection promise for every partial write.

The following functions consume protocols. The caller supplies concrete objects; core has no file reader or writer to instantiate. MSG, JAM and Squish expose the shown writer shape. Hudson needs a board-bound adapter for this generic example, because its concrete open requires a board argument that the core protocol does not express.

from golded_ftn import (
    MessageBaseReader,
    MessageBaseWriter,
    MessagePatch,
    OutgoingMessage,
    ParsedMessage,
    WriteResult,
)


def collect_messages(reader: MessageBaseReader, path: str) -> tuple[ParsedMessage, ...]:
    return tuple(reader.read(path))


def append_edit_delete(writer: MessageBaseWriter, path: str) -> WriteResult:
    writer.create(path)
    with writer.open(path) as session:
        created = session.append(
            OutgoingMessage(
                from_name="Alice",
                to_name="Bob",
                subject="Ping",
                body_text="Hello",
            )
        )
        current = session.read(created.identity.msgno)
        edited = session.update(
            current.identity,
            MessagePatch(subject="Edited"),
            current.revision,
        )
        session.delete(edited.identity, edited.revision)
        return edited

ParserException

class ParserException(RuntimeError)

Concrete readers and writer base-validation paths use this error for records they cannot parse. Concrete APIs may chain the underlying error. Filesystem failures and archive callback failures propagate separately; this is not a catch-all for every read error.

WriterError

class WriterError(RuntimeError)

Common base class for the writer-specific errors below. Concrete sessions also use it when closed or poisoned. Input validation and encoding failures can raise ValueError, LookupError or UnicodeEncodeError; filesystem/I/O failures can raise OSError. Catching only WriterError does not catch every failed write.

ConflictError

class ConflictError(WriterError)

The target is missing, belongs to another base/identity or no longer matches the expected revision. Read the current target and decide how to reconcile the change. Do not retry the same stale patch blindly. Concrete session reads also use this error for a missing message.

LockTimeoutError

class LockTimeoutError(WriterError)

The operation could not obtain the base lock within WriterOptions.lock_timeout. The operation has not begun mutating the base.

RollbackError

class RollbackError(WriterError)

An operation failed and restoring original bytes also failed. The exception reports base, operation and rollback failure. The concrete session stops accepting operations. The base may need inspection; automatic repair is outside this API.

UnsupportedOperationError

class UnsupportedOperationError(WriterError)

A requested operation or concurrency mode is unavailable. Examples include current concurrent=True requests and explicit Opus editing through the MSG writer. Unrepresentable message fields may instead raise ValueError, as documented by the concrete format.

Core does not read or write MSG, JAM, Squish or Hudson files itself. It does not implement BBS behavior, routing policy, duplicate checking, packet handling, packing, repair of damaged bases or GoldED configuration. Internal modules such as golded_ftn._writer_io are implementation details and are not public exports.

Current writer sessions are for offline editing. GoldED interoperability and concurrent reader/refresh tests are deferred; source review or passing Python tests do not establish those guarantees. Check the concrete package's format notes and the shared writer guide before choosing a format or platform.