Runtime Model

Big Picture

OptParse is organized around two related but distinct abstractions:

• AbstractValueParser{T, E} converts a single raw string token into a value of type T, with error type E
• AbstractParser{T, E, S, P, R} consumes command-line structure and eventually produces a value of type T

The high-level split in the source tree is:

• src/core/
  • parsing context
  • parse result types
  • structured errors and error rendering
• src/parsers/valueparsers/
  • string-to-value parsers such as str, integer, choice, flt, uuid, path
• src/parsers/primitives/
  • primitive parser families such as switch, toggle, option, arg, command
• src/parsers/constructors/
  • combinators that combine child parsers such as record, or, sequence, concat, combine
• src/parsers/modifiers/
  • wrappers that transform parser behavior, such as default, optional, repeated
• src/display/
  • pretty-printing of parser values

The central entrypoints live in src/entrypoints.jl:

• tryoptparse(parser, argv) / tryoptparse(::Type{T}, argv)
• optparse(parser, argv) / optparse(::Type{T}, argv)
• runparse(parser, argv; ...) / runparse(::Type{T}, argv; ...)
• partial(parser) + tryoptparse(partial(p), argv)
• lift(::Type{T}) — access the parser for a type generated by @parser

And normalize_argv(argv) lives in src/OptParse.jl.

AbstractParser{T,E,S,P,R} Type Parameters

The core parser abstraction is:

AbstractParser{T, E, S, P, R}

The parameters mean:

• T
  • the final value type returned by complete
  • equivalently, the success type of Result{T, E}
• E
  • the error type for this parser family
  • always a Union of per-parser error structs (all <: AbstractParseError)
  • composed upward: a constructor's E includes its own error plus all child errors
• S
  • the parser-family-specific state type threaded through Context{S}
  • this is the type consumed by complete(p, state::S)
• P
  • parser-family-specific extra type information
  • for leaf families this is often Nothing
  • for wrappers and constructors this is often the child parser type or tuple of child parser types
• R
  • the parser priority as a compile-time integer parameter

The invariants are:

• T is the semantic output type of the parser
• E is the union of all error types this parser family can produce (including child errors)
• S is the only state shape that parser family should interact with directly
• P should stay concrete so that parser-family-specific code can remain inferable
• R is stable for a given parser family instance and drives constructor scheduling

Helper functions expose the same information in code:

val_t(parser_or_type)
err_t(parser_or_type)
state_t(parser_or_type)
ptypes(parser_or_type)
priority(parser_or_type)

When adding a parser family, think of S, E, and R as part of the parser contract, not as incidental implementation details.

Parse Model

OptParse parsing is split into two phases.

parse

The parse phase incrementally walks the parser tree while consuming command-line tokens and updating parser-local state.

Each parser family implements a method shaped like:

parse(p::SomeParser{T, E, S}, ctx::Context{S})::InnerResult{S, E}

The result can be:

• InnerSuccess{S} with:
  • a Consumed view of the consumed tokens
  • a next Context{S}
  • counts_as_match::Bool
• InnerFailure{E} with:
  • an integer “consumed count” used for choosing better failures
  • an error of type E

complete

The complete phase collapses the final parser state into the returned value:

complete(p::SomeParser{T, E, S}, state::S)::Result{T, E}

Typical complete responsibilities:

• turn successful parser-local state into the final user-facing value
• enforce completion-time invariants, such as:
  • “required flag was never matched”
  • “repeated matched fewer than min times”
  • “one child parser failed to complete inside a constructor”
• add parser-specific error context when resurfacing child failures

Why the split exists

This split is the key to the package design:

• parse focuses on token flow and structural matching
• complete focuses on final validity and value extraction

That separation keeps combinators composable and makes parse, don't validate fit naturally into the implementation.

Parser State

Parser state is intentionally a parser-family implementation detail.

A parser family should only ever interact with its own state shape. For example:

• ArgGate works with GateState
• ArgOption works with OptionState{T, E}
• ArgCommand works with CommandState{PState}
• ModMultiple works with MultipleState{S}

This is why tight parse / complete signatures matter so much: they enforce the rule that a parser family only operates on its own state.

In practice, state shape should usually mirror the macro-state the parser can be in. Typical examples:

• a switch or required flag has two states:
  • not matched yet
  • matched successfully
• a command has states like:
  • command token not matched yet
  • command matched, inner parser not started yet
  • command matched, inner parser started and has child state
• a repetition has states like:
  • zero repetitions matched
  • one or more repetitions matched, each with its own child state snapshot

Completion reports a missing-flag error for an unmatched gate. Parsers with distinct named states use closed sum types from SumTypes.jl:

For options and arguments, E is the value-parser error type. Their Invalid variants store value errors until completion. The Selected variant contains an InnerOrState{U} wrapper for typed branch dispatch.

Use @cases to inspect these states. For example:

@cases st begin
    NotMatched => ...
    Matched(value) => ...
    Invalid(error) => ...
end

The NotMatched and Matched gate variants have no payload. Constructors use GateState'.NotMatched or OptionState{T, E}'.Matched(value) to select a variant.

Collection and constant states use aliases:

const ConstantState{X} = Val{X}
const MultipleState{S} = Vector{S}
const ObjectState{L, P} = NamedTuple{L, P}

Sequences store child states in a tuple. Transparent wrappers use their child's state type. Result and InnerResult remain result containers, separate from parser-local state.

Concrete Parser Trees And Why State Signatures Matter

The parser tree consists of concrete AbstractParser family nodes. Keep child parser fields parametric and concrete instead of storing abstract parser types.

Parser families must constrain their parse and complete signatures to their state invariants. The specialization mode controls how much of this type information the runtime uses.

For example:

function parse(
    p::ArgOption{T, E, S},
    ctx::Context{S},
) where {T, E, IE <: E, S <: OptionState{T, IE}}

is better than a looser:

function parse(p::ArgOption, ctx::Context)

because the tight signature:

• documents the actual invariant of the parser family
• avoids impossible parser-family instantiations surviving too long in inference
• gives JET less nonsense to analyze
• makes trimming behavior much more predictable

Child parser state invariants

Wrappers and constructors have an extra invariant: their child parser type must remain tied to the child state they pass around.

It is not enough for the constructor to build only valid values. The method signature must also express the relationship, otherwise inference can still consider impossible child parser / child state combinations.

For a transparent wrapper whose state is exactly the child state, write methods like:

function parse(
    p::ModHelp{T, E, S, P, _R},
    ctx::Context{S},
)::InnerResult{S, E} where {T, E, S, _R, P <: AbstractParser{<:Any, <:Any, S}}

The important part is:

P <: AbstractParser{<:Any, <:Any, S}

That tells inference that the wrapped child parser also operates on state S.

For a wrapper whose state contains a child state, bind the child parser to the inner state instead:

function parse(
    p::ModWithDefault{T, E, WithDefaultState{S}, P, _R},
    ctx::Context{WithDefaultState{S}},
)::InnerResult{WithDefaultState{S}, E} where {T, E, S, _R, P <: AbstractParser{<:Any, <:Any, S}}

Here the wrapper state is WithDefaultState{S}, but the wrapped parser operates on S. The signature must make that distinction explicit.

This applies to parse, complete, usage, focused_helpdoc, and any other method that delegates to the child parser while relying on a particular child state shape.

Context

Context{S} lives in src/core/context.jl.

Conceptually, Context is the current parser execution frame:

• it says which normalized argv buffer is being parsed
• where the parser currently is in that buffer
• what the parser-family-local state currently is
• whether global option parsing has already been terminated by --

It carries:

• buffer::Vector{String}
• pos::Int
• state::S
• usage::UsageNode
• optionsTerminated::Bool

The important point is that Context is parameterized by the parser state type. That means “update the state” is not merely a field assignment. It is often an inference checkpoint.

A parser should only ever interact with Context through the helper functions and centralized checkpoints in src/core/context.jl.

In particular, parser-family code should avoid rebuilding contexts ad hoc unless there is a very good reason. The helper API keeps context updates:

• explicit
• type-stable
• grep-friendly
• consistent across parser families

The main helpers are:

• with_state(ctx, s)
• restate(ctx, s)
• widen_state(::Type, ctx)
• widen_restate(::Type, ctx, s)
• with_options_terminated(ctx, flag)
• hasmore, hasnone, peek, remaining, remaining_len
• consume(ctx, n)

Flat context access vs optics

Current guidance:

• use direct helpers or centralized checkpoints for Context
• use optics where nested immutable state updates are actually needed

This is why:

• Context updates go through helpers like with_state and consume
• nested constructor state updates still use PropertyLens / IndexLens

As a rule of thumb:

• for Context, use the helper API
• for nested constructor state stored inside Context.state, optics are still appropriate

Consumed

Consumed lives in src/core/consumed.jl.

It is a cheap view of consumed tokens:

• it stores the shared input buffer
• it stores one or more ranges into that buffer
• it behaves like an AbstractVector{String}

This avoids eagerly materializing consumed token vectors while still making it easy to:

• inspect consumed tokens in tests
• merge consumptions from nested combinators
• preserve a precise token view when bundled short flags are expanded

Important helpers:

• consumed_empty(ctx)
• merge(::Vector{Consumed})
• as_vector(consumed)
• as_tuple(consumed)

InnerSuccess and InnerFailure

These also live in src/core/parseresult.jl.

InnerSuccess{S} carries:

• consumed::Consumed
• next::Context{S}
• counts_as_match::Bool

InnerFailure{E} carries:

• consumed::Int (number of tokens consumed before failure)
• error::E (the concrete error value)

Together they form:

const InnerResult{S, E} = Result{InnerSuccess{S}, InnerFailure{E}}

counts_as_match is subtle and important.

It exists because not every successful token consumption should count as a semantic parser match. The main current example is --:

• a primitive parser may consume --
• that should update optionsTerminated
• but it should not satisfy an or branch or one slot of a sequence

So:

• “consumed input” and “counts as a semantic match” are intentionally different concepts

Helpers:

• innerOk(ctx, n; nextctx=..., counts_as_match=true)
• innerOk(nextctx, consumed, counts_as_match=true)
• innerErr(ctx, perr; consumed=0)

Error Model

Structured errors live in src/core/errors.jl.

The core pieces are:

• AbstractParseError
  • abstract base type for all per-parser error structs
• ParseError{E}
  • @wrapped union container; E is always a Union of concrete error struct types
  • render_error(io, err::ParseError) dispatches via @unionsplit
• Per-parser error structs (all <: AbstractParseError)
  • MainError — top-level parsing errors (MAINNoProgress, MAINUnexpectedToken)
  • ConstrObjectError — record errors (OBJECTUnexpectedToken, OBJECTEndOfInput, OBJECT_MaxIter)
  • ConstrOrError — or parser errors
  • ModMultipleError — repeated errors (MULTIPLETooFew, MULTIPLETooMany)
  • ArgGateError, ArgOptionError, ArgArgumentError, ArgCommandError — primitive errors
  • StringValError, IntegerValError, FloatValError, etc. — value parser errors
• ParseException{P, E}
  • wraps parser, argv, and ParseError{E} for showerror
  • optparse prints the diagnostic to stderr and returns nothing in both modes

Every parser family or value parser family should define:

• its own error struct <: AbstractParseError with an @enum code field
• a render_error(io::IO, err::MyError) method

Error type composition happens automatically through the E type parameter: a constructor's E is Union{OwnError, child_errors...}.

The validate(::Type{E}) where {E <: AbstractParseError} helper verifies that render_error is implemented for a given error type.