Compare commits

..
Author SHA1 Message Date
ee5dfaae37 Update Rust crate aes-gcm to 0.11
Some checks failed
renovate/stability-days Updates have met minimum release age requirement
CI / checks (pull_request) Failing after 2m38s
2026-08-19 14:00:54 +03:00
a6c4e56835
[Upd] Docs
Some checks failed
CI / checks (push) Failing after 2m33s
2026-08-19 12:37:22 +02:00
14 changed files with 126 additions and 63 deletions

View file

@ -47,18 +47,29 @@ Feature summary:
| Feature | Pulls in | Enables |
| --- | --- | --- |
| `crypto` | `mtp::crypto` | AEAD, signatures, KEM, KDF, hashing |
| `host` | `mtp::host`, codec registry | QUIC host and version negotiation |
| `client` | `mtp::client` | QUIC client connections |
| `webserver` | `mtp::webserver` | HTTPS server with HTTP/1.1, HTTP/2, HTTP/3, and WebTransport MTP sessions |
| `serde` | Crypto serialization support | Serde implementations for crypto key types |
| `crypto` | `mtp::crypto` | AEAD, signatures, KEM, KDF, hashing, and connection authentication support |
| `host` | `mtp::host` | Native QUIC host and version negotiation |
| `client` | `mtp::client` | Native QUIC client connections |
| `transport` | `mtp-transport` dependency | Low-level transport support; enabled automatically by `host` and `client` |
| `pipes` | Pipe support in transport, host, client, and web server | Raw and encrypted byte streams |
| `files` | `mtp::files` | `.mk` keyrings and `.mpkb` public bundles; also enables `crypto` |
| `raw` | Raw file APIs | Legacy plaintext keyring migration APIs |
| `web-server` | `mtp::webserver` | HTTPS server with HTTP/1.1, HTTP/2, HTTP/3, and WebTransport MTP sessions |
| `full-server` | Native host and web-server surface | `host`, `web-server`, `crypto`, and `pipes` together |
| `tls` | `mtp::crypto::tls` | Development self-signed certificate generation |
| `insecure-tls` | Lower-level transport | Development-only certificate verification bypass, gated by `MTP_INSECURE_TLS=1` |
The core crates are always available: `codec`, `transport`, `common`, and `type_map`. See the [native client](./docs/NATIVE-CLIENT.md) and [native host](./docs/NATIVE-HOST.md)
The core modules always available from the facade are `codec`, `common`, and
`type_map`. Native `client` and `host` modules re-export the transport policy
types; the low-level transport crate is not exposed as `mtp::transport`. See the [native client](./docs/NATIVE-CLIENT.md) and [native host](./docs/NATIVE-HOST.md)
guides for configuration and usage. See [Security](./docs/SECURITY.md) for security boundaries.
## Sub-crates
The `mtp` facade re-exports the following modules:
`mtp::codec`, `mtp::transport`, `mtp::common`, `mtp::type_map`, `mtp::crypto`, `mtp::host`, and `mtp::client`.
`mtp::codec`, `mtp::common`, `mtp::type_map`, `mtp::crypto`, `mtp::host`,
`mtp::client`, `mtp::files`, and `mtp::webserver` when their features are enabled.
### Codec

View file

@ -1,19 +1,23 @@
# MTP Connections
Native clients and hosts share the same connection shape after the opening handshake. The client creates the connection; the host receives it from `accept()`.
Native clients and server-side hosts expose parallel connection handles after the
opening handshake. The client creates its handle; the host receives one from
`accept()`.
| Member | Native client | Native host | Web host (`WebMTPConnection`) |
| --- | --- | --- | --- |
| `version` | Compiled client version accepted by the host | Version selected by the registry | Version selected by the registry |
| `sender` | Sends `CommunicationValue` frames | Sends `CommunicationValue` frames | Sends `CommunicationValue` frames |
| `receiver` | Receives application frames | Receives application frames | Receives application frames |
| `receiver` | Underlying receiver; use `receive()` for application frames | Underlying receiver; use `receive()` for application frames | Underlying receiver; use `receive()` for application frames |
| `description` | Optional label sent during setup | Optional label received from the client | Optional label received from the client |
| `client_id` | Confirmed or assigned ID with `crypto` | Authenticated or guest client ID with `crypto` | Authenticated or guest client ID with `crypto` |
| `auth_state` | Authentication result with `crypto` | Authentication result with `crypto` | Authentication result with `crypto` |
| `request_path` | / | / | WebTransport CONNECT path (e.g. `/mtp`) |
| `path` | — | Native hosts use `/` | WebTransport CONNECT path (e.g. `/mtp`) |
| `remote_addr` | Server `SocketAddr` when available | Peer `SocketAddr` | Peer `SocketAddr` |
`WebMTPConnection`, returned by `MTPWebServer::accept()`, exposes the same members as the native host connection plus `request_path`, which contains the HTTP/3 path used for the WebTransport extended CONNECT request.
`WebMTPConnection`, returned by `MTPWebServer::accept()`, exposes the same
server-side members as the native host connection. Its `path` contains the
HTTP/3 path used for the WebTransport extended CONNECT request.
Server-side MTP connections expose `remote_addr`, the peer address observed by
QUIC. HTTP route handlers receive the peer address as `HttpRequest::remote_addr`.

View file

@ -4,23 +4,28 @@ This file documents the connection and version negotiation logic.
## Registry
The `registry` module provides a multi-version `Registry` used by the host for version negotiation. Accessed through the `mtp` facade (requires the `host` feature):
The `registry` module provides a multi-version `Registry` used by the host for
version negotiation. Accessed through the `mtp` facade (requires the `host`
feature). In this repository, `Registry::builtin()` is generated from
[`example/type-maps.yaml`](../example/type-maps.yaml), which currently contains
protocol version 3.0 only. Downstream projects can register additional versions
in their own YAML configuration.
```rust
use mtp::codec::registry::Registry;
use mtp::codec::{Version, registry::Registry};
let registry = Registry::builtin(); // loads all TypeMaps from config
let registry = Registry::builtin(); // loads all TypeMaps from the build config
// Check if a version is supported
assert!(registry.supports(&Version(1, 0)));
assert!(registry.supports(&Version(3, 0)));
// Find highest mutual version for a client
let client_versions = &[Version(0, 0), Version(1, 0)];
let client_versions = &[Version(2, 0), Version(3, 0)];
let negotiated = registry.negotiate(client_versions);
assert_eq!(negotiated, Some(Version(1, 0)));
assert_eq!(negotiated, Some(Version(3, 0)));
// Look up a version's TypeMap
let tm = registry.get(&Version(2, 0)).unwrap();
let tm = registry.get(&Version(3, 0)).unwrap();
```
The `Registry::builtin()` constructor uses the `TypeMap::vX_Y()` methods generated from the config.
@ -54,9 +59,9 @@ let mut host = MTPHost::new(config).await?;
while let Some(conn) = host.accept().await? {
// conn.version is the negotiated version
// conn.codec is a VersionedCodec scoped to that version
// conn.sender / conn.receiver for raw CommunicationValue I/O
// conn.sender / conn.receive() for application CommunicationValue I/O
let msg = conn.receiver.receive().await?;
let msg = conn.receive().await?;
}
```
@ -94,29 +99,31 @@ The client's `PROTOCOL_VERSION` constant is set by `protocol_version` in `type-m
## Version Negotiation Flow
```
Client (v2.0) Host (v0.0, v1.0, v2.0)
Client (v3.0) Host (v3.0)
| |
| QUIC connect |
|----------------------->|
| |
| CommValue{ Ident. } |
| Version -> "2.0" |
| Version -> "3.0" |
| Id -> 8765 |
| (unsigned hello; auth |
| challenge follows) |
|----------------------->|
| | registry.negotiate(&[Version(2,0)])
| | -> Some(Version(2,0))
| | registry.negotiate(&[Version(3,0)])
| | -> Some(Version(3,0))
| |
| Response | selected v2.0 TypeMap
| Response | selected v3.0 TypeMap
|<-----------------------|
| Status, version |
| |
| subsequent messages |
| use v2.0 TypeMap |
| use v3.0 TypeMap |
```
If the client sends an unsupported version (e.g. v3.0 when the host only knows up to v2.0), `negotiate` returns `None` and the connection is closed.
If the client sends an unsupported version (for example, v2.0 to the current
repository builtin host), `negotiate` returns `None` and the connection is
closed.
## Protocol Ping and Pong

View file

@ -12,6 +12,8 @@ MTP reports codec failures separately from connection and transport failures.
| `ReservedCommunicationType` | An application attempted to use a reserved communication type ID. |
| `InvalidEncoding` | Bytes do not match the MTP value or frame format. |
| `TooManyEntries` | A serialized value or frame exceeds its representable size. |
| `MissingTypeMap` | A versioned codec was asked to encode a value without a retained negotiated type map. |
| `TypeMapMismatch` | A value was created with a different protocol type map from the codec or peer operation. |
| `CryptoFailed` | Signing, verification, encryption, or decryption failed while encoding or decoding. |
| `MissingField` | A required typed field is absent. |
@ -43,4 +45,4 @@ Native builds may expose additional variants wrapping QUIC and WebTransport erro
## Authentication Rejections
The host reports unsupported or missing protocol versions through `AcceptError`. Authentication failures return `AcceptError::AuthenticationFailed` after the host sends a rejected handshake response. The authentication flow and its signed fields are defined in [Security](SECURITY.md).
The host reports unsupported or missing protocol versions through `AcceptError`. Authentication failures return `AcceptError::AuthenticationFailed` after the host sends a rejected handshake response; a handshake that exceeds the configured limit returns `AcceptError::AuthenticationTimedOut`. The authentication flow and its signed fields are defined in [Security](SECURITY.md).

View file

@ -19,7 +19,7 @@ let request = CommunicationValue::new(CommunicationType::Ping).with_id(1);
conn.sender.send(&request).await?;
let response = conn.receive().await?;
println!("received {:?}", response.id());
conn.sender.close();
conn.sender.close().await;
```
## Configuration
@ -230,7 +230,7 @@ let response = conn
Requests are routed by id through the connection's receive dispatcher. Frames with other ids remain available through `conn.receive()`.
Two send modes (configured via `mtp::transport::Policy`):
Two send modes (configured via `mtp::client::Policy`):
- `PersistentStream` (default): reuses one QUIC unidirectional stream
- `SingleStreamPerMessage`: opens a new stream per message
@ -248,12 +248,16 @@ Inbound frames are queued internally. The `receive()` method returns the next av
### Close
```rust
conn.sender.close();
conn.sender.close().await;
// or
conn.receiver.close();
```
Sends a close frame and signals the peer. The `Sender::close()` spawns an async task that sends the frame, waits for `force_close_delay` (default 300ms), then force-closes the QUIC connection if the peer has not already done so.
`Sender::close().await` gracefully finishes the active send stream, sends the
MTP close frame, and waits for `force_close_delay` (default 300ms) before
force-closing the QUIC connection if necessary. `Sender::close_immediate()` is
the fire-and-forget variant. `Receiver::close()` closes the local receive
handle without performing the sender's graceful close sequence.
### Pipes
@ -309,7 +313,7 @@ For `public_signer`, call `verify` and `into_verified` before calling `decrypt`;
The `Policy` struct controls transport behaviour:
```rust
use mtp::transport::{Policy, SendMode};
use mtp::client::{Policy, SendMode};
let policy = Policy {
send_mode: SendMode::PersistentStream,

View file

@ -130,7 +130,7 @@ while let Some(connection) = server.accept().await? {
```
> `MTPWebServer::new` consumes a `HostConfig` (not an `MTPHost` instance). It creates its own QUIC endpoint and does not share a port with a running `MTPHost`.
`server.accept()` returns `Option<WebMTPConnection>` for each WebTransport session. Ordinary HTTP routes do not surface through `accept()` because the server dispatches them internally. `WebMTPConnection` retains the negotiated version, codec, request path, remote address, description, sender, and receiver used by native MTP connections.
`server.accept()` returns `Option<WebMTPConnection>` for each WebTransport session. Ordinary HTTP routes do not surface through `accept()` because the server dispatches them internally. `WebMTPConnection` retains the negotiated version, codec, `path`, remote address, description, sender, and receiver used by native MTP connections.
## Deployment
@ -138,7 +138,7 @@ For direct browser access, leave `serve_tcp_https(true)` enabled. The server adv
When a reverse proxy or another process owns TCP, use `WebServerConfig::new().serve_tcp_https(false)`. This retains the UDP HTTP/3/WebTransport endpoint and its shared router without claiming the TCP port.
With port `0` and TCP enabled, construction binds TCP first and binds UDP to the selected TCP port, so `local_addr()` reports the common address. With TCP disabled, Quinn selects the UDP port as before. `shutdown()` stops both accept loops, gracefully finishes active HTTP requests until `drain_timeout`, closes Quinn, and then aborts remaining work. `close()` and dropping the server stop both listeners immediately.
With port `0` and TCP enabled, construction binds TCP first and binds UDP to the selected TCP port, so `local_addr()` reports the common address. With TCP disabled, Quinn selects the UDP port as before. `shutdown().await` stops both accept loops, gracefully finishes active HTTP requests until `drain_timeout`, closes Quinn, and then aborts remaining work. `close().await` and dropping the server stop both listeners immediately.
### Authentication
@ -164,12 +164,18 @@ On success, the connection has `AuthState::Authenticated`, the assigned `client_
## Errors
`MTPWebServer::new` returns `CommunicationError` for certificate parsing, certificate loading, bind failures, and rejected authentication policy.
`MTPWebServer::new` returns `CommunicationError` for certificate parsing,
certificate loading, and bind failures. Authentication policy is evaluated when
WebTransport sessions are accepted, not rejected during construction.
`accept()` returns `AcceptError` for a missing or unsupported version, a receive failure, or a send failure during the WebTransport opening handshake. HTTP route failures are reported through `WebServerMetrics::error_occurred` when metrics are configured. See [Errors](ERRORS.md) for shared error variants.
`WebServerMetrics` has these callbacks:
```rust
use std::time::Duration;
fn connection_accepted(&self)
fn connection_closed(&self, duration: Duration, reason: &str)
fn request_started(&self, path: &str)
fn request_completed(&self, path: &str, status: u16, duration: Duration)
fn error_occurred(&self, error: &WebServerError)

View file

@ -83,18 +83,18 @@ network metadata, not an authenticated client identity.
## Version Negotiation
`accept()` uses the version-bearing opening frame and registry flow in [Connector](CONNECTOR.md). The host registry is built from the type maps in `type-maps.yaml` by `Registry::builtin()`.
`accept()` uses the version-bearing opening frame and registry flow in [Connector](CONNECTOR.md). The host registry is built from the type maps in [`example/type-maps.yaml`](../example/type-maps.yaml) by `Registry::builtin()` in this repository; downstream builds can provide their own `MTP_TYPE_MAPS` configuration.
### Registry
```rust
use mtp::codec::registry::Registry;
use mtp::codec::Version;
let registry = host.registry();
assert!(registry.supports(&Version(2, 0)));
assert!(registry.supports(&Version(3, 0)));
let negotiated = registry.negotiate(&[Version(1, 0), Version(2, 0)]);
// -> Some(Version(2, 0)) if both versions are registered
let negotiated = registry.negotiate(&[Version(2, 0), Version(3, 0)]);
// -> Some(Version(3, 0)) for this repository's builtin map
```
## Authentication Flow
@ -105,13 +105,15 @@ After a successful handshake, `MTPConnection` exposes `AuthState::Authenticated`
## Handling Messages
Use `conn.sender` and `conn.receiver` for bidirectional message exchange:
Use `conn.sender` and `conn.receive()` for bidirectional message exchange. The
connection dispatcher owns the underlying receiver, especially when `pipes` is
enabled:
```rust
while let Some(conn) = host.accept().await? {
tokio::spawn(async move {
loop {
match conn.receiver.receive().await {
match conn.receive().await {
Ok(msg) => {
let response = process_message(&msg, &conn);
conn.sender.send(&response).await.ok();

View file

@ -39,4 +39,4 @@ Back up host keyrings and client keyrings as protected secrets. Test restoring a
### Graceful Shutdown
Stop accepting new connections, reject new work at the application layer, and allow active requests and pipe writers to finish. For `MTPWebServer`, call `shutdown()`; its `drain_timeout` controls graceful TCP HTTP completion and the QUIC drain period before remaining connection tasks are terminated.
Stop accepting new connections, reject new work at the application layer, and allow active requests and pipe writers to finish. For `MTPWebServer`, call `shutdown().await`; its `drain_timeout` controls graceful TCP HTTP completion and the QUIC drain period before remaining connection tasks are terminated.

View file

@ -167,6 +167,9 @@ if let Some(writer) = handle.wait().await? {
```rust
// Host
use mtp_transport::{PipeSessionParameters, accept_pipe_session};
use sha2::{Digest, Sha256};
// The streaming digest below requires `sha2` as a direct application dependency.
while let Ok(request) = conn.receive_pipe().await {
if request.description() != "file-upload" {
@ -182,7 +185,7 @@ while let Ok(request) = conn.receive_pipe().await {
let mut reader = accept_pipe_session(
reader.into_inner(), &params, &own_keyring, &client_public_bundle,
).await?;
let mut hasher = sha2::Sha256::new();
let mut hasher = Sha256::new();
while let Some(chunk) = reader.read_record().await? {
hasher.update(&chunk);
process_chunk(&chunk).await?;

View file

@ -112,4 +112,9 @@ identity-specific response for deployments where IDs are public.
The client sends one compiled-in protocol version. The host compares it with the versions in its registry and returns the selected version in the opening response. Subsequent frames use that version's type map. An unsupported version closes the connection with `AcceptError::UnsupportedVersion`.
The self-delimiting `DataValue` codec and the three-bit communication header begin at protocol version `3.0`. A peer offering an older codec version is rejected during version negotiation; the new decoder does not attempt legacy flag, ID, or crypto-container fallbacks.
The current self-delimiting `DataValue` codec and three-bit communication header
are used by the repository's protocol 3.0 map. The checked-in builtin registry
contains only 3.0, so its native clients and hosts do not provide legacy map
fallbacks. Type-map versions are configuration-driven; a custom registry may
register another version number, but its map must use the current codec format
and is not a fallback for a different legacy wire format.

View file

@ -27,7 +27,7 @@ For rotation, publish the replacement certificate or key before changing the ser
### Development Certificates
The `tls` feature exposes `mtp_crypto::tls::generate_self_signed_cert`. It creates an ECDSA P-256 server certificate for the requested domain, `127.0.0.1`, and `::1`; the certificate is valid for 13 days. `HostConfig::self_signed` provides a transport-level self-signed setup without the crypto certificate helper.
The `tls` feature exposes `mtp_crypto::tls::generate_self_signed_cert`. It creates an ECDSA P-256 server certificate for the requested domain, `127.0.0.1`, and `::1`; the certificate is valid for 13 days. The lower-level `mtp_transport::HostConfig::self_signed` provides a transport-level self-signed setup without the crypto certificate helper.
Self-signed certificates are for development. Production deployments should use a certificate trusted by the client or an explicitly pinned certificate.
@ -212,6 +212,7 @@ The crate's feature groups are:
| `serde` | Serialization support for key types |
| `wasm` | `getrandom` support for WebAssembly |
| `tls` | Development certificate generation |
| `password-kdf` | Argon2id password derivation for protected keyring files |
The main types are `Keyring`, `PublicKeyBundle`, `EncryptionType`, `HybridKem`, `XChaCha20Poly1305` (with the legacy `ChaCha20Poly1305` alias), `Aes256Gcm`, `Ed25519Signer`, and `MlDsaSigner`. Hashing and KDF helpers include `sha256`, `sha256_double`, `hkdf_extract`, `hkdf_expand`, and `derive_encryption_key`. Handshake payload builders are in `mtp_crypto::auth`.

View file

@ -59,7 +59,7 @@ When `require_pq` is true, both Ed25519 and ML-DSA-65 keys and signatures must b
**Prevention:** Treat generated type maps as versioned build artifacts.
`CodecError::UnknownVersion` means the codec was created for a version absent from its registry. `UnknownCommunicationType` and `UnknownDataType` mean the selected `TypeMap` has no mapping for the value being encoded. Select the negotiated type map and do not send an unmapped variant.
`CodecError::UnknownVersion` means the codec was created for a version absent from its registry. `UnknownCommunicationType` and `UnknownDataType` mean the selected `TypeMap` has no mapping for the value being encoded. `MissingTypeMap` means a versioned value lost its retained negotiated map; `TypeMapMismatch` means it was combined with a value or codec for another version. Select the negotiated type map and do not send an unmapped variant.
`ReservedCommunicationType` means application code attempted to use a reserved wire ID. Use generated communication types instead of assigning protocol IDs manually. `MissingField` means a required typed field was not present.

View file

@ -1,8 +1,14 @@
# Type Map
This file documents the Type Map & Registry configuration used by the MTP protocol. It will assume you are working with the [example-type-maps.yaml](./../example-type-maps.yaml).
A type musn't be the version of MTP, it stays independant.
MTP version defines the codec. The Type-Map version defines the available Types.
This file documents the type-map and registry configuration used by MTP. The
repository workspace uses [`example/type-maps.yaml`](../example/type-maps.yaml)
through [`.cargo/config.toml`](../.cargo/config.toml); that map currently
selects protocol version 3.0. The root [`example-type-maps.yaml`](../example-type-maps.yaml)
is a separate illustrative multi-version configuration used by the manual WASM
build script. Downstream applications should provide their own map.
The protocol version selects the generated codec/type-map build, while the
type-map entries define the available application types and their IDs.
## Binary Frame Format
@ -131,7 +137,7 @@ After editing the config and rebuilding, `CommunicationType` and `DataType` enum
use mtp::type_map::{CommunicationType, DataType, TypeMap};
let tm = TypeMap::v3_0();
let id = tm.data_id_enum(DataType::SomeType).unwrap();
let id = tm.data_id_enum(DataType::ExampleText).unwrap();
```
For native builds with the `registry` feature, the enums are a **union across
@ -145,30 +151,31 @@ compiled by the Vite plugin.
Encoding/decoding uses a `TypeMap` to resolve type names to wire IDs:
```rust
use mtp::codec::{encode, decode, DataValue};
use mtp::codec::{CommunicationType, CommunicationValue, DataType, DataValue};
use mtp::type_map::TypeMap;
let tm = TypeMap::v2_0();
let value = DataValue::Str("hello".into());
let tm = TypeMap::v3_0();
let value = CommunicationValue::new_with_type_map(CommunicationType::Ping, &tm)
.add_typed(DataType::Description, &tm, DataValue::Str("hello".into()));
let bytes = encode(&value, &tm).unwrap();
let decoded = decode(&bytes, &tm).unwrap();
let bytes = value.to_bytes().unwrap();
let decoded = CommunicationValue::from_bytes_with(&bytes, &tm).unwrap();
```
```rust
let tm_v3 = TypeMap::v3_0();
assert!(tm_v3.data_id_enum(DataType::SomeType).is_some());
assert!(tm_v3.data_id_enum(DataType::ExampleText).is_some());
```
When communicating with a peer on another version, encode only variants that map in the negotiated version. If an incoming frame names a type absent from the selected map, reject it as a protocol or type-map compatibility error; do not reinterpret its wire ID using another version's map. The self-delimiting codec begins at protocol version `3.0`; older versions are not codec fallbacks.
When communicating with a peer on another version, encode only variants that map in the negotiated version. If an incoming frame names a type absent from the selected map, reject it as a protocol or type-map compatibility error; do not reinterpret its wire ID using another version's map. The current repository map uses the self-delimiting codec format for protocol version `3.0`; a custom registry may register other version numbers, but those maps are not legacy wire-format fallbacks.
### Forward/Backward Compatibility Between Versions
Because enums are a union of all types across versions, a variant might exist that has no wire mapping in the *negotiated* version:
```
v3.0 client sends DataType::SomeType → host encodes with v3.0 TypeMap → wire ID 32
v3.0 host receives an unsupported pre-v3.0 peer → version negotiation error
v3.0 client sends DataType::ExampleText → host encodes with v3.0 TypeMap → wire ID 43
v3.0 host receives a version absent from the registry → version negotiation error
```
Encoding a frame with an unmapped communication or data type returns `CodecError::UnknownCommunicationType` or `CodecError::UnknownDataType`. Select a mapped variant from the compiled-in version before sending it.

View file

@ -602,8 +602,19 @@ The SDK logger receives parsed events:
```typescript
type MTPLogEvent =
| { hint: "info" | "warning"; type: string; data: unknown }
| { hint: "error"; type: string | "error"; error: string };
| {
hint: "info" | "warning";
type: string;
data: unknown;
direction?: "send" | "recv";
}
| {
hint: "error";
type: string | "error";
error: string;
data?: unknown;
direction?: "send" | "recv";
};
```
Incoming non-error frames and sent frames are logged as `info`. Error frames and transport errors are logged as `error`.