General Upgrade, NEW: WebServers, Better Docs
Some checks failed
CI / checks (push) Failing after 4m20s
Some checks failed
CI / checks (push) Failing after 4m20s
This commit is contained in:
Alex Emmet
parent
5f11d476b6
commit
02be09ef26
122 changed files with 10309 additions and 5206 deletions
48
docs/ARCHITECTURE.md
Normal file
48
docs/ARCHITECTURE.md
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
# MTP Architecture
|
||||
|
||||
MTP separates wire encoding, QUIC transport, connection policy, protocol negotiation, and application-facing clients.
|
||||
|
||||
```text
|
||||
application
|
||||
┌────────────────┴────────────────┐
|
||||
│ │
|
||||
Native client Browser SDK
|
||||
mtp-client mtp + WASM
|
||||
│ │
|
||||
└──────────────┬──────────────────┘
|
||||
│ MTP frames
|
||||
┌─────────▼─────────┐
|
||||
│ codec + type-map │
|
||||
│ versions, values │
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
┌─────────▼─────────┐
|
||||
│ QUIC transport │
|
||||
│ framing, policy │
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
┌─────────────────┴─────────────────┐
|
||||
│ │
|
||||
MTPHost MTPWebServer
|
||||
native QUIC HTTP/3 + WebTransport
|
||||
│ │
|
||||
└──────────────┬────────────────────┘
|
||||
│
|
||||
optional mtp-crypto
|
||||
authentication and E2EE
|
||||
```
|
||||
|
||||
`mtp-codec` owns `CommunicationValue` and `DataValue` serialization. A version-specific `TypeMap` translates generated type names to wire IDs.
|
||||
`mtp-transport` writes each frame as a four-byte big-endian length followed by the frame bytes and applies message, timeout, queue, and stream limits.
|
||||
|
||||
The top row represents application entry points. Native Rust code calls the client or host crates directly. Browser code calls the TypeScript SDK, which uses generated WASM bindings for the same codec and WebTransport session.
|
||||
Both clients exchange the same MTP frames with a host.
|
||||
|
||||
The middle row is shared protocol machinery. The type map determines numeric IDs, the codec serializes values, and transport framing places each serialized frame on a QUIC stream. This is why a type-map change must be compiled into both peers before the new message can be exchanged.
|
||||
|
||||
The bottom row shows the two server entry points. `MTPHost` owns a native QUIC endpoint. `MTPWebServer` owns an HTTP/3 endpoint that also accepts WebTransport MTP sessions. They cannot bind the same IP and port. `mtp-crypto` is an optional cross-cutting layer used by authenticated native connections and browser E2EE;
|
||||
TLS remains the transport security layer in both paths.
|
||||
|
||||
`mtp-host` performs version negotiation and native authentication before returning an `MTPConnection`. `mtp-webserver` routes HTTP/3 requests and WebTransport sessions through its endpoint. It currently accepts only unauthenticated WebTransport MTP sessions.
|
||||
|
||||
The [native client](NATIVE-CLIENT.md), [WASM client](WASM-CLIENT.md), [native host](NATIVE-HOST.md), and [web server](NATIVE-HOST-WEB-SERVER.md) guides cover the public APIs for each boundary.
|
||||
22
docs/CONNECTIONS.md
Normal file
22
docs/CONNECTIONS.md
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
# MTP Connections
|
||||
|
||||
Native clients and hosts share the same connection shape after the opening handshake. The client creates the connection; the host receives it from `accept()`.
|
||||
|
||||
| Member | Native client | Native host |
|
||||
| --- | --- | --- |
|
||||
| `version` | Compiled client version accepted by the host | Version selected by the registry |
|
||||
| `sender` | Sends `CommunicationValue` frames | Sends `CommunicationValue` frames |
|
||||
| `receiver` | Receives application frames | Receives application frames |
|
||||
| `description` | Optional label sent during setup | Optional label received from the client |
|
||||
| `client_id` | Confirmed or assigned ID with `crypto` | Authenticated or guest client ID with `crypto` |
|
||||
| `auth_state` | Authentication result with `crypto` | Authentication result with `crypto` |
|
||||
|
||||
The host connection also exposes a version-scoped `codec` and, for an authenticated client, its `client_public_key`. The native client connection also exposes these methods:
|
||||
|
||||
| Method | Behavior |
|
||||
| --- | --- |
|
||||
| `request` | Sends a frame and waits for a response with the same frame ID, subject to `request_timeout`. |
|
||||
| `get_ping` | Returns the latest matched protocol Ping round-trip duration. |
|
||||
| Pipe methods | Create, accept, deny, read, write, and close native pipe streams when the `pipes` feature is enabled. |
|
||||
|
||||
Connection lifecycle and keepalive behavior are defined in [Protocol Reference](PROTOCOL-REFERENCE.md). Pipe dispatch rules are in [Pipes](PIPES.md). Closing or dropping the connection stops its background tasks and closes the underlying QUIC session.
|
||||
|
|
@ -60,34 +60,10 @@ while let Some(conn) = host.accept().await? {
|
|||
}
|
||||
```
|
||||
|
||||
The host's `accept()` method:
|
||||
1. Accepts a QUIC connection
|
||||
2. If authentication is required (crypto feature): performs login/register handshake
|
||||
3. Reads the first `CommunicationValue` (always encoded with reserved type IDs)
|
||||
4. Extracts the client's protocol version from `DataType::Version` (reserved data type ID 0)
|
||||
5. Calls `registry.negotiate(&[client_version])`
|
||||
6. Returns an `AcceptError` if the version is unsupported
|
||||
7. Returns `Ok(Some(MTPConnection))` with the negotiated version otherwise
|
||||
The host reads the reserved opening frame, extracts `DataType::Version`, calls `registry.negotiate`, and returns `AcceptError::UnsupportedVersion` when no registered version matches.
|
||||
|
||||
### Login/Register Handshake
|
||||
|
||||
When `authentication_policy` is `ForceAuthentication` or `AllowAuthentication`,
|
||||
the parties run a mutually-authenticated
|
||||
**challenge-response**. The client speaks first with an *unsigned* hello:
|
||||
|
||||
- **Login** (`CommunicationType::Identification`, reserved ID 0): version, client ID
|
||||
- **Register** (`CommunicationType::Register`, reserved ID 2): version, public keys
|
||||
|
||||
The host then issues a fresh random `server_challenge` in a signed `Challenge`
|
||||
(`CommunicationType::Challenge`, reserved ID 4, carrying `ServerNonce`). The client signs
|
||||
that challenge, binding its id (login) or public keys (register), and returns a
|
||||
`ChallengeResponse` (reserved ID 5). The host verifies the proof against the challenge it
|
||||
issued and sends a signed final response, which the client verifies.
|
||||
|
||||
Because the client's proof covers the host-issued `server_challenge` (a one-time
|
||||
value held only on the accepting task's stack), a captured proof cannot be
|
||||
replayed on another connection. All signed payloads are domain-separated; see
|
||||
`mtp::crypto::auth`.
|
||||
Authentication follows the version-bearing hello when the host enables it.
|
||||
The sequence is defined in [Protocol Reference](PROTOCOL-REFERENCE.md).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -111,7 +87,7 @@ let conn = MTPClient::auth_connect(pinned.with_client_id(8765), &keys, &host_pk)
|
|||
let conn = MTPClient::auth_register(config, &keys, &host_pk).await?;
|
||||
```
|
||||
|
||||
The client's `PROTOCOL_VERSION` constant is set by `protocol_version` in `type-maps.yaml` and baked in at compile time. The client never imports the `registry` crate; it only uses `mtp::type_map` for enum types and `mtp::codec` for encoding.
|
||||
The client's `PROTOCOL_VERSION` constant is set by `protocol_version` in `type-maps.yaml` and baked in at compile time. The client uses one version and does not import the registry.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -132,13 +108,22 @@ Client (v2.0) Host (v0.0, v1.0, v2.0)
|
|||
| | registry.negotiate(&[Version(2,0)])
|
||||
| | -> Some(Version(2,0))
|
||||
| |
|
||||
| Response |
|
||||
|<-----------------------| (uses v2.0 TypeMap for encoding)
|
||||
| Status, Nonces, |
|
||||
| Signature |
|
||||
| Response | selected v2.0 TypeMap
|
||||
|<-----------------------|
|
||||
| Status, version |
|
||||
| |
|
||||
| (subsequent messages |
|
||||
| use v2.0 TypeMap) |
|
||||
| subsequent messages |
|
||||
| use v2.0 TypeMap |
|
||||
```
|
||||
|
||||
If the client sends an unsupported version (e.g. v3.0 when the host only knows up to v2.0), `negotiate` returns `None` and the connection is closed.
|
||||
|
||||
## Protocol Ping and Pong
|
||||
|
||||
See [Protocol Reference](PROTOCOL-REFERENCE.md#protocol-keepalive).
|
||||
|
||||
## Protocol Version Changes
|
||||
|
||||
Add a protocol version by adding its type-map entry and `protocol_version` to the YAML configuration, then rebuild both peers. The type-map build script generates a version-specific `TypeMap` and keeps the enum as the union of all configured type names.
|
||||
|
||||
For a backward-compatible change, keep existing communication and data IDs stable and add new types with the new version. For a breaking change, add a new version and register both versions on the host while clients migrate. A client compiles one protocol version; it can connect only when that version is present in the host registry. Remove an old version only after its clients no longer connect, because the host closes connections whose version is unsupported.
|
||||
|
|
|
|||
46
docs/ERRORS.md
Normal file
46
docs/ERRORS.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# Error Reference
|
||||
|
||||
MTP reports codec failures separately from connection and transport failures.
|
||||
|
||||
## CodecError
|
||||
|
||||
| Variant | Meaning |
|
||||
| --- | --- |
|
||||
| `UnknownVersion` | A codec was asked to use an unsupported protocol version. |
|
||||
| `UnknownCommunicationType` | A communication type has no mapping in the selected type map. |
|
||||
| `UnknownDataType` | A data type has no mapping in the selected type map. |
|
||||
| `ReservedCommunicationType` | An application attempted to use a reserved communication type ID. |
|
||||
| `InvalidEncoding` | Bytes do not match the MTP value or frame format. |
|
||||
| `TooManyEntries` | A serialized value or frame exceeds its representable size. |
|
||||
| `CryptoFailed` | Signing, verification, encryption, or decryption failed while encoding or decoding. |
|
||||
| `MissingField` | A required typed field is absent. |
|
||||
|
||||
An application should select a `TypeMap` for the negotiated version and treat `UnknownCommunicationType` and `UnknownDataType` as a type-map compatibility failure. Do not send the unmapped variant again on that connection.
|
||||
|
||||
Retry guidance: retry `ConnectionLost`, `ConnectingError`, and transient stream errors after applying backoff. Correct the request before retrying `MessageTooLarge`, `InvalidEncoding`, `MissingField`, and type-map errors.
|
||||
Retry `AuthenticationFailed` only after changing credentials or host policy;
|
||||
repeating the same proof does not repair a validation failure.
|
||||
|
||||
## CommunicationError
|
||||
|
||||
| Variant | Typical cause |
|
||||
| --- | --- |
|
||||
| `UseAfterClosed` | A send or receive operation ran after the connection handle was closed. |
|
||||
| `ClosedLocally` | The local endpoint initiated shutdown. |
|
||||
| `ClosedByPeer` | The peer closed the connection. |
|
||||
| `ConnectionLost` | The connection ended without a normal close. |
|
||||
| `ParseCommunicationValue` or `ParseError` | An incoming frame or certificate could not be parsed. |
|
||||
| `Encode` | An outgoing value could not be serialized. |
|
||||
| `MessageTooLarge` | A frame exceeds `Policy::max_message_size` or the handshake limit. |
|
||||
| `StreamClosed` or `StreamError` | A QUIC stream ended or returned an I/O error. |
|
||||
| `ConnectingError` or `ConnectionError` | The endpoint could not establish or maintain QUIC. |
|
||||
| `AuthenticationFailed` | A login, registration, signature, nonce, or host response failed validation. |
|
||||
| `CertificateParseFailed` or `CertificateLoadFailed` | TLS certificate input is malformed or unavailable. |
|
||||
| `CryptoProviderInstallFailed` | The native TLS crypto provider could not be installed. |
|
||||
| `Other` | A component returned an error without a more specific variant. |
|
||||
|
||||
Native builds may expose additional variants wrapping QUIC and WebTransport errors. WASM builds expose the transport-independent subset.
|
||||
|
||||
## Authentication Rejections
|
||||
|
||||
The host reports unsupported or missing protocol versions through `AcceptError`. Authentication failures return `AcceptError::AuthenticationFailed` after the host sends a rejected handshake response. The authentication flow and its signed fields are defined in [Security](SECURITY.md).
|
||||
|
|
@ -2,22 +2,27 @@
|
|||
|
||||
The native client is a Rust library (`mtp-client`) for connecting to an MTP host over QUIC. It uses `wtransport` under the hood and provides both unauthenticated and authenticated (crypto handshake) connection modes.
|
||||
|
||||
## Cargo Dependency
|
||||
## Prerequisites
|
||||
|
||||
Add the `mtp` umbrella crate with the `client` feature (and optionally `crypto` for authentication):
|
||||
Add the `mtp` umbrella crate with `client`. Add `crypto` for authenticated connections, `pipes` for raw streams, and `tls` for development certificate generation. The `insecure-tls` feature applies only to the lower-level transport API. The feature table is in the [README](../README.md).
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
mtp = { path = "/path/to/mtp", features = ["client"] }
|
||||
## Quick Start
|
||||
|
||||
# Add crypto for auth_connect / auth_register:
|
||||
mtp = { path = "/path/to/mtp", features = ["client", "crypto"] }
|
||||
```rust
|
||||
use mtp::client::{ClientConfig, MTPClient};
|
||||
use mtp::codec::{CommunicationType, CommunicationValue};
|
||||
|
||||
# Add pipes for raw binary streams:
|
||||
mtp = { path = "/path/to/mtp", features = ["client", "pipes"] }
|
||||
let conn = MTPClient::connect(
|
||||
ClientConfig::new("https://host.example.com:4433").with_client_id(42),
|
||||
).await?;
|
||||
let request = CommunicationValue::new(CommunicationType::Ping).with_id(1);
|
||||
conn.sender.send(&request).await?;
|
||||
let response = conn.receive().await?;
|
||||
println!("received {}", response.get_id());
|
||||
conn.sender.close();
|
||||
```
|
||||
|
||||
## ClientConfig
|
||||
## Configuration
|
||||
|
||||
```rust
|
||||
use mtp::client::{ClientConfig, ClientTlsConfig};
|
||||
|
|
@ -42,66 +47,39 @@ let config = ClientConfig::new("https://host.example.com:4433")
|
|||
| `ping_jitter` | `Option<Duration>` | `None` | Random jitter added to each interval |
|
||||
| `max_missed_pings` | `usize` | `3` | Disconnect after this many unanswered Pings |
|
||||
| `ping_timestamp` | `bool` | `true` | Include a `Timestamp` data entry in Ping |
|
||||
| `request_timeout` | `Duration` | `30s` | Max time for `MTPConnection::request` |
|
||||
| `auth_timeout` (crypto) | `Duration` | `30s` | Max time for auth handshake |
|
||||
| `require_pq` (crypto) | `bool` | `true` | Require ML-DSA-65 during authentication |
|
||||
|
||||
### TLS Certificate Handling
|
||||
|
||||
When `tls` is `ClientTlsConfig::SystemRoots` (the default), the client loads the **system's
|
||||
native root certificate store** via `rustls_native_certs`. This works with
|
||||
publicly-trusted CAs out of the box on Linux (using `openssl-probe`), macOS
|
||||
(Keychain), and Windows (Root Store).
|
||||
`ClientTlsConfig::SystemRoots` is the default. Use `ClientTlsConfig::PinnedPem` or `ClientConfig::with_pinned_pem` for a supplied certificate chain. SPKI pinning and development or insecure transport configuration are available through lower-level transport APIs. See [Security](SECURITY.md) for trust models, certificate generation, rotation, and the insecure-mode gates.
|
||||
|
||||
For development or self-signed certificates, provide one or more PEM-encoded
|
||||
certificates:
|
||||
|
||||
```rust
|
||||
let pem = std::fs::read("my-server-cert.pem")?;
|
||||
let config = ClientConfig::new("https://host.example.com:4433").with_pinned_pem(pem);
|
||||
```
|
||||
|
||||
When pinned, **only** the given certificate(s) are trusted for the TLS
|
||||
handshake.
|
||||
|
||||
## Connection Methods
|
||||
## Connecting
|
||||
|
||||
All methods return a `Result<MTPConnection, CommunicationError>`.
|
||||
|
||||
### MTPConnection
|
||||
|
||||
Shared fields and lifecycle: [MTP Connections](CONNECTIONS.md).
|
||||
|
||||
Keepalive behavior is defined in [Protocol Reference](PROTOCOL-REFERENCE.md#protocol-keepalive).
|
||||
|
||||
### Requests
|
||||
|
||||
`MTPConnection::request` sends a `CommunicationValue` and waits for a response with the same frame ID. It uses `ClientConfig::request_timeout`; timeout and connection errors reject the request.
|
||||
|
||||
The request must have a non-zero ID. The response is removed from the pending request table and is not returned by a later `conn.receive()` call. A timeout removes the pending request and returns `CommunicationError`; a response with the wrong expected type also returns an error. Frames with other IDs remain available through `conn.receive()`.
|
||||
|
||||
```rust
|
||||
pub struct MTPConnection {
|
||||
pub version: Version,
|
||||
pub sender: Sender,
|
||||
pub receiver: Receiver,
|
||||
pub description: Option<String>,
|
||||
#[cfg(feature = "crypto")]
|
||||
pub auth_state: AuthState,
|
||||
#[cfg(feature = "crypto")]
|
||||
pub client_id: u64,
|
||||
}
|
||||
let response = conn
|
||||
.request(&request_value, Some(CommunicationType::Pong))
|
||||
.await?;
|
||||
```
|
||||
|
||||
- `version` -- the negotiated protocol version
|
||||
- `sender` / `receiver` -- for message I/O
|
||||
- `description` -- the label sent during handshake (set via `ClientConfig::with_description`)
|
||||
- `client_id` -- the confirmed/assigned client identifier (crypto only)
|
||||
### Protocol keepalive
|
||||
|
||||
When `ping_interval` is non-zero, MTP sends Ping frames in the background and
|
||||
consumes their Pong responses before application message handling. `get_ping()`
|
||||
returns the round-trip duration of the latest matched Pong, or `None` until a
|
||||
Pong arrives. A connection closes when the configured unanswered Ping limit is
|
||||
reached.
|
||||
|
||||
### Ping-Pong
|
||||
|
||||
Ping/Pong is part of the protocol, not just a transport keepalive. Each Ping
|
||||
frame is matched against a Pong with the same frame id, and the client uses the
|
||||
response to update `get_ping()`. If the host does not answer within the
|
||||
configured limit, the connection closes.
|
||||
|
||||
Enable it in `ClientConfig`, then inspect the latest round-trip time on the
|
||||
connection. Pings start after the connection has been established; `None` is
|
||||
normal until the first matching Pong arrives.
|
||||
Enable it with `ClientConfig` and inspect the latest matched round-trip time with `get_ping()`. See [Protocol Reference](PROTOCOL-REFERENCE.md).
|
||||
|
||||
```rust
|
||||
use mtp::client::{ClientConfig, MTPClient};
|
||||
|
|
@ -120,11 +98,7 @@ if let Some(round_trip) = conn.get_ping() {
|
|||
}
|
||||
```
|
||||
|
||||
The client consumes the Pong frames used by this loop, so they are not returned
|
||||
by `conn.receiver.receive()`. Set `ping_interval` to `Duration::ZERO` (the
|
||||
default) to disable protocol pings. `max_missed_pings` is the number of
|
||||
outstanding Ping frames allowed before the client closes the connection; use a
|
||||
host with automatic Pong responses, or provide an equivalent responder.
|
||||
Pong dispatch and missed-Ping behavior are defined in [Protocol Reference](PROTOCOL-REFERENCE.md#protocol-keepalive). Set `ping_interval` to `Duration::ZERO` (the default) to disable protocol pings.
|
||||
|
||||
### Unauthenticated Connect
|
||||
|
||||
|
|
@ -136,8 +110,7 @@ let config = ClientConfig::new("https://host.example.com:4433").with_client_id(4
|
|||
let conn = MTPClient::connect(config).await?;
|
||||
```
|
||||
|
||||
Sends an `Identification` frame with the compiled-in protocol version and
|
||||
client ID. No cryptographic handshake is performed.
|
||||
Sends an `Identification` frame with the compiled-in protocol version and client ID. No cryptographic handshake is performed.
|
||||
|
||||
### Authenticated Login
|
||||
|
||||
|
|
@ -154,21 +127,7 @@ let config = ClientConfig::new("https://host.example.com:4433")
|
|||
let conn = MTPClient::auth_connect(config, &keys, &host_pk).await?;
|
||||
```
|
||||
|
||||
Protocol (challenge-response, the host issues the freshness):
|
||||
1. Client sends an unsigned `Identification` hello (version, client ID)
|
||||
2. Host replies with a `Challenge` carrying a fresh random `server_challenge`
|
||||
and the host's signature over it; the client verifies that signature
|
||||
3. Client generates a random `client_nonce` and signs
|
||||
`version || client_id || server_challenge || client_nonce` with Ed25519
|
||||
(and optionally ML-DSA-65)
|
||||
4. Client sends a `ChallengeResponse` frame (nonce + signature(s))
|
||||
5. Host verifies the proof against `server_challenge` and responds with
|
||||
`IdentificationResponse` (echoed nonce + host signature)
|
||||
6. Client verifies the host signature and nonce echo
|
||||
|
||||
Because the client's signature covers the host-issued `server_challenge`, a
|
||||
captured proof cannot be replayed on another connection (each connection gets a
|
||||
different challenge).
|
||||
Authentication uses the signed challenge flow in [Protocol Reference](PROTOCOL-REFERENCE.md#authentication-flow). Cryptographic fields and domain separation are defined in [Security](SECURITY.md).
|
||||
|
||||
### Registration
|
||||
|
||||
|
|
@ -186,8 +145,7 @@ let id = conn.client_id;
|
|||
let keyring_bytes = keyring.to_bytes();
|
||||
```
|
||||
|
||||
When callers already know whether a saved client id exists, the convenience
|
||||
helper chooses login or registration:
|
||||
When callers already know whether a saved client ID exists, the convenience helper uses `Some(id)` for login and `None` for registration:
|
||||
|
||||
```rust
|
||||
let conn = MTPClient::auth_connect_or_register(
|
||||
|
|
@ -198,17 +156,7 @@ let conn = MTPClient::auth_connect_or_register(
|
|||
).await?;
|
||||
```
|
||||
|
||||
Protocol (challenge-response):
|
||||
1. Client sends an unsigned `Register` hello (version, public key bundle)
|
||||
2. Host replies with a `Challenge` carrying a fresh random `server_challenge`
|
||||
(signed by the host); the client verifies that signature
|
||||
3. Client generates a random `client_nonce` and signs
|
||||
`version || server_challenge || client_nonce || public_key_bytes` with
|
||||
Ed25519 (and optionally ML-DSA-65)
|
||||
4. Client sends a `ChallengeResponse` frame (nonce + signature(s))
|
||||
5. Host verifies the proof against `server_challenge`, assigns a new client ID,
|
||||
and responds with `RegisterResponse` (the ID, echoed nonce, host signature)
|
||||
6. Client verifies the host signature and nonce echo
|
||||
Registration uses the authentication flow in [Protocol Reference](PROTOCOL-REFERENCE.md#authentication-flow).
|
||||
|
||||
## Key Material
|
||||
|
||||
|
|
@ -233,8 +181,7 @@ pub struct Keyring {
|
|||
|
||||
### PublicKeyBundle
|
||||
|
||||
The public half of a keyring, used by the host for signature verification and
|
||||
by the client for host signature verification:
|
||||
The public half of a keyring, used by the host for signature verification and by the client for host signature verification:
|
||||
|
||||
```rust
|
||||
pub struct PublicKeyBundle {
|
||||
|
|
@ -244,10 +191,11 @@ pub struct PublicKeyBundle {
|
|||
}
|
||||
```
|
||||
|
||||
Obtain the host's `PublicKeyBundle` out of band (e.g. from files exported by
|
||||
the host, or from a trusted directory).
|
||||
Obtain the host's `PublicKeyBundle` out of band (e.g. from files exported by the host, or from a trusted directory).
|
||||
|
||||
## Sending and Receiving Messages
|
||||
## Communicate
|
||||
|
||||
### Sending and Receiving Messages
|
||||
|
||||
### CommunicationValue
|
||||
|
||||
|
|
@ -264,9 +212,7 @@ let msg = CommunicationValue::new(CommunicationType::Ping)
|
|||
.to_bytes();
|
||||
```
|
||||
|
||||
When the `registry` feature is enabled (via the `host` feature), you can also
|
||||
use `add_typed` with a `TypeMap` to resolve data type names from your project's
|
||||
type-map configuration.
|
||||
When the `registry` feature is enabled (via the `host` feature), you can also use `add_typed` with a `TypeMap` to resolve data type names from your project's type-map configuration.
|
||||
|
||||
### Send
|
||||
|
||||
|
|
@ -274,9 +220,7 @@ type-map configuration.
|
|||
conn.sender.send(&msg).await?;
|
||||
```
|
||||
|
||||
For request/response flows, `MTPConnection::request` sends one frame and waits
|
||||
for a response with the same non-zero frame id. An expected response type can be
|
||||
provided for validation:
|
||||
For request/response flows, `MTPConnection::request` sends one frame and waits for a response with the same non-zero frame id. An expected response type can be provided for validation:
|
||||
|
||||
```rust
|
||||
let response = conn
|
||||
|
|
@ -284,24 +228,22 @@ let response = conn
|
|||
.await?;
|
||||
```
|
||||
|
||||
Frames with other ids are consumed by this helper. Applications that need
|
||||
subscriptions or broad routing should use one receive task and correlate there.
|
||||
Requests are routed by id through the connection's receive dispatcher. Frames with other ids remain available through `conn.receive()`.
|
||||
|
||||
Two send modes (configured via `mtp::transport::Policy`):
|
||||
- `PersistentStream` (default) -- reuses one QUIC uni-directional stream
|
||||
- `SingleStreamPerMessage` -- opens a new stream per message
|
||||
- `PersistentStream` (default): reuses one QUIC unidirectional stream
|
||||
- `SingleStreamPerMessage`: opens a new stream per message
|
||||
|
||||
### Receive
|
||||
|
||||
```rust
|
||||
match conn.receiver.receive().await {
|
||||
match conn.receive().await {
|
||||
Ok(msg) => { /* handle CommunicationValue */ }
|
||||
Err(e) => { /* connection closed or error */ }
|
||||
}
|
||||
```
|
||||
|
||||
Inbound frames are queued internally. The `receive()` method returns the next
|
||||
available message.
|
||||
Inbound frames are queued internally. The `receive()` method returns the next available message. Do not read from `conn.receiver` directly because the connection dispatcher owns the shared transport receive loop.
|
||||
|
||||
### Close
|
||||
|
||||
|
|
@ -311,121 +253,15 @@ conn.sender.close();
|
|||
conn.receiver.close();
|
||||
```
|
||||
|
||||
Sends a close frame and signals the peer. The `Sender::close()` spawns an async
|
||||
task that sends the frame, waits for `force_close_delay` (default 300ms), then
|
||||
force-closes the QUIC connection if the peer has not already done so.
|
||||
Sends a close frame and signals the peer. The `Sender::close()` spawns an async task that sends the frame, waits for `force_close_delay` (default 300ms), then force-closes the QUIC connection if the peer has not already done so.
|
||||
|
||||
## Pipes
|
||||
### Pipes
|
||||
|
||||
With the `pipes` feature enabled, the client can open **raw binary streams**
|
||||
to the host. A Pipe is a unidirectional QUIC stream that carries a lightweight
|
||||
`PipeRequest` handshake frame, then transitions to raw bytes with zero per-frame
|
||||
overhead.
|
||||
The complete pipe protocol, native API, browser API, lifecycle, and errors are documented in [Pipes](PIPES.md). Use the connection facade described there when the `pipes` feature is enabled.
|
||||
|
||||
### Enabling Pipes
|
||||
## Appendix: Crypto Containers
|
||||
|
||||
Add the `pipes` feature to your dependency:
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
mtp = { path = "/path/to/mtp", features = ["client", "pipes"] }
|
||||
```
|
||||
|
||||
### Creating a Pipe
|
||||
|
||||
```rust
|
||||
use mtp::client::MTPClient;
|
||||
use tokio::io::AsyncWriteExt;
|
||||
|
||||
let conn = MTPClient::connect(config).await?;
|
||||
|
||||
// Initiate a pipe request
|
||||
let handle = conn.create_pipe("file-transfer").await?;
|
||||
|
||||
// Wait for the host to accept or reject
|
||||
match handle.wait().await? {
|
||||
Some(mut writer) => {
|
||||
writer.write_all(b"raw binary data").await?;
|
||||
writer.finish().await?; // graceful close
|
||||
}
|
||||
None => {
|
||||
println!("host rejected the pipe");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### PipeHandle
|
||||
|
||||
```rust
|
||||
pub struct PipeHandle {
|
||||
pipe_id: u32,
|
||||
description: String,
|
||||
}
|
||||
```
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `wait()` | `Result<Option<PipeWriter>, PipeError>` | Block until the host responds. `Some(writer)` if accepted, `None` if rejected. |
|
||||
|
||||
`PipeHandle` consumes itself on `wait()`, so you cannot poll it multiple times.
|
||||
|
||||
### PipeWriter
|
||||
|
||||
```rust
|
||||
pub struct PipeWriter {
|
||||
// wraps a QUIC SendStream
|
||||
}
|
||||
```
|
||||
|
||||
`PipeWriter` implements `tokio::io::AsyncWrite`. After the handshake succeeds,
|
||||
writes go directly to the QUIC stream with no framing overhead.
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `finish()` | `Result<(), CommunicationError>` | Gracefully close the stream (sends FIN) |
|
||||
| `abort()` | `Result<(), ClosedStream>` | Abruptly reset the stream |
|
||||
|
||||
```rust
|
||||
use tokio::io::AsyncWriteExt;
|
||||
|
||||
let mut writer = handle.wait().await?.unwrap();
|
||||
writer.write_all(b"chunk 1").await?;
|
||||
writer.write_all(b"chunk 2").await?;
|
||||
writer.finish().await?;
|
||||
```
|
||||
|
||||
### PipeError
|
||||
|
||||
```rust
|
||||
pub enum PipeError {
|
||||
Rejected, // pipe request was rejected
|
||||
HandshakeTimeout, // pipe handshake timed out
|
||||
StreamClosed, // pipe stream closed unexpectedly
|
||||
IoError(String), // pipe I/O error
|
||||
ConnectionClosed, // connection closed
|
||||
}
|
||||
```
|
||||
|
||||
`PipeError` implements `std::error::Error` and can be converted from
|
||||
`CommunicationError` via `PipeError::from()`.
|
||||
|
||||
### Do Not Use `receiver.receive()` for Pipes
|
||||
|
||||
When the `pipes` feature is active, `conn.receiver.receive()` will **skip**
|
||||
`PipeResponse` frames and may return them as ordinary messages if called from
|
||||
the wrong task. Use the facade methods:
|
||||
|
||||
- `conn.receive()` to receive normal `CommunicationValue` messages
|
||||
- `conn.create_pipe(description)` to initiate a new pipe
|
||||
|
||||
These methods are internally synchronised and safe to call from separate tasks.
|
||||
|
||||
## Crypto Containers
|
||||
|
||||
With the `crypto` feature, `DataValue` supports encrypted, signed, and
|
||||
signed+encrypted containers. Encryption uses ML-KEM to encapsulate to a
|
||||
recipient's KEM public key (from their `PublicKeyBundle`); only the holder of
|
||||
the matching `Keyring` can decrypt. Signing uses the sender's Ed25519 key.
|
||||
With the `crypto` feature, `DataValue` supports encrypted, signed, and signed+encrypted containers. Encryption uses ML-KEM to encapsulate to a recipient's KEM public key (from their `PublicKeyBundle`); only the holder of the matching `Keyring` can decrypt. Signing uses the sender's Ed25519 key.
|
||||
|
||||
```rust
|
||||
use mtp::crypto::{EncryptionType, Ed25519Signer, SigAlgorithm};
|
||||
|
|
@ -454,10 +290,10 @@ let mut sec = DataValue::Container(vec![
|
|||
]);
|
||||
sec.sign_and_encrypt_container(SigAlgorithm::ED25519, &signer, enc_type, &recipient, b"aad");
|
||||
```
|
||||
> Note: DataTypeId(1) maps intenally to the reserved DataType::Id, uncareful work with reserved DataTypes & CommunicationTypes (0 - 31) may lead to unexpected behaviour.
|
||||
> Prefer registring your own.
|
||||
|
||||
On the receiving side, the recipient decrypts with its own `Keyring` (each blob
|
||||
is self-describing: its leading byte selects the algorithm and the matching KEM
|
||||
key from the keyring):
|
||||
On the receiving side, the recipient decrypts with its own `Keyring` (each blob is self-describing: its leading byte selects the algorithm and the matching KEM key from the keyring):
|
||||
|
||||
```rust
|
||||
enc.decrypt_into_container(&keyring, b"aad"); // -> Container
|
||||
|
|
@ -465,7 +301,7 @@ sig.verify_into_container(&verifier); // verifier: impl S
|
|||
sec.decrypt_signed_encrypted_container(&keyring, b"aad"); // -> SignedContainer, then verify_into_container
|
||||
```
|
||||
|
||||
## Policy Configuration
|
||||
### Policy Configuration
|
||||
|
||||
The `Policy` struct controls transport behaviour:
|
||||
|
||||
|
|
@ -474,7 +310,8 @@ use mtp::transport::{Policy, SendMode};
|
|||
|
||||
let policy = Policy {
|
||||
send_mode: SendMode::PersistentStream,
|
||||
max_message_size: 1_000_000_000,
|
||||
max_message_size: 16 * 1024 * 1024,
|
||||
handshake_max_message_size: 64 * 1024,
|
||||
open_stream_timeout: Duration::from_millis(2000),
|
||||
write_timeout: Duration::from_millis(2000),
|
||||
read_timeout: Duration::from_millis(30_000),
|
||||
|
|
@ -484,41 +321,20 @@ let policy = Policy {
|
|||
};
|
||||
```
|
||||
|
||||
To apply a custom policy, call `mtp_transport::connect()` directly instead of
|
||||
using `MTPClient`:
|
||||
Apply a custom policy with `ClientConfig::with_policy`:
|
||||
|
||||
```rust
|
||||
use mtp_transport::{connect, Policy};
|
||||
|
||||
let server_cert = match &config.tls {
|
||||
ClientTlsConfig::SystemRoots => None,
|
||||
ClientTlsConfig::PinnedPem(pem) => Some(pem.clone()),
|
||||
};
|
||||
let (sender, receiver) = connect(&config.url, server_cert, policy).await?;
|
||||
let config = config.with_policy(policy);
|
||||
let conn = MTPClient::connect(config).await?;
|
||||
```
|
||||
|
||||
Then build and send the initial `Identification` frame manually to complete
|
||||
version negotiation.
|
||||
### Version
|
||||
|
||||
## Version
|
||||
The client's protocol version is baked in at compile time via the `PROTOCOL_VERSION` constant from `mtp::codec`. The version is set by the `protocol_version` field in your `type-maps.yaml`.
|
||||
|
||||
The client's protocol version is baked in at compile time via the
|
||||
`PROTOCOL_VERSION` constant from `mtp::codec`. The version is set by the
|
||||
`protocol_version` field in your `type-maps.yaml`.
|
||||
The client never imports the `registry` module; it uses a single compiled-in version and expects the host to negotiate a compatible version.
|
||||
|
||||
The client never imports the `registry` module; it uses a single compiled-in
|
||||
version and expects the host to negotiate a compatible version.
|
||||
### Error Handling
|
||||
|
||||
## Error Handling
|
||||
|
||||
`CommunicationError` covers transport errors:
|
||||
|
||||
| Variant | Meaning |
|
||||
|-------------------------|--------------------------------------------|
|
||||
| `StreamClosed` | Connection was closed by peer or timed out |
|
||||
| `StreamError` | Transport-level I/O error |
|
||||
| `MessageTooLarge` | Frame exceeds `max_message_size` |
|
||||
| `ParseCommunicationValue` | Failed to deserialize incoming frame |
|
||||
| `AuthenticationFailed` | Nonce mismatch or invalid host signature |
|
||||
| `ConnectionError` | QUIC connection failure |
|
||||
| `UseAfterClosed` | Attempted send/receive after close |
|
||||
`CommunicationError` is summarized in the [Error Reference](ERRORS.md).
|
||||
Native builds can expose additional variants that wrap QUIC and WebTransport errors.
|
||||
|
|
|
|||
99
docs/NATIVE-HOST-WEB-SERVER.md
Normal file
99
docs/NATIVE-HOST-WEB-SERVER.md
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
# MTP Web Server
|
||||
|
||||
`MTPWebServer` serves ordinary HTTP/3 routes and WebTransport MTP sessions through one QUIC endpoint. HTTP/3 requests are handled inside the server;
|
||||
WebTransport sessions are returned by `accept()` for application messages.
|
||||
`MTPWebServer` and `MTPHost` cannot bind the same IP and port.
|
||||
|
||||
## WebServerConfig
|
||||
|
||||
| Builder | Default | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `route(path, handler)` | None | Register an exact-path HTTP/3 handler. |
|
||||
| `route_method(method, path, handler)` | None | Register a method-specific handler. |
|
||||
| `fallback(handler)` | None | Handle requests that match no route. |
|
||||
| `mtp_path(path)` | `/` | Path for WebTransport extended CONNECT. |
|
||||
| `max_request_body(bytes)` | 4 MiB | Maximum buffered HTTP/3 request body. |
|
||||
| `max_connections(count)` | 256 | Maximum concurrent HTTP/3 connections. |
|
||||
| `request_timeout(duration)` | 30 seconds | HTTP/3 request handling timeout. |
|
||||
| `drain_timeout(duration)` | 10 seconds | Shutdown drain period. |
|
||||
| `with_metrics(metrics)` | None | Receive connection, request, and error callbacks. |
|
||||
|
||||
The route and fallback builders return `Result` because duplicate routes and duplicate fallback handlers are rejected.
|
||||
|
||||
## HTTP/3 Requests and Responses
|
||||
|
||||
`Http3Request` contains `method`, `uri`, `headers`, and an optional buffered `body` represented by `bytes::Bytes`. `Http3Response::status`, `header`, and `body` build a buffered response. `try_header` returns an error for invalid header names or values. `stream` takes a `tokio::sync::mpsc::Receiver<Bytes>` for incremental response chunks.
|
||||
|
||||
```rust
|
||||
use bytes::Bytes;
|
||||
use http::{Method, StatusCode};
|
||||
use tokio::sync::mpsc;
|
||||
use mtp::webserver::{Http3Request, Http3Response, WebServerConfig};
|
||||
|
||||
async fn health(_request: Http3Request, response: Http3Response) -> Http3Response {
|
||||
response.status(StatusCode::OK).body("ok")
|
||||
}
|
||||
|
||||
async fn stream_numbers(_request: Http3Request, response: Http3Response) -> Http3Response {
|
||||
let (tx, rx) = mpsc::channel::<Bytes>(10);
|
||||
tokio::spawn(async move {
|
||||
for number in 0..10 {
|
||||
if tx.send(Bytes::from(format!("{number}\n"))).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
});
|
||||
response
|
||||
.status(StatusCode::OK)
|
||||
.header("content-type", "text/plain")
|
||||
.stream(rx)
|
||||
}
|
||||
|
||||
let web = WebServerConfig::new()
|
||||
.route("/health", health)?
|
||||
.route_method(Method::GET, "/numbers", stream_numbers)?
|
||||
.fallback(|_request, response| async move {
|
||||
response.status(StatusCode::NOT_FOUND).body("not found")
|
||||
})?
|
||||
.mtp_path("/mtp");
|
||||
```
|
||||
|
||||
## Starting and Accepting MTP Sessions
|
||||
|
||||
```rust
|
||||
use mtp::{host::HostConfig, webserver::MTPWebServer};
|
||||
|
||||
let host = HostConfig::new(
|
||||
"0.0.0.0".parse()?,
|
||||
4433,
|
||||
std::fs::read("cert.pem")?,
|
||||
std::fs::read("key.pem")?,
|
||||
);
|
||||
let mut server = MTPWebServer::new(host, web).await?;
|
||||
|
||||
while let Some(connection) = server.accept().await? {
|
||||
// connection: WebMTPConnection
|
||||
while let Ok(message) = connection.receiver.receive().await {
|
||||
println!("received MTP message {}", message.get_id());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`server.accept()` returns `Option<WebMTPConnection>` for each WebTransport session. HTTP/3 routes do not surface through `accept()` because the server dispatches them internally. `WebMTPConnection` retains the negotiated version, codec, request path, description, sender, and receiver used by native MTP connections.
|
||||
|
||||
WebTransport sessions are unauthenticated. With the `crypto` feature enabled, construction rejects any `AuthenticationPolicy` other than `Unauthenticated`. The connection has `AuthState::Unauthenticated` and a random 48-bit client ID when crypto fields are compiled in; `guest_id_generator` is not used by this adapter.
|
||||
|
||||
## Errors
|
||||
|
||||
`MTPWebServer::new` returns `CommunicationError` for certificate parsing, certificate loading, bind failures, and rejected authentication policy.
|
||||
`accept()` returns `AcceptError` for a missing or unsupported version, a receive failure, or a send failure during the WebTransport opening handshake. HTTP route failures are reported through `WebServerMetrics::error_occurred` when metrics are configured. See [Errors](ERRORS.md) for shared error variants.
|
||||
|
||||
`WebServerMetrics` has these callbacks:
|
||||
|
||||
```rust
|
||||
fn request_started(&self, path: &str)
|
||||
fn request_completed(&self, path: &str, status: u16, duration: Duration)
|
||||
fn error_occurred(&self, error: &WebServerError)
|
||||
```
|
||||
|
||||
Errors include route misses, invalid requests, body-limit failures, handler timeouts, response construction failures, and transport failures. Supply the metrics object with `WebServerConfig::with_metrics`.
|
||||
|
|
@ -1,48 +1,21 @@
|
|||
# MTP Native Host
|
||||
|
||||
The native host is a Rust library (`mtp-host`) that runs a QUIC server, accepts
|
||||
MTP client connections, negotiates protocol versions, and optionally performs a
|
||||
mutual-authentication handshake (login/register) using Ed25519 and ML-DSA-65
|
||||
signatures.
|
||||
The native host is a Rust library (`mtp-host`) that runs a QUIC server, accepts MTP client connections, negotiates protocol versions, and optionally performs a mutual-authentication handshake (login/register) using Ed25519 and ML-DSA-65 signatures.
|
||||
|
||||
## Cargo Dependency
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
mtp = { path = "/path/to/mtp", features = ["host"] }
|
||||
|
||||
# Add crypto for authenticated connections:
|
||||
mtp = { path = "/path/to/mtp", features = ["host", "crypto"] }
|
||||
|
||||
# Add pipes for raw binary streams:
|
||||
mtp = { path = "/path/to/mtp", features = ["host", "pipes"] }
|
||||
```
|
||||
Add the `mtp` umbrella crate with `host`. Add `crypto` for authenticated connections and `pipes` for raw streams. The feature table is in the [README](../README.md).
|
||||
|
||||
## HostConfig
|
||||
|
||||
```rust
|
||||
use mtp::host::HostConfig;
|
||||
use std::net::{IpAddr, Ipv4Addr};
|
||||
`HostConfig::new` takes the bind address, port, PEM certificate chain, and PEM private key. Configure authentication and transport behavior with builders:
|
||||
|
||||
let config = HostConfig::new(
|
||||
IpAddr::V4(Ipv4Addr::UNSPECIFIED),
|
||||
4433,
|
||||
std::fs::read("cert.pem")?,
|
||||
std::fs::read("key.pem")?,
|
||||
)
|
||||
.with_authentication(
|
||||
/* Keyring */,
|
||||
|client_id: u64| {
|
||||
let db = CLIENT_DB.clone();
|
||||
Box::pin(async move { db.lock().unwrap().get(&client_id).cloned() })
|
||||
},
|
||||
|bundle: PublicKeyBundle| {
|
||||
let mut db = CLIENT_DB.lock().unwrap();
|
||||
let id = next_id();
|
||||
db.insert(id, bundle);
|
||||
Box::pin(async move { id })
|
||||
},
|
||||
);
|
||||
```rust
|
||||
let config = HostConfig::new(ip, port, certificate, private_key)
|
||||
.with_pongs(true)
|
||||
.with_policy(Policy::default())
|
||||
.with_authentication(host_keyring, get_existing_client, complete_register)
|
||||
.with_authentication_policy(AuthenticationPolicy::ForceAuthentication);
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|
|
@ -54,61 +27,37 @@ let config = HostConfig::new(
|
|||
| `send_pongs` | `bool` | Sends a Pong for each received Ping (default `true`) |
|
||||
| `authentication_policy` | `AuthenticationPolicy` (crypto) | `ForceAuthentication`, `AllowAuthentication`, or `Unauthenticated` |
|
||||
| `host_keyring` | `Keyring` (crypto) | Host's signing and KEM keys |
|
||||
| `get_existing_user` | `Fn(u64) -> Pin<Box<dyn Future<Output = Option<PublicKeyBundle>> + Send>> + Send + Sync` (crypto) | Async lookup callback for login |
|
||||
| `complete_register` | `Fn(PublicKeyBundle) -> Pin<Box<dyn Future<Output = u64> + Send>> + Send + Sync` (crypto) | Async registration callback, returns new client ID |
|
||||
| `get_existing_client` | Async callback returning `Option<PublicKeyBundle>` | Receives `(client_id, description)`. `Some` supplies the stored key bundle. `description = None` is used for guest-ID collision checks. |
|
||||
| `guest_id_generator` | Async callback returning `Option<u64>` | Custom guest ID assignment. The default generates random IDs. |
|
||||
| `complete_register` | Async callback returning `u64` | Stores the public bundle and returns its assigned client ID. |
|
||||
|
||||
### AuthenticationPolicy
|
||||
|
||||
`ForceAuthentication` requires every client to complete the login/register handshake. `AllowAuthentication` accepts both authenticated and unauthenticated connections — unauthenticated clients get a random ID and `AuthState::Unauthenticated`. `Unauthenticated` rejects any client that tries to authenticate and is the default.
|
||||
|
||||
```rust
|
||||
use mtp::host::AuthenticationPolicy;
|
||||
|
||||
// Force authentication (default was `require_authentication: true`):
|
||||
let config = HostConfig::new(ip, port, cert, key)
|
||||
.with_authentication(host_keyring, get_user, register);
|
||||
|
||||
// Allow both authenticated and unauthenticated:
|
||||
let config = HostConfig::new(ip, port, cert, key)
|
||||
.with_allow_authentication(host_keyring, get_user, register);
|
||||
|
||||
// Unauthenticated only (default):
|
||||
let config = HostConfig::new(ip, port, cert, key);
|
||||
```
|
||||
`ForceAuthentication` requires every client to complete the login or registration handshake. `AllowAuthentication` accepts both authenticated and unauthenticated connections; unauthenticated clients receive an ID and `AuthState::Unauthenticated`. `Unauthenticated` rejects authentication attempts and is the default.
|
||||
Authentication policy details are in [Security](SECURITY.md).
|
||||
|
||||
### TLS
|
||||
|
||||
The host requires a TLS certificate. For development, generate a self-signed
|
||||
certificate using `rcgen`. For production, use a CA-signed certificate.
|
||||
`HostConfig::new` always uses the certificate and key supplied by the caller.
|
||||
Certificate trust and development settings are in [Security](SECURITY.md).
|
||||
|
||||
### Ping-Pong
|
||||
|
||||
The host handles protocol Ping/Pong automatically unless you disable it with
|
||||
`with_pongs(false)`. Enable the default responder explicitly when constructing
|
||||
the host if you want to make the choice visible in application configuration:
|
||||
Keepalive behavior is defined in [Protocol Reference](PROTOCOL-REFERENCE.md#protocol-keepalive).
|
||||
|
||||
```rust
|
||||
let config = HostConfig::new(ip, port, cert, key)
|
||||
.with_pongs(true);
|
||||
```
|
||||
|
||||
For every received Ping, the responder sends a Pong with the same frame id and
|
||||
copies the optional `Timestamp` data entry. Ping and Pong frames handled this
|
||||
way are not delivered by `conn.receiver.receive()`. This lets native clients
|
||||
use `ClientConfig::with_ping_interval` and `MTPConnection::get_ping()` without
|
||||
adding application-level handlers.
|
||||
|
||||
Disable it only when the application needs to handle Ping frames itself:
|
||||
Disable automatic responses only when the application needs to handle Ping frames itself:
|
||||
|
||||
```rust
|
||||
let config = HostConfig::new(ip, port, cert, key)
|
||||
.with_pongs(false);
|
||||
```
|
||||
|
||||
With automatic responses disabled, Ping frames are delivered through the normal
|
||||
receiver and the application is responsible for sending a compatible Pong (the
|
||||
same frame id, and normally the Ping's `Timestamp`) if it wants clients to
|
||||
continue their protocol ping loop.
|
||||
Follow the responder contract in [Protocol Reference](PROTOCOL-REFERENCE.md).
|
||||
|
||||
## Accepting Connections
|
||||
|
||||
|
|
@ -125,50 +74,12 @@ while let Some(conn) = host.accept().await? {
|
|||
|
||||
### MTPConnection
|
||||
|
||||
Returned by `accept()` after version negotiation (and authentication if
|
||||
enabled):
|
||||
|
||||
```rust
|
||||
pub struct MTPConnection {
|
||||
pub version: Version,
|
||||
pub codec: VersionedCodec,
|
||||
pub sender: Sender,
|
||||
pub receiver: Receiver,
|
||||
pub description: Option<String>,
|
||||
#[cfg(feature = "crypto")]
|
||||
pub auth_state: AuthState,
|
||||
#[cfg(feature = "crypto")]
|
||||
pub client_id: u64,
|
||||
#[cfg(feature = "crypto")]
|
||||
pub client_public_key: Option<PublicKeyBundle>,
|
||||
}
|
||||
```
|
||||
|
||||
- `version` -- the negotiated protocol version
|
||||
- `codec` -- a `VersionedCodec` scoped to the negotiated version (use for
|
||||
version-aware encode/decode)
|
||||
- `sender` / `receiver` -- for message I/O
|
||||
- `description` -- optional client-provided label (e.g. `"phone"`, `"desktop"`)
|
||||
- `client_id` -- the authenticated client's ID
|
||||
- `client_public_key` -- the client's public key bundle (for signature
|
||||
verification of subsequent messages)
|
||||
`accept()` returns the shared connection shape in [MTP Connections](CONNECTIONS.md)
|
||||
after version negotiation and authentication, when enabled. The host-specific `codec` is scoped to the negotiated version, and `client_public_key` is set for authenticated clients.
|
||||
|
||||
## Version Negotiation
|
||||
|
||||
When a client connects, `accept()` performs the following sequence:
|
||||
|
||||
1. Accept the QUIC connection
|
||||
2. Read the client's first `CommunicationValue` (always encoded with reserved
|
||||
type IDs)
|
||||
3. Extract the protocol version from `DataType::Version` (reserved data type ID 0) as a
|
||||
`DataValue::Str("major.minor")`
|
||||
4. Call `registry.negotiate(&[client_version])` to find the highest mutually
|
||||
supported version
|
||||
5. Return an `AcceptError` (closing the connection) if no compatible version exists
|
||||
6. Return `Ok(Some(MTPConnection))` with the negotiated version
|
||||
|
||||
The `Registry` is built automatically from all type maps defined in your
|
||||
`type-maps.yaml` via `Registry::builtin()`.
|
||||
`accept()` uses the version-bearing opening frame and registry flow in [Connector](CONNECTOR.md). The host registry is built from the type maps in `type-maps.yaml` by `Registry::builtin()`.
|
||||
|
||||
### Registry
|
||||
|
||||
|
|
@ -184,97 +95,9 @@ let negotiated = registry.negotiate(&[Version(1, 0), Version(2, 0)]);
|
|||
|
||||
## Authentication Flow
|
||||
|
||||
When `authentication_policy` is `ForceAuthentication`, `accept()` runs a mutually-authenticated
|
||||
**challenge-response** handshake before returning the connection. The host issues
|
||||
a fresh, random `server_challenge` that the client must sign, which is what makes
|
||||
the client's proof unreplayable: a captured proof is bound to a one-time challenge
|
||||
the host generates per connection and will never reissue. The challenge lives only
|
||||
on the accepting task's stack; there is no replay database or shared state.
|
||||
The connection lifecycle and authentication sequence are in [Protocol Reference](PROTOCOL-REFERENCE.md). Host callback contracts are documented below.
|
||||
|
||||
All signed payloads begin with a one-byte domain-separation tag (see
|
||||
`mtp::crypto::auth`) so a signature for one step can never be reused as another.
|
||||
|
||||
### Login
|
||||
|
||||
```
|
||||
Client Host
|
||||
| |
|
||||
| QUIC connect |
|
||||
|---------------------------------------->|
|
||||
| |
|
||||
| Identification { Version, Id } | (unsigned hello)
|
||||
|---------------------------------------->|
|
||||
| | lookup get_existing_user(id)
|
||||
| | generate random server_challenge
|
||||
| Challenge { |
|
||||
| ServerNonce(server_challenge), |
|
||||
| Signature, [PqSignature] | host signs the challenge
|
||||
| } |
|
||||
|<----------------------------------------|
|
||||
| ChallengeResponse { |
|
||||
| ClientNonce, Signature, [PqSignature]| client signs the challenge
|
||||
| } |
|
||||
|---------------------------------------->|
|
||||
| | verify proof over server_challenge
|
||||
| IdentificationResponse { |
|
||||
| Connected=true, Id, |
|
||||
| ClientNonce(echoed), |
|
||||
| Signature, [PqSignature] |
|
||||
| } |
|
||||
|<----------------------------------------|
|
||||
```
|
||||
|
||||
Payloads (`||` is concatenation, integers big-endian; `DS_*` are domain tags):
|
||||
|
||||
- Host challenge: `DS_CHALLENGE || id (8) || server_challenge (16)`
|
||||
- Client proof: `DS_LOGIN_PROOF || version_string || id (8) || server_challenge (16) || client_nonce (16)`
|
||||
- Host final: `DS_HOST_FINAL || assigned_id (8) || client_nonce (16) || server_challenge (16)`
|
||||
|
||||
### Register
|
||||
|
||||
```
|
||||
Client Host
|
||||
| |
|
||||
| QUIC connect |
|
||||
|---------------------------------------->|
|
||||
| |
|
||||
| Register { |
|
||||
| Version, | (unsigned hello)
|
||||
| PublicKeys (serialized PublicKeyBundle)
|
||||
| } |
|
||||
|---------------------------------------->|
|
||||
| | generate random server_challenge
|
||||
| Challenge { |
|
||||
| ServerNonce(server_challenge), |
|
||||
| Signature, [PqSignature] | (challenge binds id = 0)
|
||||
| } |
|
||||
|<----------------------------------------|
|
||||
| ChallengeResponse { |
|
||||
| ClientNonce, Signature, [PqSignature]|
|
||||
| } |
|
||||
|---------------------------------------->|
|
||||
| | verify proof over server_challenge
|
||||
| | call complete_register(bundle) -> new_id
|
||||
| RegisterResponse { |
|
||||
| Connected=true, Id(new_id), |
|
||||
| ClientNonce(echoed), |
|
||||
| Signature, [PqSignature] |
|
||||
| } |
|
||||
|<----------------------------------------|
|
||||
```
|
||||
|
||||
The register client proof is:
|
||||
`DS_REGISTER_PROOF || version_string || server_challenge (16) || client_nonce (16) || public_key_bytes`
|
||||
|
||||
After a successful handshake, `accept()` returns an `MTPConnection` with
|
||||
`auth_state = Authenticated`, `client_id` set, and `client_public_key`
|
||||
available for verifying subsequent signed messages from the client.
|
||||
|
||||
### Rejection
|
||||
|
||||
If verification fails or the client is not found (login), the host sends a
|
||||
rejection response with `Connected=false` and closes the send stream, returning
|
||||
`AcceptError::AuthenticationFailed` from `accept()`.
|
||||
After a successful handshake, `MTPConnection` exposes `AuthState::Authenticated`, the client ID, and the client's public key bundle when one is available.
|
||||
|
||||
## Handling Messages
|
||||
|
||||
|
|
@ -298,8 +121,7 @@ while let Some(conn) = host.accept().await? {
|
|||
|
||||
### Versioned Codec
|
||||
|
||||
The `conn.codec` is a `VersionedCodec` pre-configured with the negotiated
|
||||
version. Use it to encode/decode with version-specific type maps:
|
||||
The `conn.codec` is a `VersionedCodec` pre-configured with the negotiated version. Use it to encode/decode with version-specific type maps:
|
||||
|
||||
```rust
|
||||
let tm = conn.codec.registry().get(&conn.version).unwrap();
|
||||
|
|
@ -311,166 +133,66 @@ let value = msg.get_data(desc_id);
|
|||
|
||||
## Pipes
|
||||
|
||||
With the `pipes` feature enabled, the host can accept **raw binary streams**
|
||||
from clients. A Pipe is a unidirectional QUIC stream opened by the client that
|
||||
carries a lightweight `PipeRequest` handshake frame, then transitions to raw
|
||||
bytes with zero per-frame overhead.
|
||||
|
||||
### Enabling Pipes
|
||||
|
||||
Add the `pipes` feature to your dependency:
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
mtp = { path = "/path/to/mtp", features = ["host", "pipes"] }
|
||||
```
|
||||
|
||||
### Receiving Pipe Requests
|
||||
|
||||
When `pipes` is enabled, **do not call `conn.receiver.receive()` directly**.
|
||||
Instead, use `conn.receive()` for normal messages and `conn.receive_pipe()`
|
||||
for incoming pipe requests. A background dispatcher task routes events
|
||||
internally so the two channels do not race.
|
||||
|
||||
```rust
|
||||
use mtp::host::{MTPHost, PipeRequest};
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
while let Some(conn) = host.accept().await? {
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
tokio::select! {
|
||||
Ok(msg) = conn.receive() => {
|
||||
// handle normal CommunicationValue
|
||||
}
|
||||
Ok(req) = conn.receive_pipe() => {
|
||||
handle_pipe(req).await;
|
||||
}
|
||||
else => break,
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async fn handle_pipe(req: PipeRequest) {
|
||||
println!("Pipe {} requested: {}", req.id(), req.description());
|
||||
// Accept or deny...
|
||||
}
|
||||
```
|
||||
|
||||
### PipeRequest
|
||||
|
||||
```rust
|
||||
pub struct PipeRequest {
|
||||
// pipe_id assigned by the creator
|
||||
// description provided by the creator
|
||||
}
|
||||
```
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `id()` | `u32` | The pipe ID chosen by the creator |
|
||||
| `description()` | `&str` | Creator-provided label (e.g. `"file-transfer"`) |
|
||||
| `accept()` | `Result<PipeReader, PipeError>` | Accept the pipe; returns an `AsyncRead` stream |
|
||||
| `deny()` | `Result<(), PipeError>` | Reject the pipe |
|
||||
|
||||
### Accepting a Pipe
|
||||
|
||||
```rust
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
async fn handle_pipe(req: PipeRequest) {
|
||||
match req.accept().await {
|
||||
Ok(mut reader) => {
|
||||
let mut buf = Vec::new();
|
||||
if let Err(e) = reader.read_to_end(&mut buf).await {
|
||||
eprintln!("pipe read error: {e}");
|
||||
}
|
||||
println!("received {} bytes", buf.len());
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("pipe accept failed: {e}");
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`PipeReader` implements `tokio::io::AsyncRead`. The stream reads until the
|
||||
creator calls `PipeWriter::finish()` or the connection closes.
|
||||
|
||||
### Rejecting a Pipe
|
||||
|
||||
```rust
|
||||
async fn handle_pipe(req: PipeRequest) {
|
||||
if !should_allow(&req) {
|
||||
req.deny().await.ok();
|
||||
return;
|
||||
}
|
||||
// ... accept
|
||||
}
|
||||
```
|
||||
|
||||
### PipeError
|
||||
|
||||
```rust
|
||||
pub enum PipeError {
|
||||
Rejected, // pipe request was rejected
|
||||
HandshakeTimeout, // pipe handshake timed out
|
||||
StreamClosed, // pipe stream closed unexpectedly
|
||||
IoError(String), // pipe I/O error
|
||||
ConnectionClosed, // connection closed
|
||||
}
|
||||
```
|
||||
|
||||
`PipeError` implements `std::error::Error` and can be converted from
|
||||
`CommunicationError` via `PipeError::from()`.
|
||||
|
||||
### Important: Do Not Use `receiver.receive()` with Pipes
|
||||
|
||||
When the `pipes` feature is active, `conn.receiver.receive()` will **skip**
|
||||
`PipeRequest` frames and may return them as ordinary messages if called from
|
||||
the wrong task. Always use the facade methods:
|
||||
|
||||
- `conn.receive()` -- normal `CommunicationValue` messages
|
||||
- `conn.receive_pipe()` -- incoming `PipeRequest` objects
|
||||
|
||||
These methods are internally synchronised and safe to call from separate tasks.
|
||||
The complete pipe protocol and host API are documented in [Pipes](PIPES.md).
|
||||
|
||||
## Host Callbacks
|
||||
|
||||
### get_existing_user
|
||||
### get_existing_client
|
||||
|
||||
Called during login to retrieve a client's public key bundle for signature
|
||||
verification. Must return `Some(PublicKeyBundle)` if the client ID is known,
|
||||
or `None` to reject.
|
||||
Called during login to retrieve a client's public key bundle for signature verification, and also during guest ID generation to check whether a random candidate collides with a registered client. When used for collision checking the `description` argument is `None`.
|
||||
|
||||
Must return `Some(PublicKeyBundle)` if the client ID is known, or `None` otherwise.
|
||||
|
||||
```rust
|
||||
let get_existing_user = |id: u64| {
|
||||
// db: Arc<tokio::sync::Mutex<HashMap<u64, PublicKeyBundle>>>
|
||||
let get_existing_client = |id: u64, _description: Option<String>| {
|
||||
let db = db.clone();
|
||||
Box::pin(async move { db.lock().unwrap().get(&id).cloned() })
|
||||
Box::pin(async move { db.lock().await.get(&id).cloned() })
|
||||
};
|
||||
```
|
||||
|
||||
### guest_id_generator
|
||||
|
||||
Optional callback that controls how unauthenticated connections receive their client ID. When `None` (the default), the host generates a random 48-bit ID and checks it against `get_existing_client` to avoid collisions.
|
||||
|
||||
Return `Some(id)` to accept the guest with that ID, or `None` to reject the connection. The ID must fit in 48 bits (`id <= mtp_codec::MAX_WIRE_ID`);
|
||||
values outside that range are rejected automatically and fall back to the built-in generator.
|
||||
|
||||
```rust
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
// Sequential guest IDs:
|
||||
let counter = AtomicU64::new(1);
|
||||
let guest_id_generator = Box::new(move || {
|
||||
Box::pin(async move { Some(counter.fetch_add(1, Ordering::SeqCst)) })
|
||||
});
|
||||
|
||||
// Reject all guests (no unauthenticated connections):
|
||||
let guest_id_generator = Box::new(|| Box::pin(async { None }));
|
||||
|
||||
let config = HostConfig::new(ip, port, cert, key)
|
||||
.with_authentication(host_keyring, get_existing_client, complete_register)
|
||||
.with_authentication_policy(AuthenticationPolicy::AllowAuthentication)
|
||||
.with_guest_id_generator(guest_id_generator);
|
||||
```
|
||||
|
||||
### complete_register
|
||||
|
||||
Called during registration to persist a new client's public key bundle and
|
||||
assign a client ID. The returned `u64` becomes the client's permanent
|
||||
identifier.
|
||||
Called during registration to persist a new client's public key bundle and assign a client ID. The returned `u64` becomes the client's permanent identifier.
|
||||
|
||||
```rust
|
||||
let complete_register = |bundle: PublicKeyBundle| {
|
||||
// db: Arc<tokio::sync::Mutex<HashMap<u64, PublicKeyBundle>>>
|
||||
let complete_register = |bundle: PublicKeyBundle, _description: Option<String>| {
|
||||
let db = db.clone();
|
||||
let id = next_id.fetch_add(1, Ordering::SeqCst);
|
||||
Box::pin(async move {
|
||||
db.lock().unwrap().insert(id, bundle);
|
||||
db.lock().await.insert(id, bundle);
|
||||
id
|
||||
})
|
||||
};
|
||||
```
|
||||
|
||||
Both callbacks are called from within `accept()` and must be `Send + Sync`. They
|
||||
are `async` (returning `Pin<Box<dyn Future<...>>`) and are `.await`ed by the
|
||||
host, so they can perform I/O or other async work as needed.
|
||||
All callbacks are called from within `accept()` and must be `Send + Sync`. They are `async` (returning `Pin<Box<dyn Future<...>>`) and are `.await`ed by the host, so they can perform I/O or other async work as needed. The `complete_register` callback returns no error value. A panic aborts the normal callback flow; validate storage and ID allocation before returning the ID.
|
||||
|
||||
## Host Key Generation
|
||||
|
||||
|
|
@ -502,21 +224,17 @@ std::fs::write("host_sig_pq_pk.bin", bundle.sig_pq_public_key.as_bytes())?;
|
|||
|
||||
## Policy
|
||||
|
||||
The transport `Policy` is set to defaults internally. To customise (timeouts,
|
||||
send mode, etc.), use `mtp_transport::host()` directly instead of `MTPHost`:
|
||||
Customize transport limits and timeouts through `HostConfig::with_policy`:
|
||||
|
||||
```rust
|
||||
use mtp_transport::{host, Policy};
|
||||
|
||||
let transport = host(ip, port, cert, key, custom_policy).await?;
|
||||
// Then build version negotiation on top:
|
||||
// - accept transport.next()
|
||||
// - read first frame
|
||||
// - registry.negotiate()
|
||||
// - return MTPConnection
|
||||
let config = HostConfig::new(ip, port, cert, key)
|
||||
.with_policy(custom_policy);
|
||||
let host = MTPHost::new(config).await?;
|
||||
```
|
||||
|
||||
## Graceful Shutdown
|
||||
|
||||
Drop the `MTPHost` to stop accepting new connections. Active connections
|
||||
continue until their `Sender`/`Receiver` are dropped or the peer disconnects.
|
||||
Drop the `MTPHost` to stop accepting new connections. Active connections continue until their `Sender`/`Receiver` are dropped or the peer disconnects.
|
||||
|
||||
Run one accept loop per `MTPHost` and spawn one task per accepted connection.
|
||||
Stop the accept loop before dropping the host, then close active senders and wait for application tasks to finish. Use [Operations](OPERATIONS.md) for the deployment sequence and monitoring signals.
|
||||
|
|
|
|||
42
docs/OPERATIONS.md
Normal file
42
docs/OPERATIONS.md
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
# Operations
|
||||
|
||||
## Monitoring
|
||||
|
||||
Expose counters and gauges around the host and transport callbacks:
|
||||
|
||||
| Metric | Interpretation |
|
||||
| --- | --- |
|
||||
| Accepted and rejected connections | Compare admission failures with traffic volume. A rise in rejected connections points to certificate, version, policy, or authentication problems. |
|
||||
| Active connections and active pipe streams | Capacity currently consumed by sessions and raw streams. |
|
||||
| Authentication failures and timeouts | Credential, policy, or reachability failures during the handshake. |
|
||||
| Unsupported protocol versions | Clients that require a version still absent from the registry. |
|
||||
| Message-too-large and decode errors | Peer or schema mismatch, malformed input, or an overly small policy limit. |
|
||||
| Request latency and request timeout count | Application handler time and transport deadline pressure. |
|
||||
| Ping round-trip time and missed pings | Peer reachability and path latency. |
|
||||
| Pipe accept, reject, EOF, and reset counts | Application admission and stream completion behavior. |
|
||||
|
||||
Implement `WebServerMetrics` for HTTP/3 request and error callbacks. Record the request path, status, duration, and `WebServerError` category without logging credentials, private keys, or message contents. Export host callback results through the application's metrics system for native deployments.
|
||||
|
||||
## Tuning
|
||||
|
||||
`Policy::default()` uses a 16 MiB message limit, a 64 KiB handshake limit, a 30 second read timeout, a 30 second idle timeout, a receiver queue capacity of 1000, and 128 concurrent stream tasks.
|
||||
|
||||
For low-latency request traffic, use `SendMode::SingleStreamPerMessage`, keep message sizes bounded, use shorter read and idle timeouts, and keep queue and concurrency limits near the amount of work the application can process.
|
||||
|
||||
For high-throughput bulk traffic, use persistent streams, raise `max_message_size` only when messages require it, and size `receiver_queue_capacity` and `max_concurrent_stream_tasks` for available memory and downstream processing capacity. Use pipes for large sequential byte streams instead of increasing message limits.
|
||||
|
||||
Every queued frame consumes memory until the application reads it. Test policy changes with realistic peer counts and payload sizes before deployment.
|
||||
|
||||
## Deployment
|
||||
|
||||
### Certificate Rotation
|
||||
|
||||
Publish the replacement certificate or pin before changing the server. Update clients to trust the replacement while the current certificate remains valid, switch the server, then remove the old trust value after clients migrate. Use system roots when certificate rotation is managed by the issuing authority.
|
||||
|
||||
### Key Backup
|
||||
|
||||
Back up host keyrings and client keyrings as protected secrets. Test restoring a backup before relying on it. Keep private key files owner-only on Unix, protect backup access, and store public key bundles separately from private material.
|
||||
|
||||
### Graceful Shutdown
|
||||
|
||||
Stop accepting new connections, reject new work at the application layer, and allow active requests and pipe writers to finish. Send a normal connection close, wait for the configured drain period, then force-close remaining QUIC sessions. For `MTPWebServer`, call `shutdown()` after the accept loop stops; Headits `drain_timeout` controls the drain period.
|
||||
81
docs/PIPES.md
Normal file
81
docs/PIPES.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
# MTP Pipes
|
||||
|
||||
Pipes are unidirectional QUIC streams for raw bytes. The creator sends a `PipeRequest` communication value, the peer accepts or rejects it, and the stream then carries bytes without an MTP frame around every write.
|
||||
|
||||
The creator owns the writer. The accepting peer owns the reader. A writer finishes with a stream FIN or aborts with a stream reset. A reader returns EOF after FIN and reports a connection or stream error when the peer closes unexpectedly.
|
||||
|
||||
## Opening a Pipe
|
||||
|
||||
The creator calls `create_pipe` or the corresponding SDK `createPipe` method with a description. MTP assigns a pipe ID and sends a `PipeRequest` frame. The creator receives a handle, not an active writer, because the peer must decide whether to accept the request.
|
||||
|
||||
The request description is application metadata. It does not grant access to the stream, authenticate the creator, or negotiate an application protocol.
|
||||
Use the authenticated MTP connection and the host's admission policy when a pipe carries sensitive data.
|
||||
|
||||
## Accepting or Rejecting a Pipe
|
||||
|
||||
The receiving side reads pipe requests through `receive_pipe`, the host dispatcher, or the browser pipe callback. It calls `accept` to obtain a reader or `deny` to reject the request. A rejected request completes the creator's handle with `Rejected` and no raw byte stream becomes available.
|
||||
|
||||
With native pipes enabled, do not read the underlying `receiver` directly.
|
||||
Normal messages and pipe requests share the transport and must pass through the connection facade so a dispatcher does not deliver one event to the wrong consumer.
|
||||
|
||||
## Closing a Pipe
|
||||
|
||||
The creator closes a successful pipe with `PipeWriter::finish` or the browser writer's `close`; this sends a QUIC FIN and lets the reader observe EOF. Use `abort` when the peer should discard the stream immediately; this resets the stream and the reader receives an error instead of a clean EOF. Dropping the connection closes all active pipes.
|
||||
|
||||
The accepting side closes its reader by consuming it or dropping it. A reader does not send an application-level acknowledgement for EOF. If the application needs completion metadata, send an ordinary MTP message before finishing the pipe.
|
||||
|
||||
## Pipe Errors
|
||||
|
||||
| Error | Meaning |
|
||||
| --- | --- |
|
||||
| `Rejected` | The peer denied the request. |
|
||||
| `HandshakeTimeout` | The peer did not complete the pipe handshake in time. |
|
||||
| `StreamClosed` | The pipe stream ended unexpectedly. |
|
||||
| `IoError` | The underlying byte stream returned an I/O error. |
|
||||
| `ConnectionClosed` | The MTP connection closed while the pipe was active. |
|
||||
|
||||
Native applications use the pipe APIs on `MTPConnection`; browser applications use the SDK methods in [WASM Client](WASM-CLIENT.md#pipes). With native pipes enabled, normal messages and pipe requests must be read through the connection facade so the dispatcher can route each event to the correct queue.
|
||||
|
||||
## Native File Upload and Processing
|
||||
|
||||
The creator streams a file in chunks. The accepting side processes each chunk without buffering the complete file:
|
||||
|
||||
```rust
|
||||
// Client
|
||||
use tokio::io::AsyncWriteExt;
|
||||
|
||||
let handle = conn.create_pipe("file-upload").await?;
|
||||
if let Some(mut writer) = handle.wait().await? {
|
||||
let mut file = tokio::fs::File::open("input.bin").await?;
|
||||
tokio::io::copy(&mut file, &mut writer).await?;
|
||||
writer.finish().await?;
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
// Host
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
while let Ok(request) = conn.receive_pipe().await {
|
||||
if request.description() != "file-upload" {
|
||||
request.deny().await?;
|
||||
continue;
|
||||
}
|
||||
|
||||
let mut reader = request.accept().await?;
|
||||
let mut hasher = sha2::Sha256::new();
|
||||
let mut buffer = [0u8; 64 * 1024];
|
||||
loop {
|
||||
let count = reader.read(&mut buffer).await?;
|
||||
if count == 0 {
|
||||
break;
|
||||
}
|
||||
hasher.update(&buffer[..count]);
|
||||
process_chunk(&buffer[..count]).await?;
|
||||
}
|
||||
let digest = hasher.finalize();
|
||||
println!("processed upload with digest {digest:x}");
|
||||
}
|
||||
```
|
||||
|
||||
Send completion metadata as an ordinary MTP message after the reader observes EOF. A stream FIN means the writer finished; it does not authenticate file contents or provide a digest.
|
||||
43
docs/PROTOCOL-REFERENCE.md
Normal file
43
docs/PROTOCOL-REFERENCE.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
# Protocol Reference
|
||||
|
||||
This document owns the connection lifecycle, protocol keepalive, and application authentication flow. API guides link here for configuration.
|
||||
|
||||
## Connection Lifecycle
|
||||
|
||||
```text
|
||||
bind -> accept QUIC -> negotiate version -> authenticate if enabled
|
||||
-> dispatch application frames -> close or drain
|
||||
```
|
||||
|
||||
The opening version frame is processed before application messages. The host selects a registered type map. Authentication then completes according to the host policy. A connection is returned to the application only after these stages complete.
|
||||
|
||||
## Protocol Keepalive
|
||||
|
||||
The client sends an MTP `Ping` communication value with a frame ID. The host returns a `Pong` with the same ID when automatic responses are enabled. The client records the matched round-trip duration and closes after its configured missed-Ping limit. These frames are handled by the keepalive dispatcher and do not reach ordinary message handlers.
|
||||
|
||||
If automatic responses are disabled, the application must read Ping frames and send compatible Pong frames. Keepalive configuration is documented in the [native client](NATIVE-CLIENT.md) and [native host](NATIVE-HOST.md) guides.
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
```text
|
||||
Client Host
|
||||
| |
|
||||
| Identification or Register, unsigned |
|
||||
|------------------------------------------>|
|
||||
| | generate challenge
|
||||
| Challenge plus host signature |
|
||||
|<------------------------------------------|
|
||||
| ChallengeResponse plus client signature |
|
||||
|------------------------------------------>|
|
||||
| | verify proof and assign identity
|
||||
| IdentificationResponse plus host signature|
|
||||
|<------------------------------------------|
|
||||
```
|
||||
|
||||
Login proof binds the protocol version, client ID, host challenge, and client nonce. Registration proof binds the protocol version, public key bundle, host challenge, and client nonce. The host challenge is generated per connection.
|
||||
|
||||
`ForceAuthentication` requires login or registration. `AllowAuthentication` accepts authenticated and unauthenticated clients. `Unauthenticated` rejects authentication attempts. The connection states are `Pending`, `Authenticated`, `Unauthenticated`, and `Failed`.
|
||||
|
||||
## Version Negotiation
|
||||
|
||||
The client sends one compiled-in protocol version. The host compares it with the versions in its registry and returns the selected version in the opening response. Subsequent frames use that version's type map. An unsupported version closes the connection with `AcceptError::UnsupportedVersion`.
|
||||
163
docs/SECURITY.md
Normal file
163
docs/SECURITY.md
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
# Security
|
||||
|
||||
This document describes the security controls implemented by MTP, the crypto APIs exposed by `mtp-crypto`, and the limits that operators and application developers must account for.
|
||||
|
||||
## Reporting Vulnerabilities
|
||||
|
||||
Report suspected vulnerabilities privately to the project maintainers. Include the affected crate, feature flags, protocol path, reproducible input, and the commit or release being tested. Do not include private keys or credentials in the report.
|
||||
|
||||
## Security Boundaries
|
||||
|
||||
MTP runs over QUIC and relies on TLS for transport confidentiality and peer authentication. The native transport uses `wtransport`; the browser client uses WebTransport. MTP authentication adds application-level signatures and does not replace TLS certificate verification.
|
||||
|
||||
MTP does not provide anonymity. Client identifiers and connection metadata are visible to the host. It also cannot protect data after a client or host endpoint has been compromised.
|
||||
|
||||
## TLS Certificate Verification
|
||||
|
||||
The native client uses the system root store by default. It also supports a pinned PEM certificate or an SPKI SHA-256 pin. Browser clients use the browser root store unless `serverCertificateHashes` is configured for WebTransport.
|
||||
|
||||
| Configuration | Trusts | Intended use |
|
||||
| --- | --- | --- |
|
||||
| System roots | Certificates trusted by the operating system or browser | Publicly trusted production certificates |
|
||||
| Pinned PEM | The supplied PEM certificate chain | Private CA deployments and controlled environments |
|
||||
| SPKI hash | The public key represented by the supplied certificate | A fixed server key, with planned rotation |
|
||||
| Insecure verification | Any certificate | Local development only |
|
||||
|
||||
For rotation, publish the replacement certificate or key before changing the server, update clients to trust both values where the client API permits it, then remove the old value after all clients have migrated. A pin is a key constraint, not a substitute for a certificate rotation plan.
|
||||
|
||||
### Development Certificates
|
||||
|
||||
The `tls` feature exposes `mtp_crypto::tls::generate_self_signed_cert`. It creates an ECDSA P-256 server certificate for the requested domain, `127.0.0.1`, and `::1`; the certificate is valid for 13 days. `HostConfig::self_signed` provides a transport-level self-signed setup without the crypto certificate helper.
|
||||
|
||||
Self-signed certificates are for development. Production deployments should use a certificate trusted by the client or an explicitly pinned certificate.
|
||||
|
||||
### Insecure Verification
|
||||
|
||||
Native insecure verification has two gates:
|
||||
|
||||
1. Compile with the `insecure-tls` feature.
|
||||
2. Set `MTP_INSECURE_TLS=1` at runtime.
|
||||
|
||||
Without the runtime variable, the connection fails rather than silently disabling verification. Do not use this mode on an untrusted network.
|
||||
|
||||
## Authentication Policies
|
||||
|
||||
Hosts choose one of three policies:
|
||||
|
||||
- `ForceAuthentication` requires login or registration.
|
||||
- `AllowAuthentication` accepts authenticated and unauthenticated clients.
|
||||
- `Unauthenticated` rejects authentication attempts and is the default.
|
||||
|
||||
An unauthenticated connection receives `AuthState::Unauthenticated`. Use `ForceAuthentication` when every client must have a registered identity.
|
||||
|
||||
The native host exposes four authentication states:
|
||||
|
||||
| State | Meaning |
|
||||
| --- | --- |
|
||||
| `Unauthenticated` | The connection completed without application authentication. |
|
||||
| `Pending` | The authentication handshake is in progress. |
|
||||
| `Authenticated` | The host verified the client proof and assigned or confirmed its identity. |
|
||||
| `Failed` | Authentication started but validation failed or the handshake timed out. |
|
||||
|
||||
Authorize requests only after `Authenticated`. A failed handshake is reported through `AcceptError::AuthenticationFailed` or `AcceptError::AuthenticationTimedOut` on the host.
|
||||
|
||||
### Hybrid Signatures
|
||||
|
||||
Authenticated handshakes support Ed25519 and ML-DSA-65 dual signatures. The host and clients default to `require_pq = true`, so both signatures are required. Calling `with_require_pq(false)` permits Ed25519-only authentication and should be treated as an explicit compatibility decision.
|
||||
|
||||
The `ml-dsa` dependency is enabled by default in `mtp-crypto`. The project has not recorded an independent audit for `ml-dsa`; see [Cryptographic review status](#cryptographic-review-status).
|
||||
|
||||
### Challenge-Response Flow
|
||||
|
||||
The complete sequence is in [Protocol Reference](PROTOCOL-REFERENCE.md#authentication-flow). This section defines the signed fields and domain-separation tags used by that sequence.
|
||||
|
||||
### Domain Separation
|
||||
|
||||
Every signed handshake payload begins with a distinct byte:
|
||||
|
||||
| Tag | Payload |
|
||||
| --- | --- |
|
||||
| `0x10` | Host challenge |
|
||||
| `0x11` | Client login proof |
|
||||
| `0x12` | Client registration proof |
|
||||
| `0x13` | Host final confirmation |
|
||||
|
||||
The tags prevent a valid signature for one handshake step from being accepted as a signature for another step.
|
||||
|
||||
## Cryptographic Primitives
|
||||
|
||||
`mtp-crypto` exposes the following building blocks:
|
||||
|
||||
| Area | Implementation | Availability |
|
||||
| --- | --- | --- |
|
||||
| AEAD | XChaCha20-Poly1305 | Default |
|
||||
| AEAD | AES-256-GCM | `full` feature |
|
||||
| Classical signatures | Ed25519 | Default |
|
||||
| Post-quantum signatures | ML-DSA-65 | Default |
|
||||
| KDF and hashing | HKDF-SHA-256, SHA-256 | Default |
|
||||
| Hybrid KEM | X25519 plus ML-KEM-768 | `pqc` feature |
|
||||
|
||||
AEAD output stores the nonce before the authenticated ciphertext. Encrypted containers select their algorithm with a leading marking byte, derive an AEAD key from the KEM shared secret with HKDF, and authenticate caller-supplied AAD. Multi-recipient encryption wraps one content-encryption key separately for each recipient.
|
||||
|
||||
[mtp-crypto API](../crypto/), [native client](NATIVE-CLIENT.md), and [native host](NATIVE-HOST.md).
|
||||
|
||||
The crate's feature groups are:
|
||||
|
||||
| Feature | Adds |
|
||||
| --- | --- |
|
||||
| Default | XChaCha20-Poly1305, Ed25519, ML-DSA-65, HKDF, and SHA-256 |
|
||||
| `full` | AES-256-GCM in addition to the default features |
|
||||
| `pqc` | Hybrid X25519 and ML-KEM-768 support |
|
||||
| `serde` | Serialization support for key types |
|
||||
| `wasm` | `getrandom` support for WebAssembly |
|
||||
| `tls` | Development certificate generation |
|
||||
|
||||
The main types are `Keyring`, `PublicKeyBundle`, `EncryptionType`, `HybridKem`, `ChaCha20Poly1305`, `Aes256Gcm`, `Ed25519Signer`, and `MlDsaSigner`. Hashing and KDF helpers include `sha256`, `sha256_double`, `hkdf_extract`, `hkdf_expand`, and `derive_encryption_key`. Handshake payload builders are in `mtp_crypto::auth`.
|
||||
|
||||
## Cryptographic Review Status
|
||||
|
||||
The project records the following status for its cryptographic dependencies:
|
||||
|
||||
| Crate | Audited? | Notes |
|
||||
| --- | --- | --- |
|
||||
| `ed25519-dalek` | Yes | Used by Signal and Diem |
|
||||
| `chacha20poly1305` | Yes | NCC Group audit, December 2019 |
|
||||
| `aes-gcm` | Yes | NCC Group audit, December 2019 |
|
||||
| `ml-dsa` | No | NIST vectors pass in project tests |
|
||||
| `mlkem-tls` | No | Uses an unaudited `mlkem-rs` backend |
|
||||
| `hkdf` | No | Standard construction |
|
||||
| `sha2` | No | Standard construction |
|
||||
| `zeroize` | No | Used for secret-key containers |
|
||||
|
||||
The audit entries describe the dependency projects. MTP's crypto tests cover round trips, wrong-key failures, wrong-AAD failures, and signature failures;
|
||||
they do not replace a review of protocol composition or deployment.
|
||||
|
||||
## Browser End-to-End Encryption
|
||||
|
||||
The browser SDK's optional E2EE session uses XChaCha20-Poly1305 with message keys derived from a one-way HKDF chain. Each send and receive operation advances its chain and authenticates the message header as AAD. Initial messages can carry a hybrid KEM ciphertext for session setup.
|
||||
|
||||
This is a single-chain ratchet. It has no Diffie-Hellman ratchet step and does not provide post-compromise security. Out-of-order messages can create skipped keys; the SDK accepts a receive gap of at most 100 messages and retains at most 100 skipped keys. Consumed or evicted keys are zeroed in the SDK state where the implementation owns the buffer.
|
||||
|
||||
The session root key comes from the authenticated handshake's KEM shared secret. The initiator and responder derive separate send and receive chains.
|
||||
Each message consumes one chain key, derives one message key with HKDF, and increments its counter. `sessionStorage` stores browser session state for the current origin. `encryptedDeviceSecretProvider` supplies encrypted device secret storage when sessions must survive page reloads. The provider must protect its wrapping secret outside the SDK; the SDK does not recover a lost device secret or skipped message keys.
|
||||
|
||||
## Key Storage
|
||||
|
||||
`Keyring` contains three public and three private key values. Its private key fields use `ZeroizeOnDrop`, and serialized keyring output is held in a zeroizing buffer while it is constructed. Public key bundles contain only the three public values.
|
||||
|
||||
Applications remain responsible for storage at rest. The `files` feature writes passphrase-protected keyrings to `.mk` files and public bundles to `.mpkb` files. On Unix, keyring files are created with owner-only `0600` permissions.
|
||||
Restrict those files to the owning account and protect backups. Browser applications should treat the configured credential storage as sensitive application data.
|
||||
|
||||
## Resource Limits and Operational Controls
|
||||
|
||||
`Policy::default()` sets a 16 MiB application message limit and a 64 KiB handshake message limit. It also sets a 30 second read timeout, a 30 second maximum idle timeout, a receiver queue capacity of 1000, and a maximum of 128 concurrent stream tasks. Tune these values for the deployment and peer trust level.
|
||||
|
||||
The host does not provide a general authentication-attempt rate limiter.
|
||||
Deploy authentication endpoints behind a rate-limiting proxy or add admission control through the host callbacks, including `GuestIdGenerator` where guest connections are permitted.
|
||||
|
||||
## Security Limitations
|
||||
|
||||
- The first version-negotiation frame is sent before authentication and is not signed.
|
||||
- `AllowAuthentication` intentionally permits unauthenticated clients; it is not an authenticated-only mode.
|
||||
- Browser-side Rust panics cannot be recovered by JavaScript. The WASM client contains panic paths from internal `expect` calls.
|
||||
- The browser E2EE ratchet does not provide post-compromise security.
|
||||
108
docs/TROUBLESHOOTING.md
Normal file
108
docs/TROUBLESHOOTING.md
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
# Troubleshooting
|
||||
|
||||
Use the failure stage to narrow the cause. MTP connections pass through TLS, the opening version frame, optional authentication, and application framing in that order.
|
||||
|
||||
Each entry identifies the symptom, diagnosis, fix, and prevention. Security labels identify workarounds that change certificate verification or keepalive.
|
||||
|
||||
## Diagnosis Flow
|
||||
|
||||
```text
|
||||
Connection fails?
|
||||
-> TLS or WebTransport error? Check certificate, origin, and endpoint.
|
||||
-> UnsupportedVersion? Check compiled client version and host registry.
|
||||
-> AuthenticationFailed? Check policy, key lookup, and key bundles.
|
||||
-> CodecError? Check generated type maps and negotiated version.
|
||||
-> MessageTooLarge? Compare peer payload with Policy limits.
|
||||
-> Ping or pipe failure? Check the protocol reference or Pipes guide.
|
||||
```
|
||||
|
||||
## TLS Connection Failures
|
||||
|
||||
**Security impact:** Safe when the certificate or pin is corrected. Insecure TLS is restricted to Development Mode.
|
||||
|
||||
Check the certificate before investigating MTP frames.
|
||||
|
||||
- With native clients, confirm the certificate chains to the system roots or pass the expected PEM certificate with `ClientConfig::with_pinned_pem`.
|
||||
- With browser clients, confirm WebTransport is supported and that `serverCertificateHashes` contains the expected certificate hash when using a pinned certificate.
|
||||
- For local self-signed certificates, pin the generated PEM certificate. The lower-level insecure mode requires both the `insecure-tls` feature and `MTP_INSECURE_TLS=1`.
|
||||
- Confirm the hostname or IP address is present in the certificate's subject alternative names. A valid certificate with the wrong name still fails TLS.
|
||||
|
||||
Use [Security](SECURITY.md) for certificate trust and rotation rules.
|
||||
|
||||
## Version Negotiation Failures
|
||||
|
||||
An `AcceptError::UnsupportedVersion` means the client's compiled `PROTOCOL_VERSION` is absent from the host registry. Check that both peers were built from compatible `type-maps.yaml` files and that the host includes the required generated version.
|
||||
|
||||
An `AcceptError::MissingVersion` means the opening frame did not contain a valid `DataType::Version` string in `major.minor` form. Do not send an application frame before the opening version frame completes.
|
||||
|
||||
Use [Connector](CONNECTOR.md) for registry and migration rules.
|
||||
|
||||
## Authentication Failures
|
||||
|
||||
Check the selected `AuthenticationPolicy` first.
|
||||
|
||||
- `Unauthenticated` rejects login and registration by configuration.
|
||||
- `AllowAuthentication` accepts both authenticated and guest connections.
|
||||
- `ForceAuthentication` requires a registered login or a successful registration callback.
|
||||
|
||||
For login, verify the client ID lookup returns the expected public key bundle, the client keyring matches that bundle, and the client has the host public key used to verify the host signature. For registration, verify that `complete_register` persists the submitted public bundle and returns a valid client ID.
|
||||
|
||||
When `require_pq` is true, both Ed25519 and ML-DSA-65 keys and signatures must be available. Set `with_require_pq(false)` only for an explicit Ed25519-only compatibility deployment. Authentication sequence: [Protocol Reference](PROTOCOL-REFERENCE.md#authentication-flow).
|
||||
|
||||
## CodecError Failures
|
||||
|
||||
**Symptom:** `UnknownCommunicationType` or `UnknownDataType`.
|
||||
|
||||
**Diagnosis:** The peers use different generated type maps or the selected version does not define the value.
|
||||
|
||||
**Fix:** Build both peers from the same type-map configuration and send only types defined by the negotiated version.
|
||||
|
||||
**Prevention:** Treat generated type maps as versioned build artifacts.
|
||||
|
||||
`CodecError::UnknownVersion` means the codec was created for a version absent from its registry. `UnknownCommunicationType` and `UnknownDataType` mean the selected `TypeMap` has no mapping for the value being encoded. Select the negotiated type map and do not send an unmapped variant.
|
||||
|
||||
`ReservedCommunicationType` means application code attempted to use a reserved wire ID. Use generated communication types instead of assigning protocol IDs manually. `MissingField` means a required typed field was not present.
|
||||
|
||||
`InvalidEncoding` indicates truncated, malformed, or structurally invalid bytes. `TooManyEntries` indicates that an array, container, or frame exceeds the codec's representable count or length. `CryptoFailed` indicates that signature verification or encrypted-container processing failed. The complete variant table is in [Errors](ERRORS.md).
|
||||
|
||||
## Frames and Message Limits
|
||||
|
||||
`MessageTooLarge` means the serialized frame exceeds the configured policy. Native transport defaults are a 16 MiB application message limit and a 64 KiB handshake limit. The browser SDK defaults `maxMessageSize` to 16 MiB.
|
||||
|
||||
`ParseCommunicationValue`, `ParseError`, or `CodecError::InvalidEncoding` means the received bytes do not match the length-prefixed frame and value format. Check that the sender writes the four-byte big-endian frame length exactly once and that the receiver reads exactly that many bytes. Frame layout: [Type Map](TYPE-MAP.md).
|
||||
|
||||
## Requests and Subscriptions
|
||||
|
||||
If `request()` times out, confirm that the peer sends a response with the same non-zero frame ID. If `responseType` is set, confirm that the response uses the expected communication type after the ID matches.
|
||||
|
||||
If a subscription callback does not run, confirm that the generated type map contains the message type and that the client is connected before the sender emits the frame. Enable the SDK logger to inspect state changes and errors.
|
||||
|
||||
## Protocol Pings
|
||||
|
||||
If `get_ping()` remains `None` or the connection closes after missed pings, check the keepalive configuration and responder mode in [Protocol Reference](PROTOCOL-REFERENCE.md#protocol-keepalive).
|
||||
|
||||
## Pipes
|
||||
|
||||
If a pipe handle resolves to `null` or `PipeError::Rejected`, the peer denied the request. If `receive_pipe()` never produces a request, use the connection facade instead of reading the underlying receiver directly. If a reader sees an error instead of EOF, the writer likely aborted the stream or the connection closed. Pipe lifecycle: [Pipes](PIPES.md).
|
||||
|
||||
## Browser Diagnostics
|
||||
|
||||
Use `MTPClient.isSupported()` before creating a browser client. Inspect browser console errors, WebTransport session state, certificate pins, and the SDK logger callback. Raw WASM bindings expose lower-level callbacks, but the SDK adds timeout and lifecycle handling.
|
||||
|
||||
| Browser or WebTransport signal | Meaning and next check |
|
||||
| --- | --- |
|
||||
| `WebTransportError.source = "stream"`, `streamErrorCode = 0` | The peer closed a one-frame stream normally in the supported browser behavior. Check session state before treating it as a failure. |
|
||||
| `WebTransportError.source = "stream"`, non-zero `streamErrorCode` | A stream-level failure. Check whether the peer sent `STOP_SENDING` or reset the stream, then inspect the active pipe or frame. |
|
||||
| `WebTransportError.source = "session"` | The WebTransport session failed. Check TLS, the endpoint, the `webtransport` CONNECT path, and server logs. |
|
||||
| `WebTransportError` without `streamErrorCode` | The error is session-level or browser-specific. Inspect `error.message`, `error.source`, and the browser network panel. |
|
||||
| Close code `0`, reason `mtp-webserver shutdown` | `MTPWebServer` performed an intentional shutdown. Reconnect after the process restarts. |
|
||||
|
||||
MTP logs stream-level `STOP_SENDING` and `RESET_STREAM` events with their `streamErrorCode`. QUIC transport error numbers are implementation-specific; use the browser's source, stream code, message, and server logs together.
|
||||
|
||||
## Development Mode
|
||||
|
||||
Use `mtp_crypto::tls::generate_self_signed_cert` for local certificates. Native insecure verification requires the `insecure-tls` feature and `MTP_INSECURE_TLS=1`; it disables certificate verification and is not a production fix.
|
||||
|
||||
## Getting More Help
|
||||
|
||||
Capture the negotiated version, connection state, error variant, endpoint, and relevant server log entries. Enable the SDK logger or Rust tracing, then remove credentials, private keys, and message contents before sharing a report.
|
||||
|
|
@ -1,6 +1,26 @@
|
|||
# Type Map
|
||||
|
||||
This file documents the Type Map & Registry configuration used by the MTP protocol.
|
||||
This file documents the Type Map & Registry configuration used by the MTP protocol. It will assume you are working with the [example-type-maps.yaml](./../example-type-maps.yaml).
|
||||
|
||||
## Binary Frame Format
|
||||
|
||||
Every transport frame is a four-byte big-endian length followed by one `CommunicationValue`. The length counts all bytes after the length field.
|
||||
|
||||
```text
|
||||
u32 length
|
||||
u16 communication_type
|
||||
u8 flags
|
||||
u32 id if flag 0x04 is set
|
||||
u48 sender if flag 0x01 is set
|
||||
u48 receiver if flag 0x02 is set
|
||||
u8 signature_type if flag 0x10 is set
|
||||
... signature if flag 0x10 is set, length depends on signature_type
|
||||
... data container or encrypted payload
|
||||
```
|
||||
|
||||
The flag values are `0x01` for sender, `0x02` for receiver, `0x04` for frame ID, `0x08` for encrypted data, `0x10` for a frame signature, and `0x20` for a signed encrypted container. Sender and receiver IDs are six-byte unsigned big-endian values. The `communication_type` and every container field use IDs from the negotiated `TypeMap`.
|
||||
|
||||
Data values begin with a one-byte kind marker. MTP assigns `0x01` and `0x02` to boolean true and false, `0x03` to signed integers, `0x04` to unsigned integers, `0x05` to floats, `0x06` to UTF-8 strings, `0x07` to bytes, `0x08` to arrays, `0x09` to containers, `0x0A` through `0x0C` to crypto containers, and `0xFF` to null. Length-prefixed values use a four-byte big-endian payload length; container and array counts use two-byte big-endian counts.
|
||||
|
||||
## TypeMap & Compile-Time Type Safety
|
||||
|
||||
|
|
@ -34,7 +54,7 @@ let tm = TypeMap::v2_0();
|
|||
let id = tm.data_id_enum(DataType::SomeType).unwrap();
|
||||
```
|
||||
|
||||
The enums are a **union across all versions**; every type name from every version is a variant. The version-specific `TypeMap` maps each variant to the correct wire ID for that version. Types not defined in a version return `None`:
|
||||
The enums are a **union across all versions**; every type name from every version is a variant. The version-specific `TypeMap` maps each variant to the correct wire ID for that version. For a type absent from a selected version, the lookup returns `None`.
|
||||
|
||||
Encoding/decoding uses a `TypeMap` to resolve type names to wire IDs:
|
||||
|
||||
|
|
@ -58,6 +78,8 @@ let tm_v1 = TypeMap::v1_0();
|
|||
assert!(tm_v1.data_id_enum(DataType::ExampleType).is_some()); // defined in v1.0
|
||||
```
|
||||
|
||||
When communicating with a peer on another version, encode only variants that map in the negotiated version. If an incoming frame names a type absent from the selected map, reject it as a protocol or type-map compatibility error; do not reinterpret its wire ID using another version's map. Keep old IDs stable, register both versions during migration, and remove a version only after its clients have moved.
|
||||
|
||||
### Forward/Backward Compatibility Between Versions
|
||||
|
||||
Because enums are a union of all types across versions, a variant might exist that has no wire mapping in the *negotiated* version:
|
||||
|
|
@ -67,7 +89,7 @@ v2.0 client sends DataType::SomeType → host encodes with v2.0 TypeMap → w
|
|||
v2.0 host receives DataType::ExampleType (from v1.0 client) → not in v2.0 TypeMap → None → Error
|
||||
```
|
||||
|
||||
This is by design: the host maps unknown types to `Error`, and the client should only send types that exist in its compiled-in version.
|
||||
Encoding a frame with an unmapped communication or data type returns `CodecError::UnknownCommunicationType` or `CodecError::UnknownDataType`. Select a mapped variant from the compiled-in version before sending it.
|
||||
|
||||
## Registry
|
||||
|
||||
|
|
@ -95,20 +117,11 @@ let decoded = codec.decode(&bytes, Version(2, 0)).unwrap();
|
|||
|
||||
## Customizing Type Maps in Downstream Projects
|
||||
|
||||
External projects must provide their own type map configuration. Browser projects should install `mtp` and configure `mtp/vite`; they do not need to publish, fork, or copy a generated WASM package.
|
||||
|
||||
```typescript
|
||||
import { defineConfig } from "vite";
|
||||
import { mtp } from "mtp/vite";
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [mtp({ typeMaps: "./type-maps.yaml" })],
|
||||
});
|
||||
```
|
||||
External projects must provide their own type map configuration. Browser projects use the Vite plugin from [Defining Type Maps](#defining-type-maps) and do not need to publish, fork, or copy a generated WASM package.
|
||||
|
||||
For Rust builds, or when invoking `wasm-pack` manually, set the `MTP_TYPE_MAPS` environment variable. If the variable points to an invalid file, the build fails. If `MTP_TYPE_MAPS` is not set, the build script emits a warning and generates reserved protocol types only; application-specific communication and data types will not be available.
|
||||
|
||||
1. Create a `type-maps.yaml` in your project root
|
||||
1. Create a `type-maps.yaml` in your project root (or anywhere else, adapt the path accordingly)
|
||||
2. Set the `MTP_TYPE_MAPS` environment variable in `.cargo/config.toml`:
|
||||
|
||||
```toml
|
||||
|
|
|
|||
|
|
@ -2,6 +2,17 @@
|
|||
|
||||
The browser client is exposed through the `mtp` npm package. Most applications should use the SDK-first `MTPClient` API; direct generated WASM bindings remain available from `mtp/raw` for advanced integrations.
|
||||
|
||||
## Browser Compatibility
|
||||
|
||||
The SDK requires the browser to expose `WebTransport`. `MTPClient.isSupported()` is the runtime check. A browser without WebTransport cannot connect through this client.
|
||||
|
||||
| Requirement | Check |
|
||||
| --- | --- |
|
||||
| WebTransport API | `MTPClient.isSupported()` |
|
||||
| Certificate trust | Browser validation or `serverCertificateHashes` |
|
||||
| Secure context | Serve the application from HTTPS where required by the browser |
|
||||
| Generated bindings | Run the Vite integration during development and build |
|
||||
|
||||
## Package Entry Points
|
||||
|
||||
```typescript
|
||||
|
|
@ -17,20 +28,12 @@ import { mtp } from "mtp/vite";
|
|||
|
||||
## Vite Type-Map Workflow
|
||||
|
||||
Browser apps provide their own type map. The Vite plugin runs `wasm-pack` during dev/build with `MTP_TYPE_MAPS` set, writes generated output under `node_modules/.vite/mtp/` by default, and aliases `mtp/raw` plus `mtp/type-map` to that generated output.
|
||||
|
||||
```typescript
|
||||
// vite.config.ts
|
||||
import { defineConfig } from "vite";
|
||||
import { mtp } from "mtp/vite";
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [mtp({ typeMaps: "./type-maps.yaml" })],
|
||||
});
|
||||
```
|
||||
Browser apps provide their own type map. The Vite plugin runs `wasm-pack` during dev and build with `MTP_TYPE_MAPS` set, writes generated output under `node_modules/.vite/mtp/` by default, and aliases `mtp/raw` plus `mtp/type-map` to that generated output. Configuration: [Type Map](TYPE-MAP.md).
|
||||
|
||||
You do not need to publish, fork, or copy an app-specific generated WASM package.
|
||||
|
||||
The [web client example](../example/web-client/src/main.ts) shows the entry point. Its [Vite configuration](../example/web-client/vite.config.ts) shows the generated binding integration.
|
||||
|
||||
## SDK Quick Start
|
||||
|
||||
```typescript
|
||||
|
|
@ -80,6 +83,41 @@ if (!MTPClient.isSupported()) {
|
|||
}
|
||||
```
|
||||
|
||||
## MTPClient Options
|
||||
|
||||
| Option | Default | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `url` | Required | WebTransport endpoint. |
|
||||
| `descriptor` | None | Client label sent during connection setup. |
|
||||
| `hostPublicKey` | None | Host public key bundle for authenticated login or registration. |
|
||||
| `credentials` | None | Existing client ID and serialized keyring. |
|
||||
| `credentialsStorageKey` | `mtp:credentials` | Key used by configured credential storage. |
|
||||
| `storage` | None | Sync or async credential storage adapter. |
|
||||
| `serverCertificateHashes` | Omitted | WebTransport certificate pins. |
|
||||
| `maxMessageSize` | 16 MiB | Inbound and outbound frame limit. Values below frame overhead are rejected by the transport. |
|
||||
| `authTimeoutMs` | No SDK timeout | Login and registration timeout. `undefined` leaves the promise pending until transport or peer failure. |
|
||||
| `requestTimeoutMs` | 30 seconds | Default `request()` timeout. |
|
||||
| `pings` | `false` | Protocol pings, or an object with `intervalMs`. |
|
||||
| `logger` | No-op | Receives SDK state and error events. |
|
||||
| `sessionStorage` | In-memory | E2EE session state storage. |
|
||||
| `encryptedDeviceSecretProvider` | In-memory | Device-secret storage for E2EE. |
|
||||
|
||||
`wasm` selects a custom generated WASM module. `MTPClient.create` validates positive safe-integer values for the numeric limits and timeout options.
|
||||
|
||||
## Differences from Native Client
|
||||
|
||||
The browser SDK uses WebTransport and JavaScript promises. The native client uses Rust futures, direct QUIC configuration, and `MTPConnection` handles. Browser pipes expose promise-based readers and writers; native pipes implement Tokio I/O traits.
|
||||
|
||||
### Native and Browser Credential Persistence
|
||||
|
||||
The `storage` option supplies the credential adapter. The adapter stores the client ID and serialized keyring after registration and returns them for later connections. The SDK does not select `localStorage` or IndexedDB for an application. Treat the serialized keyring as private key material.
|
||||
|
||||
`sessionStorage` and `encryptedDeviceSecretProvider` are separate E2EE session stores. The latter exchanges `EncryptedDeviceSecretRecord` values through `setEncryptedDeviceSecret` and `getEncryptedDeviceSecret`; the application chooses the backing store and protects its wrapping key.
|
||||
|
||||
### Native and Browser Certificate Checks
|
||||
|
||||
WebTransport certificate pins must match the server certificate hash. A pin mismatch is a TLS failure, not an MTP authentication failure. Check the browser network panel, endpoint origin, and WebTransport CONNECT path before inspecting frames.
|
||||
|
||||
## Credentials And Storage
|
||||
|
||||
Authenticated connections need stable key material. Pass `credentials` when you already have a client ID and serialized keyring, or pass a small `storage` object and let the SDK persist credentials after registration.
|
||||
|
|
@ -148,12 +186,12 @@ If hashes are omitted, the browser uses its normal TLS root store.
|
|||
|
||||
`maxMessageSize` caps inbound and outbound MTP frames before buffering/sending.
|
||||
`authTimeoutMs` bounds connect/login/register promises at the SDK layer.
|
||||
`requestTimeoutMs` sets the default timeout for `request()` calls; a request can override it with `timeoutMs` in its options.
|
||||
|
||||
## Streams
|
||||
|
||||
The browser client uses one WebTransport session per `MTPClient` instance.
|
||||
`send()`, `request()`, and `subscribe()` all operate over that session; the SDK
|
||||
does not expose browser stream objects directly.
|
||||
`send()`, `request()`, and `subscribe()` all operate over that session; the SDK does not expose browser stream objects directly.
|
||||
|
||||
Use the normal message APIs to send and receive over that session:
|
||||
|
||||
|
|
@ -169,18 +207,9 @@ await client.send("SomeType", { value: "hello" });
|
|||
unsubscribe();
|
||||
```
|
||||
|
||||
Internally, each outbound MTP frame is written to a new WebTransport
|
||||
unidirectional stream as a four-byte big-endian length followed by the frame,
|
||||
then that stream is closed. Incoming frames are read from the session's
|
||||
incoming unidirectional streams. The reader accepts both one-frame streams and
|
||||
native peers that place several frames on a persistent stream, so browser and
|
||||
native clients interoperate without stream configuration.
|
||||
Internally, each outbound MTP frame is written to a new WebTransport unidirectional stream as a four-byte big-endian length followed by the frame, then that stream is closed. Incoming frames are read from the session's incoming unidirectional streams. The reader accepts both one-frame streams and native peers that place several frames on a persistent stream, so browser and native clients interoperate without stream configuration.
|
||||
|
||||
The SDK deliberately owns stream lifetime and framing. Do not create browser
|
||||
streams for MTP frames yourself through the SDK. For direct generated bindings,
|
||||
use `client.raw.client` or import `WasmClient` from `mtp/raw`; a `WasmClient`
|
||||
still owns one active WebTransport session, so create another instance for an
|
||||
independent connection.
|
||||
The SDK owns stream lifetime and framing. Do not create browser streams for MTP frames yourself through the SDK. For direct generated bindings, use `client.raw.client` or import `WasmClient` from `mtp/raw`; a `WasmClient` still owns one active WebTransport session, so create another instance for an independent connection.
|
||||
|
||||
## Sending, Requests, Subscriptions, And Pings
|
||||
|
||||
|
|
@ -200,7 +229,7 @@ await client.send("SomeType", { value: "hello" }, {
|
|||
});
|
||||
```
|
||||
|
||||
`request` sends one frame and resolves with the parsed response carrying the same frame id. `responseType` is validated after the id match:
|
||||
`request` sends one frame and resolves with the parsed response carrying the same frame id. If the matching response has a different `responseType`, the promise rejects with a response-type error. A timeout rejects the promise and removes the pending request:
|
||||
|
||||
```typescript
|
||||
const response = await client.request(
|
||||
|
|
@ -220,7 +249,7 @@ const unsubscribe = client.subscribe("SomeType", (message) => {
|
|||
unsubscribe();
|
||||
```
|
||||
|
||||
Protocol pings are real MTP `Ping` frames sent by the WASM client, not just transport keepalives:
|
||||
Protocol ping behavior is defined in [Protocol Reference](PROTOCOL-REFERENCE.md#protocol-keepalive). The SDK configuration is:
|
||||
|
||||
```typescript
|
||||
await MTPClient.create({
|
||||
|
|
@ -359,6 +388,8 @@ interface ParsedFrame {
|
|||
}
|
||||
```
|
||||
|
||||
### Frames
|
||||
|
||||
Raw message helpers that remain available include:
|
||||
|
||||
- `build_frame(messageType, data, options?)`
|
||||
|
|
@ -377,6 +408,8 @@ const parsed = codec.decode(frame);
|
|||
const display = codec.format(frame);
|
||||
```
|
||||
|
||||
### Crypto
|
||||
|
||||
Raw crypto and key helpers include:
|
||||
|
||||
- `ed25519_generate()`
|
||||
|
|
@ -409,48 +442,25 @@ const confirmedId = await rawClient.auth_connect(
|
|||
);
|
||||
```
|
||||
|
||||
### Raw Pipes
|
||||
### Pipes
|
||||
|
||||
The raw `WasmClient` exposes the same pipe operations as the SDK wrapper:
|
||||
The raw `WasmClient` exposes the same pipe operations as the SDK wrapper. The shared lifecycle is in [Pipes](PIPES.md); raw bindings use snake_case names.
|
||||
|
||||
```typescript
|
||||
// Incoming pipe requests
|
||||
rawClient.set_on_pipe_request((event) => {
|
||||
const { pipeId, description } = event;
|
||||
// accept or deny
|
||||
void rawClient.accept_pipe(event.pipeId);
|
||||
});
|
||||
|
||||
// Outgoing pipe
|
||||
const handle = await rawClient.create_pipe("file-transfer");
|
||||
const writer = await handle.wait();
|
||||
if (writer) {
|
||||
await writer.write(new Uint8Array([0x01, 0x02]));
|
||||
await writer.write(chunk);
|
||||
await writer.close();
|
||||
}
|
||||
|
||||
// Accept incoming pipe
|
||||
const reader = await rawClient.accept_pipe(pipeId);
|
||||
const chunk = await reader.read();
|
||||
|
||||
// Deny incoming pipe
|
||||
await rawClient.deny_pipe(pipeId);
|
||||
```
|
||||
|
||||
Raw `PipeWriter` and `PipeReader` have the same interface as the SDK types:
|
||||
|
||||
```typescript
|
||||
interface PipeWriter {
|
||||
write(data: Uint8Array): Promise<void>;
|
||||
close(): Promise<void>;
|
||||
abort(): void;
|
||||
readonly pipeId: number;
|
||||
}
|
||||
|
||||
interface PipeReader {
|
||||
read(): Promise<Uint8Array | null>;
|
||||
readonly pipeId: number;
|
||||
readonly description: string;
|
||||
}
|
||||
```
|
||||
|
||||
A `WasmClient` manages one active WebTransport session. Create a new instance for independent connections, and call `free()` or `[Symbol.dispose]()` on raw WASM objects when you want to release memory eagerly.
|
||||
|
||||
### State Management
|
||||
|
||||
A `WasmClient` owns one active WebTransport session. Create a separate client for each independent connection. Call `free()` or `[Symbol.dispose]()` on raw WASM objects when the application no longer needs them. SDK session and device secret persistence are documented in [Security](SECURITY.md#browser-end-to-end-encryption).
|
||||
|
|
|
|||
Loading…
Reference in a new issue