[WIP] Security work While on holiday
This commit is contained in:
parent
a81ac4efca
commit
7f0231e3f1
SSH key fingerprint:
SHA256:D1+Ub8o0v4K5y1JNivW8IxEOelqLSvPmUzBbDIoZkRQ
109 changed files with 19694 additions and 5210 deletions
|
|
@ -38,12 +38,33 @@ MTP separates wire encoding, QUIC transport, connection policy, protocol negotia
|
|||
The top row represents application entry points. Native Rust code calls the client or host crates directly. Browser code calls the TypeScript SDK, which uses generated WASM bindings for the same codec and WebTransport session.
|
||||
Both clients exchange the same MTP frames with a host.
|
||||
|
||||
The middle row is shared protocol machinery. The type map determines numeric IDs, the codec serializes values, and transport framing places each serialized frame on a QUIC stream. This is why a type-map change must be compiled into both peers before the new message can be exchanged.
|
||||
The middle row is shared protocol machinery. The type map determines numeric IDs, the codec serializes self-delimiting `DataValue` payloads, and transport framing places each serialized frame on a QUIC stream. `CommunicationValue` contains only routing metadata and one generic payload. Protection is a composable value property (`Signed<Value>` or `Encrypted<Value>`), not a transport or communication-frame mode, so the frame and transport layers never infer encryption or signature state from header flags. This is why a type-map or codec change must be compiled into both peers before the new message can be exchanged.
|
||||
|
||||
The bottom row shows the two server entry points. `MTPHost` is a native QUIC endpoint for native MTP clients. `MTPWebServer` owns TCP HTTPS and UDP HTTP/3/WebTransport listeners on the same numeric port, reuses one `HostConfig` and router, and provides the same `accept()`-based MTP session API. Its QUIC listener still uses only the `h3` ALPN, so it cannot share its UDP address with the native MTP ALPN endpoint. Choose `MTPHost` for native clients and `MTPWebServer` for browser-facing HTTP and WebTransport.
|
||||
|
||||
`mtp-crypto` is an optional cross-cutting layer used by authenticated native connections, WebTransport connections, and browser E2EE; TLS remains the transport security layer in both paths.
|
||||
|
||||
MTP exposes protection as independent capabilities rather than prescribing an
|
||||
application topology:
|
||||
|
||||
- A stateless protected `DataValue` composes `Signed<Value>` and
|
||||
`Encrypted<Value>` in the order selected by the application.
|
||||
- A direct protected frame carries a protected value under its application
|
||||
communication type and routes it straight to the frame receiver.
|
||||
- A sealed relay uses the reserved `Relay` communication type, an absent outer
|
||||
sender, and separately protected metadata and content. Applications choose
|
||||
the next hop, final recipient, and both recipient sets.
|
||||
- A stateful encrypted session advances symmetric send and receive chains for
|
||||
an active exchange.
|
||||
- An encrypted pipe protects an ordered byte stream with transcript-bound
|
||||
records and an authenticated final record; forward-secure duplex setup is an
|
||||
explicit option.
|
||||
|
||||
These constructions are peers. Relay is optional and is not the default path
|
||||
for encrypted application messages. Use direct protected frames when no
|
||||
intermediate component needs relay metadata; use sealed relay when routing or
|
||||
store-and-forward topology requires a distinct metadata-access boundary.
|
||||
|
||||
`mtp-host` performs version negotiation and native authentication before returning an `MTPConnection`. `mtp-webserver` routes HTTP/1.1, HTTP/2, and HTTP/3 requests through one route table and surfaces WebTransport sessions through `accept()`. WebTransport MTP sessions support the same optional cryptographic authentication as native hosts when the `crypto` feature is enabled.
|
||||
|
||||
The [native client](NATIVE-CLIENT.md), [WASM client](WASM-CLIENT.md), [native host](NATIVE-HOST.md), and [web server](NATIVE-HOST-WEB-SERVER.md) guides cover the public APIs for each boundary. The web server guide should be read as the host API for browser-facing deployments; it accepts the same `HostConfig` and authentication callbacks as the native host.
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ Native clients and hosts share the same connection shape after the opening hands
|
|||
| `description` | Optional label sent during setup | Optional label received from the client | Optional label received from the client |
|
||||
| `client_id` | Confirmed or assigned ID with `crypto` | Authenticated or guest client ID with `crypto` | Authenticated or guest client ID with `crypto` |
|
||||
| `auth_state` | Authentication result with `crypto` | Authentication result with `crypto` | Authentication result with `crypto` |
|
||||
| `request_path` | — | — | WebTransport CONNECT path (e.g. `/mtp`) |
|
||||
| `request_path` | / | / | WebTransport CONNECT path (e.g. `/mtp`) |
|
||||
| `remote_addr` | Server `SocketAddr` when available | Peer `SocketAddr` | Peer `SocketAddr` |
|
||||
|
||||
`WebMTPConnection`, returned by `MTPWebServer::accept()`, exposes the same members as the native host connection plus `request_path`, which contains the HTTP/3 path used for the WebTransport extended CONNECT request.
|
||||
|
|
|
|||
|
|
@ -124,6 +124,6 @@ See [Protocol Reference](PROTOCOL-REFERENCE.md#protocol-keepalive).
|
|||
|
||||
## Protocol Version Changes
|
||||
|
||||
Add a protocol version by adding its type-map entry and `protocol_version` to the YAML configuration, then rebuild both peers. The type-map build script generates a version-specific `TypeMap` and keeps the enum as the union of all configured type names.
|
||||
Add a protocol version by adding its type-map entry and `protocol_version` to the YAML configuration, then rebuild both peers. The type-map build script generates a version-specific `TypeMap`. Native hosts built with the registry feature keep an enum union across configured versions; a browser client and its generated `mtp/type-map` declarations use only the map selected by that client's `protocol_version`, plus reserved names.
|
||||
|
||||
For a backward-compatible change, keep existing communication and data IDs stable and add new types with the new version. For a breaking change, add a new version and register both versions on the host while clients migrate. A client compiles one protocol version; it can connect only when that version is present in the host registry. Remove an old version only after its clients no longer connect, because the host closes connections whose version is unsupported.
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ let conn = MTPClient::connect(
|
|||
let request = CommunicationValue::new(CommunicationType::Ping).with_id(1);
|
||||
conn.sender.send(&request).await?;
|
||||
let response = conn.receive().await?;
|
||||
println!("received {}", response.get_id());
|
||||
println!("received {:?}", response.id());
|
||||
conn.sender.close();
|
||||
```
|
||||
|
||||
|
|
@ -259,48 +259,51 @@ Sends a close frame and signals the peer. The `Sender::close()` spawns an async
|
|||
|
||||
The complete pipe protocol, native API, browser API, lifecycle, and errors are documented in [Pipes](PIPES.md). Use the connection facade described there when the `pipes` feature is enabled.
|
||||
|
||||
## Appendix: Crypto Containers
|
||||
## Appendix: Composable Data Protection
|
||||
|
||||
With the `crypto` feature, `DataValue` supports encrypted, signed, and signed+encrypted containers. Encryption uses ML-KEM to encapsulate to a recipient's KEM public key (from their `PublicKeyBundle`); only the holder of the matching `Keyring` can decrypt. Signing uses the sender's Ed25519 key.
|
||||
With the `crypto` feature, any `DataValue` can be signed or encrypted. The operations return typed errors and compose by operation order. `Encrypted(Signed(Value))` keeps the signer identity inside the encrypted plaintext; `Signed(Encrypted(Value))` leaves it visible. The example uses different keyrings for the signer and recipient to make the ownership explicit.
|
||||
|
||||
```rust
|
||||
use mtp::crypto::{EncryptionType, Ed25519Signer, SigAlgorithm};
|
||||
use mtp::codec::{ProtectionPurpose, DataTypeId, DataValue};
|
||||
use mtp::crypto::{Ed25519Signer, Keyring};
|
||||
|
||||
let enc_type = EncryptionType::MlKemChaCha20Poly1305;
|
||||
let signer = Ed25519Signer::new(&keyring.sig_cl_secret_key)?;
|
||||
|
||||
// `recipient` is the PublicKeyBundle of whoever should be able to decrypt
|
||||
// (e.g. the host's bundle, obtained out of band).
|
||||
|
||||
// Encrypted container
|
||||
let mut enc = DataValue::Container(vec![
|
||||
(DataTypeId(1), DataValue::Str("secret".into())),
|
||||
let sender_keyring = Keyring::generate();
|
||||
let recipient_keyring = Keyring::generate();
|
||||
let signer = Ed25519Signer::new(&sender_keyring.sig_cl_secret_key)?;
|
||||
let recipient = recipient_keyring.public_key_bundle();
|
||||
let sender_public_keys = sender_keyring.public_key_bundle();
|
||||
let value = DataValue::Container(vec![
|
||||
(DataTypeId(32), DataValue::Str("secret".into())),
|
||||
]);
|
||||
enc.encrypt_container(enc_type, &recipient, b"aad");
|
||||
|
||||
// Signed container
|
||||
let mut sig = DataValue::Container(vec![
|
||||
(DataTypeId(1), DataValue::Str("signed".into())),
|
||||
]);
|
||||
sig.sign_container(SigAlgorithm::ED25519, &signer);
|
||||
|
||||
// Signed + encrypted
|
||||
let mut sec = DataValue::Container(vec![
|
||||
(DataTypeId(1), DataValue::Str("both".into())),
|
||||
]);
|
||||
sec.sign_and_encrypt_container(SigAlgorithm::ED25519, &signer, enc_type, &recipient, b"aad");
|
||||
// The outer encrypted wrapper hides the signer metadata.
|
||||
let private_signer = value.clone().sign(7, ProtectionPurpose::from(1), &signer)?;
|
||||
let sealed = private_signer.encrypt_for(
|
||||
std::slice::from_ref(&recipient),
|
||||
ProtectionPurpose::from(2),
|
||||
)?;
|
||||
```
|
||||
> Note: DataTypeId(1) maps intenally to the reserved DataType::Id, uncareful work with reserved DataTypes & CommunicationTypes (0 - 31) may lead to unexpected behaviour.
|
||||
> Prefer registring your own.
|
||||
|
||||
On the receiving side, the recipient decrypts with its own `Keyring` (each blob is self-describing: its leading byte selects the algorithm and the matching KEM key from the keyring):
|
||||
Reverse the calls when the signer identity should remain visible to the recipient before opening the encrypted value:
|
||||
|
||||
```rust
|
||||
enc.decrypt_into_container(&keyring, b"aad"); // -> Container
|
||||
sig.verify_into_container(&verifier); // verifier: impl SignatureScheme
|
||||
sec.decrypt_signed_encrypted_container(&keyring, b"aad"); // -> SignedContainer, then verify_into_container
|
||||
let encrypted = value.encrypt_for(
|
||||
std::slice::from_ref(&recipient),
|
||||
ProtectionPurpose::from(2),
|
||||
)?;
|
||||
let public_signer = encrypted.sign(7, ProtectionPurpose::from(1), &signer)?;
|
||||
```
|
||||
|
||||
Opening and verification are explicit and return the inner value without mutating the wrapper:
|
||||
|
||||
```rust
|
||||
let signed = sealed.decrypt(&recipient_keyring, ProtectionPurpose::from(2))?;
|
||||
signed.verify(7, &sender_public_keys, ProtectionPurpose::from(1))?;
|
||||
let plain = signed.into_verified(7, &sender_public_keys, ProtectionPurpose::from(1))?;
|
||||
```
|
||||
|
||||
For `public_signer`, call `verify` and `into_verified` before calling `decrypt`; its outer signature is available before the encrypted value is opened.
|
||||
|
||||
### Policy Configuration
|
||||
|
||||
The `Policy` struct controls transport behaviour:
|
||||
|
|
|
|||
|
|
@ -124,7 +124,7 @@ let mut server = MTPWebServer::new(host_config, web).await?;
|
|||
while let Some(connection) = server.accept().await? {
|
||||
// connection: WebMTPConnection
|
||||
while let Ok(message) = connection.receive().await {
|
||||
println!("received MTP message {}", message.get_id());
|
||||
println!("received MTP message {:?}", message.id());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
|
@ -146,7 +146,7 @@ With port `0` and TCP enabled, construction binds TCP first and binds UDP to the
|
|||
|
||||
| Policy | Behavior |
|
||||
|--------|----------|
|
||||
| `Unauthenticated` (default) | No authentication handshake is performed. The connection has `AuthState::Unauthenticated` and a random 48-bit client ID. `guest_id_generator` is not used by this adapter. |
|
||||
| `Unauthenticated` (default) | No authentication handshake is performed. The connection has `AuthState::Unauthenticated` and a random full-width `u64` client ID. `guest_id_generator` is not used by this adapter. |
|
||||
| `AllowAuthentication` | The server accepts the first message. If it is an `Identification` or `Register` message, a full challenge-response handshake is performed. If it is an ordinary opening message, the connection remains unauthenticated. |
|
||||
| `ForceAuthentication` | The server requires a valid `Identification` or `Register` message as the first frame and performs the challenge-response handshake. Any other opening message is rejected. |
|
||||
|
||||
|
|
|
|||
|
|
@ -157,10 +157,9 @@ let get_existing_client = |id: u64, _description: Option<String>| {
|
|||
|
||||
### guest_id_generator
|
||||
|
||||
Optional callback that controls how unauthenticated connections receive their client ID. When `None` (the default), the host generates a random 48-bit ID and checks it against `get_existing_client` to avoid collisions.
|
||||
Optional callback that controls how unauthenticated connections receive their client ID. When `None` (the default), the host generates a random full-width `u64` ID and checks it against `get_existing_client` to avoid collisions.
|
||||
|
||||
Return `Some(id)` to accept the guest with that ID, or `None` to reject the connection. The ID must fit in 48 bits (`id <= mtp_codec::MAX_WIRE_ID`);
|
||||
values outside that range are rejected automatically and fall back to the built-in generator.
|
||||
Return `Some(id)` to accept the guest with that full-width `u64` ID, or `None` to reject the connection.
|
||||
|
||||
```rust
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
|
|
|||
146
docs/PIPES.md
146
docs/PIPES.md
|
|
@ -1,6 +1,14 @@
|
|||
# MTP Pipes
|
||||
|
||||
Pipes are unidirectional QUIC streams for raw bytes. The creator sends a `PipeRequest` communication value, the peer accepts or rejects it, and the stream then carries bytes without an MTP frame around every write.
|
||||
Pipes are unidirectional QUIC/WebTransport streams. The transport primitive is
|
||||
byte-oriented, but raw pipe bytes are not confidential or authenticated by
|
||||
MTP. The creator sends a `PipeRequest` communication value, the peer accepts
|
||||
or rejects it, and an application that carries sensitive data must place the
|
||||
encrypted record layer described below on top of the accepted stream.
|
||||
|
||||
The request's `Description` and `PipeRequest` type remain clear transport
|
||||
metadata. Do not put identities, call details, file names, or other sensitive
|
||||
protocol information in them.
|
||||
|
||||
The creator owns the writer. The accepting peer owns the reader. A writer finishes with a stream FIN or aborts with a stream reset. A reader returns EOF after FIN and reports a connection or stream error when the peer closes unexpectedly.
|
||||
|
||||
|
|
@ -11,6 +19,88 @@ The creator calls `create_pipe` or the corresponding SDK `createPipe` method wit
|
|||
The request description is application metadata. It does not grant access to the stream, authenticate the creator, or negotiate an application protocol.
|
||||
Use the authenticated MTP connection and the host's admission policy when a pipe carries sensitive data.
|
||||
|
||||
The browser SDK's `createEncryptedPipe` and `acceptEncryptedPipe` convenience
|
||||
methods derive the local identity, actual pipe ID, random session ID, and
|
||||
default application purpose from MTP state. Use the lower-level session
|
||||
functions only when integrating a custom pipe transport. The low-level API
|
||||
checks that a supplied pipe ID matches the actual pipe; it does not infer a
|
||||
caller-provided sender or recipient identity.
|
||||
|
||||
The convenience methods intentionally require registered client credentials
|
||||
because their endpoint identity is the transport client's registered MTP
|
||||
identity. An application that needs a cryptographic identity independent from
|
||||
transport registration must use the lower-level session functions and provide
|
||||
the endpoint IDs and key material explicitly.
|
||||
|
||||
## Endpoint Encryption
|
||||
|
||||
`initiate_pipe_session`/`accept_pipe_session` in the native transport, or
|
||||
`initiateMTPPipeSession`/`acceptMTPPipeSession` in the browser SDK, perform the
|
||||
pipe-establishment step. The initiator sends an
|
||||
`Encrypted(Signed(Array<...>))` offer containing a fresh 32-byte initial chain key,
|
||||
session ID, pipe ID, direction, purpose, and both endpoint IDs. The recipient
|
||||
decrypts it with its keyring, resolves the expected sender bundle, verifies
|
||||
the signature, and checks every expected field before returning the record
|
||||
reader. The offer is bounded and separately framed from application records.
|
||||
|
||||
The helpers then return `EncryptedPipeWriter`/`EncryptedPipeReader` (or their
|
||||
browser equivalents) without changing the raw QUIC/WebTransport adapter. The
|
||||
context contains the unique pipe/session identity, endpoint identities,
|
||||
direction, and application protocol purpose. Do not derive the initial chain
|
||||
key from the clear description or pipe ID alone.
|
||||
|
||||
The receiver's signature verification policy is explicit and independent from
|
||||
its decryption keyring. Configure `signaturePolicy` on the browser accept
|
||||
helper, or use the client's `defaultSignatureVerificationPolicy`. The
|
||||
initiator and responder signing `signatureSuite` remain separate from this
|
||||
receive policy. Both sides default to Ed25519; choose `signatureSuite: "dual"`
|
||||
and a matching `signaturePolicy: "dual"` explicitly when hybrid signatures
|
||||
are required.
|
||||
|
||||
Each record is encoded as:
|
||||
|
||||
```text
|
||||
[4-byte big-endian ciphertext length]
|
||||
[1-byte record type: DATA=0, FINAL=1]
|
||||
[XChaCha20-Poly1305 nonce || ciphertext || tag]
|
||||
```
|
||||
|
||||
The AEAD associated data is `MTP-PIPE-E2EE-1 || purpose || direction ||
|
||||
transcript-hash || sequence || record length || record type`. The transcript
|
||||
hash binds the session ID, pipe ID, sender, recipient, purpose, and direction.
|
||||
The sequence starts at zero and advances only after successful authentication.
|
||||
A missing, duplicated, reordered, or modified record causes authentication to
|
||||
fail. Each record derives a one-use message key and the next chain key with
|
||||
HKDF using the authenticated context and sequence number; the bootstrap key is
|
||||
never used directly as an AEAD key. The record layer caps one encoded record at
|
||||
16 MiB.
|
||||
|
||||
`FINAL` is an authenticated empty record. A reader returns clean EOF only
|
||||
after validating it; transport EOF before `FINAL` is truncation.
|
||||
Authentication, framing, sequence, and I/O failures permanently poison the
|
||||
encrypted reader or writer and erase its current chain key. This is a one-way
|
||||
chain, not a Diffie-Hellman ratchet, so the ordinary offer does not provide
|
||||
forward secrecy.
|
||||
|
||||
The wrapper exposes `writeRecord`/`readRecord`. Callers that already have an
|
||||
independently authenticated session may still construct it directly with a
|
||||
key and context; otherwise use the establishment helpers.
|
||||
|
||||
For more than two members, native `initiate_group_pipe_session` and the browser
|
||||
`initiateMTPPipeSession` recipient-array form encrypt one fresh session key to
|
||||
each current member. Membership changes are rekeys: create a new session ID
|
||||
and offer with the new recipient set, and stop using the old record chain. A
|
||||
removed member must never receive a later session key; an added member must
|
||||
not receive historical records.
|
||||
|
||||
When a live call needs forward secrecy, use the duplex handshake
|
||||
`initiate_forward_secure_pipe_session`/`accept_forward_secure_pipe_session` or
|
||||
the browser `initiateMTPForwardSecurePipeSession`/
|
||||
`acceptMTPForwardSecurePipeSession`. The responder contributes a fresh
|
||||
ephemeral hybrid-KEM key, while long-term signing keys authenticate the
|
||||
exchange. These helpers require a bidirectional stream and bind the handshake
|
||||
transcript into the record context.
|
||||
|
||||
## Accepting or Rejecting a Pipe
|
||||
|
||||
The receiving side reads pipe requests through `receive_pipe`, the host dispatcher, or the browser pipe callback. It calls `accept` to obtain a reader or `deny` to reject the request. A rejected request completes the creator's handle with `Rejected` and no raw byte stream becomes available.
|
||||
|
|
@ -20,7 +110,12 @@ Normal messages and pipe requests share the transport and must pass through the
|
|||
|
||||
## Closing a Pipe
|
||||
|
||||
The creator closes a successful pipe with `PipeWriter::finish` or the browser writer's `close`; this sends a QUIC FIN and lets the reader observe EOF. Use `abort` when the peer should discard the stream immediately; this resets the stream and the reader receives an error instead of a clean EOF. Dropping the connection closes all active pipes.
|
||||
The creator closes a successful encrypted pipe with `EncryptedPipeWriter::finish`
|
||||
or the browser writer's `close`; this authenticates `FINAL` and then sends a
|
||||
QUIC FIN. Use `abort` when the peer should discard the stream immediately; this
|
||||
resets the stream and the reader receives an error instead of a clean EOF.
|
||||
Dropping the connection closes all active pipes. Raw pipe FIN is not an
|
||||
authenticated application completion signal.
|
||||
|
||||
The accepting side closes its reader by consuming it or dropping it. A reader does not send an application-level acknowledgement for EOF. If the application needs completion metadata, send an ordinary MTP message before finishing the pipe.
|
||||
|
||||
|
|
@ -38,23 +133,40 @@ Native applications use the pipe APIs on `MTPConnection`; browser applications u
|
|||
|
||||
## Native File Upload and Processing
|
||||
|
||||
The creator streams a file in chunks. The accepting side processes each chunk without buffering the complete file:
|
||||
The creator streams a file in encrypted records. The accepting side processes
|
||||
each decrypted chunk without buffering the complete file. The `session_key`
|
||||
below is obtained from the authenticated pipe-establishment protocol:
|
||||
|
||||
```rust
|
||||
// Client
|
||||
use tokio::io::AsyncWriteExt;
|
||||
use mtp_transport::{PipeSessionParameters, initiate_pipe_session};
|
||||
use tokio::io::AsyncReadExt;
|
||||
|
||||
let handle = conn.create_pipe("file-upload").await?;
|
||||
if let Some(mut writer) = handle.wait().await? {
|
||||
let pipe_id = handle.pipe_id();
|
||||
if let Some(writer) = handle.wait().await? {
|
||||
let params = PipeSessionParameters::new(
|
||||
format!("file-upload/{pipe_id}"), pipe_id, own_client_id, host_client_id, 0x40, 0,
|
||||
)?;
|
||||
let mut writer = initiate_pipe_session(
|
||||
writer.into_inner(), params, &own_keyring, &host_public_bundle,
|
||||
).await?;
|
||||
let mut file = tokio::fs::File::open("input.bin").await?;
|
||||
tokio::io::copy(&mut file, &mut writer).await?;
|
||||
let mut buffer = [0u8; 64 * 1024];
|
||||
loop {
|
||||
let count = file.read(&mut buffer).await?;
|
||||
if count == 0 {
|
||||
break;
|
||||
}
|
||||
writer.write_record(&buffer[..count]).await?;
|
||||
}
|
||||
writer.finish().await?;
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
// Host
|
||||
use tokio::io::AsyncReadExt;
|
||||
use mtp_transport::{PipeSessionParameters, accept_pipe_session};
|
||||
|
||||
while let Ok(request) = conn.receive_pipe().await {
|
||||
if request.description() != "file-upload" {
|
||||
|
|
@ -62,16 +174,18 @@ while let Ok(request) = conn.receive_pipe().await {
|
|||
continue;
|
||||
}
|
||||
|
||||
let mut reader = request.accept().await?;
|
||||
let pipe_id = request.id();
|
||||
let reader = request.accept().await?;
|
||||
let params = PipeSessionParameters::new(
|
||||
format!("file-upload/{pipe_id}"), pipe_id, client_id, own_client_id, 0x40, 0,
|
||||
)?;
|
||||
let mut reader = accept_pipe_session(
|
||||
reader.into_inner(), ¶ms, &own_keyring, &client_public_bundle,
|
||||
).await?;
|
||||
let mut hasher = sha2::Sha256::new();
|
||||
let mut buffer = [0u8; 64 * 1024];
|
||||
loop {
|
||||
let count = reader.read(&mut buffer).await?;
|
||||
if count == 0 {
|
||||
break;
|
||||
}
|
||||
hasher.update(&buffer[..count]);
|
||||
process_chunk(&buffer[..count]).await?;
|
||||
while let Some(chunk) = reader.read_record().await? {
|
||||
hasher.update(&chunk);
|
||||
process_chunk(&chunk).await?;
|
||||
}
|
||||
let digest = hasher.finalize();
|
||||
println!("processed upload with digest {digest:x}");
|
||||
|
|
|
|||
|
|
@ -17,6 +17,47 @@ The client sends an MTP `Ping` communication value with a frame ID. The host ret
|
|||
|
||||
If automatic responses are disabled, the application must read Ping frames and send compatible Pong frames. Keepalive configuration is documented in the [native client](NATIVE-CLIENT.md) and [native host](NATIVE-HOST.md) guides.
|
||||
|
||||
## Relay metadata version
|
||||
|
||||
Protected relay metadata declares the reserved `RelayVersion` field as an unsigned integer. Builders currently emit version `1` automatically. Receivers select the metadata schema from this field before interpreting any version-specific fields. Missing versions are unsupported legacy relays, and unknown versions are rejected.
|
||||
|
||||
Relay format versions are independent of application type-map versions. A type-map version selects application-defined communication and data types. It does not select the protected relay metadata schema.
|
||||
|
||||
## Relay `CreatedAt`
|
||||
|
||||
The reserved `CreatedAt` field in relay metadata is an unsigned integer containing milliseconds elapsed since `1970-01-01T00:00:00Z`. It is not an ISO timestamp and it is not measured in seconds.
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
2026-08-11T12:00:00.000Z
|
||||
↓
|
||||
Unix epoch milliseconds
|
||||
↓
|
||||
CreatedAt = 1786449600000
|
||||
```
|
||||
|
||||
Native relay builders and browser relay senders use this unit. Verified browser metadata exposes `createdAt` as a `bigint`; native verified metadata exposes `u64`.
|
||||
|
||||
## Direct protected envelope
|
||||
|
||||
The high-level direct protected API signs an MTP-owned envelope before it is
|
||||
encrypted for the recipient. Its reserved fields are `ProtectedVersion`,
|
||||
`MessageType`, `FinalRecipientId`, `MessageId`, `CreatedAt`, and `Content`.
|
||||
Receivers verify the envelope before dispatching application content and require
|
||||
the signed message type and final recipient to match the outer communication
|
||||
type and receiver. If the outer sender is present, it must match the signed
|
||||
signer ID. `MessageId` and `CreatedAt` are authenticated; callers can pass a
|
||||
replay guard to reject a previously accepted `(signerId, MessageId)` pair.
|
||||
Native and browser replay guards both receive `CreatedAt` as authenticated
|
||||
metadata, but the timestamp is not part of the replay key.
|
||||
Verified SDK results expose the authenticated `protectedVersion` and
|
||||
`finalRecipientId` alongside the application content.
|
||||
|
||||
Native applications use the same schema through `ProtectedMessageBuilder` and
|
||||
`open_protected`; language bindings delegate envelope construction and opening
|
||||
to this codec boundary.
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
```text
|
||||
|
|
@ -41,3 +82,5 @@ Login proof binds the protocol version, client ID, host challenge, and client no
|
|||
## Version Negotiation
|
||||
|
||||
The client sends one compiled-in protocol version. The host compares it with the versions in its registry and returns the selected version in the opening response. Subsequent frames use that version's type map. An unsupported version closes the connection with `AcceptError::UnsupportedVersion`.
|
||||
|
||||
The self-delimiting `DataValue` codec and the three-bit communication header begin at protocol version `3.0`. A peer offering an older codec version is rejected during version negotiation; the new decoder does not attempt legacy flag, ID, or crypto-container fallbacks.
|
||||
|
|
|
|||
131
docs/SECURITY.md
131
docs/SECURITY.md
|
|
@ -95,9 +95,100 @@ The tags prevent a valid signature for one handshake step from being accepted as
|
|||
| Classical signatures | Ed25519 | Default |
|
||||
| Post-quantum signatures | ML-DSA-65 | Default |
|
||||
| KDF and hashing | HKDF-SHA-256, SHA-256 | Default |
|
||||
| Password KDF for `.mk` files | Argon2id | `files` feature |
|
||||
| Hybrid KEM | X25519 plus ML-KEM-768 | `pqc` feature |
|
||||
|
||||
AEAD output stores the nonce before the authenticated ciphertext. Encrypted containers select their algorithm with a leading marking byte, derive an AEAD key from the KEM shared secret with HKDF, and authenticate caller-supplied AAD. Multi-recipient encryption wraps one content-encryption key separately for each recipient.
|
||||
AEAD output stores the nonce before the authenticated ciphertext. `DataValue::Encrypted` uses one canonical multi-recipient envelope and derives a content key through authenticated KEM key wrapping. `DataValue::Signed` authenticates a domain-separated purpose, signer ID, and exact serialized inner value. MTP does not accept caller-supplied AAD as a replacement for this context.
|
||||
|
||||
| Protection | Authenticated fields |
|
||||
| --- | --- |
|
||||
| `Signed<Value>` | `MTP-DATA-SIGN-1`, signature algorithm, purpose, signer ID, and the exact serialized inner value. |
|
||||
| `Encrypted<Value>` | `MTP-DATA-ENC-1`, encryption suite, purpose, recipient count, recipient table, and the ciphertext. Each wrapped content key also authenticates `MTP-DATA-WRAP-1`, suite, purpose, and its KEM ciphertext. |
|
||||
|
||||
The communication header is routing metadata, not automatically part of either
|
||||
generic value wrapper's authenticated data. The high-level direct protected API
|
||||
adds an MTP-owned signed envelope that binds its application type, final
|
||||
recipient, message ID, creation time, and content to the outer route. Callers
|
||||
using the generic protection primitives must bind any routing or message
|
||||
metadata they require in their own signed value.
|
||||
|
||||
Protection composition is significant: `Encrypted(Signed(Value))` hides signer metadata until decryption and is the construction used for sealed-sender payloads; `Signed(Encrypted(Value))` exposes the signer metadata while protecting the contents. A sealed-sender frame simply omits the outer communication sender, routes with its receiver field, and carries an `Encrypted(Signed(Value))` payload. There is no sealed-sender frame flag or wire type.
|
||||
|
||||
### Protected Frame Visibility
|
||||
|
||||
Before opening an `Encrypted(Signed(Value))` payload, a component with access to the MTP frame can read the frame length, communication type, presence flags, transport correlation ID, and next-hop receiver. Relayable application messages use the generic reserved `Relay` communication type; operation-specific names are inside the ciphertext. The outer encrypted value also reveals its encryption suite, generic relay protection purpose, recipient count, unlabeled KEM ciphertext and wrapped-key entries, and ciphertext length. Recipient entries contain no recipient IDs, although recipient count and the cryptographic entry material remain visible.
|
||||
|
||||
The signer algorithm, signature purpose, signer ID, signature, and application-defined inner value are encrypted. They become available only after a recipient opens the encrypted value. The recipient must still verify the inner signature before trusting its signer ID or contents.
|
||||
|
||||
Sealed sender is therefore a construction rule, not an anonymity guarantee or a separate protocol type. The frame sender is absent, the next-hop receiver remains visible for routing, and MTP does not inspect application containers to infer identities or protection flags.
|
||||
|
||||
Connection authentication and protected identity are separate. For a sealed
|
||||
relay sent over an authenticated connection, the host knows the connection's
|
||||
registered MTP identity even though the outer relay sender is absent. The
|
||||
protected signer remains hidden until a metadata recipient decrypts and
|
||||
verifies the relay metadata.
|
||||
|
||||
For a sealed relay sent over an unauthenticated connection, the host receives
|
||||
no registered MTP identity from connection authentication. The outer relay
|
||||
sender is still absent, and the protected signer is still hidden until metadata
|
||||
decryption and verification. The network connection nevertheless has observable
|
||||
metadata such as peer addressing, timing, sizes, and the visible frame fields
|
||||
described above. Neither case provides network anonymity.
|
||||
|
||||
### Relay access model and replay protection
|
||||
|
||||
Relay messages separate metadata recipients from content recipients. A relay
|
||||
service can receive the metadata key, verify the authenticated signer and
|
||||
message identifiers, index the opaque encrypted-content value, and forward the
|
||||
frame without receiving a content key. Only a content recipient can open the
|
||||
content. The final recipient and application message type remain inside the
|
||||
protected metadata/content structure; the outer frame exposes only the chosen
|
||||
next hop.
|
||||
|
||||
The receiver must consume the authenticated `(signer ID, MessageId)` pair with
|
||||
a replay guard. `CreatedAt` is authenticated metadata that the guard receives
|
||||
for retention or observability, but it is not part of the replay identity and
|
||||
must not be used as the replay defense. The native codec exposes `ReplayGuard`
|
||||
and the browser SDK exposes the matching `MTPReplayGuard` contract. Both
|
||||
high-level APIs use bounded process-local guards by default for direct and
|
||||
relay subscriptions. Those defaults are duplicate suppression only while an
|
||||
entry remains in the fixed cache: eviction, reloads, or multiple receiver
|
||||
processes can permit a previously accepted message again. Low-level relay
|
||||
metadata opening remains replay-optional for callers reopening stored frames.
|
||||
Use a durable guard when replay state must survive cache eviction, reloads, or
|
||||
process boundaries. A guard should atomically record a new ID before
|
||||
dispatching application content. Transport frame IDs must not be used for
|
||||
this purpose.
|
||||
|
||||
`VerifiedRelayMetadata` is an authenticated capability rather than a caller
|
||||
constructed data transfer object. Rust fields are private and the browser
|
||||
implementation keeps authenticated state behind a branded class. Content
|
||||
opening consumes that authenticated state, so changing a message ID or
|
||||
recipient in a normal object cannot make unrelated encrypted content inherit
|
||||
those fields. Browser callers can call `dispose()` or `free()` on the metadata
|
||||
capability for deterministic native-handle release; finalization remains a
|
||||
fallback.
|
||||
|
||||
### Signature policy
|
||||
|
||||
Verification takes a receiver-side `SignaturePolicy`/`ProtectionPolicy`.
|
||||
`AnySupported` is useful for compatibility at the low-level codec boundary,
|
||||
but protocol receivers should select `Ed25519` or `Dual`. The browser SDK uses
|
||||
an explicit `ed25519` default and permits an operation or client override. It
|
||||
never derives receive policy from the recipient keyring. The sender's
|
||||
signature suite remains a separate choice. Signature policy must be applied
|
||||
independently to relay metadata, relay content, and pipe session establishment.
|
||||
|
||||
### Key history and rotation
|
||||
|
||||
Recipient KEM key history is tried locally without adding a stable recipient
|
||||
key identifier to the visible encrypted-recipient table. Signing-key resolvers
|
||||
receive a claimed, unverified signer ID only as a trusted-key lookup key; the
|
||||
relay helpers authenticate that ID when they verify against the returned
|
||||
history. Deployments should retain old
|
||||
verification keys for at least as long as stored signed messages remain
|
||||
accepted, and should make key-history lookup an authorization decision rather
|
||||
than accepting any key supplied with a message.
|
||||
|
||||
[mtp-crypto API](../crypto/), [native client](NATIVE-CLIENT.md), and [native host](NATIVE-HOST.md).
|
||||
|
||||
|
|
@ -112,7 +203,7 @@ The crate's feature groups are:
|
|||
| `wasm` | `getrandom` support for WebAssembly |
|
||||
| `tls` | Development certificate generation |
|
||||
|
||||
The main types are `Keyring`, `PublicKeyBundle`, `EncryptionType`, `HybridKem`, `ChaCha20Poly1305`, `Aes256Gcm`, `Ed25519Signer`, and `MlDsaSigner`. Hashing and KDF helpers include `sha256`, `sha256_double`, `hkdf_extract`, `hkdf_expand`, and `derive_encryption_key`. Handshake payload builders are in `mtp_crypto::auth`.
|
||||
The main types are `Keyring`, `PublicKeyBundle`, `EncryptionType`, `HybridKem`, `XChaCha20Poly1305` (with the legacy `ChaCha20Poly1305` alias), `Aes256Gcm`, `Ed25519Signer`, and `MlDsaSigner`. Hashing and KDF helpers include `sha256`, `sha256_double`, `hkdf_extract`, `hkdf_expand`, and `derive_encryption_key`. Handshake payload builders are in `mtp_crypto::auth`.
|
||||
|
||||
## Cryptographic Review Status
|
||||
|
||||
|
|
@ -139,19 +230,45 @@ The browser SDK's optional E2EE session uses XChaCha20-Poly1305 with message key
|
|||
This is a single-chain ratchet. It has no Diffie-Hellman ratchet step and does not provide post-compromise security. Out-of-order messages can create skipped keys; the SDK accepts a receive gap of at most 100 messages and retains at most 100 skipped keys. Consumed or evicted keys are zeroed in the SDK state where the implementation owns the buffer.
|
||||
|
||||
The session root key comes from the authenticated handshake's KEM shared secret. The initiator and responder derive separate send and receive chains.
|
||||
Each message consumes one chain key, derives one message key with HKDF, and increments its counter. `sessionStorage` stores browser session state for the current origin. `encryptedDeviceSecretProvider` supplies encrypted device secret storage when sessions must survive page reloads. The provider must protect its wrapping secret outside the SDK; the SDK does not recover a lost device secret or skipped message keys.
|
||||
Each message consumes one chain key, derives one message key with HKDF, and increments its counter. `sessionStorage` stores browser session state for the current origin. `encryptedSecretProvider` is an independent caller-managed encrypted-secret facility; it is not automatically used by `MTPSessionStorage` or `MTPSessionManager`. Applications that need encrypted session persistence must coordinate those stores explicitly. The provider must protect its wrapping secret outside the SDK; the SDK does not recover a lost secret or skipped message keys.
|
||||
|
||||
Relay envelopes, browser session E2EE, and encrypted pipes are separate
|
||||
protocols:
|
||||
|
||||
| Model | State | Intended use |
|
||||
| --- | --- | --- |
|
||||
| `RelayEnvelope` | Stateless `Encrypted(Signed(Value))`, multi-recipient | Store-and-forward messages and routing |
|
||||
| `SessionE2EE` | Stateful symmetric ratchet in `sessionStorage` | Active browser exchanges |
|
||||
| `EncryptedPipeSession` | Authenticated setup plus ordered record chain | Protected streams |
|
||||
|
||||
Encrypted pipes bind the pipe/session transcript, direction, purpose, sequence,
|
||||
record length, and record type to each record. `FINAL` is authenticated and
|
||||
unexpected EOF is reported as truncation. The ordinary signed/KEM offer is not
|
||||
forward-secure; the native and browser duplex helpers use an ephemeral
|
||||
authenticated KEM exchange before deriving the record chain. Group membership
|
||||
changes require a new session key and recipient set.
|
||||
|
||||
## Key Storage
|
||||
|
||||
`Keyring` contains three public and three private key values. Its private key fields use `ZeroizeOnDrop`, and serialized keyring output is held in a zeroizing buffer while it is constructed. Public key bundles contain only the three public values.
|
||||
|
||||
Applications remain responsible for storage at rest. The `files` feature writes passphrase-protected keyrings to `.mk` files and public bundles to `.mpkb` files. On Unix, keyring files are created with owner-only `0600` permissions.
|
||||
Role-specific protocol boundaries should validate only the material they need:
|
||||
`validate_encryption()` checks that a KEM public/private pair corresponds, while
|
||||
`validate_full()` additionally requires a complete hybrid signing identity.
|
||||
This keeps partial browser keyrings usable without allowing an envelope sender
|
||||
to proceed with an invalid local decryption key.
|
||||
|
||||
Applications remain responsible for storage at rest. The `files` feature writes passphrase-protected keyrings to `.mk` files and public bundles to `.mpkb` files. Protected `.mk` files store the Argon2id identifier, parameters, salt, and AEAD ciphertext; they do not derive their key with HKDF. On Unix, keyring files are created with owner-only `0600` permissions.
|
||||
Restrict those files to the owning account and protect backups. Browser applications should treat the configured credential storage as sensitive application data.
|
||||
|
||||
## Resource Limits and Operational Controls
|
||||
|
||||
`Policy::default()` sets a 16 MiB application message limit and a 64 KiB handshake message limit. It also sets a 30 second read timeout, a 30 second maximum idle timeout, a receiver queue capacity of 1000, and a maximum of 128 concurrent stream tasks. Tune these values for the deployment and peer trust level.
|
||||
|
||||
The recursive codec applies additional defaults while parsing untrusted values:
|
||||
maximum nesting depth 64, 65,536 value nodes, 16 MiB per blob or envelope,
|
||||
and 64 encrypted recipients. Decrypted values are parsed with the same limits.
|
||||
|
||||
The host does not provide a general authentication-attempt rate limiter.
|
||||
Deploy authentication endpoints behind a rate-limiting proxy or add admission control through the host callbacks, including `GuestIdGenerator` where guest connections are permitted.
|
||||
|
||||
|
|
@ -161,3 +278,9 @@ Deploy authentication endpoints behind a rate-limiting proxy or add admission co
|
|||
- `AllowAuthentication` intentionally permits unauthenticated clients; it is not an authenticated-only mode.
|
||||
- Browser-side Rust panics cannot be recovered by JavaScript. The WASM client contains panic paths from internal `expect` calls.
|
||||
- The browser E2EE ratchet does not provide post-compromise security.
|
||||
- The ordinary encrypted-pipe offer does not provide forward secrecy; use the
|
||||
duplex handshake when recorded-call confidentiality after long-term KEM
|
||||
compromise is required.
|
||||
- Replay state is process-local by default for high-level subscriptions. Use a
|
||||
durable replay guard when protection must survive reloads or coordinate
|
||||
multiple receiver processes.
|
||||
|
|
|
|||
|
|
@ -63,7 +63,7 @@ When `require_pq` is true, both Ed25519 and ML-DSA-65 keys and signatures must b
|
|||
|
||||
`ReservedCommunicationType` means application code attempted to use a reserved wire ID. Use generated communication types instead of assigning protocol IDs manually. `MissingField` means a required typed field was not present.
|
||||
|
||||
`InvalidEncoding` indicates truncated, malformed, or structurally invalid bytes. `TooManyEntries` indicates that an array, container, or frame exceeds the codec's representable count or length. `CryptoFailed` indicates that signature verification or encrypted-container processing failed. The complete variant table is in [Errors](ERRORS.md).
|
||||
`InvalidEncoding` indicates truncated, malformed, duplicate-field, reserved-kind, or structurally invalid bytes. `TooManyEntries` indicates that an array, container, or frame exceeds the codec's representable count or length. Protection operations return typed errors for malformed envelopes, authentication failures, invalid signatures, and missing recipients. The complete variant table is in [Errors](ERRORS.md).
|
||||
|
||||
## Frames and Message Limits
|
||||
|
||||
|
|
|
|||
119
docs/TYPE-MAP.md
119
docs/TYPE-MAP.md
|
|
@ -6,21 +6,84 @@ This file documents the Type Map & Registry configuration used by the MTP protoc
|
|||
|
||||
Every transport frame is a four-byte big-endian length followed by one `CommunicationValue`. The length counts all bytes after the length field.
|
||||
|
||||
This is the only transport frame length prefix. Transports write the
|
||||
`CommunicationValue` bytes directly and do not add another length before this
|
||||
field. The close-frame sentinel occupies the same four-byte position.
|
||||
|
||||
```text
|
||||
u32 length
|
||||
u16 communication_type
|
||||
u8 flags
|
||||
u32 id if flag 0x04 is set
|
||||
u48 sender if flag 0x01 is set
|
||||
u48 receiver if flag 0x02 is set
|
||||
u8 signature_type if flag 0x10 is set
|
||||
... signature if flag 0x10 is set, length depends on signature_type
|
||||
... data container or encrypted payload
|
||||
[4 bytes total length]
|
||||
[2 bytes communication type]
|
||||
[1 byte flags]
|
||||
bit 0 = has ID
|
||||
bit 1 = has sender ID
|
||||
bit 2 = has receiver ID
|
||||
bits 3-7 must be zero
|
||||
[4 bytes ID] if bit 0
|
||||
[8 bytes sender ID] if bit 1
|
||||
[8 bytes receiver ID] if bit 2
|
||||
[DataValue payload]
|
||||
```
|
||||
|
||||
The flag values are `0x01` for sender, `0x02` for receiver, `0x04` for frame ID, `0x08` for encrypted data, `0x10` for a frame signature, and `0x20` for a signed encrypted container. Sender and receiver IDs are six-byte unsigned big-endian values. The `communication_type` and every container field use IDs from the negotiated `TypeMap`.
|
||||
The only defined flag values are `0x01` for ID, `0x02` for sender, and `0x04` for receiver. Unknown flag bits are rejected. IDs are full-width unsigned big-endian values: the correlation ID is `u32`, while sender and receiver IDs are `u64`. Encryption and signing are properties of the `DataValue` payload, never of the frame header.
|
||||
|
||||
Data values begin with a one-byte kind marker. MTP assigns `0x01` and `0x02` to boolean true and false, `0x03` to signed integers, `0x04` to unsigned integers, `0x05` to floats, `0x06` to UTF-8 strings, `0x07` to bytes, `0x08` to arrays, `0x09` to containers, `0x0A` through `0x0C` to crypto containers, and `0xFF` to null. Length-prefixed values use a four-byte big-endian payload length; container and array counts use two-byte big-endian counts.
|
||||
`Relay` is the reserved opaque application communication type. Relay frames
|
||||
omit the outer sender, expose only the next-hop receiver and transport
|
||||
correlation data, and carry the actual operation and application metadata in
|
||||
their protected payload.
|
||||
|
||||
## DataValue Wire Format
|
||||
|
||||
Every `DataValue` begins with a one-byte kind marker. MTP assigns `0x01` and `0x02` to boolean true and false, `0x03` to signed `i128`, `0x04` to unsigned `u128`, `0x05` to `f64`, `0x06` to UTF-8 strings, `0x07` to bytes, `0x08` to arrays, `0x09` to containers, `0x0A` to `Encrypted<Value>`, `0x0B` to `Signed<Value>`, and `0xFF` to null. Kind `0x0C` is reserved and rejected. All multibyte numeric values, counts, and lengths are big-endian.
|
||||
|
||||
Strings and bytes have a four-byte byte length. Arrays have a two-byte element count followed by that many self-delimiting values. The protection wrappers have the following canonical layouts.
|
||||
|
||||
```text
|
||||
Container
|
||||
|
||||
09
|
||||
[2 bytes element count]
|
||||
|
||||
repeat for each element:
|
||||
[2 bytes DataTypeId]
|
||||
[DataValue]
|
||||
```
|
||||
|
||||
Container field IDs must be unique. Each nested value is self-delimiting, so container elements have no generic per-element payload length.
|
||||
|
||||
```text
|
||||
Signed
|
||||
|
||||
0B
|
||||
[4 bytes wrapper length]
|
||||
|
||||
[1 byte signature algorithm]
|
||||
[1 byte purpose]
|
||||
[8 bytes signer ID]
|
||||
[signature]
|
||||
[DataValue]
|
||||
```
|
||||
|
||||
The wrapper length counts the bytes after the length field. Signature length is determined by the signature algorithm. The signature covers `MTP-DATA-SIGN-1 || algorithm || purpose || signer ID || serialized inner value`.
|
||||
|
||||
```text
|
||||
Encrypted
|
||||
|
||||
0A
|
||||
[4 bytes envelope length]
|
||||
|
||||
[1 byte encryption suite]
|
||||
[1 byte purpose]
|
||||
[2 bytes recipient count]
|
||||
|
||||
[recipient entry]
|
||||
...
|
||||
|
||||
[encrypted DataValue bytes]
|
||||
```
|
||||
|
||||
The envelope length counts the bytes after the length field. A recipient entry is an unlabeled fixed-size KEM ciphertext and wrapped content-encryption key; both lengths are determined by the selected suite. The encrypted bytes are the AEAD output for the complete serialized inner `DataValue`.
|
||||
|
||||
Protection nesting directly represents both signer-visibility choices: `Encrypted(Signed(Container))` keeps signer metadata private, while `Signed(Encrypted(Container))` exposes it. A frame with no outer sender and an `Encrypted(Signed(Container))` payload uses sealed sender. Sealed sender adds no flag or distinct wire type.
|
||||
|
||||
## TypeMap & Compile-Time Type Safety
|
||||
|
||||
|
|
@ -43,6 +106,12 @@ export default defineConfig({
|
|||
|
||||
Rust and manual WASM builds can set `MTP_TYPE_MAPS` directly (see [Customizing Type Maps in Downstream Projects](#customizing-type-maps-in-downstream-projects)).
|
||||
|
||||
For browser builds, `protocol_version` selects the one application map compiled
|
||||
into that WASM client. The Vite-generated `mtp/type-map` module contains the
|
||||
reserved MTP names and the application names from that selected version only;
|
||||
the selected version must be present in `type_maps`. This keeps its TypeScript
|
||||
unions aligned with the client runtime.
|
||||
|
||||
### Using Generated Enums
|
||||
|
||||
After editing the config and rebuilding, `CommunicationType` and `DataType` enums are generated automatically. Use them in code:
|
||||
|
|
@ -50,11 +119,17 @@ After editing the config and rebuilding, `CommunicationType` and `DataType` enum
|
|||
```rust
|
||||
use mtp::type_map::{CommunicationType, DataType, TypeMap};
|
||||
|
||||
let tm = TypeMap::v2_0();
|
||||
let tm = TypeMap::v3_0();
|
||||
let id = tm.data_id_enum(DataType::SomeType).unwrap();
|
||||
```
|
||||
|
||||
The enums are a **union across all versions**; every type name from every version is a variant. The version-specific `TypeMap` maps each variant to the correct wire ID for that version. For a type absent from a selected version, the lookup returns `None`.
|
||||
For native builds with the `registry` feature, the enums are a **union across
|
||||
all versions**; every type name from every version is a variant. The
|
||||
version-specific `TypeMap` maps each variant to the correct wire ID for that
|
||||
version. For a type absent from a selected version, the lookup returns `None`.
|
||||
Browser-generated TypeScript unions intentionally differ: they contain only
|
||||
the selected `protocol_version` plus reserved names, matching the WASM client
|
||||
compiled by the Vite plugin.
|
||||
|
||||
Encoding/decoding uses a `TypeMap` to resolve type names to wire IDs:
|
||||
|
||||
|
|
@ -70,23 +145,19 @@ let decoded = decode(&bytes, &tm).unwrap();
|
|||
```
|
||||
|
||||
```rust
|
||||
let tm_v2 = TypeMap::v2_0();
|
||||
assert!(tm_v2.data_id_enum(DataType::SomeType).is_some()); // defined in v2.0
|
||||
assert!(tm_v2.data_id_enum(DataType::ExampleType).is_none()); // NOT in v2.0
|
||||
|
||||
let tm_v1 = TypeMap::v1_0();
|
||||
assert!(tm_v1.data_id_enum(DataType::ExampleType).is_some()); // defined in v1.0
|
||||
let tm_v3 = TypeMap::v3_0();
|
||||
assert!(tm_v3.data_id_enum(DataType::SomeType).is_some());
|
||||
```
|
||||
|
||||
When communicating with a peer on another version, encode only variants that map in the negotiated version. If an incoming frame names a type absent from the selected map, reject it as a protocol or type-map compatibility error; do not reinterpret its wire ID using another version's map. Keep old IDs stable, register both versions during migration, and remove a version only after its clients have moved.
|
||||
When communicating with a peer on another version, encode only variants that map in the negotiated version. If an incoming frame names a type absent from the selected map, reject it as a protocol or type-map compatibility error; do not reinterpret its wire ID using another version's map. The self-delimiting codec begins at protocol version `3.0`; older versions are not codec fallbacks.
|
||||
|
||||
### Forward/Backward Compatibility Between Versions
|
||||
|
||||
Because enums are a union of all types across versions, a variant might exist that has no wire mapping in the *negotiated* version:
|
||||
|
||||
```
|
||||
v2.0 client sends DataType::SomeType → host encodes with v2.0 TypeMap → wire ID 32
|
||||
v2.0 host receives DataType::ExampleType (from v1.0 client) → not in v2.0 TypeMap → None → Error
|
||||
v3.0 client sends DataType::SomeType → host encodes with v3.0 TypeMap → wire ID 32
|
||||
v3.0 host receives an unsupported pre-v3.0 peer → version negotiation error
|
||||
```
|
||||
|
||||
Encoding a frame with an unmapped communication or data type returns `CodecError::UnknownCommunicationType` or `CodecError::UnknownDataType`. Select a mapped variant from the compiled-in version before sending it.
|
||||
|
|
@ -109,10 +180,10 @@ let registry = Registry::builtin();
|
|||
let codec = VersionedCodec::new(registry);
|
||||
|
||||
// Encode with a specific version
|
||||
let bytes = codec.encode(&value, Version(2, 0)).unwrap();
|
||||
let bytes = codec.encode(&value, Version(3, 0)).unwrap();
|
||||
|
||||
// Decode with a specific version
|
||||
let decoded = codec.decode(&bytes, Version(2, 0)).unwrap();
|
||||
let decoded = codec.decode(&bytes, Version(3, 0)).unwrap();
|
||||
```
|
||||
|
||||
## Customizing Type Maps in Downstream Projects
|
||||
|
|
|
|||
|
|
@ -30,6 +30,11 @@ import { mtp } from "mtp/vite";
|
|||
|
||||
Browser apps provide their own type map. The Vite plugin runs `wasm-pack` during dev and build with `MTP_TYPE_MAPS` set, writes generated output under `node_modules/.vite/mtp/` by default, and aliases `mtp/raw` plus `mtp/type-map` to that generated output. Configuration: [Type Map](TYPE-MAP.md).
|
||||
|
||||
The browser build uses the map named by `protocol_version` and includes the
|
||||
reserved MTP names. It does not advertise application names from other map
|
||||
versions, because the generated WASM client is compiled for that one protocol
|
||||
version. The selected version must exist in `type_maps`.
|
||||
|
||||
You do not need to publish, fork, or copy an app-specific generated WASM package.
|
||||
|
||||
The [web client example](../example/web-client/src/main.ts) shows the entry point. Its [Vite configuration](../example/web-client/vite.config.ts) shows the generated binding integration.
|
||||
|
|
@ -100,7 +105,8 @@ if (!MTPClient.isSupported()) {
|
|||
| `pings` | `false` | Protocol pings, or an object with `intervalMs`. |
|
||||
| `logger` | No-op | Receives SDK state and error events. |
|
||||
| `sessionStorage` | In-memory | E2EE session state storage. |
|
||||
| `encryptedDeviceSecretProvider` | In-memory | Device-secret storage for E2EE. |
|
||||
| `encryptedSecretProvider` | In-memory | Independent caller-managed encrypted secret storage. |
|
||||
| `defaultSignatureVerificationPolicy` | `"ed25519"` | Receiver policy for protected signatures. |
|
||||
|
||||
`wasm` selects a custom generated WASM module. `MTPClient.create` validates positive safe-integer values for the numeric limits and timeout options.
|
||||
|
||||
|
|
@ -112,7 +118,222 @@ The browser SDK uses WebTransport and JavaScript promises. The native client use
|
|||
|
||||
The `storage` option supplies the credential adapter. The adapter stores the client ID and serialized keyring after registration and returns them for later connections. The SDK does not select `localStorage` or IndexedDB for an application. Treat the serialized keyring as private key material.
|
||||
|
||||
`sessionStorage` and `encryptedDeviceSecretProvider` are separate E2EE session stores. The latter exchanges `EncryptedDeviceSecretRecord` values through `setEncryptedDeviceSecret` and `getEncryptedDeviceSecret`; the application chooses the backing store and protects its wrapping key.
|
||||
`sessionStorage` and `encryptedSecretProvider` are separate caller-managed
|
||||
stores. The latter exchanges `MTPEncryptedSecretRecord` values through
|
||||
`setEncryptedSecret`, `getEncryptedSecret`, and `deleteEncryptedSecret`.
|
||||
`MTPSessionManager` does not automatically route session state through the
|
||||
provider. If session material must be encrypted at rest, the caller must make
|
||||
that coordination explicit in its `MTPSessionStorage` implementation. Secret
|
||||
IDs are opaque to MTP, so a caller can map its own state to the ID while
|
||||
choosing the backing store and protecting its wrapping key.
|
||||
|
||||
### Direct Protected Messages
|
||||
|
||||
Use `sendProtected` when the destination is the frame receiver and no
|
||||
intermediate relay needs a separately encrypted metadata layer. It keeps the
|
||||
application communication type on the outer frame and encrypts an MTP-owned
|
||||
signed envelope for the exact recipient bundles supplied by the caller. The
|
||||
envelope authenticates `ProtectedVersion`, `MessageType`, `FinalRecipientId`,
|
||||
`MessageId`, `CreatedAt`, and `Content`. The opening operation checks the
|
||||
authenticated type and final recipient against the outer frame.
|
||||
|
||||
```typescript
|
||||
await client.sendProtected("ProtectedMessage", { Content: "hello" }, {
|
||||
receiverId: recipientId,
|
||||
recipients: [recipientPublicKey],
|
||||
signaturePurpose: 0x40,
|
||||
encryptionPurpose: 0x41,
|
||||
exposeSender: false,
|
||||
});
|
||||
```
|
||||
|
||||
The protection purposes are application-defined domain-separation values.
|
||||
`exposeSender` controls only the outer frame sender; the protected value remains
|
||||
signed in either case. If `identity` is omitted, the SDK uses stored registered
|
||||
credentials and rejects the operation when no usable protection identity is
|
||||
available.
|
||||
|
||||
An unauthenticated connection can still send a protected value when the caller
|
||||
provides an explicit `identity` with the signer ID and keyring. The connection's
|
||||
authentication state and the protected signer's identity are independent.
|
||||
|
||||
When `signatureSuite` is omitted, protected send helpers use Ed25519 even when
|
||||
the signing keyring also contains post-quantum keys. This matches the default
|
||||
receiver policy. Use `signatureSuite: "dual"` together with
|
||||
`signaturePolicy: "dual"` when both sides explicitly require hybrid
|
||||
signatures.
|
||||
|
||||
Open a direct protected frame with the recipient keyring and a resolver that
|
||||
receives the claimed, unverified signer ID only as a trusted-key lookup key:
|
||||
|
||||
```typescript
|
||||
const message = await client.openProtected(frame, {
|
||||
recipient: {
|
||||
id: recipientId,
|
||||
keyring: recipientKeyring,
|
||||
keyringHistory: previousRecipientKeyrings,
|
||||
},
|
||||
expectedReceiverId: recipientId,
|
||||
expectedSignerId: signerId,
|
||||
resolveSignerPublicKeys: (id) => signerDirectory.get(id) ?? [],
|
||||
signaturePolicy: "dual",
|
||||
signaturePurpose: 0x40,
|
||||
encryptionPurpose: 0x41,
|
||||
replayGuard,
|
||||
});
|
||||
|
||||
console.log(message.type, message.signerId, message.messageId, message.data);
|
||||
```
|
||||
|
||||
`protectedVersion`, `finalRecipientId`, `signerId`, `messageId`, and `createdAt`
|
||||
are taken from the verified protected envelope. `outerSender`, when present,
|
||||
must equal the authenticated signer.
|
||||
Protected application data may be any supported MTP `DataValue`, including
|
||||
scalar, byte, array, and container values. Direct opening uses a bounded
|
||||
process-local duplicate-suppression guard by default. The bounded cache can
|
||||
evict old entries, so supply a durable `replayGuard` keyed by authenticated
|
||||
signer and message ID when replay protection must survive eviction, reloads, or
|
||||
multiple receiver processes. The guard also receives authenticated
|
||||
`createdAt` metadata, which is not part of the replay key.
|
||||
`subscribeProtected` uses the same opening and verification path:
|
||||
|
||||
```typescript
|
||||
const unsubscribe = client.subscribeProtected(
|
||||
"ProtectedMessage",
|
||||
(message, frame) => handleMessage(message.data, frame),
|
||||
{
|
||||
recipient: { id: recipientId, keyring: recipientKeyring },
|
||||
resolveSignerPublicKeys: (id) => signerDirectory.get(id) ?? [],
|
||||
signaturePolicy: "dual",
|
||||
signaturePurpose: 0x40,
|
||||
encryptionPurpose: 0x41,
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
Each `subscribeProtected` registration owns its own bounded default replay
|
||||
guard, so multiple handlers receive the same raw frame through the WASM
|
||||
fan-out dispatcher. Pass the same caller-owned `replayGuard` deliberately when
|
||||
several subscriptions should share replay state.
|
||||
|
||||
### Sealed Relay Messages
|
||||
|
||||
`sendSealedRelay` uses the reserved opaque `Relay` communication type. Its
|
||||
inner message type must be an application communication type, not an MTP
|
||||
control type. The outer frame contains no sender and exposes only the next-hop
|
||||
receiver. The
|
||||
signed relay metadata contains the generic `signerId`, `finalRecipientId`,
|
||||
`messageId`, `createdAt`, application `metadata`, and an opaque encrypted
|
||||
content value. `createdAt` is generated as Unix epoch milliseconds. For
|
||||
example, `2026-08-11T12:00:00.000Z` is `1786449600000`.
|
||||
|
||||
```typescript
|
||||
const data = { Content: "hello" };
|
||||
|
||||
await client.sendSealedRelay("ProtectedMessage", data, {
|
||||
finalRecipientId,
|
||||
nextHopId,
|
||||
metadataRecipients: [
|
||||
relayPublicKey,
|
||||
recipientPublicKey,
|
||||
],
|
||||
contentRecipients: [
|
||||
recipientPublicKey,
|
||||
],
|
||||
metadata: {
|
||||
ExampleMetadata: "routing context",
|
||||
},
|
||||
});
|
||||
|
||||
client.subscribeSealedRelay(
|
||||
"ProtectedMessage",
|
||||
(message, frame) => handleMessage(message.data, frame),
|
||||
{
|
||||
recipient: {
|
||||
id: finalRecipientId,
|
||||
keyring: recipientKeyring,
|
||||
},
|
||||
expectedSignerId: signerId,
|
||||
resolveSignerPublicKeys: () => [senderPublicKey],
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
The caller supplies the exact metadata and content recipient sets; the SDK
|
||||
does not infer application topology. Set `signaturePolicy: "dual"` to require
|
||||
hybrid signatures explicitly, and install a durable `replayGuard` so a valid
|
||||
`(signerId, messageId)` is dispatched only once.
|
||||
|
||||
Each sealed-relay or metadata subscription likewise gets an independent
|
||||
bounded default guard. This preserves fan-out when multiple handlers inspect
|
||||
the same outer `Relay` frame; an explicitly supplied guard is shared by the
|
||||
subscriptions that receive it.
|
||||
|
||||
Applications choose between direct protected delivery and sealed relay based
|
||||
on topology and metadata-access requirements. Prefer `sendProtected` for a
|
||||
direct destination. Use `sendSealedRelay` when a next hop must route or store a
|
||||
message and the application needs metadata recipients to differ from content
|
||||
recipients. Neither construction requires connection authentication, although
|
||||
the host can associate an authenticated connection with its registered MTP
|
||||
identity.
|
||||
|
||||
For metadata-only access, call `openRelayMetadata` or subscribe with
|
||||
`subscribeRelayMetadata`. These operations authenticate the metadata and
|
||||
expose `encryptedContent` for forwarding without attempting content
|
||||
decryption. A final recipient calls `openRelayContent` after metadata
|
||||
verification; the returned `MTPVerifiedRelayContent` includes the application
|
||||
type and data plus `signerId`, `finalRecipientId`, `messageId`, `createdAt`,
|
||||
and generic metadata fields. These are authenticated protected identities, not
|
||||
the clear outer sender and next-hop receiver.
|
||||
Relay content inherits the authenticated metadata's `signaturePolicy` when no
|
||||
content override is supplied. A different content policy is rejected so the
|
||||
two relay layers cannot be verified under conflicting rules.
|
||||
Metadata passed to a `subscribeRelayMetadata` handler is callback-scoped and is
|
||||
disposed after the handler resolves. Do not retain it for a later
|
||||
`openRelayContent` call; use `openRelayMetadata` directly when a longer-lived
|
||||
verified capability is needed, and call `dispose()` when finished.
|
||||
When signer key history is used, `signerPublicKeys` exposes the trusted
|
||||
candidates, `matchedSignerKeyIndex` identifies the key that verified the
|
||||
metadata, and `matchedSignerPublicKey` returns that exact bundle.
|
||||
|
||||
Protected receive operations accept an optional `recipient` decryption
|
||||
identity. Its `keyring` controls decryption and its optional `id` is used only
|
||||
for final-recipient validation. The identity is independent from connection
|
||||
authentication. Metadata opening does not require the identity ID to match the
|
||||
clear next-hop receiver, so a forwarded frame can be opened by a metadata
|
||||
recipient or final recipient with the appropriate keyring. When `recipient` is
|
||||
omitted, stored registered credentials remain the convenience fallback.
|
||||
|
||||
To open values encrypted for a rotated recipient, provide `keyringHistory` on
|
||||
the decryption identity. The current `keyring` is tried first, followed by
|
||||
history entries from newest to oldest. Exact duplicate byte sequences are
|
||||
removed without changing the caller's input arrays. An empty current keyring
|
||||
or an empty history entry is rejected.
|
||||
|
||||
Generic MTP `DataValue` inputs accept `bigint` for exact integer values. An
|
||||
integral JavaScript `number` outside the safe-integer range is rejected, so it
|
||||
cannot silently become an imprecise float. Use `bigint` for large signed or
|
||||
unsigned integers.
|
||||
|
||||
For streams, prefer `createEncryptedPipe` and `acceptEncryptedPipe`; they bind
|
||||
the actual pipe ID and local identity automatically. The lower-level
|
||||
`initiateMTPPipeSession` API also accepts multiple recipient bundles for a
|
||||
group bootstrap. Group membership changes require a fresh session ID and
|
||||
recipient set. Live calls that need forward secrecy can use the exported
|
||||
duplex `initiateMTPForwardSecurePipeSession` and
|
||||
`acceptMTPForwardSecurePipeSession` helpers.
|
||||
|
||||
The convenience pipe methods intentionally require registered client
|
||||
credentials because they use the connection's registered identity as the
|
||||
endpoint identity. Use the lower-level session functions when transport
|
||||
authentication and cryptographic endpoint identity must remain independent.
|
||||
|
||||
Receive-side signature policy is independent from the recipient keyring. Use
|
||||
`signaturePolicy` on protected receive and encrypted-pipe accept operations,
|
||||
or configure `defaultSignatureVerificationPolicy` on the client. The sender's
|
||||
`signatureSuite` selects how local values are signed and is a separate choice.
|
||||
Both sender and receiver default to Ed25519; `dual` is always an explicit
|
||||
choice on each side.
|
||||
|
||||
### Native and Browser Certificate Checks
|
||||
|
||||
|
|
@ -262,7 +483,10 @@ Use `pings: true` for the default interval.
|
|||
|
||||
## Pipes
|
||||
|
||||
Pipes are raw binary streams over QUIC. A pipe starts with a lightweight `PipeRequest` handshake frame, then the stream carries raw bytes with zero per-frame overhead. Pipes are unidirectional; the peer that initiates the pipe writes, and the peer that accepts it reads.
|
||||
Pipes are byte-oriented streams over WebTransport. The `PipeRequest` type and
|
||||
description are clear transport metadata; raw stream bytes are not protected
|
||||
by MTP. For sensitive calls, files, or application streams, wrap the accepted
|
||||
pipe with `MTPEncryptedPipeWriter` or `MTPEncryptedPipeReader`.
|
||||
|
||||
### Outgoing Pipes
|
||||
|
||||
|
|
@ -284,6 +508,41 @@ await writer.close();
|
|||
|
||||
`writer.close()` sends a QUIC stream FIN. `writer.abort()` resets the stream abruptly. Each `write` resolves when the chunk has been handed to the transport; it does not wait for the peer to consume it.
|
||||
|
||||
### Encrypted Pipe Records
|
||||
|
||||
`initiateMTPPipeSession` and `acceptMTPPipeSession` perform the signed/KEM
|
||||
protected pipe-session offer and return the encrypted record wrapper. The
|
||||
offer binds the session ID, pipe ID, endpoint IDs, direction, and purpose. Do
|
||||
not derive the initial chain key from the clear description or pipe ID alone.
|
||||
|
||||
```typescript
|
||||
import {
|
||||
initiateMTPPipeSession,
|
||||
} from "mtp";
|
||||
|
||||
const encryptedWriter = await initiateMTPPipeSession(
|
||||
writer,
|
||||
{
|
||||
sessionId: new TextEncoder().encode(`file-transfer/${writer.pipeId}`),
|
||||
pipeId: writer.pipeId,
|
||||
senderId: ownClientId,
|
||||
recipientId: hostClientId,
|
||||
purpose: 0x40,
|
||||
direction: 0,
|
||||
},
|
||||
ownKeyring,
|
||||
hostPublicKeyBundle,
|
||||
);
|
||||
await encryptedWriter.writeRecord(chunk);
|
||||
await encryptedWriter.close();
|
||||
```
|
||||
|
||||
`writeRecord` and `readRecord` use XChaCha20-Poly1305 with ordered sequence
|
||||
numbers bound to the session context. Each record advances an HKDF chain and
|
||||
uses a one-use message key. Record insertion, removal, reordering, or
|
||||
modification fails authentication. The wrapper is intentionally separate from
|
||||
the raw `PipeWriter`/`PipeReader` transport primitives.
|
||||
|
||||
The handle and writer expose `pipeId` and `description`:
|
||||
|
||||
```typescript
|
||||
|
|
@ -330,8 +589,8 @@ console.log(reader.pipeId, reader.description);
|
|||
|
||||
1. The initiator calls `createPipe(description)`; the SDK sends a `PipeRequest` frame with a random `pipeId` and the description.
|
||||
2. The receiver's `setOnPipeRequest` callback fires with `{ pipeId, description }`.
|
||||
3. The receiver calls `acceptPipe(pipeId)`; the SDK sends a `PipeResponse` with `Accepted = true` and opens a new unidirectional stream for raw data.
|
||||
4. The initiator's `handle.wait()` resolves with a `PipeWriter` bound to that stream.
|
||||
3. The receiver calls `acceptPipe(pipeId)`; the SDK sends a `PipeResponse` with `Accepted = true` and opens a new unidirectional stream for byte transport.
|
||||
4. The initiator's `handle.wait()` resolves with a `PipeWriter` bound to that stream. Sensitive applications then perform their signed/encrypted session-key setup and construct an encrypted record wrapper.
|
||||
5. If the receiver calls `denyPipe(pipeId)`, `handle.wait()` resolves with `null`.
|
||||
|
||||
Pipes share the same WebTransport session as message frames; they do not need a separate connection.
|
||||
|
|
@ -417,9 +676,13 @@ Raw crypto and key helpers include:
|
|||
- `keyring_generate()`
|
||||
- `keyring_from_ed25519(secretKey, publicKey)`
|
||||
- `WasmKeyring.from_bytes(bytes)` and `keyring.to_bytes()`
|
||||
- `keyring.validate_encryption()` for envelope decryption roles
|
||||
- `keyring.validate_full()` for complete hybrid identities
|
||||
- `WasmPublicKeyBundle.from_bytes(bytes)` and `bundle.to_bytes()`
|
||||
- `WasmEd25519Signer`
|
||||
- `WasmChaCha20Poly1305`
|
||||
- `sign_data_value_with_keyring` and `verify_data_value_with_policy` (both require an explicit signature suite), plus `encrypt_data_value`, `encrypt_data_value_for_recipients`, and `decrypt_data_value`
|
||||
- `parse_data_value` and `encode_data_value`
|
||||
- `wasm_sha256`, `wasm_sha256_double`, `wasm_hkdf_expand`, and `wasm_derive_encryption_key`
|
||||
|
||||
Raw authenticated login and registration map directly to the Rust WASM layer:
|
||||
|
|
@ -463,4 +726,4 @@ A `WasmClient` manages one active WebTransport session. Create a new instance fo
|
|||
|
||||
### State Management
|
||||
|
||||
A `WasmClient` owns one active WebTransport session. Create a separate client for each independent connection. Call `free()` or `[Symbol.dispose]()` on raw WASM objects when the application no longer needs them. SDK session and device secret persistence are documented in [Security](SECURITY.md#browser-end-to-end-encryption).
|
||||
A `WasmClient` owns one active WebTransport session. Create a separate client for each independent connection. Call `free()` or `[Symbol.dispose]()` on raw WASM objects when the application no longer needs them. SDK session and encrypted secret persistence are documented in [Security](SECURITY.md#browser-end-to-end-encryption).
|
||||
|
|
|
|||
Loading…
Reference in a new issue