Initial commit

This commit is contained in:
Alex Emmet 2026-06-20 01:55:31 +02:00
commit 0bbcab5727
24 changed files with 1416 additions and 0 deletions

2
.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
**target/
**Cargo.lock

12
Cargo.toml Normal file
View file

@ -0,0 +1,12 @@
[workspace]
members = [
"common",
"crypto",
"registry",
"type-map",
"codec",
"transport",
"host",
"client",
]
resolver = "3"

60
README.md Normal file
View file

@ -0,0 +1,60 @@
# **M**ethanium **T**ransport **P**rotocol
**MTP** is a **m**odular **t**ransport **p**rotocol by Methanium.
## Crates
- Common
- Crypto
- TypeMap
- Registry
- Codec
- Transport
- Host
- Client
## Registry & TypeMaps
A TypeMap maps Communication-Types to a byte, and Data-Types to 2 bytes.
A Communication- or Data-Type is a String to be used Plain in the code, while the Byte is transmitted.
The Registries map a TypeMap to a specific version of MTP.
## Codec
The Codec crate handles encoding and decoding of MTP packets using Communication Types and Data Types from the Registry.
The Codec has these Data Values:
- Container
- Encrypted Container
- Signed Integer
- Unsigned Integer
- Boolean
- Signed Float
- String
- Array
- Binary (List of Bytes)
The Codec uses the Crypto crate to encrypt and decrypt Encrypted Containers.
## Transport
The Transport crate wraps QUIC.
## Host & Client
The host creates a quic Server that listens for incoming connections.
The host also manages the registry and versions to allow for backwards compatibility with clients.
The client connects to a quic Server and uses a registry to communicate with the host.
The Client may be Outdated while the Host is updated.
## Common
Common Handles Logging and Error Handling. Personal information reported to Common will be anonmized.
## Crypto Stack
| Crate | Audited? | Notes |
| ---------------- | -------- | ------------------------------------------------ |
| ml-dsa | No | NIST vectors pass; regression bug fixed Jan 2026 |
| ed25519-dalek | Yes | Used by Signal, Diem |
| chacha20poly1305 | Yes | NCC Group audit, Dec 2019 |
| aes-gcm | Yes | NCC Group audit, Dec 2019 |
| hkdf | No | Simple construction; well-reviewed |
| sha2 | No | Standard construction; widely reviewed |
| zeroize | No | Simple; widely used |
| mlkem-tls | No | mlkem-rs backend unaudited |

6
client/Cargo.toml Normal file
View file

@ -0,0 +1,6 @@
[package]
name = "client"
version = "0.1.0"
edition = "2024"
[dependencies]

14
client/src/lib.rs Normal file
View file

@ -0,0 +1,14 @@
pub fn add(left: u64, right: u64) -> u64 {
left + right
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
let result = add(2, 2);
assert_eq!(result, 4);
}
}

19
codec/Cargo.toml Normal file
View file

@ -0,0 +1,19 @@
[package]
name = "mtp-codec"
version = "0.1.0"
edition = "2024"
[dependencies]
type-map = { path = "../type-map" }
common = { path = "../common" }
mtp-crypto = { path = "../crypto", optional = true }
base64 = "*"
byteorder = "*"
rand = { version = "*", features = ["std", "std_rng"] }
[features]
default = []
# Enables the Registry module and versioned codec wrapper
registry = []
# Enables EncryptedContainer support via mtp-crypto
crypto = ["dep:mtp-crypto"]

View file

@ -0,0 +1,25 @@
// Communication types used in the MTP protocol.
// The numeric values are part of the onwire format.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[allow(non_camel_case_types)]
pub enum CommunicationType {
ping = 0x01,
message_send = 0x02,
update = 0x03,
}
impl CommunicationType {
pub fn as_number(self) -> u8 {
self as u8
}
pub fn from_number(n: u8) -> Self {
match n {
0x01 => CommunicationType::ping,
0x02 => CommunicationType::message_send,
0x03 => CommunicationType::update,
_ => CommunicationType::ping,
}
}
}

View file

@ -0,0 +1,300 @@
use byteorder::{BigEndian, ReadBytesExt, WriteBytesExt};
use std::collections::BTreeMap;
use std::io::{Cursor, Read};
use crate::data_value::DataValue;
use crate::rand_u32;
use type_map::{CommTypeId, DataTypeId};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CommunicationValue {
id: u32,
comm_type: CommTypeId,
sender: u64,
receiver: u64,
data: BTreeMap<DataTypeId, DataValue>,
}
#[allow(dead_code)]
impl CommunicationValue {
pub fn new(comm_type: CommTypeId) -> Self {
Self {
id: rand_u32(),
comm_type,
sender: 0,
receiver: 0,
data: BTreeMap::new(),
}
}
pub fn with_id(mut self, p0: u32) -> Self {
self.id = p0;
self
}
pub fn get_id(&self) -> u32 {
self.id
}
pub fn with_sender(mut self, sender: u64) -> Self {
self.sender = sender;
self
}
pub fn get_sender(&self) -> u64 {
self.sender
}
pub fn with_receiver(mut self, receiver: u64) -> Self {
self.receiver = receiver;
self
}
pub fn get_receiver(&self) -> u64 {
self.receiver
}
pub fn get_type(&self) -> CommTypeId {
self.comm_type
}
pub fn is_type(&self, p0: CommTypeId) -> bool {
self.comm_type == p0
}
pub fn add_data(mut self, data: DataTypeId, value: DataValue) -> Self {
self.data.insert(data, value);
self
}
pub fn get_data(&self, data_type: DataTypeId) -> &DataValue {
self.data.get(&data_type).unwrap_or(&DataValue::Null)
}
pub fn get_data_container(&self) -> &BTreeMap<DataTypeId, DataValue> {
&self.data
}
}
impl CommunicationValue {
/*
* Frame format (strict new format):
* [4 bytes u32 total_length] // number of bytes after this field
* [1 byte communication_type]
* [1 byte flags]
* [optional 4 bytes id] // if flags bit2 set
* [optional 6 bytes sender] // if flags bit0 set
* [optional 6 bytes receiver]// if flags bit1 set
* [data container bytes...]
*
* Flags:
* bit0 => has sender
* bit1 => has receiver
* bit2 => has id
*/
pub fn to_bytes(&self) -> Vec<u8> {
let mut payload = Vec::new();
let has_sender = self.sender != 0;
let has_receiver = self.receiver != 0;
let has_id = self.id != 0;
let mut flags: u8 = 0;
if has_sender {
flags |= 0b0000_0001;
}
if has_receiver {
flags |= 0b0000_0010;
}
if has_id {
flags |= 0b0000_0100;
}
payload.push(self.comm_type.0);
payload.push(flags);
if has_id {
let _ = payload.write_u32::<BigEndian>(self.id);
}
if has_sender {
let sender_be = self.sender.to_be_bytes();
payload.extend_from_slice(&sender_be[2..]); // 6 bytes
}
if has_receiver {
let receiver_be = self.receiver.to_be_bytes();
payload.extend_from_slice(&receiver_be[2..]); // 6 bytes
}
let container_value = DataValue::container_from_map(&self.data);
let container_bytes = container_value.to_bytes();
payload.extend_from_slice(&container_bytes);
let mut frame = Vec::with_capacity(4 + payload.len());
let _ = frame.write_u32::<BigEndian>(payload.len() as u32);
frame.extend_from_slice(&payload);
frame
}
pub fn from_bytes(bytes: &[u8]) -> Option<Self> {
let mut cursor = Cursor::new(bytes);
let total_len = cursor.read_u32::<BigEndian>().ok()? as usize;
if bytes.len() < 4 + total_len {
return None;
}
let frame_end = 4 + total_len;
let comm_type_num = cursor.read_u8().ok()?;
let comm_type = CommTypeId(comm_type_num);
let flags = cursor.read_u8().ok()?;
let has_sender = (flags & 0b0000_0001) != 0;
let has_receiver = (flags & 0b0000_0010) != 0;
let has_id = (flags & 0b0000_0100) != 0;
let id = if has_id {
cursor.read_u32::<BigEndian>().ok()?
} else {
0
};
let sender = if has_sender {
let mut buf = [0u8; 8];
cursor.read_exact(&mut buf[2..]).ok()?;
u64::from_be_bytes(buf)
} else {
0
};
let receiver = if has_receiver {
let mut buf = [0u8; 8];
cursor.read_exact(&mut buf[2..]).ok()?;
u64::from_be_bytes(buf)
} else {
0
};
let pos = cursor.position() as usize;
if pos > frame_end {
return None;
}
let data_bytes = &bytes[pos..frame_end];
let data_value = DataValue::from_bytes(data_bytes)?;
let data = data_value.as_map()?;
Some(Self {
id,
comm_type,
sender,
receiver,
data,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::data_value::DataValue;
use type_map::{CommTypeId, DataTypeId};
fn roundtrip(cv: CommunicationValue) -> CommunicationValue {
let bytes = cv.to_bytes();
let decoded = CommunicationValue::from_bytes(&bytes).expect("failed to deserialize");
let bytes2 = decoded.to_bytes();
assert_eq!(bytes, bytes2);
decoded
}
#[test]
fn test_flags_and_order_without_optional() {
let cv = CommunicationValue::new(CommTypeId(1)).with_id(0);
let bytes = cv.to_bytes();
// [u32 len][type][flags]...
assert!(bytes.len() >= 6);
let mut c = Cursor::new(bytes.as_slice());
let total_len = c.read_u32::<BigEndian>().expect("read len");
assert_eq!(total_len as usize + 4, bytes.len());
let typ = c.read_u8().expect("read type");
assert_eq!(typ, 1);
let flags = c.read_u8().expect("read flags");
assert_eq!(flags & 0b0000_0111, 0);
}
#[test]
fn test_flags_and_order_with_all_optional() {
let cv = CommunicationValue::new(CommTypeId(2))
.with_id(0xAABBCCDD)
.with_sender(0x0000_1122_3344_5566)
.with_receiver(0x0000_6677_8899_AABB);
let bytes = cv.to_bytes();
let mut c = Cursor::new(bytes.as_slice());
let total_len = c.read_u32::<BigEndian>().expect("len");
assert_eq!(total_len as usize + 4, bytes.len());
let typ = c.read_u8().expect("type");
assert_eq!(typ, 2);
let flags = c.read_u8().expect("flags");
assert_eq!(flags & 0b0000_0111, 0b0000_0111);
let id = c.read_u32::<BigEndian>().expect("id");
assert_eq!(id, 0xAABBCCDD);
let mut sender6 = [0u8; 6];
c.read_exact(&mut sender6).expect("sender");
assert_eq!(sender6, [0x11, 0x22, 0x33, 0x44, 0x55, 0x66]);
let mut receiver6 = [0u8; 6];
c.read_exact(&mut receiver6).expect("receiver");
assert_eq!(receiver6, [0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB]);
}
#[test]
fn test_roundtrip_complex() {
let cv = CommunicationValue::new(CommTypeId(3))
.with_id(1234)
.with_sender(111)
.with_receiver(222)
.add_data(DataTypeId(1), DataValue::Str("alice".to_string()))
.add_data(DataTypeId(2), DataValue::SignedNumber(42))
.add_data(DataTypeId(3), DataValue::BoolTrue)
.add_data(
DataTypeId(4),
DataValue::Array(vec![DataValue::SignedNumber(1), DataValue::SignedNumber(2)]),
);
let decoded = roundtrip(cv.clone());
assert_eq!(decoded.get_id(), 1234);
assert_eq!(decoded.get_sender(), 111);
assert_eq!(decoded.get_receiver(), 222);
assert_eq!(decoded.get_type(), CommTypeId(3));
assert_eq!(
decoded.get_data(DataTypeId(1)),
&DataValue::Str("alice".to_string())
);
assert_eq!(
decoded.get_data(DataTypeId(2)),
&DataValue::SignedNumber(42)
);
}
#[test]
fn test_corrupted_length_returns_none() {
let mut bad = vec![0u8; 8];
// total_length claims more than available
bad[0..4].copy_from_slice(&(1000u32.to_be_bytes()));
assert!(CommunicationValue::from_bytes(&bad).is_none());
}
}

663
codec/src/data_value.rs Normal file
View file

@ -0,0 +1,663 @@
use base64::Engine;
use base64::engine::general_purpose;
use byteorder::{BigEndian, ReadBytesExt, WriteBytesExt};
use std::collections::BTreeMap;
use std::hash::{Hash, Hasher};
use std::io::Cursor;
use type_map::DataTypeId;
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum DataKind {
Bool,
SignedNumber,
UnsignedNumber,
Float,
Str,
Bytes,
Array(Box<DataKind>),
Container,
#[cfg(feature = "crypto")]
EncryptedContainer,
Null,
}
#[derive(Debug, Clone, Eq)]
pub enum DataValue {
BoolTrue,
BoolFalse,
Bool(bool),
SignedNumber(i128),
UnsignedNumber(u128),
Float(u8, u32),
Str(String),
Bytes(Vec<u8>),
Array(Vec<DataValue>),
Container(Vec<(DataTypeId, DataValue)>),
#[cfg(feature = "crypto")]
EncryptedContainer(Vec<(DataTypeId, DataValue)>),
Null,
}
impl DataValue {
/*
* Container format:
* [2 bytes u16 entry_count] // number of key-value pairs
* [1 byte kind] // DataValue kind marker
* [if kind == BOOL_TRUE or BOOL_FALSE:]
* [1 byte key] // DataTypes discriminant
* [else:]
* [2 bytes u16 payload_len] // length of the value payload
* [1 byte key] // DataTypes discriminant
* [payload_len bytes payload] // value data (interpreted based on kind)
*
* Kind markers:
* 0x01 => BoolTrue
* 0x02 => BoolFalse
* 0x03 => Number (i64, 8 bytes big-endian)
* 0x04 => Str (UTF-8 bytes)
* 0x05 => Array (nested container format)
* 0x06 => Container (nested container format)
* 0x07 => Null
*/
const KIND_BOOL_TRUE: u8 = 0x01;
const KIND_BOOL_FALSE: u8 = 0x02;
const KIND_SIGNED_NUMBER: u8 = 0x03;
const KIND_UNSIGNED_NUMBER: u8 = 0x04;
const KIND_FLOAT: u8 = 0x05;
const KIND_STR: u8 = 0x06;
const KIND_BYTES: u8 = 0x07;
const KIND_ARRAY: u8 = 0x08;
const KIND_CONTAINER: u8 = 0x09;
#[cfg(feature = "crypto")]
const KIND_ENCRYPTED_CONTAINER: u8 = 0x0A;
const KIND_NULL: u8 = 0x0B;
pub fn container_from_map(map: &BTreeMap<DataTypeId, DataValue>) -> DataValue {
let mut container = Vec::new();
for (key, value) in map {
container.push((key.clone(), value.clone()));
}
DataValue::Container(container)
}
pub fn kind(&self) -> DataKind {
match self {
DataValue::Bool(_) | DataValue::BoolTrue | DataValue::BoolFalse => DataKind::Bool,
DataValue::SignedNumber(_) => DataKind::SignedNumber,
DataValue::UnsignedNumber(_) => DataKind::UnsignedNumber,
DataValue::Float(_, _) => DataKind::Float,
DataValue::Str(_) => DataKind::Str,
DataValue::Array(a) => {
if let Some(first) = a.first() {
DataKind::Array(Box::new(first.kind()))
} else {
DataKind::Array(Box::new(DataKind::Null))
}
}
DataValue::Bytes(_) => DataKind::Bytes,
DataValue::Container(_) => DataKind::Container,
#[cfg(feature = "crypto")]
DataValue::EncryptedContainer(_) => DataKind::EncryptedContainer,
DataValue::Null => DataKind::Null,
}
}
pub fn as_bool(&self) -> Option<bool> {
match self {
DataValue::BoolTrue => Some(true),
DataValue::BoolFalse => Some(false),
DataValue::Bool(v) => Some(*v),
_ => None,
}
}
pub fn as_str(&self) -> Option<&str> {
match self {
DataValue::Str(s) => Some(s),
_ => None,
}
}
pub fn as_string(&self) -> Option<String> {
self.as_str().map(|s| s.to_string())
}
pub fn as_signed_number(&self) -> Option<i128> {
match self {
DataValue::SignedNumber(n) => Some(*n),
_ => None,
}
}
pub fn as_unsigned_number(&self) -> Option<u128> {
match self {
DataValue::UnsignedNumber(n) => Some(*n),
_ => None,
}
}
pub fn as_float(&self) -> Option<(u8, u32)> {
match self {
DataValue::Float(a, b) => Some((*a, *b)),
_ => None,
}
}
pub fn as_array(&self) -> Option<Vec<DataValue>> {
match self {
DataValue::Array(a) => Some(a.clone()),
_ => None,
}
}
pub fn as_bytes(&self) -> Option<Vec<u8>> {
match self {
DataValue::Bytes(b) => Some(b.clone()),
_ => None,
}
}
pub fn as_container(&self) -> Option<Vec<(DataTypeId, DataValue)>> {
match self {
DataValue::Container(c) => Some(c.clone()),
_ => None,
}
}
#[cfg(feature = "crypto")]
pub fn as_encrypted_container(&self) -> Option<Vec<(DataTypeId, DataValue)>> {
match self {
DataValue::EncryptedContainer(c) => Some(c.clone()),
_ => None,
}
}
pub fn as_map(&self) -> Option<BTreeMap<DataTypeId, DataValue>> {
match self {
DataValue::Container(c) => {
let mut out = BTreeMap::new();
for (k, v) in c {
out.insert(k.clone(), v.clone());
}
Some(out)
}
_ => None,
}
}
pub fn to_bytes(&self) -> Vec<u8> {
match self {
DataValue::Container(entries) => Self::encode_container(entries),
DataValue::Array(arr) => Self::encode_array(arr),
_ => {
let mut out = Vec::new();
if Self::write_value_payload(&mut out, self).is_none() {
return Vec::new();
}
out
}
}
}
pub fn from_bytes(bytes: &[u8]) -> Option<Self> {
let mut cursor = Cursor::new(bytes);
let value = Self::read_value(&mut cursor, true)?;
if cursor.position() as usize != bytes.len() {
return None;
}
Some(value)
}
pub fn to_base64(&self) -> String {
general_purpose::STANDARD.encode(self.to_bytes())
}
pub fn from_base64(base64_str: &str) -> Option<Self> {
let bytes = general_purpose::STANDARD.decode(base64_str).ok()?;
Self::from_bytes(&bytes)
}
fn encode_container(entries: &[(DataTypeId, DataValue)]) -> Vec<u8> {
let mut out = Vec::new();
if out
.write_u16::<BigEndian>(u16::try_from(entries.len()).ok().unwrap_or(0))
.is_err()
{
return Vec::new();
}
for (key, value) in entries {
if !Self::write_container_entry(&mut out, key.clone(), value) {
return Vec::new();
}
}
out
}
fn write_container_entry(buf: &mut Vec<u8>, key: DataTypeId, value: &DataValue) -> bool {
let kind = Self::kind_marker(value);
buf.push(kind);
if kind == Self::KIND_BOOL_TRUE || kind == Self::KIND_BOOL_FALSE {
let key = u8::try_from(key.0).unwrap_or(0);
buf.push(key);
return true;
}
let mut payload = Vec::new();
if Self::write_value_payload(&mut payload, value).is_none() {
return false;
}
let len_u16 = match u16::try_from(payload.len()) {
Ok(v) => v,
Err(_) => return false,
};
if buf.write_u16::<BigEndian>(len_u16).is_err() {
return false;
}
let key = u8::try_from(key.0).unwrap_or(0);
buf.push(key);
buf.extend_from_slice(&payload);
true
}
fn encode_array(arr: &[DataValue]) -> Vec<u8> {
let mut out = Vec::new();
if out
.write_u16::<BigEndian>(u16::try_from(arr.len()).ok().unwrap_or(0))
.is_err()
{
return Vec::new();
}
for value in arr {
if !Self::write_array_entry(&mut out, value) {
return Vec::new();
}
}
out
}
fn write_array_entry(buf: &mut Vec<u8>, value: &DataValue) -> bool {
let kind = Self::kind_marker(value);
buf.push(kind);
if kind == Self::KIND_BOOL_TRUE || kind == Self::KIND_BOOL_FALSE {
return true;
}
let mut payload = Vec::new();
if Self::write_value_payload(&mut payload, value).is_none() {
return false;
}
let len_u16 = match u16::try_from(payload.len()) {
Ok(v) => v,
Err(_) => return false,
};
if buf.write_u16::<BigEndian>(len_u16).is_err() {
return false;
}
buf.extend_from_slice(&payload);
true
}
fn write_value_payload(buf: &mut Vec<u8>, value: &DataValue) -> Option<()> {
match value {
DataValue::BoolTrue => Some(()),
DataValue::BoolFalse => Some(()),
DataValue::Bool(v) => {
if *v {
Some(())
} else {
Some(())
}
}
DataValue::SignedNumber(n) => {
buf.write_i128::<BigEndian>(*n).ok()?;
Some(())
}
DataValue::UnsignedNumber(n) => {
buf.write_u128::<BigEndian>(*n).ok()?;
Some(())
}
DataValue::Float(a, b) => {
buf.write_u8(*a).ok()?;
buf.write_u32::<BigEndian>(*b).ok()?;
Some(())
}
DataValue::Str(s) => {
buf.extend_from_slice(s.as_bytes());
Some(())
}
DataValue::Array(arr) => {
let bytes = Self::encode_array(arr);
buf.extend_from_slice(&bytes);
Some(())
}
DataValue::Bytes(b) => {
buf.extend_from_slice(b);
Some(())
}
DataValue::Container(entries) => {
let bytes = Self::encode_container(entries);
buf.extend_from_slice(&bytes);
Some(())
}
#[cfg(feature = "crypto")]
DataValue::EncryptedContainer(entries) => {
let bytes = Self::encode_container(entries);
buf.extend_from_slice(&bytes);
Some(())
}
DataValue::Null => Some(()),
}
}
fn read_value(cursor: &mut Cursor<&[u8]>, top_level: bool) -> Option<Self> {
if top_level {
let start = cursor.position() as usize;
let remaining = cursor.get_ref().len().checked_sub(start)?;
if remaining < 2 {
return None;
}
let snapshot = cursor.clone();
if let Some(container) = Self::try_read_container(cursor) {
return Some(container);
}
*cursor = snapshot;
let array = Self::read_array(cursor)?;
return Some(array);
}
let kind = cursor.read_u8().ok()?;
Self::read_value_by_kind(cursor, kind, None)
}
fn try_read_container(cursor: &mut Cursor<&[u8]>) -> Option<Self> {
let count = cursor.read_u16::<BigEndian>().ok()? as usize;
let mut entries = Vec::with_capacity(count);
for _ in 0..count {
let kind = cursor.read_u8().ok()?;
if kind == Self::KIND_BOOL_TRUE || kind == Self::KIND_BOOL_FALSE {
let key = DataTypeId(cursor.read_u16::<BigEndian>().ok()?);
let value = if kind == Self::KIND_BOOL_TRUE {
DataValue::BoolTrue
} else {
DataValue::BoolFalse
};
entries.push((key, value));
continue;
}
let len = cursor.read_u16::<BigEndian>().ok()? as usize;
let key = DataTypeId(cursor.read_u16::<BigEndian>().ok()?);
let start = cursor.position() as usize;
let end = start.checked_add(len)?;
if end > cursor.get_ref().len() {
return None;
}
let payload = &cursor.get_ref()[start..end];
let mut inner = Cursor::new(payload);
let value = Self::read_value_by_kind(&mut inner, kind, Some(len))?;
if inner.position() as usize != len {
return None;
}
cursor.set_position(end as u64);
entries.push((key, value));
}
Some(DataValue::Container(entries))
}
fn read_array(cursor: &mut Cursor<&[u8]>) -> Option<Self> {
let count = cursor.read_u16::<BigEndian>().ok()? as usize;
let mut out = Vec::with_capacity(count);
for _ in 0..count {
let kind = cursor.read_u8().ok()?;
if kind == Self::KIND_BOOL_TRUE {
out.push(DataValue::BoolTrue);
continue;
}
if kind == Self::KIND_BOOL_FALSE {
out.push(DataValue::BoolFalse);
continue;
}
let len = cursor.read_u16::<BigEndian>().ok()? as usize;
let start = cursor.position() as usize;
let end = start.checked_add(len)?;
if end > cursor.get_ref().len() {
return None;
}
let payload = &cursor.get_ref()[start..end];
let mut inner = Cursor::new(payload);
let value = Self::read_value_by_kind(&mut inner, kind, Some(len))?;
if inner.position() as usize != len {
return None;
}
cursor.set_position(end as u64);
out.push(value);
}
Some(DataValue::Array(out))
}
fn read_value_by_kind(
cursor: &mut Cursor<&[u8]>,
kind: u8,
payload_len: Option<usize>,
) -> Option<Self> {
match kind {
Self::KIND_BOOL_TRUE => Some(DataValue::BoolTrue),
Self::KIND_BOOL_FALSE => Some(DataValue::BoolFalse),
Self::KIND_SIGNED_NUMBER => Some(DataValue::SignedNumber(
cursor.read_i128::<BigEndian>().ok()?,
)),
Self::KIND_UNSIGNED_NUMBER => Some(DataValue::UnsignedNumber(
cursor.read_u128::<BigEndian>().ok()?,
)),
Self::KIND_FLOAT => {
let a = cursor.read_u8().ok()?;
let b = cursor.read_u32::<BigEndian>().ok()?;
Some(DataValue::Float(a, b))
}
Self::KIND_STR => {
let len = payload_len?;
let start = cursor.position() as usize;
let end = start.checked_add(len)?;
if end > cursor.get_ref().len() {
return None;
}
let s = std::str::from_utf8(&cursor.get_ref()[start..end])
.ok()?
.to_string();
cursor.set_position(end as u64);
Some(DataValue::Str(s))
}
Self::KIND_ARRAY => {
let start = cursor.position() as usize;
let len = payload_len?;
let end = start.checked_add(len)?;
if end > cursor.get_ref().len() {
return None;
}
let mut inner = Cursor::new(&cursor.get_ref()[start..end]);
let arr = Self::read_array(&mut inner)?;
if inner.position() as usize != len {
return None;
}
cursor.set_position(end as u64);
Some(arr)
}
Self::KIND_CONTAINER => {
let start = cursor.position() as usize;
let len = payload_len?;
let end = start.checked_add(len)?;
if end > cursor.get_ref().len() {
return None;
}
let mut inner = Cursor::new(&cursor.get_ref()[start..end]);
let c = Self::try_read_container(&mut inner)?;
if inner.position() as usize != len {
return None;
}
cursor.set_position(end as u64);
Some(c)
}
#[cfg(feature = "crypto")]
Self::KIND_ENCRYPTED_CONTAINER => {
let start = cursor.position() as usize;
let len = payload_len?;
let end = start.checked_add(len)?;
if end > cursor.get_ref().len() {
return None;
}
let mut inner = Cursor::new(&cursor.get_ref()[start..end]);
let c = Self::try_read_container(&mut inner)?;
if inner.position() as usize != len {
return None;
}
cursor.set_position(end as u64);
if let DataValue::Container(entries) = c {
Some(DataValue::EncryptedContainer(entries))
} else {
None
}
}
Self::KIND_NULL => Some(DataValue::Null),
_ => None,
}
}
fn kind_marker(value: &DataValue) -> u8 {
match value {
DataValue::BoolTrue => Self::KIND_BOOL_TRUE,
DataValue::BoolFalse => Self::KIND_BOOL_FALSE,
DataValue::Bool(v) => {
if *v {
Self::KIND_BOOL_TRUE
} else {
Self::KIND_BOOL_FALSE
}
}
DataValue::SignedNumber(_) => Self::KIND_SIGNED_NUMBER,
DataValue::UnsignedNumber(_) => Self::KIND_UNSIGNED_NUMBER,
DataValue::Float(_, _) => Self::KIND_FLOAT,
DataValue::Str(_) => Self::KIND_STR,
DataValue::Array(_) => Self::KIND_ARRAY,
DataValue::Bytes(_) => Self::KIND_BYTES,
DataValue::Container(_) => Self::KIND_CONTAINER,
#[cfg(feature = "crypto")]
DataValue::EncryptedContainer(_) => Self::KIND_ENCRYPTED_CONTAINER,
DataValue::Null => Self::KIND_NULL,
}
}
}
impl PartialEq for DataValue {
fn eq(&self, other: &Self) -> bool {
use DataValue::*;
match (self, other) {
(BoolTrue, BoolTrue) | (BoolFalse, BoolFalse) => true,
(BoolTrue, Bool(true)) | (Bool(true), BoolTrue) => true,
(BoolFalse, Bool(false)) | (Bool(false), BoolFalse) => true,
(Bool(a), Bool(b)) => a == b,
(SignedNumber(a), SignedNumber(b)) => a == b,
(UnsignedNumber(a), UnsignedNumber(b)) => a == b,
(Float(a, b), Float(c, d)) => a == c && b == d,
(Str(a), Str(b)) => a == b,
(Array(a), Array(b)) => a == b,
(Bytes(a), Bytes(b)) => a == b,
(Container(a), Container(b)) => a == b,
#[cfg(feature = "crypto")]
(EncryptedContainer(a), EncryptedContainer(b)) => a == b,
(Null, Null) => true,
_ => false,
}
}
}
impl Hash for DataValue {
fn hash<H: Hasher>(&self, state: &mut H) {
use DataValue::*;
match self {
BoolTrue | Bool(true) => {
0u8.hash(state);
true.hash(state);
}
BoolFalse | Bool(false) => {
0u8.hash(state);
false.hash(state);
}
SignedNumber(n) => {
1u8.hash(state);
n.hash(state);
}
UnsignedNumber(n) => {
2u8.hash(state);
n.hash(state);
}
Float(n, m) => {
3u8.hash(state);
n.hash(state);
m.hash(state);
}
Str(s) => {
2u8.hash(state);
s.hash(state);
}
Array(a) => {
3u8.hash(state);
a.hash(state);
}
Bytes(a) => {
4u8.hash(state);
a.hash(state);
}
Container(c) => {
5u8.hash(state);
c.hash(state);
}
#[cfg(feature = "crypto")]
EncryptedContainer(c) => {
6u8.hash(state);
c.hash(state);
}
Null => {
6u8.hash(state);
}
}
}
}

26
codec/src/lib.rs Normal file
View file

@ -0,0 +1,26 @@
pub mod communication_types;
pub mod communication_value;
pub mod data_value;
pub mod util;
pub use communication_value::CommunicationValue;
pub use data_value::{DataKind, DataValue};
pub use util::rand_u32;
pub use type_map::{CommTypeId, DataTypeId, TypeMap, Version};
use common::CodecError;
pub fn encode(_value: &DataValue, _typemap: &TypeMap) -> Result<Vec<u8>, CodecError> {
// write header using typemap.data_id(), serialize value
todo!()
}
pub fn decode(_bytes: &[u8], _typemap: &TypeMap) -> Result<DataValue, CodecError> {
// read header, look up type names from typemap, build DataValue
todo!()
}
#[cfg(feature = "registry")]
pub mod registry;

67
codec/src/registry.rs Normal file
View file

@ -0,0 +1,67 @@
use crate::{CodecError, DataValue, decode, encode};
use std::collections::BTreeMap;
use type_map::{TypeMap, Version};
#[derive(Clone, Debug)]
pub struct Registry {
versions: BTreeMap<Version, TypeMap>,
}
impl Registry {
pub fn new() -> Self {
Self {
versions: BTreeMap::new(),
}
}
pub fn register(&mut self, typemap: TypeMap) {
self.versions.insert(typemap.version.clone(), typemap);
}
pub fn get(&self, version: Version) -> Option<&TypeMap> {
self.versions.get(&version)
}
pub fn negotiate(&self, client_versions: &[Version]) -> Option<Version> {
client_versions
.iter()
.filter(|v| self.versions.contains_key(v))
.max()
.cloned()
}
pub fn latest(&self) -> Option<&TypeMap> {
self.versions.last_key_value().map(|(_, v)| v)
}
}
#[derive(Clone, Debug)]
pub struct VersionedCodec {
registry: Registry,
}
impl VersionedCodec {
pub fn new(registry: Registry) -> Self {
Self { registry }
}
pub fn encode(&self, value: &DataValue, version: Version) -> Result<Vec<u8>, CodecError> {
let typemap = self
.registry
.get(version)
.ok_or(CodecError::UnknownVersion)?;
encode(value, typemap)
}
pub fn decode(&self, bytes: &[u8], version: Version) -> Result<DataValue, CodecError> {
let typemap = self
.registry
.get(version)
.ok_or(CodecError::UnknownVersion)?;
decode(bytes, typemap)
}
pub fn negotiate(&self, client_versions: &[Version]) -> Option<Version> {
self.registry.negotiate(client_versions)
}
}

9
codec/src/util.rs Normal file
View file

@ -0,0 +1,9 @@
use rand::random;
pub fn rand_u32() -> u32 {
random::<u32>()
}
pub fn rand_u64() -> u64 {
random::<u64>()
}

6
common/Cargo.toml Normal file
View file

@ -0,0 +1,6 @@
[package]
name = "common"
version = "0.1.0"
edition = "2024"
[dependencies]

12
common/src/lib.rs Normal file
View file

@ -0,0 +1,12 @@
pub enum RegistryError {
ReservedCommId(u8, String),
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum CodecError {
UnknownVersion,
UnknownCommunicationType(String),
UnknownDataType(String),
ReservedCommunicationType(u8),
InvalidEncoding,
}

6
crypto/Cargo.toml Normal file
View file

@ -0,0 +1,6 @@
[package]
name = "mtp-crypto"
version = "0.1.0"
edition = "2024"
[dependencies]

14
crypto/src/lib.rs Normal file
View file

@ -0,0 +1,14 @@
pub fn add(left: u64, right: u64) -> u64 {
left + right
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
let result = add(2, 2);
assert_eq!(result, 4);
}
}

6
host/Cargo.toml Normal file
View file

@ -0,0 +1,6 @@
[package]
name = "host"
version = "0.1.0"
edition = "2024"
[dependencies]

14
host/src/lib.rs Normal file
View file

@ -0,0 +1,14 @@
pub fn add(left: u64, right: u64) -> u64 {
left + right
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
let result = add(2, 2);
assert_eq!(result, 4);
}
}

7
registry/Cargo.toml Normal file
View file

@ -0,0 +1,7 @@
[package]
name = "registry"
version = "0.1.0"
edition = "2024"
[dependencies]
common = { path = "../common" }

94
registry/src/lib.rs Normal file
View file

@ -0,0 +1,94 @@
use common::RegistryError;
use std::collections::HashMap;
#[repr(transparent)]
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct CommTypeId(pub u8);
#[repr(transparent)]
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct DataTypeId(pub u16);
/*
* # Reserved Internal Namespace
* These are NEVER assigned by a RegistryConfig. They are fixed
* across all versions for version negotiation & security.
*/
pub const INTERNAL_COMM_RESERVED: std::ops::Range<u8> = 0..16;
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum InternalCommType {
VersionNegotiate = 0,
SecurityHandshake = 1,
RegistrySync = 2,
// 3-15 reserved for future internal use
}
impl InternalCommType {
pub fn as_id(self) -> CommTypeId {
CommTypeId(self as u8)
}
}
#[derive(Clone, Debug)]
// (major , minor , patch );
pub struct Version(pub u16, pub u16, pub u16);
pub struct Registry {
pub version: Version,
comm_name_to_id: HashMap<String, u8>,
comm_id_to_name: HashMap<u8, String>,
data_name_to_id: HashMap<String, u16>,
data_id_to_name: HashMap<u16, String>,
}
pub struct RegistryConfig {
pub version: Version,
pub communication_types: HashMap<String, u8>,
pub data_types: HashMap<String, u16>,
}
impl Registry {
pub fn from_config(cfg: RegistryConfig) -> Result<Self, RegistryError> {
for (name, &id) in &cfg.communication_types {
if INTERNAL_COMM_RESERVED.contains(&id) {
return Err(RegistryError::ReservedCommId(id, name.clone()));
}
}
let mut comm_id_to_name = HashMap::with_capacity(cfg.communication_types.len());
for (name, id) in &cfg.communication_types {
comm_id_to_name.insert(*id, name.clone());
}
let mut data_id_to_name = HashMap::with_capacity(cfg.data_types.len());
for (name, id) in &cfg.data_types {
data_id_to_name.insert(*id, name.clone());
}
Ok(Self {
version: cfg.version,
comm_name_to_id: cfg.communication_types,
comm_id_to_name,
data_name_to_id: cfg.data_types,
data_id_to_name,
})
}
pub fn resolve_comm(&self, name: &str) -> Option<CommTypeId> {
self.comm_name_to_id.get(name).copied().map(CommTypeId)
}
pub fn parse_comm(&self, id: CommTypeId) -> Option<&str> {
self.comm_id_to_name.get(&id.0).map(|s| s.as_str())
}
pub fn resolve_data(&self, name: &str) -> Option<DataTypeId> {
self.data_name_to_id.get(name).copied().map(DataTypeId)
}
pub fn parse_data(&self, id: DataTypeId) -> Option<&str> {
self.data_id_to_name.get(&id.0).map(|s| s.as_str())
}
}

6
transport/Cargo.toml Normal file
View file

@ -0,0 +1,6 @@
[package]
name = "transport"
version = "0.1.0"
edition = "2024"
[dependencies]

14
transport/src/lib.rs Normal file
View file

@ -0,0 +1,14 @@
pub fn add(left: u64, right: u64) -> u64 {
left + right
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
let result = add(2, 2);
assert_eq!(result, 4);
}
}

6
type-map/Cargo.toml Normal file
View file

@ -0,0 +1,6 @@
[package]
name = "type-map"
version = "0.1.0"
edition = "2024"
[dependencies]

28
type-map/src/lib.rs Normal file
View file

@ -0,0 +1,28 @@
use std::collections::HashMap;
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct CommTypeId(pub u8);
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct DataTypeId(pub u16);
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Version(pub u16);
/// A single protocol version's type dictionary.
/// Compiled into the client, or loaded by the host via the registry.
#[derive(Clone, Debug)]
pub struct TypeMap {
pub version: Version,
pub comm_types: HashMap<String, u8>,
pub data_types: HashMap<String, u16>,
}
impl TypeMap {
pub fn comm_id(&self, name: &str) -> Option<u8> {
self.comm_types.get(name).copied()
}
pub fn data_id(&self, name: &str) -> Option<u16> {
self.data_types.get(name).copied()
}
}