How Morcom works.

Decisions at a glance

Decision

Choice

Rejected

Manifest and signing

OpenSSF Model Signing (OMS) on Sigstore

Rejected: Custom format, raw TUF or in-toto at manifest level

Primary byte transport

BitTorrent v2 (libtorrent, BEP 52)

Rejected: IPFS/Bitswap, tokenised storage

Fallback transport

Signed HTTP mirrors with range requests

Rejected: None. The fallback is mandatory

Watch-list transport

Iroh (BLAKE3 over QUIC)

Rejected: No alternative under consideration

Client compatibility

Hugging Face Hub REST surface via an HF_ENDPOINT swap

Rejected: Xet protocol, not needed for MVP

Index

Postgres + Meilisearch

Rejected: Elastic, custom search

Index mirroring

Signed append-only log + TUF-signed snapshots

Rejected: Ad hoc dumps

Enterprise mirror path

OCI artifacts via ORAS, optional

Rejected: OCI as canonical distribution

Manifest format and signing

Adopt OMS verbatim. It is a detached signature in the Sigstore bundle format wrapping an in-toto statement whose predicate is a per-file SHA-256 manifest. It is PKI-agnostic: keyless Sigstore with OIDC and the Rekor transparency log, an enterprise CA, or self-signed. That matches the federated seeder model, where a university signs with its own institutional identity. NVIDIA has signed all NGC models with OMS since March 2025 and Google is prototyping it on Kaggle, so Morcom manifests are verifiable by tooling the ecosystem is already building.

Locator sidecar

The one thing Morcom adds. A small JSON file with the BitTorrent v2 infohash and magnet, HTTP mirror URLs, an optional Iroh ticket, file sizes, and licence and provenance fields. The OMS signature covers the weight hashes; the locator is signed separately by the same key so mirrors can be added without re-signing weights.

TUF and in-toto

TUF is applied at the index snapshot layer, not to individual manifests. in-toto is already inside OMS.

OCI and ORAS

OCI registries are proven for model artifacts (Harbor with CNAI, KAITO, ModelPack, Ollama), but OCI is a client-server pull model and recreates the central host. Supported as an optional mirror target an institution can expose, never as the canonical path.

P2P distribution layer

Five transports were evaluated against what a multi-hundred-gigabyte checkpoint needs.

Why BitTorrent v2

BEP 52's per-file Merkle tree is exactly what a multi-hundred-GB safetensors file needs. Any 16 KiB block verifies on arrival, torrents stay small, and identical files across repos share root hashes, giving natural dedup in the same spirit as Hugging Face's Xet content addressing. libtorrent 2.0+ is mature and embeddable. Academic Torrents has run this exact HTTP plus swarm hybrid since 2013 and moves over 3 TB a day.

Why the HTTP fallback is non-negotiable

It guarantees the system degrades to "just works" behind hostile firewalls and on the long tail, and it is what makes the HF_ENDPOINT swap transparent.

Iroh

Prototyped in parallel, never blocked on. libtorrent large-swarm throughput is measured directly; no primary-source gigabit benchmarks were located.

Seeding incentives and durability

Seed-node operator program

Public badges on model pages and org profiles, tied to measurable SLAs: uptime, bytes served, models pinned.

Morcom Anchor tier

For institutions committing to seed a defined model set under an availability SLA, analogous to Debian official mirrors. Anchors get a governance voice.

Long-tail fallback pinning

Models with zero live seeders fall back to a foundation-run store on zero-egress object storage, budgeted at a few hundred TB. This prevents the classic dead-swarm failure.

Seed-node daemon

A packaged libtorrent-based service for institutions with configurable pin sets. It reports metrics to the index for badge and SLA tracking.

Abuse, malware, and licensing

This is the highest-risk area.

Safetensors-preferred policy

Pickle files are warned on and quarantined. Ingestion runs picklescan and ClamAV, but scanning is treated as necessary and not sufficient: 7z evasion and ShadowPickle-class attacks defeat multiple scanners as of 2026. Trusted-signer provenance is the real defence.

Mandatory licence field

Every manifest carries an SPDX or OMS licence field. Morcom refuses to index without it.

Notice and delist

Handled at the index layer. Morcom cannot remove bytes from a swarm, but it can delist a manifest, the same posture Meta took against the LLaMA torrent on Hugging Face.

Namespace-reuse protection

Deleted org names are never recycled, closing the model namespace reuse attack documented by Unit 42.

Hugging Face API compatibility

The huggingface_hub client builds file URLs from a template whose base is read from HF_ENDPOINT:

{ENDPOINT}/{repo_id}/resolve/{revision}/{filename}

One line redirects everything:

export HF_ENDPOINT=https://hub.morcom.ai

hf-mirror.com proves this works transparently for millions of users.

Minimal gateway surface

GET /api/models

Listing. Also GET /api/models/{repo_id} for metadata: siblings, tags, sha.

GET /api/models/{repo_id}/revision/{revision}

Revision resolution.

GET /{repo_id}/resolve/{revision}/{filename}

The critical resolver. Small files are served directly; large files 302-redirect to a signed HTTP mirror URL, LFS style. A Morcom-specific header carries the magnet and Iroh ticket so Morcom-aware clients prefer P2P.

HEAD on the resolver

Returns X-Repo-Commit, Content-Length, ETag, and the LFS X-Linked-Size and X-Linked-Etag semantics the client expects.

GET /api/whoami-v2

Token endpoints, permissive for anonymous public read.

/.well-known/openapi.json

Discoverability.

Xet

Not required. The client falls back to plain LFS-style resolve when the endpoint is not Xet-enabled. Morcom's own content addressing, BitTorrent v2 Merkle or Iroh BLAKE3, is philosophically identical and is the Morcom-native fast path. A Xet-compatible CAS is a later optimisation.

Index and search stack

Postgres

The canonical registry: repos, revisions, manifest hashes, signatures, seeder and mirror lists, licence metadata.

Meilisearch

Typo-tolerant search, trivial to self-host, built on Tantivy so a Rust gateway can embed the engine directly if preferred. Typesense is the alternative.

Mirrorability

A first-class requirement, in three parts. A signed append-only transparency log of manifest registrations, Rekor-style, so nobody can silently rewrite history. Periodic signed snapshots distributed by git-style replication and torrent. TUF roles (root, snapshot, timestamp, targets) over the snapshots so a compromised mirror cannot serve stale or forged index state, with root keys multi-party and held by distinct member organisations.

If Morcom's servers vanish, any mirror reconstitutes the full signed index. Precedents: PyPI mirroring with TUF (PEP 458), the Debian mirror network, kernel.org mirrors.

Cost model

Monthly, order of magnitude.

Index API, Postgres, Meilisearch

Low thousands USD

Tracker and DHT bootstrap

Metadata only, bandwidth-light

Hundreds USD

Signing and verification

Sigstore public infrastructure, Rekor

Near zero

Metadata mirrors and snapshots

Trivial

Fallback pinning, about 300 TB

Backblaze B2, free egress via Cloudflare Bandwidth Alliance

About $1,800

Cloudflare R2, zero egress

About $4,500

Same 300 TB on S3 + CloudFront

Egress alone, at roughly one full read per month

$15,000 to $27,000

That is the economics Morcom is built to escape. For contrast, Hugging Face's own docs cite 45 PB on the legacy LFS and S3 backend serving more than 3 million models to more than 18 million developers, an egress-dominated CDN operation. Morcom externalises byte serving to seeders and pays only for the long tail.

Technical caveats

• The Iroh throughput figure comes from a single GitHub issue, not an official benchmark.
• libtorrent large-swarm numbers should be measured, not assumed.
• picklescan and safetensors do not fully solve malware. Provenance is the defence.
• Hugging Face does not publish absolute storage or CDN costs; the contrast is inferred from published pricing and scale figures.