• Home
• Reference
• Components
• Host client

Host client

The host client receives indexed blockchain data from multiple indexers over P2P. It verifies the data using attestation records, runs WASM lens transforms to produce view documents, and serves those documents over GraphQL.

If indexers are data producers, hosts are consumers and servers. The separation lets you scale serving independently from indexing.

Architecture

Components

Key source files

ShinzoHub event subscription

No webhooks. The host opens a persistent WebSocket connection to ShinzoHub's CometBFT node and subscribes to transaction events:

cancel, channel, err := shinzohub.StartEventSubscription(wsURL)

This subscribes to two query filters:

• "tm.event='Tx' AND Registered.key EXISTS": view registration events.
• "tm.event='Tx' AND EntityRegistered.key EXISTS": new indexers or hosts joining.

Events arrive on the returned channel. The host's main loop reads from this channel and dispatches to the view processor or network handler as appropriate.

Attestation system

When a host receives data from multiple indexers for the same block, it creates an AttestationRecord using a P-counter CRDT.

AttestationRecord schema

type Ethereum__Mainnet__AttestationRecord {
    attested_doc: String @index
    source_doc: String
    CIDs: [String]
    doc_type: String @index
    vote_count: Int @crdt(type: pcounter)
}

The @crdt(type: pcounter) annotation tells DefraDB to use a Positive Counter merge strategy. Each node tracks its own increments separately, and merges are deterministic:

Host A: {A: 1, B: 0}    (saw 1 indexer)
Host B: {A: 0, B: 1}    (saw 1 indexer)
Merge:  {A: 1, B: 1} -> total = 2

Attestation flow

The upsert pattern in GraphQL:

mutation {
    upsert_AttestationRecord(
        create: { vote_count: 1 },
        update: { vote_count: 1 },
        filter: { attested_doc: { _eq: "block_1000" } }
    ) { _docID }
}

View-specific attestation collections

The host creates separate attestation collections per view:

collectionName := fmt.Sprintf("Ethereum__Mainnet__AttestationRecord_%s", viewName)

So you get AttestationRecord_Block for primitive attestations and AttestationRecord_TokenTransfer for a specific view.

CID array merging doesn't work right

When multiple hosts create AttestationRecords for the same document but with different CID sets, they may create separate documents instead of merging. The P-counter for vote_count merges correctly, but the CID array ([]string) has no CRDT merge strategy. Documented in the host-client ADR-03.

Re-org handling

When a blockchain re-org happens, different indexers may briefly have different versions of the same block:

Indexer A: Block #1000 with hash 0xaaa (pre-reorg)
Indexer B: Block #1000 with hash 0xbbb (post-reorg)

These produce separate documents (different CIDs), each with its own AttestationRecord. The post-reorg version accumulates more votes as indexers converge, and applications pick the one with higher consensus.

View discovery

Hosts watch for view registrations through two paths.

At startup, the host calls FetchAllRegisteredViews(), which queries CometBFT with tx_search?query="Registered.key EXISTS", paginates through all historical registration transactions, and processes each view bundle.

At runtime, the host subscribes to tm.event='Tx' AND Registered.key EXISTS over WebSocket. When a new event arrives, it extracts the key, creator, and view attributes and processes the bundle.

There is a known bug here: the View Registry precompile emits "ViewRegistered" with attributes view_address/view_name/creator/data, but the host subscribes to "Registered" and expects attributes key/creator/view. Neither event type nor attribute names match. Fixed in ShinzoHub v2.

View processing pipeline

When the host receives a view bundle (from either discovery path):

1. ProcessViewFromWireFormat(): base64-decodes the wire bytes, parses VWL format, extracts the view name from SDL via regex.
2. PostWasmToFile(): for each lens, base64-decodes the WASM bytes, validates the WASM magic number (0x00 0x61 0x73 0x6D), and writes to disk with a sha256-derived filename.
3. SetupLensInDefraDB(): builds a LensConfig (source collection, destination, lens path, arguments) and calls defraNode.DB.AddLens() to register with LensVM.
4. ConfigureLens(): calls defraNode.DB.AddView() with or without a transform CID, auto-corrects field names for schema compatibility.
5. SubscribeTo(): enables P2P replication for the view's collection via CreateP2PCollections().
6. SaveViewToRegistry(): persists view metadata to views.json in the lens registry directory so the host can recover on restart.

Lens transforms

The host runs LensVM (source-gh/lens) to execute WASM modules that transform primitives into view documents.

WASM runtimes in use:

Lens binary size varies by source language. AssemblyScript produces ~73 KB WASM. Rust produces ~200 KB.

GraphQL serving

Example query:

{
  USDCTransfer(filter: { blockNumber: { _gte: 19540000 } }) {
    from
    to
    amount
    blockNumber
  }
}

Earnings

Hosts receive the Compute Factor component of view pricing. There is an immediate payment per query served, plus a potential bonus at epoch end if the host's coverage (uptime and data availability) exceeds the network average.

Document filtering

Hosts can filter incoming documents by contract address, event type, or function signature. A host running only ERC-20 transfer views does not need to store every log from every contract.

Metrics

Port 8080 exposes Prometheus metrics: block processing rates, attestation counts, view status (active, syncing, errored), and P2P peer counts.

Resource requirements

Key files

Lens integration is imported from source-gh/lens. DefraDB configuration comes from shinzo-gh/shinzo-app-sdk.