Skip to main content

Substrate Processor

This section applies to squid processors indexing Substrate-based chains, including:

  • Polkadot
  • Kusama
  • Moonbeam (Substrate data). For EVM data, use EVM Processor
  • Moonriver (Substrate data). For EVM data, use EVM Processor
  • Astar
  • Acala

Overview and the data model​

A squid processor is a Node.js process that fetches historical on-chain data from an Archive, performs arbitrary transformations and saves the result. By convention, the processor entry point is src/processor.ts. SubstrateBatchProcessor is the central class that handles Substrate data extraction, transformation and persistence. A single batch handler function supplied to SubstrateBatchProcessor.run() is responsible for transforming data from multiple events and transactions in a single in-memory batch.

Processor treats historical on-chain data as an ordered execution log. It subscribes to log items and receives items in batches. The order of items within the batch is identical to the order of data within the historical chain blocks. The batch handler function is called once for each batch.

For Substrate-based chains execution log items can be of the following kinds:

  • Event items, corresponding to matching Substrate runtime events.
  • Call items, corresponding to matching calls executed by the Substrate runtime. All events emitted within a call are placed before the call. All the child calls are placed before the parent call. By default, both successful and failed calls will be handled.

Additional support is available for log items produced by the Frontier EVM pallet (see EVM support), the Contracts pallet (see Ink! support) and the Gear Messages pallet.

Further, processor can extract additional data by querying the historical runtime state and indeed any external API.