Skip to main content
Version: Old ArrowSquid docs

Indexing unfinalized blocks

Starting with the ArrowSquid release, Subsquid SDK supports indexing blocks before they are finalized, enabling real-time use cases. To take advantage of this feature, a squid processor must

  1. have a node RPC endpoint as one of its data sources and
  2. use a Database with hot blocks support (e.g. TypeormDatabase) in its processor.run() call, and
  3. (EVM only) not set the useArchiveOnly(true) setting.

With this, the processor will consider some blocks to be "hot":

The processor will periodically (interval setting for EVM, Substrate) poll the RPC endpoint for changes in consensus. When the consensus changes, it will re-run the batch handler with the new consensus data and ask the Database to adjust its state. The Database then must roll back the changes made due to orphaned blocks and apply the new changes. With this, the state of the Database reflects the current blockchain consensus at all times.

RPC ingestion can create a heavy load on node endpoints. With Archives the load is typically short and the total number of requests is low, but their frequency may be sufficient to trigger http 429 responses. Use private endpoints and rate limit your requests with the rateLimit chain source option (EVM/Substrate).