This section applies to squids indexing EVM chains. See the supported networks page for a full list.
Overview and the data model
A squid processor is a Node.js process that fetches historical on-chain data from an Archive and/or a chain node RPC endpoint, performs arbitrary transformations and saves the result.
EvmBatchProcessor is the central class that handles EVM data extraction, transformation and persistence. By convention, the processor entry point is
src/main.ts; it is started by calling
EvmBatchProcessor.run() there. A single batch handler function supplied to that method is responsible for transforming data from multiple blocks in a single in-memory batch.
A batch provides iterables to access all items requested in processor configuration, which may include logs, transactions, traces and contract state diffs; see the batch context and block data pages for details. Further, the processor can extract additional data by querying the historical chain state and indeed any external API.
Starting with the ArrowSquid release, the processor can ingest data either from an Archive or directly from an RPC endpoint. If both an Archive and an RPC endpoint are provided, the processor will use the Archive until it reaches the highest block available there, then index the few remaining blocks using the RPC endpoint. This allows squids to combine low sync times with near real-time chain data access. It is, however, possible to use either just the Archive (e.g. for analytics) or just the RPC endpoint (e.g. for local development).