Skip to main content
Version: Firesquid

Develop a squid

Below is a general outline of the squid development steps.

0. Prerequisites

1. Model the data with a schema file

Start the development by defining the schema of the target database in the schema.graphql in the squid root folder. The schema file also defines the shape of the GraphQL API of the squid. The schema definition consists of regular graphql type declarations annotated with custom directives to define:

  • relations between the entities
  • entity properties, property types and constraints
  • indexes to be created in the database

A full description of the schema.graphql dialect is available here.

2. Generate TypeORM classes

The squid processor data handlers use TypeORM entities to interact with the target database during the data processing. All necessary entity classes are generated by the squid framework from schema.graphql with make codegen.

By convention, the generated model classes are kept in src/model/generated by default. Custom user-defined entities can be added in src/model/index.ts.

3. Generate the database migrations

The database schema changes (including the initial DDL) are applied through migration files located at db/migrations. The migrations are generated by

npx squid-typeorm-migration generate

The migrations are generated by squid-typeorm-migration(1) to match the difference between the database entity classes generated at the previous step and the local database schema. Consult schema updates for more details.

4. Define the squid processor and the data handlers

A squid processor is a separate node.js process that fetches historical on-chain data, performs arbitrary transformations and saves the result into the target database schema defined above. By convention, the processor entry point is src/processor.ts.

Subsquid SDK currently offers support only for Substrate-based chains with two base classes for data processing: SubstrateBatchProcessor for batch processing and SubstrateProcessor. We recommend using SubstrateBatchProcessor as it enables up to a 100x performance improvement compared to SubstrateProcessor.

The structure for working with a processor instance is the following:

  • Initialize the processor by defining the archive data source for the network to be indexed.
  • Specify which data should be fetched by the processor. The data selectors may depend on the type of the source chain. For Substrate chains one can subscribe for block data, call data, or runtime events. Additional options are available for substrate chains supporting EVM and WASM contracts to index only transactions or event logs emitted by a specific contract. See the processor configuration section.
  • Generate typescript definitions for the Substrate, EVM and WASM data types together with the data desereliazation boilerplate. See Typegen for more details.
  • Define the processor handler(s) that transform the data and save it into the target database schema. See the data handlers section.
  • Run the processor with make process.

A detailed walk-through is in the dedicated Squid Processor section

5. Run locally and test the GraphQL endpoint

Follow Run a Squid page for details. Reiterate the steps 1, 2, 3 if there's a need to change the data model.

What's next?