Storage calls and state queries

It is sometimes impossible to extract the required data with only event and call data without querying the runtime state. The context exposes a lightweight gRPC client to the chain node accessible via ctx._chain. It exposes low-level methods for accessing the storage. However, the recommended way to query the storage is by generating type-safe access classes with Substrate typegen.

To enable the gRPC client, one must provide a chain data source to the processor:

const processor = new SubstrateBatchProcessor()
  .setDataSource({
    archive: lookupArchive("kusama")
    chain: 'wss://kusama-rpc.polkadot.io'
  })

tip

We recommend using private endpoints for better performance and stability of your squids. The standard approach is to keep the endpoint URL in an environment variable and set it via secrets when deploying to Aquarium.

Type-safe storage access with typegen

Substrate typegen tool generates storage access classes at src/types/storage.ts. The classes take historical runtime upgrades into account by exposing versioned getters (like asVxx). The generated access methods support single- and multi-key queries and allow listing storage keys and pairs.

Note that the generated getters always query the historical blockchain state at the "current" block derived the context. This is the recommended way to access the storage.

To generate storage classes with typegen:

• Set the specVersions to an archive endpoint URL. Use archive registry to get an up-to-date list of public archives.
• List fully qualified names of the storage items at the storage section of the typegen config. The format is ${PalleteName}.${KeyName}.
• Rerun the typegen with

sqd typegen

Here's an example of typegen.json for generating an access class for Balances.Reserves:

typegen.json

{
  "outDir": "src/types",
  "specVersions": "https://kusama.archive.subsquid.io/graphql", 
  "storage": [
    "Balances.Account"
  ]
}

To generate classes for all available storage calls, set "storage": true.

Inspect the generated storage access classes at src/types/storage.ts. The file contents should look similar to this:

src/types/storage.ts

/**
 * This code is generated by typegen
 */
export class SystemAccountStorage extends StorageBase {
  //.. constructors

  get isVxx(): boolean {
    return this.getTypeHash() === '2208f857b7cd6fecf78ca393cf3d17ec424773727d0028f07c9f0dc608fc1b7a'
  }

  get asVxx(): SystemAccountStorageVxx {
    assert(this.isVxx)
    return this as any
  }
}

export interface SystemAccountStorageVxx {
  get(key: Uint8Array): Promise<vxx.AccountInfo>
  getAll(): Promise<vxx.AccountInfo[]>
  getMany(keys: Uint8Array[]): Promise<vxx.AccountInfo[]>
  getKeys(): Promise<Uint8Array[]>
  getKeys(key: Uint8Array): Promise<Uint8Array[]>
  getKeysPaged(pageSize: number): AsyncIterable<Uint8Array[]>
  getKeysPaged(pageSize: number, key: Uint8Array): AsyncIterable<Uint8Array[]>
  getPairs(): Promise<[k: Uint8Array, v: vxx.AccountInfo][]>
  getPairs(key: Uint8Array): Promise<[k: Uint8Array, v: vxx.AccountInfo][]>
  getPairsPaged(pageSize: number): AsyncIterable<[k: Uint8Array, v: vxx.AccountInfo][]>
  getPairsPaged(pageSize: number, key: Uint8Array): AsyncIterable<[k: Uint8Array, v: vxx.AccountInfo][]>
}

The generated access interface provides methods for accessing:

• a single storage value with get(key)
• multiple values in a batch call with getMany(keys[])
• all storage values with getAll()
• all storage keys with getKeys()
• all keys with a given prefix with getKeys(keyPrefix) (only if the storage keys are decodable)
• paginated keys via getKeysPaged(pageSize) and getKeysPaged(pageSize, keyPrefix)
• key-value pairs via getPairs() and getPairs(keyPrefix)
• paginated key-value pairs via getPairsPaged(pageSize) and getPairsPaged(pageSize, keyPrefix)

Access Storage items

As previously mentioned, storage items are always retrieved at the "current" block height of BatchContext, the first argument of the storage class constructor. The height is determined by the block hash taken from the second constructor argument of type {hash: string}, typically a block header.

Example

processor.ts

import {BalancesAccountStorage} from "./types/storage"

const processor = new SubstrateBatchProcessor()
  .setBatchSize(50)
  .setBlockRange({ from: 9_000_000, to: 9_100_000})
  .setDataSource({
    // Lookup archive by the network name in the Subsquid registry
    archive: lookupArchive("kusama"),
    // use a private endpoint for production
    chain: 'wss://kusama-rpc.polkadot.io'
  }).includeAllBlocks({ from: 9_000_000, to: 9_100_000})

processor.run(new TypeormDatabase(), async ctx => {
  for (const block of ctx.blocks) { 
    // block.header is of type { hash: string }
    let storage = new BalancesAccountStorage(ctx, block.header)
    let aliceAddress = ss58.decode('5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY').bytes
    let aliceBalance = (await storage.asV1050.get(aliceAddress)).free
    ctx.log.info(`Alice free account balance at block ${block.header.height}: ${aliceBalance.toString()}`)
  }
})