Substrate Processor

Search…
Substrate Processor
The main explanation and overview for the Processor component of a Squid has been treated in the Key Concepts section. This page will go over the most important customizations a developer would want to make, when building their API.
Importing and instantiating
The Substrate Processor is defined in an npm package that needs to be installed before being able to use it:
Note: the subsquid-template already has this package in its dependencies
1
npm install @subsquid/substrate-processor
Copied!
Then the SubstrateProcessor class can be imported from the package
1
import {SubstrateProcessor} from "@subsquid/substrate-processor"
Copied!
Then, it's finally possible to declare an instance of it:
1
const processor = new SubstrateProcessor('kusama_balances')
Copied!
Note: all of the code snippets in this page can be found in the processor.ts file in the subsquid-template project
Setting data sources
The SubstrateProcessor class exposes a method to customize its data source, by specifying a blockchain RPC WebSocked URL and an Archive endpoint URL:
1
processor.setDataSource({
2
    archive: 'https://kusama.indexer.gc.subsquid.io/v4/graphql',
3
    chain: 'wss://kusama-rpc.polkadot.io'
4
})
Copied!
The argument of the function is an interface defined in the processor's source, intuitively called DataSource:
processor.ts
1
export interface DataSource {
2
    /**
3
     * Archive endpoint URL
4
     */
5
    archive: string
6
    /**
7
     * Chain node RPC websocket URL
8
     */
9
    chain: string
10
}
11
​
12
// ...
13
​
14
export class SubstrateProcessor {
15
​
16
    // ...
17
    setDataSource(src: DataSource): void {
18
        this.assertNotRunning()
19
        this.src = src
20
    }
21
}
Copied!
Batch sizes
Subsquids leaves to the developer the choice to set the amount of blocks to process in a single batch, which can be tweaked for performance reasons, for example.
Similarly to setting a data source, the SubstrateProcessor exposes a method for this:
1
processor.setBatchSize(500)
Copied!
Easily enough, the function is defined to accept a number as an argument:
processor.ts
1
export class SubstrateProcessor {
2
​
3
    // ...
4
    setBatchSize(size: number): void {
5
        this.assertNotRunning()
6
        assert(size > 0)
7
        this.batchSize = size
8
    }
9
}
Copied!
Start block/Global execution range
Subsquid also gives developers the power to reduce the scope of their blockchain exploration, if they want to. It is in fact possible to configure the SubstrateProcessor to handle a range of block. The starting block is mandatory, when invoking this method, but the end block can be omitted, which means the Squid will continue to process new blocks as they are written to the blockchain.
1
processor.setBlockRange({from: 583000});
Copied!
The argument of the function is an interface, defined in the processor's source code, and once again intuitively called Range.
range.ts
1
export interface Range {
2
    /**
3
     * Start of segment (inclusive)
4
     */
5
    from: number
6
    /**
7
     * End of segment (inclusive). Defaults to infinity.
8
     */
9
    to?: number
10
}
Copied!
And used in the function definition.
processor.ts
1
export class SubstrateProcessor {
2
​
3
    // ...
4
    setBlockRange(range: Range): void {
5
        this.assertNotRunning()
6
        this.blockRange = range
7
    }
8
}
Copied!
Types Bundle
It is mandatory for the Processor correct execution to specify the types Bundle it needs to take into account. I can be done this way:
1
processor.setTypesBundle('kusama');
Copied!
The function parameter can either be
the name of a blockchain, for which Subsquid has built-in typesBundle​
the path to a JSON file containing the definitions
an object, containing the definitions, respecting the typing of the OldTypesBundle interface
Here is how it's defined in the SubstrateProcessor class:
processor.ts
1
export class SubstrateProcessor {
2
​
3
    // ...
4
    setTypesBundle(bundle: string | OldTypesBundle): void {
5
        this.assertNotRunning()
6
        if (typeof bundle == 'string') {
7
            this.typesBundle = getOldTypesBundle(bundle) || readOldTypesBundle(bundle)
8
        } else {
9
            this.typesBundle = bundle
10
        }
11
    }
12
}
Copied!
Where the two getOldTypesBundle and readOldTypesBundle functions are defined as such:
io.ts
1
export function getOldTypesBundle(chain: string): OldTypesBundle | undefined {
2
    switch(chain) {
3
        // ...
4
        case 'kusama':
5
            return require('./old/definitions/kusama').bundle
6
        // ...
7
        case 'polkadot':
8
            return require('./old/definitions/polkadot').bundle
9
        default:
10
            return undefined
11
    }
12
}
13
​
14
​
15
export function readOldTypesBundle(file: string): OldTypesBundle {
16
    let content: string
17
    try {
18
        content = fs.readFileSync(file, 'utf-8')
19
    } catch(e: any) {
20
        throw new OldTypesBundleError(`Failed to read ${file}: ${e}`)
21
    }
22
    let json: any
23
    try {
24
        json = JSON.parse(content)
25
    } catch(e: any) {
26
        throw new OldTypesBundleError(`Failed to parse ${file}: ${e}`)
27
    }
28
    return json
29
}
Copied!
Which, respectively, collect a built-in typesBundle from the available list, or process a JSON file, given its path.
API Extensions
EVM Processor
Last modified 1mo ago
Copy link
Contents
Importing and instantiating
Setting data sources
Batch sizes
Start block/Global execution range
Types Bundle