SQD
SQD is a decentralized data lake and query engine designed to provide permissionless access to blockchain data at scale, serving as the infrastructure layer for AI agents and decentralized applications to efficiently query and process large volumes of on-chain data [1].
Overview
SQD (formerly known as Subsquid) functions as an emergent and open database network that addresses one of the fundamental challenges in blockchain technology: accessing and processing exponentially growing amounts of on-chain data. The platform enables developers to query and aggregate blockchain data, application data (decoded smart contract states), and relevant off-chain data in a decentralized manner [2].
Unlike traditional centralized data lakes used in Web2 (such as BigQuery, Snowflake, or Apache), SQD provides a decentralized alternative specifically designed for Web3 applications. The network is built with several key properties: infinite horizontal scalability, permissionless data access, credible neutrality, trust-minimized queries, and low maintenance costs [2].
The SQD ecosystem consists of multiple components working together to provide a comprehensive data infrastructure solution. These include the SQD Network (a decentralized data lake), Squid SDK (a developer toolkit for indexing blockchain data), and SQD Cloud (an enterprise-grade hosted indexing service) [1].
Technology
The SQD Network architecture is built around several key components that work together to provide a decentralized data infrastructure:
Data Providers
Data Providers produce the data served by the SQD Network. Currently, the network focuses on on-chain data from EVM and Substrate chains, with plans to add support for Cosmos and Solana. During the bootstrapping phase, Subsquid Labs GmbH acts as the primary data provider, serving as a proxy for chains from which data is ingested block-by-block. The ingested data is validated, split into compressed chunks, and saved into persistent storage for distribution among workers [3].
Workers
Workers contribute storage and compute resources to the network, serving data in a peer-to-peer manner. Each worker must register on-chain by bonding 100,000 SQD tokens, which can be slashed if the worker violates protocol rules. SQD holders can delegate tokens to specific workers to signal reliability and earn a portion of the rewards. Worker rewards are distributed each epoch and depend on factors such as online time, data served, delegated tokens, fairness, and liveness [4].
Scheduler
The Scheduler distributes data chunks submitted by data providers among the workers. It monitors updates to datasets and worker sets, sending requests to workers to download new chunks or redistribute existing data chunks based on capacity and target redundancy for each dataset [5].
Logs Collector
The Logs Collector gathers liveness pings and query execution logs from workers, batches them, and saves them into public persistent storage. These logs, signed by workers' P2P identities and pinned to IPFS, are stored for at least six months and used by other network participants [6].
Reward Manager
The Reward Manager calculates and submits worker rewards on-chain for each epoch. Rewards are based on worker liveness, delegated tokens, served queries (measured in bytes), and liveness since registration. The Reward Manager accesses logs, calculates rewards, and submits claimable commitments on-chain for each epoch [7].
Query Validation
SQD Network provides economic guarantees for the validity of queried data, with the possibility of validating specific queries on-chain. All query responses are signed by the worker who executed the query, acting as a commitment. The smart contract validation logic may be dataset-specific, with options including Proof by Authority, Optimistic on-chain validation, and Zero-Knowledge proofs [8].
SQD Token
The SQD token is an ERC-20 protocol token native to the SQD Network ecosystem. The token was deployed on the Ethereum mainnet and bridged to Arbitrum One. It serves multiple functions within the ecosystem:
- Incentive Alignment: SQD rewards node operators who contribute computation and storage resources to the network.
- Network Curation: Through delegation, SQD enables permissionless selection of trustworthy operators for rewards.
- Resource Consumption: Users can lock SQD tokens to increase their rate limits for data access.
- Governance: SQD tokenholders can participate in decision-making regarding protocol changes and proposals [9].
Tokenomics
The SQD token has a maximum supply of 1,337,000,000 tokens. The token includes a deflationary mechanism where protocol revenues are partially used to reduce the supply over time [10].
For data consumers, the SQD token implements a locking mechanism with boosters to incentivize longer lock periods. The longer tokens are locked, the more compute units (CUs) are allocated per SQD/year. This virtual yield system allows gateway operators to submit more queries to the network based on their locked tokens [11].
The network rewards are paid to workers and delegators for each epoch from a rewards pool. During the initial 3-year bootstrapping phase, the reward cap and total supply of SQD are fixed. Afterward, the reward cap drops significantly until governance determines the inflation schedule and new tokens are minted to replenish the reward pool [12].
Use Cases
SQD is designed to serve various use cases across the blockchain ecosystem:
- AI Agents: SQD provides the data infrastructure for autonomous AI agents that require access to blockchain data. The platform enables these agents to query and process large volumes of data efficiently [1].
- DeFi Applications: Financial applications can use SQD to access historical transaction data, smart contract states, and other relevant information for analytics and decision-making [13].
- Gaming: Blockchain games can leverage SQD for efficient data access, enabling better player experiences and game mechanics [14].
- Privacy Solutions: Applications focused on privacy can utilize SQD's infrastructure to process data while maintaining user privacy [15].
- Social Applications: Decentralized social platforms can use SQD to manage and query user data and interactions [16].
The platform's architecture makes it particularly suitable for applications that require high-throughput, low-latency access to blockchain data, especially in scenarios where traditional centralized solutions would be prohibitively expensive or technically challenging to implement. [17] [18] [19] [20] [21]
