Storage Challenge System

Introduction

The storage challenge system is a fundamental component of Gitopia's decentralized storage infrastructure, designed to ensure the reliability and availability of stored repository data. This system employs a sophisticated verification mechanism that balances efficiency with security, ensuring that storage providers maintain genuine copies of the data they commit to storing.

The Challenge-Response Protocol

At the heart of our storage verification system lies the challenge-response protocol, which leverages Merkle trees to efficiently verify data availability. This approach, inspired by industry leaders like Filecoin and Arweave, provides a robust method for verifying storage without overwhelming the blockchain with excessive data.

Merkle Trees: The Foundation of Verification

Merkle trees serve as the backbone of our verification system. These binary trees organize data in a way that enables efficient verification of individual data chunks. Each leaf node contains a cryptographic hash of a data chunk, while non-leaf nodes contain hashes of their child nodes combined. The root node provides a single hash that represents the entire dataset.

This approach dramatically reduces the amount of data needed for verification. For example, verifying a 1 GiB file split into 256 KiB chunks requires only about 416 bytes of proof data, making the verification process both fast and cost-effective.

                    Root Hash (H1234)
                    /            \
                   /              \
            H12                    H34
           /  \                   /  \
          /    \                 /    \
        H1      H2            H3      H4
        |       |             |       |
    Data1    Data2        Data3    Data4

In this simplified example:

• Data1-4 represent chunks of the original data
• H1-4 are hashes of individual data chunks
• H12 and H34 are hashes of combined child hashes
• Root Hash (H1234) represents the entire dataset

A Merkle proof allows verification that a specific data chunk is part of the dataset without needing the entire dataset. It includes the data chunk being proven and the sibling hashes needed to reconstruct the path to the root. This process is extremely efficient; for a 1 GiB file with 256 KiB chunks (4,096 chunks), each proof requires only 12 hashes (384 bytes) and 12 hash computations. The storage provider sends the requested 256 KiB chunk and 384 bytes of proof data, totaling approximately 256.4 KiB for verification.

Challenge Generation and Verification

The blockchain generates challenges through a deterministic process that ensures both unpredictability and verifiability. Using the previous block's timestamp as a seed, the system creates random challenges that are distributed to storage providers.

Challenge Parameters

Each challenge includes several key components:

• The selected storage provider
• The target packfile or release asset to verify
• A specific chunk index within the packfile or release asset
• A response deadline (challenge_duration configured in the storage module params)

The random number generator uses the previous block's timestamp as a seed to ensure unpredictable but verifiable challenge selection.

Verification Process

When a challenge is initiated, the blockchain creates a challenge record containing the required CID, chunk index, response deadline, and the expected Merkle root. Storage providers must respond within the challenge period by submitting:

• The requested hash of the data chunk
• A Merkle proof demonstrating that the hash belongs to the stored packfile

Validators then verify that:

• The response arrived within the time window
• The provided hash and Merkle proof are valid against the on-chain root hash

This process ensures timely challenges, with proofs occurring off the main consensus path to prevent blockchain bloat and maintain transaction throughput.

Economic Incentives

The challenge system is supported by a carefully designed economic model that encourages reliable storage while discouraging malicious behavior. This Proof of Storage mechanism verifies data availability, rewards honest providers, and penalizes malpractice, ensuring the network is both secure and economically sustainable.

The incentive model has two phases:

Phase 1: Bootstrap Phase

In the bootstrap phase, the network jump-starts provider participation using token subsidies from the Ecosystem Incentives pool.

• Genesis Allocation: 7.5 million LORE.

• Block Emissions: 15% of block emissions are allocated to ecosystem incentives. These subsidies are distributed via a challenge-response protocol.

• Challenge Frequency: The network issues a challenge every challenge_interval_blocks (initially 1000 blocks, or ~30 minutes), resulting in ~48 challenges per day. Each challenge randomly targets a stored data chunk.

• Challenge Reward: A successful response earns a reward, this is calculated based on the reward_per_day` parameter and the number of active providers. This generous reward covers provider costs and provides a strong profit motive before user fee revenue becomes significant.

• Slashing: To deter cheating, providers must stake collateral. Failure to respond to a challenge correctly and on time results in a penalty. The slash amount is set high relative to the reward (e.g., 1250 LORE slashed for a 125 LORE reward) to make honesty the only rational strategy.

Table 1: Example Challenge Parameters (Bootstrap Phase)

Parameter	Bootstrap Value (Year 1)	Rationale/Notes
Max Providers	5
Challenge Interval	1000 blocks (≈30 min)	Balances verification frequency with network overhead.
Reward per Day	6000 LORE ($5 at current price)	Generous subsidy to attract providers.
Minimum Stake	1,000,000 LORE	Ensures providers have "skin in the game." Kept moderate initially to lower the barrier to entry.
Slashing Amount	1250 LORE	Penalty for a failed challenge. Set at 5x the reward to strongly disincentivize cheating.
Consecutive Fails Threshold	3	Number of consecutive failures for additional slashing.
Consecutive Fails Slash Percentage	1%	Percentage of stake slashed for consecutive failures.
Unstake Cooldown Blocks	28 days	Number of blocks to wait before completing unstaking.
Storage Price Per MB	0.01 LORE	Price per MB for storage updates beyond free storage.
Free Storage MB	150 MB	Free storage per user.
Max Providers	5	Maximum number of providers allowed in the network.
Enable Storage Challenges	true	Enable storage challenges.

Phase 2: Long-Term User-Funded Model

As the user base grows, the network transitions to a sustainable model where provider compensation is primarily funded by user fees.

• Phasing Out Subsidies: The challenge_reward from the Ecosystem Incentives pool is gradually tapered down via on-chain governance as user fee revenue increases.
• User Fees: Users pay for storage and bandwidth beyond a free tier.
• Pricing & Market Rates: User fees will be benchmarked against other services and are adjustable by governance.
• Reward Distribution from Fees: User fees are collected into a "Storage Fee Pool," and the challenge_reward is dynamically calculated based on the pool's size, distributing user payments to providers in proportion to their verifiable storage.

Free Usage Tier

To encourage developer adoption, a free usage tier is provided, offering up to 150 MB of persistent storage per user. This cost is subsidized by the network's Ecosystem Incentives pool. The free_storage_mb parameter is enforced on-chain, rejecting storage beyond the limit unless the user has sufficient balance.

Stake Requirements

Storage providers must maintain a minimum stake of 1,000,000 LORE (approximately $850 at current prices). This requirement ensures significant economic commitment, provides adequate slashing capacity, creates a meaningful barrier against Sybil attacks, and remains accessible for serious providers.

Challenge Mechanics

The challenge system operates through precisely defined parameters governing verification frequency, rewards, and penalties, balancing network security with operational efficiency.

1. Challenge Schedule: Challenges are automatically generated at regular intervals (every challenge_interval_blocks), ensuring a consistent verification process.
2. Challenge Structure and Rewards: Challenges are distributed to storage providers, who must respond within challenge_duration (e.g., 10 seconds). Providers earn rewards for successful responses, which are dynamically calculated based on the reward_per_day and the number of active providers. Providers must supply a valid Merkle proof and the requested data chunk for automatic verification and rewards on the blockchain.
3. Graduated Penalty System:
   • Minor Infractions: Failed challenges (invalid proof/data) or missed challenges (no response) result in a -1250 LORE penalty.
   • Major Infractions: consecutive_fails_threshold consecutive failures trigger a consecutive_fails_slash_percentage stake slash (e.g., 1% of stake).
4. Exit Conditions: There is a mandatory unstake_cooldown_blocks unstaking period (e.g., 28 days).
5. Implementation Safeguards: Challenges are invalidated if the packfile is updated during the challenge window.

These mechanics create a robust verification system that ensures consistent data availability, provides meaningful economic incentives, maintains network security, scales efficiently with network growth, and accommodates temporary network issues.

System Parameters and Governance

All system parameters are stored in the x/storage module and can be modified through governance proposals. This flexibility allows the network to adapt to changing market conditions and optimize performance based on real-world operation data.