Quickstart

Setup

Clone the repo:

gh repo clone ethereum-optimism/Retro-Funding

Navigate to the repo:

cd Retro-Funding

Install the dependencies:

poetry install

Activate the virtual environment:

poetry env activate

Then, you can execute the commands below to test the simulation pipeline.

Simulation Pipeline

Data

Note

We have provided raw data files in the data directory. There are test datasets as well as the latest data for actual projects available.

You can also query Retro Funding data directly from OSO by following the guides here.

Each month, we will dump the latest data into the data directory.

Note that the project_id for each project is generated by OSO and is not the same as the project_id used by OP Atlas. To get the OP Atlas identifier for a project, query the project_name field in the projects_v1 table. The display_name is the name of the project as it will appear on OP Atlas's website.

Weights

The weights for each model are stored in the eval-algos/S7/weights directory.

Be sure to use the right data snapshot and simulation period for your simulation.

data_snapshot:
  data_dir: 'eval-algos/S7/data/M1/onchain/'
  projects_file: 'onchain_builders.csv'
  metrics_file: 'onchain_metrics_by_project.csv'

simulation:
  periods:
    previous: 'Jan 2025'
    current: 'Feb 2025'

You can also modify the weights for each model as you see fit:

  metrics:
    amortized_contract_invocations_monthly: 0.25
    gas_fees_monthly: 0.25
    average_tvl_monthly: 0.25
    active_farcaster_users_monthly: 0.25

    active_addresses_monthly: 0.0
    contract_invocations_monthly: 0.0
  
  # Variant weights must sum to 1.0
  metric_variants:
    adoption: 0.20    # Current period value
    growth: 0.20      # Change from previous period
    retention: 0.60   # Minimum of current and previous

The available metrics can be found in the onchain_metrics_by_project.csv file for a given data snapshot.

Run the Pipeline

Then, from the root directory, run:

poetry run python eval-algos/S7/models/onchain_builders.py
poetry run python eval-algos/S7/models/devtooling_openrank.py

This will run the model and save the results to the corresponding data directories using the default "testing" data and weights.

To run the model with an existing set of data and weights, include the YAML config file as an argument:

poetry run python eval-algos/S7/models/onchain_builders.py onchain_builders_goldilocks.yaml
poetry run python eval-algos/S7/models/devtooling_openrank.py devtooling_arcturus.yaml

You can now make your own changes to the weights and run the pipeline again, eg:

poetry run python eval-algos/S7/models/onchain_builders.py <my_weights.yaml>

You can also simulate the funding allocation process by running:

poetry run python eval-algos/scripts/process_onchain_builders.py <my_weights.yaml>
poetry run python eval-algos/scripts/process_devtooling_openrank.py <my_weights.yaml>

Note that the budget and min/max caps are hardcoded in the scripts, so you will need to make any changes to the budget or caps directly in the scripts for now.

alloc = AllocationConfig(
    budget=8_000_000/6,
    min_amount_per_project=200, 
    max_share_per_project=0.05
)

Here is a Loom Video tutorial for further guidance.

Simulate Funding Allocations

We have also provided a module to allocate funding to the projects based on the results from the simulation pipeline.

You can see how this works by looking at the TestAlgo notebook.

For example:

import pandas as pd
import sys

sys.path.append('./eval-algos/S7/models/')
from onchain_builders import OnchainBuildersCalculator, load_config, load_data
from utils.allocator import AllocationConfig, allocate_with_constraints

# Load the data and config
ds, sim_cfg = load_config('eval-algos/S7/weights/onchain_builders_testing.yaml')
calculator = OnchainBuildersCalculator(sim_cfg)
df_data = load_data(ds)

# Run the analysis
analysis = calculator.run_analysis(df_data)

# Get the scores (and use the display_name as the index)
scores = analysis['final_results']['weighted_score'].reset_index().set_index('display_name')['weighted_score']

# Configure the budget and allocation constraints
alloc = AllocationConfig(
  budget=1_000_000,
  min_amount_per_project=200,
  max_share_per_project=0.05
)

# Allocate the funding
rewards = allocate_with_constraints(scores, alloc)

Model Details

Onchain Builders

The Onchain Builders model analyzes and scores active Superchain projects based on their onchain activity. It processes raw metrics through a multi-step pipeline to produce normalized, weighted scores that reflect project performance across different chains and time periods.

Pipeline Steps

1. Filter and Pivot Raw Metrics

• Takes raw metrics data with non-zero weights for specified measurement periods
• Pivots data by chain and metric to create a structured view
• Groups by project_id, project_name, display_name, and chain

2. Sum and Weight by Chain

• Applies chain-specific weights (e.g., different weights for OP Mainnet vs other chains)
• Sums weighted metrics across all chains for each project
• Preserves project metadata in the aggregation

3. Calculate Metric Variants

For each metric, computes three variants:

• Adoption: Current period value
• Growth: Positive difference between current and previous period values
• Retention: Minimum value between current and previous periods

4. Normalize Metric Variants

• Applies min-max normalization to each metric variant
• Scales values to [0,1] range while preserving null values (e.g., TVL for non-DeFi projects)
• Uses fallback center value (0.5) when range is zero

5. Apply Weights

• Multiplies each normalized metric variant by:
  • Its metric-specific weight
  • Its variant-specific weight (adoption/growth/retention)

6. Aggregate Final Scores

• Combines weighted variants using power mean (p=2)
• Normalizes final scores to sum to 1.0

7. Prepare Results

• Flattens multi-level columns for readability
• Merges intermediate results for transparency
• Sorts projects by final weighted score

Devtooling OpenRank

The Devtooling model evaluates open-source developer tools by analyzing their relationships with onchain projects through package dependencies and developer contributions. It uses an EigenTrust-based algorithm released by OpenRank to propagate value through the graph.

Pipeline Steps

1. Build Unweighted Graph

Constructs a directed graph with three types of edges:

• Package Dependencies: Onchain projects → Devtooling projects
• Commit Events: Onchain projects → Developers
• GitHub Engagement: Developers → Devtooling projects

Removes duplicate edges when an onchain project is also a devtooling project.

2. Compute Onchain Project Pretrust

• Uses economic metrics from onchain projects
• Applies log transformation and min-max scaling
• Combines metrics using configured weights
• Normalizes scores to sum to 1.0

3. Compute Devtooling Project Pretrust

• Uses GitHub metrics (num packages, stars, forks, etc.)
• Applies log transformation and min-max scaling
• Combines metrics using configured weights
• Normalizes scores to sum to 1.0

4. Compute Developer Reputation

• Developers are pre-filtered to focus on active developers committing code to onchain project repos in Rust, Solidity, TypeScript, and Vyper
• Distributes onchain project pretrust to developers based on commit activity

5. Weight Edges

Applies weights based on:

• Link type (package dependency, commit, GitHub engagement)
• Event type (NPM, CARGO, COMMIT_CODE)
• Time decay for non-static relationships
• More recent interactions contribute more weight in most cases

6. Apply EigenTrust

• Combines pretrust scores from all sources
• Runs EigenTrust propagation on weighted graph
• Computes final trust scores for each node

7. Rank and Evaluate Projects

• Ranks devtooling projects by final EigenTrust scores
• Applies eligibility criteria:
  • Minimum package dependency count
  • Minimum developer link count
• Normalizes scores among eligible projects

8. Serialize Value Flow

• Uses iterative proportional fitting (IPF)
• Creates detailed value flow attribution
• Ensures contribution sums match:
  • Per devtool: Sum equals its overall score
  • Per onchain project: Sum equals its pretrust

Data Updates

Credentials

Add your BigQuery credentials to the root directory as credentials.json. Update the GCP_CREDENTIALS and GCP_PROJECT_ID in eval-algos/scripts/config.py to match your project.

See here for more information on how to get your BigQuery credentials.

Retrieve the Latest Data

Note

This feature is currently broken. We are working on fixing it. Instead, get the latest data from OSO by following the guides here.

From the root directory, run:

python eval-algos/scripts/fetch_data.py

This will download the data from BigQuery and save it to the data directory. You can review the included models in eval-algos/scripts/config.py.

Note: the models in the config file may not always be the same as the latest ones used for the algorithms, so for the time being, we recommend using the raw data files in the data directory.