Skip to content

Commit 9a0a02a

Browse files
shimkivshimkiv
authored
Merge branch 'main' into feat/remove-pre-genesis-timestamp
2 parents d820335 + 786a620 commit 9a0a02a

22 files changed

+1934
-21
lines changed

‎docs/berkeley-upgrade/appendix.mdx

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
title: Appendix
3+
sidebar_label: Appendix
4+
hide_title: true
5+
description: Berkeley Upgrade Appendix
6+
keywords:
7+
- Berkeley
8+
- upgrade
9+
- appendix
10+
---
11+
12+
# Appendix
13+
14+
## Migration from o1labs/client-sdk to mina-signer
15+
16+
The signing library `o1labs/client-sdk` was deprecated some time ago and will stop working after the Mina mainnet upgrade. All users should upgrade to use the [mina-signer](https://www.npmjs.com/package/mina-signer) library.
17+
18+
Below you will find an example of how to use the `mina-signer` library. Please keep in mind the following:
19+
20+
1. Make sure to adjust the `nonce` to the correct nonce on the account you want to use as "sender"
21+
1. Update the `url` variable with an existing Mina Node GraphQL endpoint
22+
23+
```javascript
24+
import { Client } from 'mina-signer';
25+
26+
// create the client and define the keypair
27+
28+
const client = new Client({ network: 'testnet' }); // Mind the `network` client configuration option
29+
30+
const senderPrivateKey = 'EKFd1Gx...'; // Sender's private key
31+
const senderPublicKey = 'B62qrDM...'; // Sender's public key, perhaps derived from the private key using `client.derivePublicKey(senderPrivateKey)`;
32+
33+
// define and sign payment
34+
35+
let payment = {
36+
from: senderPublicKey,
37+
to: 'B62qkBw...', // Recipient public key
38+
amount: 100,
39+
nonce: 1,
40+
fee: 1000000,
41+
};
42+
43+
const signedPayment = client.signPayment(payment, senderPrivateKey);
44+
45+
// send payment to graphql endpoint
46+
47+
const url = 'https://qanet.minaprotocol.network/graphql';
48+
49+
const sendPaymentMutationQuery = `
50+
mutation SendPayment($input: SendPaymentInput!, $signature: SignatureInput!) {
51+
sendPayment(input: $input, signature: $signature) {
52+
payment {
53+
hash
54+
}
55+
}
56+
}
57+
`;
58+
const graphQlVariables = {
59+
input: signedPayment.data,
60+
signature: signedPayment.signature,
61+
};
62+
const body = JSON.stringify({
63+
query: sendPaymentMutationQuery,
64+
variables: graphQlVariables,
65+
operationName: 'SendPayment',
66+
});
67+
68+
const paymentResponse = await fetch(url, {
69+
method: 'POST',
70+
headers: { 'Content-Type': 'application/json' },
71+
body
72+
});
73+
74+
const paymentResponseJson = await paymentResponse.json();
75+
if (paymentResponse.ok) {
76+
console.log(`Transaction hash: ${paymentResponseJson.data.sendPayment.payment.hash}`);
77+
} else {
78+
console.error(JSON.stringify(paymentResponseJson));
79+
}
80+
81+
82+
```

‎docs/berkeley-upgrade/archive-migration/appendix.mdx

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
---
2+
title: Appendix
3+
sidebar_label: Appendix
4+
hide_title: true
5+
description: archive node schema changes between Mainnet and Berkeley
6+
keywords:
7+
- Berkeley
8+
- upgrade
9+
- archive migration
10+
- appendix
11+
- mina archive node
12+
- archive node
13+
---
14+
15+
# Appendix
16+
17+
## Archive node schema changes
18+
19+
If you are using the Archive Node database directly for your system integrations, then you should understand all the changes that might impact your applications. The most important change is that the `balances` table in the Berkeley schema will no longer exist. In the new schema, it is replaced with the table `accounts_accessed` - from an application semantics point of view, the data in `accounts_accessed` is still the same.
20+
21+
In the Berkeley protocol, accounts can now have the same public key but a different token_id. This means accounts are identified by both their public key and token_id, not just the public key. Consequently, the foreign key for the account in all tables is account_identifier_id instead of public_key_id.
22+
23+
### Schema differences
24+
- **Removed Types**
25+
- The options `create_token`, `create_account`, and `mint_tokens` have been removed from the user_command_type enumeration.
26+
- Indexes Dropped
27+
- We've removed several indexes from tables, this may affect how you search and organize data:
28+
- `idx_public_keys_id`
29+
- `idx_public_keys_value`
30+
- `idx_snarked_ledger_hashes_value`
31+
- `idx_blocks_id`
32+
- `idx_blocks_state_hash`
33+
- **Table Removed**
34+
- The `balances` table is no longer available.
35+
- **New Tables Added**
36+
- We've introduced the following new tables:
37+
- `tokens`
38+
- `token_symbols`
39+
- `account_identifiers`
40+
- `voting_for`
41+
- `protocol_versions`
42+
- `accounts_accessed`
43+
- `accounts_created`
44+
- `zkapp_commands`
45+
- `blocks_zkapp_commands`
46+
- `zkapp_field`
47+
- `zkapp_field_array`
48+
- `zkapp_states_nullable`
49+
- `zkapp_states`
50+
- `zkapp_action_states`
51+
- `zkapp_events`
52+
- `zkapp_verification_key_hashes`
53+
- `zkapp_verification_keys`
54+
- `zkapp_permissions`
55+
- `zkapp_timing_info`
56+
- `zkapp_uris`
57+
- `zkapp_updates`
58+
- `zkapp_balance_bounds`
59+
- `zkapp_nonce_bounds`
60+
- `zkapp_account_precondition`
61+
- `zkapp_accounts`
62+
- `zkapp_token_id_bounds`
63+
- `zkapp_length_bounds`
64+
- `zkapp_amount_bounds`
65+
- `zkapp_global_slot_bounds`
66+
- `zkapp_epoch_ledger`
67+
- `zkapp_epoch_data`
68+
- `zkapp_network_precondition`
69+
- `zkapp_fee_payer_body`
70+
- `zkapp_account_update_body`
71+
- `zkapp_account_update`
72+
- `zkapp_account_update_failures`
73+
- **Updated Tables**
74+
- The following tables have been updated
75+
- `timing_info`
76+
- `user_commands`
77+
- `internal_commands`
78+
- `epoch_data`
79+
- `blocks`
80+
- `blocks_user_commands`
81+
- `blocks_internal_commands`
82+
83+
### Differences per table
84+
- **`timing_info`**
85+
- Removed columns:
86+
- `token`
87+
- `initial_balance`
88+
- **`user_commands`**
89+
- Removed columns:
90+
- `fee_token`
91+
- `token`
92+
- **`internal_commands`**
93+
- Removed columns:
94+
- `token`
95+
- Renamed column
96+
- `command_type` to `type`
97+
- **`epoch_data`**
98+
- Added columns:
99+
- `total_currency`
100+
- `start_checkpoint`
101+
- `lock_checkpoint`
102+
- `epoch_length`
103+
- **`blocks`**
104+
- Added columns:
105+
- `last_vrf_output`
106+
- `min_window_density`
107+
- `sub_window_densities`
108+
- `total_currency`
109+
- `global_slot_since_hard_fork`
110+
- `global_slot_since_genesis`
111+
- `protocol_version_id`
112+
- `proposed_protocol_version_id`
113+
- Removed column:
114+
- `global_slot`
115+
- **`blocks_user_commands`**
116+
- Removed columns:
117+
- `fee_payer_account_creation_fee_paid`
118+
- `receiver_account_creation_fee_paid`
119+
- `created_token`
120+
- `fee_payer_balance`
121+
- `source_balance`
122+
- `receiver_balance`
123+
- Added index:
124+
- `idx_blocks_user_commands_sequence_no`
125+
- **`blocks_internal_commands`**
126+
- Removed columns:
127+
- `receiver_account_creation_fee_paid`
128+
- `receiver_balance`
129+
- Added indexes:
130+
- `idx_blocks_internal_commands_sequence_no`
131+
- `idx_blocks_internal_commands_secondary_sequence_no`
132+
133+
### Rosetta API new operations
134+
135+
The Berkeley upgrade introduces two new operation types:
136+
- `zkapp_fee_payer_dec`
137+
- `zkapp_balance_change`

‎docs/berkeley-upgrade/archive-migration/archive-migration-installation.mdx

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
---
2+
title: Installing the archive migration package
3+
sidebar_label: Installing archive migration package
4+
hide_title: false
5+
description: Satisfying the archive migration prerequisites.
6+
keywords:
7+
- Berkeley
8+
- upgrade
9+
- archive migration
10+
- installing
11+
- prerequisites
12+
- mina archive node
13+
- archive node
14+
---
15+
16+
The archive node Berkeley migration package is sufficient for satisfying the migration from Devnet/Mainnet to Berkeley.
17+
However, it has some limitations. For example, the migration package does not migrate a non-canonical chain and it skips orphaned blocks that are not part of a canonical chain.
18+
19+
To mitigate these limitations, the archive node maintenance package is available for use by archive node operators who want to maintain a copy of their Devnet and Mainnet databases for historical reasons.
20+
21+
## Install with Google Cloud SDK
22+
23+
The Google Cloud SDK installer does not always register a `google-cloud-sdk` apt package. The best way to install gsutil is using the apt repostory:
24+
25+
```sh
26+
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
27+
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
28+
sudo apt-get update && sudo apt-get install google-cloud-sdk
29+
```
30+
31+
## Download the o1labs Mainnet archive database
32+
33+
We strongly encourage you to perform the migration on your own data to preserve the benefits of decentralization. However, if you want to use the archive data that o1labs runs (for example, to bootstrap a new archive from SQL without waiting all day for the chain to download and replay), you can use the following steps:
34+
35+
1. Download the Devnet/Mainnet archive data using cURL or gsutil:
36+
37+
- cURL:
38+
39+
For Devnet:
40+
```sh
41+
curl https://storage.googleapis.com/mina-archive-dumps/devnet-archive-dump-{date}_0000.sql.tar.gz
42+
```
43+
44+
For Mainnet:
45+
```sh
46+
curl https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-{date}_0000.sql.tar.gz
47+
```
48+
49+
To filter the dumps by date, replace `{date}` using the required `yyyy-dd-mm` format. For example, for March 15, 2024, use `2024-03-15`.
50+
51+
:warning: The majority of backups have the `0000` suffix. If a download with that name suffix is not available, try incrementing it. For example, `0001`, `0002`, and so on.
52+
53+
- gsutil:
54+
55+
```sh
56+
gsutil cp gs://mina-archive-dumps/mainnet-archive-dump-2024-01-15* .
57+
```
58+
59+
2. Extract the tar package.
60+
61+
```sh
62+
tar -xvzf {network}-archive-dump-{date}_0000.sql.tar.gz {network}-archive-dump-{date}_0000.sql
63+
```
64+
65+
3. Import the Devnet/Mainnet archive dump into the Berkeley database.
66+
67+
Run this command at the database server:
68+
69+
```sh
70+
psql -U {user} -f {network}-archive-dump-{date}_0000.sql
71+
```
72+
73+
The database in the dump **archive_balances_migrated** is created with the Devnet/Mainnet archive schema.
74+
75+
Note: This database does not have any Berkeley changes.
76+
77+
## Ensure the location of Google Cloud bucket with the Devnet/Mainnet precomputed blocks
78+
79+
The recommended method is to perform migration on your own data to preserve the benefits of decentralization.
80+
81+
`gsutil cp gs://mina_network_block_data/{network}-*.json .`
82+
83+
:warning: Precomputed blocks for the Mainnet network take ~800 GB of disk space. Plan for adequate time to download these blocks. The Berkeley migration app downloads them incrementally only when needed.
84+
85+
## Validate the Devnet/Mainnet database
86+
87+
The correct Devnet/Mainnet database state is crucial for a successful migration.
88+
89+
[Missing blocks](/berkeley-upgrade/archive-migration/mainnet-database-maintenance#missing-blocks) is one the most frequent issues when dealing with the Devnet/Mainnet archive. Although this step is optional, it is strongly recommended that you verify the archive condition before you start the migration process.
90+
91+
To learn how to maintain archive data, see [Devnet/Mainnet database maintenance](/berkeley-upgrade/archive-migration/mainnet-database-maintenance).
92+
93+
## Download the migration applications
94+
95+
Migration applications are distributed as part of the archive migration Docker and Debian packages.
96+
97+
Choose the packages that are appropriate for your environment.
98+
99+
### Debian packages
100+
101+
To get the Debian packages:
102+
103+
```
104+
CODENAME=bullseye
105+
CHANNEL=stable
106+
VERSION=3.0.1-e848ecb
107+
108+
echo "deb [trusted=yes] http://packages.o1test.net $CODENAME $CHANNEL" | tee /etc/apt/sources.list.d/mina.list
109+
apt-get update
110+
apt-get install --allow-downgrades -y "mina-archive-migration=$VERSION"
111+
```
112+
113+
### Docker image
114+
115+
To get the Docker image:
116+
117+
```
118+
docker pull gcr.io/o1labs-192920/mina-archive-migration:3.0.1-e848ecb-{codename}
119+
```
120+
121+
Where supported codenames are:
122+
- bullseye
123+
- focal
124+
- buster
125+
126+
127+
## Devnet/Mainnet genesis ledger
128+
129+
The Mina Devnet/Mainnet genesis ledger is stored in GitHub in the `mina` repository under the `genesis_ledgers` subfolder. However, if you are already running a daemon that is connected to the Mina Mainnet or the Devnet network, you already have the genesis ledger locally.
130+
131+
## Berkeley database schema files
132+
133+
You can get the Berkeley schema files from different locations:
134+
135+
- GitHub repository from the `berkeley` branch.
136+
137+
Note: The `berkeley` branch can contain new updates regarding schema files, so always get the latest schema files instead of using an already downloaded schema.
138+
139+
- Archive/Rosetta Docker from `berkeley` version
140+
141+
### Example: Downloading schema sources from GitHub
142+
143+
```sh
144+
wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql
145+
146+
wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql
147+
```
148+
149+
## Next steps
150+
151+
Congratulations on completing the essential preparation and verification steps. You are now ready to perform the migration steps in [Migrating Devnet/Mainnet Archive to Berkeley Archive](/berkeley-upgrade/archive-migration/migrating-archive-database-to-berkeley).

0 commit comments

Comments
 (0)