Skip to content

Commit 095268d

Browse files
tomusdrwnodebreaker0-0
authored andcommitted
Increase priority of ImOnline heartbeats (paritytech#3972)
* Make sure im-online reports are high priority. * Bump runtime. base-code edit option environment block num - test metrics fn push and finalized listing best_block_num listing Add translate API for storage values (paritytech#3947) * Add translate item. * fix * doc * fix doc * A test added. * Apply suggestions from code review Co-Authored-By: Bastian Köcher <[email protected]> * address suggestion im-online: account for block authorship (paritytech#3973) * im-online: account for block authorship * im-online: add test for block authorship onlineness * im-online: cleanup * im-online: fix test grandpa: Use storage proofs for Grandpa authorities (paritytech#3734) * grandpa: Write Grandpa authorities to well known key. Instead of requiring execution proofs for Grandpa authorities, this enables much simpler storage proofs. * grandpa: Introduce named AuthorityList type. * grandpa: Storage migration for srml-grandpa module. * Remove no-longer-used GrandpaApi runtime API. * grandpa: Write AuthorityList to storage with encoding version. We expect the AuthorityList type may change (eg. key changes). To make upgrades smoother, include a version in the stored value. * Bump node runtime spec version. * Update srml/grandpa/src/lib.rs Co-Authored-By: André Silva <[email protected]> Add the code for compiling node-cli for WASM-browser (paritytech#3974) * Extract CLI to separate module in node/cli * Make node/cli compile for WASM * More work on node/cli browser * More work on browser node * More work * More work * Purge a bit the CI script * More clean up * Remove substrate-finality-grandpa from the CI Its tests use tokio, which fails to compile. * Address review * Add rocksdb feature to the service * Fix substrate-service WASM CI * Apply suggestions from code review Co-Authored-By: Bastian Köcher <[email protected]> * Don't WASM-compile substrate-service altogether Remove NetworkSpecialization::on_event (paritytech#3976) and backend reference to rpc builder (paritytech#3979) Improve doc for storages in srml-support (paritytech#3982) * improve doc * Apply suggestions from code review Retire `storage_items!` (paritytech#3950) * Retire storage_items * Add storage_items! tests to srml/support/tests * Assimilate genesis config Revert "grandpa: Use storage proofs for Grandpa authorities (paritytech#3734)" (paritytech#3983) This reverts commit c3b1a98. Print warning again if polling network is too long (paritytech#3984) Add events for im_online (paritytech#3991) * Add AllGood event for im_online * Another event just in case. * Bump runtime authority-discovery: futures 03 Future (paritytech#3848) * authority-discovery: futures 03 Future * make ci happy * use futures timer instead of tokio timer * Update core/authority-discovery/src/lib.rs Co-Authored-By: Bastian Köcher <[email protected]> * remove tokio 01 runtime * trigger build * kill futures01 * rename futures Integrate Wasmtime for runtime execution (paritytech#3869) * executor: Use non wasmi-specific execution in tests. * executor: Move all runtime execution tests into tests file. * executor: Use test_case macro to easily execute tests with different Wasm execution methods. * executor: Convert errors to strings with Display, not Debug. * node-executor: Rewrite benchmarks with criterion. They were not passing compilation before and criterion seems to be more widely used in Substrate. * executor: Begin implementation of Wasm runtime. The implementation demonstrates the outline of the execution, but does not link against the external host functions. * executor: Define and implement basic FunctionExecutor. The SandboxCapabilities::invoke is still left unimplemented. * executor: Implement host function trampoline generation. * executor: Instantiate and link runtime module to env module. * executor: Provide input data during wasmtime execution. * executor: Implement SandboxCapabilites::invoke for wasmtime executor. * executor: Integrate and test wasmtime execution method. * executor: Improve FunctionExecution error messages. * Scope the unsafe blocks to be smaller. * Rename TrampolineState to EnvState. * Let EnvState own its own compiler instead of unsafe lifetime cast. * Refactor out some common wasmi/wasmtime logic. * Typos and cosmetic changes. * More trampoline comments. * Cargo.lock update. * cli: CLI option for running Substrate with compiled Wasm execution. * executor: Switch dependency from fork to official wasmtime repo. * Quiet down cranelift logs. * Explicitly catch panics during host calls. We do this to ensure that panics do not cross language boundaries. * Additional checks and clarifications in make_trampoline. * Fixes after merge from master and panic safety for wasmtime instantiation. Implement Debug for some structures (paritytech#3941) * Implement Debug for some structures `NetworkConfiguration`, `TransportConfig`, `NodeKeyConfig` and `Secret`. Needs a new release of the `rust-libp2p` crate. This PR is just a reminder. * Explicitly separate `std` and `core` * Add manual implementation for Secret Possible fix to storage cache (paritytech#3989) * Comment local_cache propagation * Add test * Deny cache when modifications are unknown * Fix indentation Don't panic in Offchain test context, when we are already panicking (paritytech#3996) executor: Move runtime version caching out of WasmRuntime interface. (paritytech#3993) * executor: Move runtime caching out of WasmRuntime interface. The runtime version is now fetched and cached at a higher level, not within the WasmRuntime trait implementations. * executor: Require successful querying of runtime version. Move config path generation into the service config for reusability (paritytech#3978) * Move config path generation into the service config for reusability * Make NoCostum Default and fix tests * Apply suggestions from code review Co-Authored-By: Bastian Köcher <[email protected]> * remove function not used anymore * Make path into an option * remove database_path function and call it directly * remove helper functions, use consts expose offchain worker storage prefix (paritytech#3977) * expose offchain worker storage prefix * add docs * move STORAGE_PREFIX to primitives Optional serde for phragmen support (paritytech#3994) * Add serde to phragmen * Update lock file * and bump a version * and bump a version again * Apply suggestions from code review Co-Authored-By: Bastian Köcher <[email protected]> * revert impl-tarit for tuple update * revert session. * Revert "revert session." This reverts commit 98086c9. * Revert "revert impl-tarit for tuple update" This reverts commit 28a7fdd. doc (paritytech#3995) Implement color output for wasm builder (paritytech#4004) * Implement color output for wasm builder * Fix `Cargo.lock` CI test executor with wasmtime (paritytech#4005) Avoid sending heartbeat if we are already considered online. (paritytech#3981) * Don't send a heartbeat if already online. * Remove env_logger. * Update lock. * Bump runtime. * Merge master Deprecate ValidateUnsigned and prevent duplicate heartbeats (paritytech#3975) * Add pre-dispatch checks for ValidateUnsigned * Deprecate ValidateUnsigned. * Bump specversion. * Fix test. fix formula in comment (paritytech#4006) Substrate EVM (paritytech#3927) * srml-evm: init the basic structures * srml-evm: finish executor implementation * srml-evm: implement balance deposit and withdraw * srml-evm: implement the actuall call/create * srml-evm: use crates.io version of evm * srml-evm: fix no-std compile * Remove dependency patch * Update to evm 0.14 * Use double map for account storage * Add precompiles support * Add some basic docs * Use runtime_io::chain_id() * Update srml/evm/src/lib.rs Co-Authored-By: Xiliang Chen <[email protected]> * Update srml/evm/src/lib.rs Co-Authored-By: Xiliang Chen <[email protected]> * Fix WithdrawReason * Unique saturate balance to u128 * Unique saturate withdraw to u128 * Remove extern crate alloc * Move account code to a separate storage and use ref for convert_account_id * More match cause for error message * Fix potential interger overflow * Use decode_len for fetching code length support crypto primitives for no_std introducing `full_crypto` feature (paritytech#3778) * introduced "with_crypto" feature and applied switches like in substrate-api-client fork * introduced "with_crypto" feature and applied switches like in substraTEE-worker fork * distinguishing core::hash vs std::hash * @bkchr's review requests fulfilled * fixes * revert dependency upgrade ed25519-dalek * added full_crypto features to all crates using app_crypto! macro * fixing CI complaints. * fix again * adding CI test for with_crypto feature * added full_crypto for ecdsa. now builds wit h--no-deafault-features --features with_crypto * remove --release from CI test * @bkchr requested changes. moved full_crypto CI test to build stage * fixing no_std issue * CI fresh copy from srml-staking * gitlab CI with +nightly * solved no-feature-in-macro dilemma * cosmetics * Update core/application-crypto/src/sr25519.rs Co-Authored-By: Bastian Köcher <[email protected]> * Update core/application-crypto/src/ed25519.rs Co-Authored-By: Bastian Köcher <[email protected]> * even more simple * undo line delete * refactoring app_crypto macro. splitting functionalities based on full_crypto feature * whitespace cosmetics refactor code: remove unused import (paritytech#4010) * remove unused import * fix test error Update RPC deps. (paritytech#4012) * Bump rpc deps * Update core/test-runtime/src/genesismap.rs Co-Authored-By: Bastian Köcher <[email protected]> Enable parallel block download (paritytech#4014) Traffic statistics (paritytech#4017) * Network stats * Fixed tests Limit longevity of im-online heartbeats. (paritytech#4011) * Limit longevity of im-online heartbeats. * Unused import. * Use parameter for session duration. Header-only sync for old forks (paritytech#3942) * Header-only sync for old forks * Simplified blocks-count * Update core/consensus/common/src/block_import.rs Co-Authored-By: Marcio Diaz <[email protected]> Revert "Header-only sync for old forks (paritytech#3942)" (paritytech#4022) This reverts commit ac78c90. Don't re-compile on every `cargo run` (paritytech#4019) - Add new crate `substrate-build-script-utils` to unify the code of `node`, `node-template` and `polkadot-node`. - The `node-cli` build script needs to search upwards for the `.git/HEAD` file to find it. fix warnings (paritytech#4024) Less verbose console output (paritytech#4029) Apply breaking changes of new libp2p versions (paritytech#3877) * Apply breaking changes of new libp2p versions * Oops, forgot to update version * Fix tests * Fix imports with WASM * Fix WASM for real * Update core/network/src/debug_info.rs Co-Authored-By: Roman Borschel <[email protected]> * Fix compilation Update `syn`, `proc-macro2` and `quote` to `1.x` (paritytech#4028) Make `TestExternalities` implement `Send` (paritytech#4030) * Make `TestExternalities` implement `Send` + `Sync` * Fixes offchain * Make it just `Send` Treasury rewards should pay the remainder of the 10% (paritytech#4026) * Treasury rewards should pay the remainder of the 10%.. * Event for deposits coming into treasury * Fix tests * Remove OnDilution Also replace paths in [build-dependencies] (paritytech#4039) * Also replace paths in [build-dependencies] * Update scripts/node-template-release/src/main.rs Co-Authored-By: Bastian Köcher <[email protected]> * Slightly reduce memory usage * Update scripts/node-template-release/src/main.rs Co-Authored-By: Bastian Köcher <[email protected]> Decrease peer reputation on bad transactions (paritytech#4035) * Decrease reputation on bad transactions * Don't punish on duplicate transactions Add `ExistenceRequirement` to `Currency` trait (paritytech#4000) * Added a public transfer_some function and a private transfer_inner fn * Move transfer_some to the end of the module impl to fix failing contracts test * Change whitespace * Remove needless change to transfer logic * Fix error * Update srml/balances/src/lib.rs Co-Authored-By: joe petrowski <[email protected]> * Improve documentation and add test * Update srml/balances/src/lib.rs Co-Authored-By: Kian Paimani <[email protected]> * Switch to changing Currency trait instead Ban incoming invalid transactions (paritytech#4037) Improve auto-docs a little. (paritytech#4032) Check polkadot compilation. (paritytech#3980) * Check polkadot compilation. * Fix git grep. * Make sure to update properly. Use commit hash instead. Refactor out MaxPossibleReward, fix staking arithmetic (paritytech#4041) * Refactor out MaxPossibleReward, fix staking arithmetic * Fix rounding error in test grandpa: Use storage proofs for Grandpa authorities (paritytech#3985) Allow import withouth state verification (paritytech#4031) * Allow import without state verification * Explicit None Co-Authored-By: Robert Habermeier <[email protected]> Correctly serialize code in chain spec as hex (paritytech#4025) * Correctly serialize code in chain spec as hex. Due to a bug, the runtime code was previously serialized as a JSON array of numbers, pretty printed one byte per line. * Remove panic in macro and whitelist attribute types for storage genesis config lines. * Use syn::Error to enforce whitelisted attributes on genesis config. * Blacklist genesis extra config line attributes instead of whitelist. clean node/cli/Cargo.toml (paritytech#4046) * clean node/cli/Cargo.toml * minor fix * clean node/runtime/Cargo.toml fix two typos (paritytech#4048) Increase parallel downloads to 5 (paritytech#4045) * Increase parallel downloads to 5 * CLI param Remove dependencies on OpenSSL (paritytech#4036) * Remove dependency on hyper-tls This removes one of the dependencies on native-tls, and thus on OpenSSL. I will remove the other in a separate commit. * Remove the `HttpClient` enum It only had one variant. * Apply suggestions from code review Co-Authored-By: Pierre Krieger <[email protected]> fix inmemory (paritytech#4049) grandpa: progressively increase target gossip peers (paritytech#4050) * grandpa: stricter gossip message filtering * gossip: remove filtered message on send_message * gossip: add test for tracking of broadcast attempts * grandpa: only restrict gossip if we're connected to more than 5 authorities * grandpa: add test for progressive gossip * grandpa: add test for gossip filtering on local non-authority node * grandpa: fix doc * gossip, grandpa: tabify * grandpa: relax filtering logic for global messages Introduce thread pool for transaction validation. (paritytech#4051) Batch gossip messages (paritytech#4055) Fix minor comment typo "do" -> "do not" (paritytech#4054) Fix sync downloading ancient chains (paritytech#4060) * Update best block on announcement * Added a test grandpa: remove the periodic block announcer (paritytech#4062) * grandpa: remove the periodic block announcer * grandpa: remove periodic block announcer test Fix the ui tests (paritytech#4066) Improved cache documentation (paritytech#4067) Additional logging for the transaction pool. (paritytech#4068) * Additional logging for the pool. * Long line. Pass startup_time to telemetry (paritytech#4069) Further fix for common block update (paritytech#4071) Add a --no-private-ipv4 CLI option (paritytech#4042) * Add a --no-private-ipv4 CLI option * Fix tests * Fix tests Change max width to 100. (paritytech#4072) Don't update common block on ancient block import (paritytech#4073) Don't search for authority set change block if delay is zero (paritytech#4076) srml-timestamp: define max timestamp drift in millis (paritytech#4077) * srml-timestamp: define max timestamp drift in millis * srml-timestamp: suffix MAX_TIMESTAMP_DRIFT with unit Substrate runtime interface 2.0 (paritytech#4057) * Adds first version of traits for generating the host functions * First steps of the procedural macro * Implements generation of the host extern functions * Prefix ext host function with snake case trait name * Implement host functions implementation on the host * Change `HostFunctions` interface * Implement `HostFunctions` for tuples * Make `WasmExecutor` generic over the host functions * Begin to add a test and make it compile * Make the test succeed * Add test to ensure that host functions are not found * It's alive! Make the `set_storage` test work * Add test for mutable references * Code cleanup and documentation etc * Add marker trait for types that should be passed as SCALE encoded * Inherit the visibility from the trait and more improvements * More impls and move them into their own file * Code simplification by dropping one trait * Give it a better name * Implement traits for arrays * Refactor code to support pass by codec/inner * Docs * Implement pass by inner for some crypto types and add a test * Implement exchangeable function support * Rewrite sr-io with as runtime interface * Start reworking after master merge * Adds `PassByCodec` derive * Adds `PassByInner` derive * Fix compilation errors * More implementations * Implement runtime interface traits for `str` * Make `sr-io` compile again * Fix more compilation errors * More progress on getting stuff back to compile * More compilation fixes * Fix warnings * Remove le conversions * Add support for `wasm_only` interfaces * Implement `Allocator` interface * Improve error message * Move `WasmAllocator` to `sr-io` and more clean ups * Use correct function signature for wasm functions * Store the host functions with the Wasm runtime * Docs update * Fix compilation after master merge * Remove `sr-io/without_std` * Make `srml-support` tests run again * More compilation error fixes * Use correct doc syntax * Fix test-runtime * Fix compilation * Catch native panics when executing the wasm runtime As with the native runtime, we now catch all native panics when we execute the wasm runtime. The panics inside the wasm runtime were already catched before by the wasm executor automatically, but any panic in the host functions could bring down the node. The recent switch to execute the native counterpart of the host function in `sr-io`, makes this change required. The native `sr-io` functions just `panic` when something is not provided or any other error occured. * Fix compilation * Don't panic in a panic * Move `sr-sandbox` to new runtime interface * Fixes tests after sandbox changes * Make sure we detect invalid utf8 * Fixes after master merge * Adds pass by enum strategy * Fix wasmtime integration * Some macro structure clean up * Rework and test exchangebale host functions * PassBy derive macros documentation * Docs for `runtime_interface` macro * Support wild card argument names * Adds ui tests * Make sure that we are backwards compatible to the old runtime interfaces * Documentation * Fixes after latest master merge * Make `wasmtime` happy * Make `full_crypto` work * Make the new interface versionable * Rename `Sanboxing` to `Sandbox` * Don't finalize in test while importing * Fix Performance regression * Fix test Remove Self vote from phragmen (paritytech#4081) * first draft of everything that works * Some test fixes Allow ancient fork download after ancestry search (paritytech#4080) Typed chain state queries over rpc. (paritytech#4079) * Create typed client helpers for querying chain state storage items declared by 'decl_storage!'. * Update substrate-rpc-custom functions to use async await syntax. * The implementation of substrate-rpc-custom was a bit verbose and repetitive. This commit makes the implementation simpler by intruducing a struct which represents query for a typed value in storage. The new struct is called StorageQuery. A StorageQuery wraps a raw StorageKey but is not directy constructable. To construct a StorageQuery, the user must supply an implementation of a srml_support::storage::generator trait such as StorageValue or StorageMap. A type implementing one of the generator traits can be aquired by: A) marking a storage item as pub within a call to decl_storage (recommended) or B) implementing one of the generator types manually. While option B may sometimes me necessary, it's not recommended because separate manual implementaions may lose sync with the original definition. * drop unused dependency * fmt * Remove unnecessary pub from Authorities field in test-runtime storage declaration. This field was added to support a test in an earlier commit. The test no longer relies on test-runtime so the change can be reverted. * Move it to srml as support extension. * Fix long lines. * Fix. Cleanup filtered messages (paritytech#4082) Don't request old obsolete blocks (paritytech#4084) core/consensus: fix console output for slot duration (paritytech#4085) [WIP] .gitlab-ci.yml: Fix a check for polkadot to work on forked repos v2 (paritytech#4078) * Fix a check for polkadot to work on forked repos. * .gitlab-ci.yml: Add `git diff` for CI debugging * More future proof and less redundant. * .gitlab-ci.yml: Improve comments and refactor sed command * .gitlab-ci.yml: Remove two commas in a row * .gitlab-ci.yml: Ensure to match branch statements at the end * .gitlab-ci.yml: Reference concrete commit When one does not specify the concrete commit, cargo-update tries to checkout 'master' in the Substrate repository. * .gitlab-ci.yml: Remove 'git diff' debug line Fix runtime interface docs and remove accidentally added file (paritytech#4092) EVM should store contracts' balances in an account. (paritytech#4090) update tiny-keccak (paritytech#4093) * update tiny-keccak * fix * Update Cargo.toml * update Cargo.lock * remove keccak-hasher Do nothing for zero imbalances (paritytech#4089) Fix sync stalling on moving head (best block) prior to the tip of the chain sometimes (paritytech#4091) * Work around finalization woes * Fixed check_block * Added a test Weight annotation for block hooks. (paritytech#4058) * Initial version that works with proper tests. * get rid of todos and grumbles and such. * Cleanup and fix line-width * fix test runtime test Sensible limits for known blocks and extrinsics (paritytech#4094) Randomized tests on storage cache (paritytech#4070) * randomized tests * destroy clones * remove commented * fixes * add finalization fork State machine call proof backend (paritytech#3945) * drafting a proof extraction at the hashdb level (to include everything for the 'call' proof case). * use full proof by default (replace previous proof recorder). * fix warnings. * Cache value not found in proof recorder. * Remove need or internal backend struct. * fix type. * doc Translation for linked map (paritytech#4052) * make linked_map generic functions better * implement translation of linked map * proc-macro for linked map updated * test linked map migration * account for instances * address grumbles * cut map short if migration fails Do not download blocks too far ahead (paritytech#4095) Keep tack of best imported block precisely (paritytech#4100) Additional wasm diagnostics (paritytech#4097) * Wasm diagnostics * Pass the error * Make errno optional * Cargo.lock * Log the error Implement Copy, Deref and FromStr to (ed25519|sr25519)::Public (paritytech#3998) *: Refactor authority discovery (key mngmt, runtime API) (paritytech#3955) * {core,srml}/authority-discovery: Move generic to specific session keys * {srml,core}/authority-discovery: Verify signature outside of runtime Given that the `core/authority-discovery` uses concrete authority identifiers and signatures, one can verify a signature with the authority discovery within `core`. Given the above, the `verify` runtime api is obsolete and thus removed. * *: Add authority discovery to the set of session keys * *: Sign authority discovery DHT payload with keystore instead of runtime Instead of calling a runtime function to sign a dht payload, which then invokes the keystore, pass the keystore to the authority discovery module and use it directly. * core/authority-discovery: Give libp2p Kademlia time to start up * core/authority-discovery: Move authorities priority group name to const * node/runtime/src/lib.rs: Bump runtime spec version * *: Fix lints and node/testing test failures * *: Fix formatting * core/authority-discovery: Box dht event channel in unit tests * node/cli/src/service.rs: Fix future import * node/cli/src/service.rs: Replace unwrap by expect with proof * node/cli/src/chain_spec: Remove TODO for testnet key generation * core/authority-discovery/src/lib: Remove scale encoding TODOs * srml/authority-discovery: Make comment a doc comment * core/authority-discovery: Remove unused StreamExt import * node/runtime: Bump impl version to debug CI * Test ci. * Change the line width to 100. * Revert "Change the line width to 100." This reverts commit edff1f8. * Fix a check for polkadot to work on forked repos. * Revert "node/runtime: Bump impl version to debug CI" This reverts commit 1a90903. * Revert "Test ci." This reverts commit a2c9df5. * Cargo.lock: Fix wrong lock file merge * srml/authority-discovery: Keep track of new validator set not upcoming * core/authority-discovery: Document key retrieval functions Reorganising the repository - external renames and moves (paritytech#4074) * Adding first rough ouline of the repository structure * Remove old CI stuff * add title * formatting fixes * move node-exits job's script to scripts dir * Move docs into subdir * move to bin * move maintainence scripts, configs and helpers into its own dir * add .local to ignore * move core->client * start up 'test' area * move test client * move test runtime * make test move compile * Add dependencies rule enforcement. * Fix indexing. * Update docs to reflect latest changes * Moving /srml->/paint * update docs * move client/sr-* -> primitives/ * clean old readme * remove old broken code in rhd * update lock * Step 1. * starting to untangle client * Fix after merge. * start splitting out client interfaces * move children and blockchain interfaces * Move trie and state-machine to primitives. * Fix WASM builds. * fixing broken imports * more interface moves * move backend and light to interfaces * move CallExecutor * move cli off client * moving around more interfaces * re-add consensus crates into the mix * fix subkey path * relieve client from executor * starting to pull out client from grandpa * move is_decendent_of out of client * grandpa still depends on client directly * lemme tests pass * rename srml->paint * Make it compile. * rename interfaces->client-api * Move keyring to primitives. * fixup libp2p dep * fix broken use * allow dependency enforcement to fail * move fork-tree * Moving wasm-builder * make env * move build-script-utils * fixup broken crate depdencies and names * fix imports for authority discovery * fix typo * update cargo.lock * fixing imports * Fix paths and add missing crates * re-add missing crates base-code(rebased) set global/local metric + peers_num add prres_num,best_heght,finallity_heght add TARGET_SYN_NUMBER add README.md add README.md add README.md TARGET_SYN_NUMBER add README.md add README.md fix: justification and newline at EOF PEERS_NUM README.md update README.md update README.md update README.md update README.md README.md README.md translation of introduction feat: add prometheus! in substrate feat: add match for opts options SaturatedConversion remove refactor prometheus/lib.rs chore: make macros for widgets TODO : counter metrics type add->remove chore: add prometheus! to consensus chore: update README chore: refactor README chore: Refactor README chore: Refactor README chore: Refactor README
1 parent 0865253 commit 095268d

File tree

856 files changed

+27613
-14877
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

856 files changed

+27613
-14877
lines changed

.dockerignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ doc
33
.idea/
44
Dockerfile
55
.dockerignore
6+
.local

.editorconfig

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ tab_width=4
66
end_of_line=lf
77
charset=utf-8
88
trim_trailing_whitespace=true
9-
max_line_length=120
9+
max_line_length=100
1010
insert_final_newline=true
1111

1212
[*.yml]

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,4 @@ rls*.log
2121
*.orig
2222
*.rej
2323
**/wip/*.stderr
24+
.local

.gitlab-ci.yml

Lines changed: 97 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ variables:
5151
- /^[0-9]+$/ # PRs
5252
retry:
5353
max: 2
54-
when:
54+
when:
5555
- runner_system_failure
5656
- unknown_failure
5757
- api_failure
@@ -78,7 +78,7 @@ check-runtime:
7878
GITLAB_API: "https://gitlab.parity.io/api/v4"
7979
GITHUB_API_PROJECT: "parity%2Finfrastructure%2Fgithub-api"
8080
script:
81-
- ./scripts/gitlab/check_runtime.sh
81+
- ./.maintain/gitlab/check_runtime.sh
8282
allow_failure: true
8383

8484

@@ -89,7 +89,7 @@ check-line-width:
8989
only:
9090
- /^[0-9]+$/
9191
script:
92-
- ./scripts/gitlab/check_line_width.sh
92+
- ./.maintain/gitlab/check_line_width.sh
9393
allow_failure: true
9494

9595

@@ -117,7 +117,7 @@ cargo-check-subkey:
117117
except:
118118
- /^v[0-9]+\.[0-9]+.*$/ # i.e. v1.0, v2.1rc1
119119
script:
120-
- cd ./subkey
120+
- cd ./bin/subkey
121121
- BUILD_DUMMY_WASM_BINARY=1 time cargo check --release
122122
- sccache -s
123123

@@ -133,7 +133,7 @@ test-linux-stable: &test-linux
133133
variables:
134134
- $DEPLOY_TAG
135135
script:
136-
- time cargo test --all --release --verbose --locked |
136+
- WASM_BUILD_NO_COLOR=1 time cargo test --all --release --verbose --locked |
137137
tee output.log
138138
- sccache -s
139139
after_script:
@@ -145,8 +145,17 @@ test-linux-stable: &test-linux
145145
paths:
146146
- ${CI_COMMIT_SHORT_SHA}_warnings.log
147147

148+
test-dependency-rules: &test-linux
149+
stage: test
150+
<<: *docker-env
151+
allow_failure: true
152+
except:
153+
variables:
154+
- $DEPLOY_TAG
155+
script:
156+
- .maintain/ensure-deps.sh
148157

149-
test-srml-staking: &test-srml-staking
158+
test-paint-staking: &test-paint-staking
150159
stage: test
151160
<<: *docker-env
152161
variables:
@@ -158,14 +167,25 @@ test-srml-staking: &test-srml-staking
158167
variables:
159168
- $DEPLOY_TAG
160169
script:
161-
- cd srml/staking/
162-
- time cargo test --release --verbose --no-default-features --features std
170+
- cd paint/staking/
171+
- WASM_BUILD_NO_COLOR=1 time cargo test --release --verbose --no-default-features --features std
163172
- sccache -s
164173

165-
166-
167-
168-
174+
test-wasmtime: &test-wasmtime
175+
stage: test
176+
<<: *docker-env
177+
variables:
178+
# Enable debug assertions since we are running optimized builds for testing
179+
# but still want to have debug assertions.
180+
RUSTFLAGS: -Cdebug-assertions=y
181+
RUST_BACKTRACE: 1
182+
except:
183+
variables:
184+
- $DEPLOY_TAG
185+
script:
186+
- cd client/executor
187+
- WASM_BUILD_NO_COLOR=1 time cargo test --release --verbose --features wasmtime
188+
- sccache -s
169189

170190
test-linux-stable-int:
171191
<<: *test-linux
@@ -177,7 +197,7 @@ test-linux-stable-int:
177197
script:
178198
- echo "___Logs will be partly shown at the end in case of failure.___"
179199
- echo "___Full log will be saved to the job artifacts only in case of failure.___"
180-
- RUST_LOG=sync=trace,consensus=trace,client=trace,state-db=trace,db=trace,forks=trace,state_db=trace,storage_cache=trace
200+
- WASM_BUILD_NO_COLOR=1 RUST_LOG=sync=trace,consensus=trace,client=trace,state-db=trace,db=trace,forks=trace,state_db=trace,storage_cache=trace
181201
time cargo test -p node-cli --release --verbose --locked -- --ignored --test-threads=1
182202
&> ${CI_COMMIT_SHORT_SHA}_int_failure.log
183203
- sccache -s
@@ -202,25 +222,13 @@ check-web-wasm:
202222
- time cargo web build -p sr-io
203223
- time cargo web build -p sr-primitives
204224
- time cargo web build -p sr-std
205-
- time cargo web build -p substrate-chain-spec
206225
- time cargo web build -p substrate-client
207226
- time cargo web build -p substrate-consensus-aura
208227
- time cargo web build -p substrate-consensus-babe
209228
- time cargo web build -p substrate-consensus-common
210-
- time cargo web build -p substrate-keyring
211-
- time cargo web build -p substrate-keystore
212-
- time cargo web build -p substrate-executor
213-
- time cargo web build -p substrate-network
214-
- time cargo web build -p substrate-offchain
215-
- time cargo web build -p substrate-panic-handler
216-
- time cargo web build -p substrate-peerset
217-
- time cargo web build -p substrate-primitives
218-
- time cargo web build -p substrate-rpc-servers
219-
- time cargo web build -p substrate-serializer
220-
- time cargo web build -p substrate-state-db
221-
- time cargo web build -p substrate-state-machine
222229
- time cargo web build -p substrate-telemetry
223-
- time cargo web build -p substrate-trie
230+
# Note: the command below is a bit weird because several Cargo issues prevent us from compiling the node in a more straight-forward way.
231+
- time cargo build --manifest-path=bin/node/cli/Cargo.toml --no-default-features --features "browser" --target=wasm32-unknown-unknown
224232
- sccache -s
225233

226234
node-exits:
@@ -229,7 +237,27 @@ node-exits:
229237
except:
230238
- /^v[0-9]+\.[0-9]+.*$/ # i.e. v1.0, v2.1rc1
231239
script:
232-
- ./ci/check_for_exit.sh
240+
- ./.maintain/check_for_exit.sh
241+
242+
243+
test-full-crypto-feature: &test-full-crypto-feature
244+
stage: test
245+
<<: *docker-env
246+
variables:
247+
# Enable debug assertions since we are running optimized builds for testing
248+
# but still want to have debug assertions.
249+
RUSTFLAGS: -Cdebug-assertions=y
250+
RUST_BACKTRACE: 1
251+
except:
252+
variables:
253+
- $DEPLOY_TAG
254+
script:
255+
- cd primitives/core/
256+
- time cargo +nightly build --verbose --no-default-features --features full_crypto
257+
- cd ../application-crypto
258+
- time cargo +nightly build --verbose --no-default-features --features full_crypto
259+
- sccache -s
260+
233261

234262
#### stage: build
235263

@@ -242,7 +270,7 @@ build-linux-substrate:
242270
variables:
243271
- $DEPLOY_TAG
244272
script:
245-
- time cargo build --release --verbose
273+
- WASM_BUILD_NO_COLOR=1 time cargo build --release --verbose
246274
- mkdir -p ./artifacts/substrate/
247275
- mv ./target/release/substrate ./artifacts/substrate/.
248276
- echo -n "Substrate version = "
@@ -255,8 +283,8 @@ build-linux-substrate:
255283
fi
256284
- sha256sum ./artifacts/substrate/substrate | tee ./artifacts/substrate/substrate.sha256
257285
- printf '\n# building node-template\n\n'
258-
- ./scripts/node-template-release.sh ./artifacts/substrate/substrate-node-template.tar.gz
259-
- cp -r scripts/docker/substrate.Dockerfile ./artifacts/substrate/
286+
- ./.maintain/node-template-release.sh ./artifacts/substrate/substrate-node-template.tar.gz
287+
- cp -r .maintain/docker/substrate.Dockerfile ./artifacts/substrate/
260288
- sccache -s
261289

262290
build-linux-subkey:
@@ -270,7 +298,7 @@ build-linux-subkey:
270298
script:
271299
- cd ./subkey
272300
- BUILD_DUMMY_WASM_BINARY=1 time cargo build --release --verbose
273-
- cd ..
301+
- cd -
274302
- sccache -s
275303
- mkdir -p ./artifacts/subkey
276304
- mv ./target/release/subkey ./artifacts/subkey/.
@@ -279,7 +307,7 @@ build-linux-subkey:
279307
sed -n -r 's/^subkey ([0-9.]+.*)/\1/p' |
280308
tee ./artifacts/subkey/VERSION;
281309
- sha256sum ./artifacts/subkey/subkey | tee ./artifacts/subkey/subkey.sha256
282-
- cp -r scripts/docker/subkey.Dockerfile ./artifacts/subkey/
310+
- cp -r .maintain/docker/subkey.Dockerfile ./artifacts/subkey/
283311
- sccache -s
284312

285313
build-rust-doc-release:
@@ -295,7 +323,7 @@ build-rust-doc-release:
295323
<<: *build-only
296324
script:
297325
- rm -f ./crate-docs/index.html # use it as an indicator if the job succeeds
298-
- BUILD_DUMMY_WASM_BINARY=1 RUSTDOCFLAGS="--html-in-header $(pwd)/rustdoc-header.html" time cargo +nightly doc --release --all --verbose
326+
- BUILD_DUMMY_WASM_BINARY=1 RUSTDOCFLAGS="--html-in-header $(pwd).maintain/rustdoc-header.html" time cargo +nightly doc --release --all --verbose
299327
- cp -R ./target/doc ./crate-docs
300328
- echo "<meta http-equiv=refresh content=0;url=substrate_service/index.html>" > ./crate-docs/index.html
301329
- sccache -s
@@ -319,6 +347,35 @@ check_warnings:
319347
fi
320348
allow_failure: true
321349

350+
# Check whether Polkadot 'master' branch builds using this Substrate commit.
351+
check_polkadot:
352+
stage: build
353+
<<: *docker-env
354+
allow_failure: true
355+
dependencies:
356+
- test-linux-stable
357+
script:
358+
- COMMIT_HASH=$(git rev-parse HEAD)
359+
- SUBSTRATE_PATH=$(pwd)
360+
# Clone the current Polkadot master branch into ./polkadot.
361+
- git clone --depth 1 https://gitlab.parity.io/parity/polkadot.git
362+
- cd polkadot
363+
# Within Polkadot 'master' alter each Cargo.toml that references the
364+
# Substrate 'polkadot-master' branch:
365+
# 1. Replace the 'branch = "polkadot-master"' statements with the rev of our
366+
# commit.
367+
# 2. Replace 'git = "https://.*"' with 'git = "file://.*"' (the local
368+
# checked out Substrate repository one folder above).
369+
# 3. Remove any trailing commas.
370+
- git grep -l "polkadot-master" | grep toml | xargs sed -i "s/branch.*=.*\"polkadot-master\"/rev = \"$COMMIT_HASH\"/; s~https://github.com/paritytech/substrate~file://$SUBSTRATE_PATH~; s/,\s*}/ }/"
371+
# Make sure 'Cargo.lock' matches 'Cargo.toml'. It's enough to update one
372+
# package, others are updated along the way.
373+
- cargo update -p sr-io
374+
# Check whether Polkadot 'master' branch builds with this Substrate commit.
375+
- time cargo check
376+
- cd -
377+
- sccache -s
378+
322379
#### stage: publish
323380

324381
.publish-docker-release: &publish-docker-release
@@ -485,10 +542,10 @@ publish-gh-doc:
485542
- echo "Substrate version = ${DEPLOY_TAG}"
486543
# or use helm to render the template
487544
- helm template
488-
--values ./scripts/kubernetes/values.yaml
545+
--values ./.maintain/kubernetes/values.yaml
489546
--set image.tag=${DEPLOY_TAG}
490547
--set validator.keys=${VALIDATOR_KEYS}
491-
./scripts/kubernetes | kubectl apply -f - --dry-run=false
548+
./.maintain/kubernetes | kubectl apply -f - --dry-run=false
492549
- echo "# substrate namespace ${KUBE_NAMESPACE}"
493550
- kubectl -n ${KUBE_NAMESPACE} get all
494551
- echo "# substrate's nodes' external ip addresses:"
@@ -550,16 +607,16 @@ deploy-ue1-tag:
550607
validator 1 4:
551608
<<: *validator-deploy
552609
script:
553-
- ./scripts/flamingfir-deploy.sh flamingfir-validator1
610+
- ./.maintain/flamingfir-deploy.sh flamingfir-validator1
554611
validator 2 4:
555612
<<: *validator-deploy
556613
script:
557-
- ./scripts/flamingfir-deploy.sh flamingfir-validator2
614+
- ./.maintain/flamingfir-deploy.sh flamingfir-validator2
558615
validator 3 4:
559616
<<: *validator-deploy
560617
script:
561-
- ./scripts/flamingfir-deploy.sh flamingfir-validator3
618+
- ./.maintain/flamingfir-deploy.sh flamingfir-validator3
562619
validator 4 4:
563620
<<: *validator-deploy
564621
script:
565-
- ./scripts/flamingfir-deploy.sh flamingfir-validator4
622+
- ./.maintain/flamingfir-deploy.sh flamingfir-validator4
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

scripts/docker/subkey.Dockerfile renamed to .maintain/docker/subkey.Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ LABEL io.parity.image.authors="[email protected]" \
88
io.parity.image.vendor="Parity Technologies" \
99
io.parity.image.title="parity/subkey" \
1010
io.parity.image.description="Subkey: key generating utility for Substrate." \
11-
io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/scripts/docker/subkey.Dockerfile" \
11+
io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/.maintain/docker/subkey.Dockerfile" \
1212
io.parity.image.revision="${VCS_REF}" \
1313
io.parity.image.created="${BUILD_DATE}" \
1414
io.parity.image.documentation="https://github.com/paritytech/substrate/tree/${VCS_REF}/subkey"

scripts/docker/substrate.Dockerfile renamed to .maintain/docker/substrate.Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ LABEL io.parity.image.authors="[email protected]" \
88
io.parity.image.vendor="Parity Technologies" \
99
io.parity.image.title="parity/substrate" \
1010
io.parity.image.description="Substrate: The platform for blockchain innovators." \
11-
io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/scripts/docker/Dockerfile" \
11+
io.parity.image.source="https://github.com/paritytech/substrate/blob/${VCS_REF}/.maintain/docker/Dockerfile" \
1212
io.parity.image.revision="${VCS_REF}" \
1313
io.parity.image.created="${BUILD_DATE}" \
1414
io.parity.image.documentation="https://wiki.parity.io/Parity-Substrate"

.maintain/ensure-deps.sh

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
#!/usr/bin/env bash
2+
3+
# The script is meant to check if the rules regarding packages
4+
# dependencies are satisfied.
5+
# The general format is:
6+
# [top-lvl-dir]<[crate-name-prefix]
7+
8+
# For instance no crate within `./client` directory
9+
# is allowed to import any crate with a directory path containing `paint`.
10+
# Such rule is just: `client<paint`.
11+
12+
# The script should be run from the main repo directory!
13+
14+
set -u
15+
16+
PLEASE_DONT=(
17+
"client<paint"
18+
"client<node"
19+
"paint<node"
20+
"paint<client"
21+
"primitives<paint"
22+
"primitives<client"
23+
)
24+
25+
VIOLATIONS=()
26+
PACKAGES=()
27+
28+
for rule in "${PLEASE_DONT[@]}"
29+
do
30+
from=$(echo $rule | cut -f1 -d\<)
31+
to=$(echo $rule | cut -f2 -d\<)
32+
33+
cd $from
34+
echo "Checking rule $rule"
35+
packages=$(find -name Cargo.toml | xargs grep -wn "path.*$to")
36+
has_references=$(echo -n $packages | wc -c)
37+
if [ "$has_references" != "0" ]; then
38+
VIOLATIONS+=("$rule")
39+
# Find packages that violate:
40+
PACKAGES+=("$packages")
41+
fi
42+
cd - > /dev/null
43+
done
44+
45+
# Display violations and fail
46+
I=0
47+
for v in "${VIOLATIONS[@]}"
48+
do
49+
cat << EOF
50+
51+
===========================================
52+
======= Violation of rule: $v
53+
===========================================
54+
${PACKAGES[$I]}
55+
56+
57+
EOF
58+
I=$I+1
59+
done
60+
61+
exit ${#VIOLATIONS[@]}
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)