Skip to content

Refactor metrics to handle multiple subnetworks #936

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ eth2_ssz = "0.4.0"
ethereum-types = "0.12.1"
ethportal-api = { path = "ethportal-api" }
jsonrpsee = "0.20.0"
lazy_static = "1.4.0"
parking_lot = "0.11.2"
portalnet = { path = "portalnet" }
prometheus_exporter = "0.8.4"
Expand Down
6 changes: 6 additions & 0 deletions book/src/users/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,12 @@ cargo run -p trin -- \
--web3-transport http
```

### Updating metrics dashboard
If there are new changes to the metrics dashboard template that you want to view in
an already-existing dashboard. The simplest way to update your dashboard is to delete
your `prometheus` datasource and `Trin App metrics` dashboard, and re-run the
`create-dashboard` command.

### View metrics remotely
Trin metrics on a remote machine can be monitored by listening to the grafana
address on a local machine.
Expand Down
2 changes: 2 additions & 0 deletions ethportal-api/src/dashboard/grafana.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ pub struct GrafanaAPI {
address: String,
}

// todo: automatically update datasource/dashboard via `create-dashboard` command
// rather than deleting and recreating them
Comment on lines +15 to +16
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To issue?

Copy link
Collaborator Author

@njgheorghita njgheorghita Oct 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well... Imo, this is a very low priority issue. Given the increased activity of our stale issue bot, I don't think it will be addressed anytime soon and will simply be marked as stale before anybody fixes this.

I updated the docs (in book/src/users/monitoring.md) so there shouldn't be any surprises for users wanting to update their dashboard. I'm leaning towards leaving this todo here & not opening an issue. Simply as a reference for any future devs who might be working on improving our grafana workflow, whenever that time may come around.

What do you think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh, I guess I'm still not fully sold on the stale bot. The idea that we are hesitant to create issues that aren't going to be quickly resolved feels like an anti-pattern to me. I don't really see why TODOs peppered throughout the code is any better than a backlog of issues that have been around for a while (I'd argue it is worse, in fact. Partially because it's hard to have a conversation in a code TODO). But I guess the solution I'm leaning toward would handle the TODOs in code: #932 so I'm fine leaving it for now

impl GrafanaAPI {
pub fn new(username: String, password: String, address: String) -> Self {
let basic_auth_string = format!("{username}:{password}");
Expand Down
52 changes: 52 additions & 0 deletions portalnet/src/config.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
use std::net::SocketAddr;

use ethereum_types::H256;

use ethportal_api::types::bootnodes::Bootnodes;
use ethportal_api::types::cli::TrinConfig;
use ethportal_api::types::distance::Distance;

/// Capacity of the cache for observed `NodeAddress` values.
/// Provides capacity for 32 full k-buckets. This capacity will be shared among all active portal
/// subnetworks.
const NODE_ADDR_CACHE_CAPACITY: usize = discv5::kbucket::MAX_NODES_PER_BUCKET * 32;

#[derive(Clone)]
pub struct PortalnetConfig {
pub external_addr: Option<SocketAddr>,
pub private_key: H256,
pub listen_port: u16,
pub bootnodes: Bootnodes,
pub data_radius: Distance,
pub internal_ip: bool,
pub no_stun: bool,
pub node_addr_cache_capacity: usize,
}

impl Default for PortalnetConfig {
fn default() -> Self {
Self {
external_addr: None,
private_key: H256::random(),
listen_port: 4242,
bootnodes: Bootnodes::default(),
data_radius: Distance::MAX,
internal_ip: false,
no_stun: false,
node_addr_cache_capacity: NODE_ADDR_CACHE_CAPACITY,
}
}
}

impl PortalnetConfig {
pub fn new(trin_config: &TrinConfig, private_key: H256) -> Self {
Self {
external_addr: trin_config.external_addr,
private_key,
listen_port: trin_config.discovery_port,
no_stun: trin_config.no_stun,
bootnodes: trin_config.bootnodes.clone(),
..Default::default()
}
}
}
25 changes: 14 additions & 11 deletions portalnet/src/discovery.rs
Original file line number Diff line number Diff line change
@@ -1,29 +1,32 @@
use super::types::messages::{PortalnetConfig, ProtocolId};
use crate::socket;
use std::hash::{Hash, Hasher};
use std::net::Ipv4Addr;
use std::path::PathBuf;
use std::str::FromStr;
use std::{convert::TryFrom, fmt, fs, io, net::SocketAddr, sync::Arc};

use anyhow::anyhow;
use async_trait::async_trait;
use bytes::BytesMut;
use discv5::{
enr::{CombinedKey, EnrBuilder, NodeId},
ConfigBuilder, Discv5, Event, ListenConfig, RequestError, TalkRequest,
};
use ethportal_api::types::enr::Enr;
use ethportal_api::utils::bytes::hex_encode;
use ethportal_api::NodeInfo;
use lru::LruCache;
use parking_lot::RwLock;
use rlp::RlpStream;
use serde_json::{json, Value};
use std::hash::{Hash, Hasher};
use std::net::Ipv4Addr;
use std::path::PathBuf;
use std::str::FromStr;
use std::{convert::TryFrom, fmt, fs, io, net::SocketAddr, sync::Arc};
use tokio::sync::mpsc;
use tracing::{debug, info, warn};
use trin_utils::version::get_trin_version;
use utp_rs::{cid::ConnectionPeer, udp::AsyncUdpSocket};

use super::config::PortalnetConfig;
use super::types::messages::ProtocolId;
use crate::socket;
use ethportal_api::types::enr::Enr;
use ethportal_api::utils::bytes::hex_encode;
use ethportal_api::NodeInfo;
use trin_utils::version::get_trin_version;

/// Size of the buffer of the Discv5 TALKREQ channel.
const TALKREQ_CHANNEL_BUFFER: usize = 100;

Expand Down
1 change: 1 addition & 0 deletions portalnet/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#![warn(clippy::unwrap_used)]

pub mod config;
pub mod discovery;
pub mod events;
pub mod find;
Expand Down
2 changes: 2 additions & 0 deletions portalnet/src/metrics/mod.rs
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
pub mod labels;
pub mod overlay;
pub mod portalnet;
pub mod storage;
Loading