Enhanced Traffic Graph Widget with Multi-timeframe Support and Data Persistence #866

rebroad · 2025-04-09T23:00:18Z

This PR improves the network traffic graph widget in the Debug window to provide:

Multiple time range options (from 5 minutes to 28 days)
Data persistence between sessions
Interactive visualization features including tooltips and logarithmic scaling

Motivation

The existing network traffic graph has limited utility with its fixed time range and lack of historical data preservation. This enhancement allows users to:

Analyze network traffic patterns over different timeframes
Preserve historical traffic data between session restarts
Interact with the graph to get specific data points
Better visualize varying traffic volumes with logarithmic scaling

These improvements are valuable for:

Network debugging and troubleshooting
Understanding Bitcoin Core's network behaviour
Monitoring traffic patterns for optimization
Research purposes

Implementation

The implementation preserves all existing functionality while adding new features:

Added a pre-configured timeframe selection (13 different timeframes)
Implemented traffic data serialization and deserialization
Enhanced the visualization with interactive features
Improved tooltip information with precise timestamps and traffic rates

Supporting changes:

Added formatBytesps function for a prettier display of traffic rates
Added FormatISO8601Time for better time display

Testing

Tested on Linux with various network conditions. The new functionality can be exercised by:

Using the slider (or arrow keys) to select different time ranges
Restarting the application to verify data persistence
Clicking on the graph to toggle between linear and logarithmic scales
Hovering over data points to view detailed information

Documentation

The changes are mostly self-documenting through the UI and are constrained to the Qt interface without affecting core functionality.

Compatibility

This PR maintains compatibility with existing functionality. The data persistence file uses proper serialization versioning to allow for future format changes if needed.

DrahtBot · 2025-04-09T23:00:22Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

No conflicts as of last run.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

if (m_tt_point && m_tt_point <= DESIRED_SAMPLES) -> if (m_tt_point > 0 && m_tt_point <= m_time_stamp[m_value].size()) [Index check should use the actual queue size, not a constant]
if (m_tt_point > DESIRED_SAMPLES) m_tt_point = 0; -> if (m_tt_point > m_time_stamp[m_value].size()) m_tt_point = 0; [Index check should use the actual queue size, not a constant]
while (m_time_stamp[i].size() > DESIRED_SAMPLES) -> while (m_time_stamp[i].size() > static_cast<int>(DESIRED_SAMPLES * static_cast<double>(m_values[i]) / m_values[0])) [The maximum number of samples stored per range should scale with the range duration, not a fixed constant, making the queue management logic unclear.]
static uint64_t last_time_ms; -> uint64_t last_time_ms; [Static variable within a loop across ranges leads to incorrect state persistence, corrupting data loading.]
if (!dst_point || (!is_peak && !is_dip)) return dst_point; -> if (dst_point == 0 || (!is_peak && !is_dip)) return 0; [Inconsistent indexing (0-based vs 1-based) and potential out-of-bounds access (accessing at dst_point - 1 when dst_point could be 0) makes the function logic unclear and potentially buggy.]

DrahtBot · 2025-04-09T23:49:49Z

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin-core/gui/runs/40286962281}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

luke-jr

Merging it all together likes this makes review very annoying. It was better as separate PRs.

luke-jr · 2025-04-14T22:10:54Z

src/qt/trafficgraphwidget.cpp

+            // Restore the saved total bytes counts to the node if they exist
+            if (m_total_bytes_recv > 0 || m_total_bytes_sent > 0) {
+                model->node().setTotalBytesRecv(m_total_bytes_recv);
+                model->node().setTotalBytesSent(m_total_bytes_sent);
+            }


The GUI should not be changing low-level internals.

If for some reason the clientModel is changed, its data should be taken as authoritative, even if it contradicts the history so far (if that's an issue, wipe the history).

Alternatively, maybe the history should be stored with the node, and loaded into the GUI (or RPC, in another hypothetical PR).

The idea is that the data is not persistent - but yes, I see your point - it should probably not be something that only happens when the GUI is running, and should perhaps also happen in bitcoind also.

The GUI should not be changing low-level internals.

If for some reason the clientModel is changed, its data should be taken as authoritative, even if it contradicts the history so far (if that's an issue, wipe the history).

Alternatively, maybe the history should be stored with the node, and loaded into the GUI (or RPC, in another hypothetical PR).

@luke-jr I've removed the setTotalBytes functions as I agree with your reasoning. The GUI keeps a local track of the totals from the last session and simply adds them to the totals provided by the node now. I did think about storing the data on the node, but I am not sure it makes much sense given the data is currently stored on the GUI and so it makes more sense to keep the totals local with the traffic history for now. Perhaps someone thinks it's worth storing this on the node in the future, but it doesn't seem like an urgent need for now.

luke-jr · 2025-04-14T22:12:04Z

src/qt/forms/debugwindow.ui

@@ -694,16 +703,6 @@
             </property>
            </widget>
           </item>
-           <item>
-            <widget class="QPushButton" name="btnClearTrafficGraph">


If we're deleting this, maybe there should be a way for the user to insert a reference line?

For now, I'd move removal of anything to a separate PR.

The user can effectively remove the data by deleting the .dat file - is there any basis for being able to do this while the client is running?

luke-jr · 2025-04-14T22:12:48Z

src/qt/rpcconsole.cpp

@@ -578,7 +576,7 @@ RPCConsole::RPCConsole(interfaces::Node& node, const PlatformStyle *_platformSty
    // based timer interface
    m_node.rpcSetTimerInterfaceIfUnset(rpcTimerInterface);

-    setTrafficGraphRange(INITIAL_TRAFFIC_GRAPH_MINS);


Why are we losing this default constant?

because it isn't needed - on startup it "defaults" to the first non-full range

src/util/time.cpp

rebroad · 2025-04-15T23:02:39Z

Merging it all together likes this makes review very annoying. It was better as separate PRs.

I vaguely remember people complaining that I was raising several PRs that could be combined when I first raised the PRs years ago.... I guess one cannot please everyone all of the time.

@luke-jr As a middle-ground, I can move the distinct functional changes into different commits.

rebroad · 2025-05-01T23:11:37Z

I've removed the code that changed the totals on the node - the GUI just keeps track now without writing values to the node. Also, added some checks for corrupt save files so that it doesn't load bad data.

DrahtBot · 2025-05-01T23:15:59Z

🚧 At least one of the CI tasks failed.
_{Task lint: https://github.com/bitcoin-core/gui/runs/41512420338}
_{LLM reason (✨ experimental): The CI failure is caused by a lint check failure due to the use of tabs instead of spaces in the codebase.}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

rebroad · 2025-05-02T06:36:04Z

I realise this PR is starting to look a bit messy with all the updates. I also realise I ought to separate various functionality changes into separate commits. Is it better to raise a new PR once this is done, force push to this one?

rebroad · 2025-05-03T19:46:41Z

Have just added some more GUI improvements - the text on the graph now has a black outline around it making it easier to read (and is printed AFTER the graph rather than before) .Also, the grey line at the bottom of the graph is now printed after the graph also, causing the graph not to spill onto that line.

Also, the bright green and red lines no longer surround the sides of the graph but only draw an outline along actual graph values. This makes it easier to distinguish about the graph going off the edge of the display areas, vs a sudden drop in traffic (which previously looked the same). The now graph also fills the area fully to the edges at the side now (which it did not before due to rounding issues).

It would be useful to know whether I should split all these changes into individual pull requests, or individual commits, or better all together like this.

DrahtBot · 2025-05-04T21:26:04Z

🚧 At least one of the CI tasks failed.
_{Task previous releases, depends DEBUG: https://github.com/bitcoin-core/gui/runs/41616791301}
_{LLM reason (✨ experimental): The CI failed due to a compilation error caused by treating warnings as errors, specifically a sign comparison warning in trafficgraphwidget.cpp.}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

Add a new GUI utility function to format bytes per second values with appropriate units (B/s, kB/s, MB/s, GB/s) and precision, ensuring consistent and readable display of network traffic rates in the traffic graph widget.

Add a new time formatting function that matches the existing date and datetime ISO8601 formatting functions, providing consistent time display for the traffic graph widget tooltips and other UI elements that need time-only display.

…persistence This commit significantly improves the network traffic graph widget with: 1. Multiple timeframe support - View traffic data across different time periods (5 minutes to 28 days) using an enhanced slider interface 2. Traffic data persistence - Save and restore traffic information between sessions, preserving historical traffic patterns 3. Interactive visualization features: - Logarithmic scale toggle (mouse click) for better visualization of varying traffic volumes - Interactive tooltips showing detailed traffic information at specific points - Yellow highlight indicators for selected data points 4. Smooth transitions between different time ranges with animated scaling These improvements allow users to better monitor and analyze network traffic patterns over time, which is especially useful for debugging connectivity issues or understanding network behavior under different conditions. The implementation includes proper thread-safety considerations and handles edge cases like time jumps or missing data appropriately.

DrahtBot added the CI failed label Apr 9, 2025

DrahtBot mentioned this pull request Apr 10, 2025

Replace libevent with our own HTTP and socket-handling implementation bitcoin/bitcoin#32061

Open

rebroad force-pushed the trafficgraphwidget-rebased branch from 530adae to 31b1ffc Compare April 10, 2025 16:33

DrahtBot removed the CI failed label Apr 10, 2025

rebroad marked this pull request as ready for review April 10, 2025 21:35

rebroad changed the title ~~qt: Improve TrafficGraphWidget functionality~~ qt: Enhanced Traffic Graph Widget with Multi-timeframe Support and Data Persistence Apr 11, 2025

luke-jr suggested changes Apr 14, 2025

View reviewed changes

rebroad requested a review from luke-jr April 15, 2025 23:03

rebroad force-pushed the trafficgraphwidget-rebased branch from 31b1ffc to ab7bbd0 Compare April 16, 2025 21:37

laanwj changed the title ~~qt: Enhanced Traffic Graph Widget with Multi-timeframe Support and Data Persistence~~ Enhanced Traffic Graph Widget with Multi-timeframe Support and Data Persistence Apr 24, 2025

laanwj added the Feature label Apr 24, 2025

rebroad force-pushed the trafficgraphwidget-rebased branch 6 times, most recently from 280b15c to de2f01a Compare May 1, 2025 14:35

rebroad force-pushed the trafficgraphwidget-rebased branch 2 times, most recently from cf274a3 to 0982a88 Compare May 1, 2025 23:15

DrahtBot added the CI failed label May 1, 2025

rebroad force-pushed the trafficgraphwidget-rebased branch from 0982a88 to 47732ed Compare May 2, 2025 06:34

rebroad force-pushed the trafficgraphwidget-rebased branch from 47732ed to c308d55 Compare May 2, 2025 06:43

DrahtBot removed the CI failed label May 2, 2025

rebroad force-pushed the trafficgraphwidget-rebased branch from c308d55 to b3155ba Compare May 2, 2025 08:25

rebroad force-pushed the trafficgraphwidget-rebased branch 4 times, most recently from 4d14caf to 54d90c9 Compare May 2, 2025 15:30

rebroad marked this pull request as draft May 2, 2025 16:01

rebroad force-pushed the trafficgraphwidget-rebased branch from 54d90c9 to 28ac517 Compare May 3, 2025 19:40

rebroad force-pushed the trafficgraphwidget-rebased branch from 28ac517 to c0b9aff Compare May 4, 2025 20:28

DrahtBot added the CI failed label May 4, 2025

rebroad force-pushed the trafficgraphwidget-rebased branch 9 times, most recently from 48b175e to b1aa3b8 Compare May 6, 2025 14:58

DrahtBot removed the CI failed label May 6, 2025

rebroad force-pushed the trafficgraphwidget-rebased branch from b1aa3b8 to a896748 Compare May 6, 2025 20:29

rebroad added 2 commits May 8, 2025 16:36

qt: Add formatBytesps function for bytes per second display

34794f8

Add a new GUI utility function to format bytes per second values with appropriate units (B/s, kB/s, MB/s, GB/s) and precision, ensuring consistent and readable display of network traffic rates in the traffic graph widget.

rebroad force-pushed the trafficgraphwidget-rebased branch from a896748 to 15fd680 Compare May 8, 2025 15:37

rebroad force-pushed the trafficgraphwidget-rebased branch from 15fd680 to bc13d01 Compare May 8, 2025 21:54

Enhanced Traffic Graph Widget with Multi-timeframe Support and Data Persistence #866

Are you sure you want to change the base?

Enhanced Traffic Graph Widget with Multi-timeframe Support and Data Persistence #866

Uh oh!

Conversation

rebroad commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Implementation

Testing

Documentation

Compatibility

Uh oh!

DrahtBot commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews

Conflicts

LLM Linter (✨ experimental)

Uh oh!

DrahtBot commented Apr 9, 2025

Uh oh!

luke-jr left a comment

Choose a reason for hiding this comment

Uh oh!

luke-jr Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

rebroad Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

rebroad Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

luke-jr Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

rebroad Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

luke-jr Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

rebroad Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rebroad commented Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rebroad commented May 1, 2025

Uh oh!

DrahtBot commented May 1, 2025

Uh oh!

rebroad commented May 2, 2025

Uh oh!

rebroad commented May 3, 2025

Uh oh!

DrahtBot commented May 4, 2025

Uh oh!

Uh oh!

rebroad commented Apr 9, 2025 •

edited

Loading

DrahtBot commented Apr 9, 2025 •

edited

Loading

rebroad commented Apr 15, 2025 •

edited

Loading