ringbuf: Add event counters (#1621)

Ringbufs provide a fixed-size buffer for storing diagnostic events,
which may include data. This is very useful for debugging. However,
because the ring buffer has a fixed size, only the last `N` events are
visible at any given time. If a ringbuf has a capacity of, say, 16
events, and a given event has occurred 10,000 times over the lifespan of
the task, there's currently no way to know that the event has happened
more than 16 times. Furthermore, there's no way to use the ringbuf to
determine whether a given event has _ever_ occurred during the task's
lifetime: if a given variant is not _currently_ present in the ringbuf,
it may still have occurred previously and been "drowned out" by
subsequent events. Finally, some targets disable the ringbuf entirely,
for space reasons, and on those targets, it may still be desirable to be
able to get some less granular diagnostic data out of the ringbuf events
a task records.

Therefore, this commit extends the `ringbuf` crate to support generating
counts of the number of times a particular event variant has been
recorded, in addition to a fixed-size buffer of events. These counters
provide less detailed data than inspecting the ringbuf contents, because
they don't record the values of any _fields_ on that event. However,
they allow recording very large numbers of events in a fixed amount of
space, and provide historical data for events that may have occurred in
the past and then "fallen off" the end of the ringbuf as new events were
recorded. By inspecting these counters, we can determine if a given
variant has *ever* been recorded, even if it isn't currently in the
buffer, and we can see the total number of times it has been recorded
over the task's entire lifetime.

Event counters are implemented using a new `CountedRingbuf` type, which
marries a ring buffer with a type which can count occurances of entry
variants. `CountedRingbuf`s may be declared using the `counted_ringbuf!`
macro, and entries can be recorded using the existing `ringbuf_entry!`
macro.

`CountedRingbuf` requires that the entry type implement a new `Count`
trait, which defines the counter type, an initializer for creating new
instances of the counter type, and a method to increment the counter
type with a given instance of the entry type. An implementation of
`ringbuf::Count` can be generated for an entry type using the
`#[derive(ringbuf::Count)]` attribute, which generates a `struct` with
an `AtomicU32` field for each of the deriving `enum`'s variants. This
`struct` can then be loaded by Humility to provide a view of the event
counters.

Because recording event counters requires only `4 * <N_VARIANTS>` bytes,
counters are currently recorded even when the `ringbuf` crate has the
`disabled` feature set. This way, we can still record some diagnostic
data on targets that don't have the space to store the ringbuf itself.
If it becomes necessary to also disable counters, we could add a
separate `disable-counters` feature as well, so that a target can pick
and choose between recording ringbuf entries, counts, or both.

As a proof of concept, I've also updated `gimlet-seq-server` to use the
new event counters for its `Trace` ringbuf. Subsequent commits will roll
it out elsewhere.

Future work will include:

- [ ] Actually adopting event counters in existing ringbufs
- [ ] Updating `humility ringbuf` to also dump ringbuf event counts (see
  oxidecomputer/humility#449).

Depends on #1624, both to reduce merge conflicts and to mitigate an
increase in size due to the changes in this branch.

Author: hawkw

Cargo.lock

@@ -4,5 +4,5 @@
 
 fn main() {
     build_util::expose_target_board();
-    build_util::expose_m_profile();
+    build_util::expose_m_profile().unwrap();
 }

app/gimlet/build.rs

@@ -56,7 +56,7 @@ pub fn has_feature(s: &str) -> bool {
 ///
 /// This will set one of `cfg(armv6m)`, `cfg(armv7m)`, or `cfg(armv8m)`
 /// depending on the value of the `TARGET` environment variable.
-pub fn expose_m_profile() {
+pub fn expose_m_profile() -> Result<()> {
     let target = crate::target();
 
     if target.starts_with("thumbv6m") {
@@ -67,9 +67,9 @@ pub fn expose_m_profile() {
     } else if target.starts_with("thumbv8m") {
         println!("cargo:rustc-cfg=armv8m");
     } else {
-        println!("Don't know the target {}", target);
-        std::process::exit(1);
+        bail!("Don't know the target {target}");
     }
+    Ok(())
 }
 
 /// Returns the `HUBRIS_BOARD` envvar, if set.

build/util/src/lib.rs

@@ -45,7 +45,7 @@ include!(concat!(env!("OUT_DIR"), "/i2c_config.rs"));
 )]
 mod payload;
 
-#[derive(Copy, Clone, PartialEq)]
+#[derive(Copy, Clone, PartialEq, ringbuf::Count)]
 enum Trace {
     Ice40Rails(bool, bool),
     IdentValid(bool),
@@ -112,7 +112,7 @@ enum Trace {
     None,
 }
 
-ringbuf!(Trace, 128, Trace::None);
+counted_ringbuf!(Trace, 128, Trace::None);
 
 #[export_name = "main"]
 fn main() -> ! {

drv/gimlet-seq-server/src/main.rs

@@ -3,5 +3,8 @@
 // file, You can obtain one at https://mozilla.org/MPL/2.0/.
 
 fn main() {
-    build_util::expose_m_profile();
+    match build_util::expose_m_profile() {
+        Ok(_) => (),
+        Err(e) => println!("cargo:warn={e}"),
+    }
 }

lib/armv6m-atomic-hack/build.rs

@@ -7,9 +7,16 @@ edition = "2021"
 # To disable a ring buffer (but leave it otherwise present), enable the
 # "disabled" feature
 disabled = []
+# Enable deriving the `Count` trait
+derive = ["ringbuf-macros"]
+default = ["derive"]
 
 [dependencies]
 static-cell = { path = "../static-cell" }
+ringbuf-macros = { path = "macros", optional = true }
+
+[target.'cfg(target_arch = "arm")'.dependencies]
+armv6m-atomic-hack = { path = "../armv6m-atomic-hack" }
 
 [lib]
 test = false

lib/ringbuf/Cargo.toml

@@ -0,0 +1,48 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+//! Demonstrates the use of `counted_ringbuf!` and friends.
+//!
+//! This example is primarily intended to be used with `cargo expand` to show
+//! the macro-generated code for `#[derive(ringbuf::Count)]` and friends.
+use ringbuf::*;
+
+#[derive(ringbuf::Count, Debug, Copy, Clone, PartialEq, Eq)]
+pub enum Event {
+    NothingHappened,
+    SomethingHappened,
+    SomethingElse(u32),
+    SecretThirdThing { secret: () },
+}
+
+counted_ringbuf!(Event, 16, Event::NothingHappened);
+counted_ringbuf!(MY_NAMED_RINGBUF, Event, 16, Event::NothingHappened);
+
+ringbuf!(NON_COUNTED_RINGBUF, Event, 16, Event::NothingHappened);
+
+pub fn example() {
+    ringbuf_entry!(Event::SomethingHappened);
+    ringbuf_entry!(NON_COUNTED_RINGBUF, Event::SomethingHappened);
+}
+
+pub fn example_named() {
+    ringbuf_entry!(MY_NAMED_RINGBUF, Event::SomethingElse(420));
+}
+
+pub mod nested {
+    use super::Event;
+
+    ringbuf::counted_ringbuf!(Event, 16, Event::NothingHappened);
+
+    pub fn example() {
+        ringbuf::ringbuf_entry!(Event::SomethingHappened);
+        ringbuf::ringbuf_entry_root!(Event::SomethingElse(666));
+        ringbuf::ringbuf_entry_root!(
+            MY_NAMED_RINGBUF,
+            Event::SecretThirdThing { secret: () }
+        );
+    }
+}
+
+fn main() {}

lib/ringbuf/examples/counts.rs

@@ -0,0 +1,12 @@
+[package]
+name = "ringbuf-macros"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+proc-macro = true
+
+[dependencies]
+proc-macro2 = "1.0.78"
+quote = "1.0.35"
+syn = "2.0.48"

lib/ringbuf/macros/Cargo.toml

@@ -0,0 +1,99 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+extern crate proc_macro;
+use proc_macro::TokenStream;
+use proc_macro2::{Ident, Span};
+use quote::{quote, ToTokens};
+use syn::{parse_macro_input, DeriveInput};
+
+/// Derives an implementation of the `ringbuf::Count` trait for the annotated
+/// `enum` type.
+///
+/// Note that this macro can currently only be used on `enum` types.
+#[proc_macro_derive(Count)]
+pub fn derive_count(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input as DeriveInput);
+    match gen_count_impl(input) {
+        Ok(tokens) => tokens.to_token_stream().into(),
+        Err(err) => err.to_compile_error().into(),
+    }
+}
+
+fn gen_count_impl(input: DeriveInput) -> Result<impl ToTokens, syn::Error> {
+    let name = &input.ident;
+    let data_enum = match input.data {
+        syn::Data::Enum(ref data_enum) => data_enum,
+        _ => {
+            return Err(syn::Error::new_spanned(
+                input,
+                "`ringbuf::Count` can only be derived for enums",
+            ));
+        }
+    };
+    let variants = &data_enum.variants;
+    let len = variants.len();
+    let mut variant_names = Vec::with_capacity(len);
+    let mut variant_patterns = Vec::with_capacity(len);
+    for variant in variants {
+        let ident = &variant.ident;
+        variant_patterns.push(match variant.fields {
+            syn::Fields::Unit => quote! { #name::#ident => &counters.#ident },
+            syn::Fields::Named(_) => {
+                quote! { #name::#ident { .. } => &counters.#ident }
+            }
+            syn::Fields::Unnamed(_) => {
+                quote! { #name::#ident(..) => &counters.#ident }
+            }
+        });
+        variant_names.push(ident.clone());
+    }
+    let counts_ty = counts_ty(name);
+    let code = quote! {
+        #[doc = concat!(" Ringbuf entry total counts for [`", stringify!(#name), "`].")]
+        #[allow(nonstandard_style)]
+        pub struct #counts_ty {
+            #(
+                #[doc = concat!(
+                    " The total number of times a [`",
+                    stringify!(#name), "::", stringify!(#variant_names),
+                    "`] entry"
+                )]
+                #[doc = " has been recorded by this ringbuf."]
+                pub #variant_names: core::sync::atomic::AtomicU32
+            ),*
+        }
+
+        #[automatically_derived]
+        impl ringbuf::Count for #name {
+            type Counters = #counts_ty;
+
+            // This is intended for use in a static initializer, so the fact that every
+            // time the constant is used it will be a different instance is not a
+            // problem --- in fact, it's the desired behavior.
+            //
+            // `declare_interior_mutable_const` is really Not My Favorite Clippy
+            // Lint...
+            #[allow(clippy::declare_interior_mutable_const)]
+            const NEW_COUNTERS: #counts_ty = #counts_ty {
+                #(#variant_names: core::sync::atomic::AtomicU32::new(0)),*
+            };
+
+            fn count(&self, counters: &Self::Counters) {
+                #[cfg(all(target_arch = "arm", armv6m))]
+                use ringbuf::rmv6m_atomic_hack::AtomicU32Ext;
+
+                let counter = match self {
+                    #(#variant_patterns),*
+                };
+                counter.fetch_add(1, core::sync::atomic::Ordering::Relaxed);
+            }
+        }
+    };
+    Ok(code)
+}
+
+fn counts_ty(ident: &Ident) -> Ident {
+    Ident::new(&format!("{ident}Counts"), Span::call_site())
+}