Sender: Lands UDP votes #5064

akhinvasara-jumptrading · 2025-05-09T19:33:32Z

This PR cleans up the sender tile such that we can land votes. As of now, they're sent over UDP to the advertised udp vote port. This has been tested against jump's private test cluster.

This is not the final state of sender...there is ongoing work to add quic support and clean up the data structures used. Landing this PR will help further code evolve around this plumbing and for continuous testing with vote=true.

src/disco/metrics/metrics.xml

src/discof/poh/fd_pohi_tile.c

src/app/firedancer/topology.c

mmcgee-jump · 2025-05-12T03:07:16Z

src/discof/sender/fd_sender_tile.c


  fd_epoch_leaders_t const * lsched = fd_stake_ci_get_lsched_for_slot( ctx->stake_ci, poh_slot );
-  if( FD_UNLIKELY( !lsched      ) ) { return -1; }
+  if( FD_UNLIKELY( !lsched      ) ) {
+    ctx->metrics.leader_sched_not_found++;


Don't love logging metrics in this function. Just return an error code, and have the parent track the result codes in an enum metric is a little cleaner.

<enum name="GetLeaderResult"> <Success> <ErrNoScheduleForSlot> <ErrNoContactInfo> <ErrUnroutable> </enum>

This will also change soon, so going to leave as is for now

mmcgee-jump · 2025-05-12T03:07:39Z

src/discof/sender/fd_sender_tile.c


  fd_pubkey_t const * slot_leader = fd_epoch_leaders_get( lsched, poh_slot );
-  if( FD_UNLIKELY( !slot_leader ) ) { return -1 ; } /* Count this as bad slot too */
+  if( FD_UNLIKELY( !slot_leader ) ) {
+    ctx->metrics.leader_not_found++;


I believe per contract, this cannot fail so you can FD_LOG_ERR here and abort.

this will also change with quic PR, and ill incorporate all this feedback in that

src/discof/sender/fd_sender_tile.c

mmcgee-jump · 2025-05-12T03:10:10Z

src/discof/sender/fd_sender_tile.c


  fd_shred_dest_t * sdest = fd_stake_ci_get_sdest_for_slot( ctx->stake_ci, poh_slot );
  fd_shred_dest_idx_t sdest_idx = fd_shred_dest_pubkey_to_idx( sdest, slot_leader );
  if( FD_UNLIKELY( sdest_idx==FD_SHRED_DEST_NO_DEST ) ) {
+    ctx->metrics.leader_contact_not_found++;


I'd also check in this fn that the ip address and port are nonzero, otherwise it's an err (nonroutable) as well.

src/discof/sender/fd_sender_tile.c

mmcgee-jump

Thanks! Some good improvements

src/discof/repair/fd_repair_tile.c

src/app/firedancer/topology.c

src/discof/sender/fd_sender_tile.c

src/tango/dcache/fd_dcache.h

ripatel-fd · 2025-05-16T03:03:09Z

src/discof/sender/Local.mk

This dir should be renamed to discof/send

ripatel-fd · 2025-05-16T03:03:25Z

src/disco/metrics/generate/types.py

@@ -25,7 +25,8 @@ class Tile(Enum):
    GOSSIP = 19
    NETLNK = 20
    SOCK = 21,
-    REPAIR = 22
+    REPAIR = 22,
+    SENDER = 23


Suggested change

SENDER = 23

SEND = 23,

ripatel-fd · 2025-05-16T03:03:39Z

src/disco/metrics/metrics.xml

@@ -210,6 +210,13 @@ metric introduced.
    <counter name="PktRetransmissions" summary="Number of QUIC packets that retransmitted." />
 </tile>

+<tile name="sender">


Suggested change

<tile name="sender">

<tile name="send">

ripatel-fd

Looks good. Just for future reference, I think we should have uniform naming. Either refer to this thing everywhere as send or as sender.

Since the existing name is sender unrelated to this change, I won't block the PR on it.

I haven't reviewed the application logic, so please get an approval from Charlie/Michael, too.

ripatel-fd · 2025-05-16T03:03:49Z

src/disco/verify/fd_verify_tile.c

@@ -7,7 +7,7 @@
 #define IN_KIND_QUIC   (0UL)
 #define IN_KIND_BUNDLE (1UL)
 #define IN_KIND_GOSSIP (2UL)
-
+#define IN_KIND_SENDER (3UL)


Suggested change

#define IN_KIND_SENDER (3UL)

#define IN_KIND_SEND (3UL)

ripatel-fd · 2025-05-16T03:04:03Z

src/discof/gossip/fd_gossip_tile.c

+  fd_wksp_t * sender_contact_out_mem;
+  ulong       sender_contact_out_chunk0;
+  ulong       sender_contact_out_wmark;
+  ulong       sender_contact_out_chunk;


rename to send_

ripatel-fd · 2025-05-16T03:04:08Z

src/discof/gossip/fd_gossip_tile.c

@@ -588,12 +588,12 @@ after_credit( fd_gossip_tile_ctx_t * ctx,

    ulong tvu_peer_cnt = 0;
    ulong repair_peers_cnt = 0;
-    ulong voter_peers_cnt = 0;
+    ulong sender_peers_cnt = 0;


Suggested change

ulong sender_peers_cnt = 0;

ulong send_peers_cnt = 0;

ripatel-fd · 2025-05-16T03:04:22Z

src/discof/replay/fd_replay_tile.c

-  err = fd_blockstore_block_hash_query( ctx->blockstore, vote_slot, vote_block_hash );
+
+  /* guaranteed to be on frontier from caller check */
+  const fd_fork_t * fork = fd_forks_query_const( ctx->forks, vote_slot );


Suggested change

const fd_fork_t * fork = fd_forks_query_const( ctx->forks, vote_slot );

fd_fork_t const * fork = fd_forks_query_const( ctx->forks, vote_slot );

ripatel-fd reviewed May 11, 2025

View reviewed changes

src/disco/metrics/metrics.xml Outdated Show resolved Hide resolved

ripatel-fd reviewed May 11, 2025

View reviewed changes

src/disco/metrics/metrics.xml Outdated Show resolved Hide resolved