gc optimize #1151

georgehao · 2025-03-20T09:49:13Z

when send too many transaction to txpool during my benchmark, the golang gc will take 1/3 CPU profile.

I want to use the ballast mechanism to reduce the gc times to reduce golang GC CPU cost. I try GOGC = 500 and GOMEMLIMIT=10G, but it doesn't work. mainly because this section of code limits the GOGC to never be over 100.

go-ethereum/cmd/utils/flags.go

Lines 1788 to 1802 in b393386

    
           mem, err := gopsutil.VirtualMemory() 
        
           if err == nil { 
        
           	if 32<<(^uintptr(0)>>63) == 32 && mem.Total > 2*1024*1024*1024 { 
        
           		log.Warn("Lowering memory allowance on 32bit arch", "available", mem.Total/1024/1024, "addressable", 2*1024) 
        
           		mem.Total = 2 * 1024 * 1024 * 1024 
        
           	} 
        
           	allowance := int(mem.Total / 1024 / 1024 / 3) 
        
           	if cache := ctx.GlobalInt(CacheFlag.Name); cache > allowance { 
        
           		log.Warn("Sanitizing cache to Go's GC limits", "provided", cache, "updated", allowance) 
        
           		ctx.GlobalSet(CacheFlag.Name, strconv.Itoa(allowance)) 
        
           	} 
        
           } 
        
           // Ensure Go's GC ignores the database cache for trigger percentage 
        
           cache := ctx.GlobalInt(CacheFlag.Name) 
        
           gogc := math.Max(20, math.Min(100, 100/(float64(cache)/1024)))

Assuming the memory is 30G, GOGC value is 20 after the calculation, this will make the Golang GC very frequent, wasting the CPU resources, so suggest removing this part.

If we wonder about the large number of txs that trigger the OOM, the suggestion is to set GOMEMLIMIT depending on the realistic situation.

For my benchmark, I have enough memory, I don't want the CPU waste on GC, I want to use ballast to reduce the GC times, I need a big GOGC value. here is the performance after the adjustment.

The throughput increased 25% from 45Mgas/s to 55Mgas/s

Before update

After update.

And most of the time, just keeping the GOGC = 100 is a good decision

Summary by CodeRabbit

New Features
- Introduced new debugging and web extension functionality that allows manual configuration of process memory limits.
Refactor
- Removed automatic memory management adjustments, shifting control to manual configuration for memory usage and garbage collection tuning.

coderabbitai · 2025-03-20T09:49:22Z

Walkthrough

This update removes automatic memory management in the SetEthConfig function by eliminating code that adjusted cache allowance and garbage collector settings based on system memory. Additionally, it introduces manual memory limit control by adding a new method SetMemoryLimit in the debug API and a corresponding setMemoryLimit method in the web3 extension’s debug property.

Changes

File(s)	Change Summary
cmd/utils/flags.go	Removed the code block that auto-adjusted cache allowance and GC settings based on system memory.
internal/debug/api.go, internal/web3ext/web3ext.go	Added new memory limit functions: `SetMemoryLimit` in the debug API and `setMemoryLimit` in the web3 extension.

Sequence Diagram(s)

sequenceDiagram
  participant U as User
  participant H as HandlerT
  participant D as debug.SetMemoryLimit
  U->>H: SetMemoryLimit(limit)
  H->>D: SetMemoryLimit(limit)
  D-->>H: previousLimit
  H-->>U: previousLimit

sequenceDiagram
  participant U as User
  participant W as Web3 Extension
  participant API as debug_setMemoryLimit
  U->>W: setMemoryLimit(limit)
  W->>API: debug_setMemoryLimit(limit)
  API-->>W: previousLimit
  W-->>U: previousLimit

Poem

I'm a bunny hopping in the code's bright light,
Memory now dances to new limits set just right.
No more hidden tweaks in auto GC's sway,
Manual control leads my playful day.
CodeRabbit cheers — hop on, come what may!

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b393386 and 29a9bc1.

📒 Files selected for processing (3)

cmd/utils/flags.go (0 hunks)
internal/debug/api.go (1 hunks)
internal/web3ext/web3ext.go (1 hunks)

💤 Files with no reviewable changes (1)

cmd/utils/flags.go

⏰ Context from checks skipped due to timeout of 90000ms (2)

GitHub Check: test
GitHub Check: Analyze (go)

🔇 Additional comments (2)

internal/debug/api.go (1)

247-250: Excellent addition of SetMemoryLimit method to address the GC optimization objective.

The implementation correctly exposes the Go runtime's memory limit control, which aligns with the PR's goal of reducing garbage collection frequency and CPU overhead. This is a valuable addition that will allow users to tune memory settings based on their environment.

internal/web3ext/web3ext.go (1)

311-315: Good implementation of the web3 extension method for memory limit control.

This addition to the debug API exposes the memory limit functionality through the RPC interface, making it accessible to users. The placement right after the GC percent method is logical and consistent with the codebase organization.

✨ Finishing Touches

📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

colinlyguo · 2025-03-20T10:03:21Z

cmd/utils/flags.go

-	}
-	// Ensure Go's GC ignores the database cache for trigger percentage
-	cache := ctx.GlobalInt(CacheFlag.Name)
-	gogc := math.Max(20, math.Min(100, 100/(float64(cache)/1024)))


What about just removing this dynamic setting? and keep other logic. e.g. the Lowering memory allowance on 32bit arch logic.

omerfirmak · 2025-03-20T10:23:28Z

this might make your peak numbers better, but I doubt it will help with the sustained throughput.

georgehao · 2025-03-20T10:26:46Z

this might make your peak numbers better, but I doubt it will help with the sustained throughput.

yes, the CPU will not cost at gc at least. I will investigate the sustained throughput bottleneck.

georgehao added 2 commits March 20, 2025 15:10

optimize gc

cd7cc0c

fix typo

29a9bc1

georgehao requested review from Thegaram, omerfirmak and colinlyguo March 20, 2025 09:49

colinlyguo reviewed Mar 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gc optimize #1151

gc optimize #1151

georgehao commented Mar 20, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Mar 20, 2025 •

edited

Loading

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

colinlyguo Mar 20, 2025 •

edited

Loading

omerfirmak commented Mar 20, 2025

georgehao commented Mar 20, 2025

	mem, err := gopsutil.VirtualMemory()
	if err == nil {
	if 32<<(^uintptr(0)>>63) == 32 && mem.Total > 210241024*1024 {
	log.Warn("Lowering memory allowance on 32bit arch", "available", mem.Total/1024/1024, "addressable", 2*1024)
	mem.Total = 2 * 1024 * 1024 * 1024
	}
	allowance := int(mem.Total / 1024 / 1024 / 3)
	if cache := ctx.GlobalInt(CacheFlag.Name); cache > allowance {
	log.Warn("Sanitizing cache to Go's GC limits", "provided", cache, "updated", allowance)
	ctx.GlobalSet(CacheFlag.Name, strconv.Itoa(allowance))
	}
	}
	// Ensure Go's GC ignores the database cache for trigger percentage
	cache := ctx.GlobalInt(CacheFlag.Name)
	gogc := math.Max(20, math.Min(100, 100/(float64(cache)/1024)))

gc optimize #1151

Are you sure you want to change the base?

gc optimize #1151

Conversation

georgehao commented Mar 20, 2025 • edited by coderabbitai bot Loading

Before update

After update.

Summary by CodeRabbit

coderabbitai bot commented Mar 20, 2025 • edited Loading

Walkthrough

Changes

Sequence Diagram(s)

Poem

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

colinlyguo Mar 20, 2025 • edited Loading

Choose a reason for hiding this comment

omerfirmak commented Mar 20, 2025

georgehao commented Mar 20, 2025

georgehao commented Mar 20, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Mar 20, 2025 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)

colinlyguo Mar 20, 2025 •

edited

Loading