Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node:worker_threads low-hanging fruit #18758

Merged
merged 55 commits into from
Apr 8, 2025
Merged

node:worker_threads low-hanging fruit #18758

merged 55 commits into from
Apr 8, 2025

Conversation

190n
Copy link
Collaborator

@190n 190n commented Apr 3, 2025

What does this PR do?

Fixes some of the easier tests from test/parallel/test-worker-*

  • Ensure Worker with execArgv: [] receives an empty array for process.execArgv instead of inheriting the parent thread's array (I haven't found this covered in the Node tests yet so I added it to ours)
  • Progress toward Bun crash with Worker #18139 (but not fixed; I may move that to another PR if I get a complete fix in)
  • Added support for reading the environment variable BUN_DESTRUCT_VM_ON_EXIT=1, which tells Bun to destroy the main thread's JavaScript VM when the process exits. Normally, Worker VMs are the only ones that get destroyed. The intent of this flag is that running a test or program with it will expose bugs that would otherwise only have occurred if the code was running inside a Worker that exited. For instance, the changes from Fix UDPSocket not being closed when VM destructs #18687 were motivated by running a Node UDP test with BUN_DESTRUCT_VM_ON_EXIT=1.
  • Makes the name of the node:fs/promises FileHandle constructor be FileHandle instead of FileHandle2, and adds a property Symbol(messaging_transfer_symbol) (this is progress towards test-worker-message-port-transfer-fake-js-transferable-internal.js but does not fix it just yet)
  • Makes postMessage avoid calling a user-overridden Array.prototype[Symbol.iterator]
  • Makes autoSelectFamilyAttemptTimeoutDefault in node:net unique per-thread. Our code doesn't use this value anywhere, but Node's test harness multiplies the value by a fixed amount before the tests run. This means that if you have a Node test that repeatedly spawns new Workers, each one will greatly increase the global autoSelectFamilyAttemptTimeoutDefault, until eventually a TypeError is thrown because the harness attempts to increase it past the u32 limit.
  • Makes worker.getHeapSnapshot() error if called on a Worker that isn't running or if the options object is invalid
  • Makes the online event fire right before a Worker starts running its entrypoint instead of right after
  • Ensures the exit event is fired with code 1 instead of 0 on a Worker which throws an uncaught exception
  • Accept and call a callback function passed to worker.terminate, and emit a deprecation warning if you do so
  • Makes Symbol arguments passed in the argv and execArgv arrays coerce to Symbol(description) instead of throwing an error
  • Makes process.argv contain [worker eval] instead of a Blob URL for Workers spawned with eval: true
  • Adds error if the env option spawning a Worker is not the right type
  • Adjusts the error message when closing an already-closed MessagePort, and makes MessagePort.close accept a callback
  • Fix never freeing the source code for eval: true Workers

TODOs:

  • calling terminate doesn't fire the exit event (this affects several tests)
  • find out which tests are wrongly categorized as worker_threads (several of Node's worker tests are just "do XYZ, but in a worker that terminates at some inconvenient time," and many of those tests run into bugs with our implementation of XYZ before they run into any worker bugs)
  • ensure there are no regressions
  • possibly fix more tests

How did you verify your code works?

Node tests

190n added 30 commits March 14, 2025 13:41
src/bun_js.zig Outdated
@@ -68,6 +68,7 @@ pub const Run = struct {
.args = ctx.args,
.graph = graph_ptr,
.is_main_thread = true,
.destruct_main_thread_on_exit = bun.getenvTruthy("BUN_DESTRUCT_VM_ON_EXIT"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.destruct_main_thread_on_exit = bun.getenvTruthy("BUN_DESTRUCT_VM_ON_EXIT"),
.destruct_main_thread_on_exit = bun.getRuntimeFeatureFlag("BUN_DESTRUCT_VM_ON_EXIT"),

Copy link
Collaborator

@Jarred-Sumner Jarred-Sumner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good but a few comments

auto* globalObject = defaultGlobalObject(lexicalGlobalObject);
if (globalObject && callback.isCallable()) {
if (auto* process = jsDynamicCast<Bun::Process*>(globalObject->processObject())) {
process->queueNextTick(globalObject, AsyncContextFrame::withAsyncContextIfNeeded(globalObject, callback));
Copy link
Collaborator

@Jarred-Sumner Jarred-Sumner Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check when Node calls this callback? It's unlikely called on process.nextTick

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/nodejs/node/blob/24e4a5447197c1adaec1538b0d55732e21b11a87/src/node_messaging.cc#L709

https://github.com/nodejs/node/blob/24e4a5447197c1adaec1538b0d55732e21b11a87/src/handle_wrap.cc#L81-L83

They attach it to the message port to be called later. I didn't look very hard into where it gets called from there; I'll undo the change for now and investigate more for a later PR

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#18768 is where the fixed version of this will go

@190n 190n marked this pull request as ready for review April 7, 2025 21:09
@Jarred-Sumner Jarred-Sumner merged commit eee5d4f into main Apr 8, 2025
67 of 70 checks passed
@Jarred-Sumner Jarred-Sumner deleted the ben/workers branch April 8, 2025 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants