Skip to content

Issues with data model compilation with worker threads #9285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
igorlukanin opened this issue Feb 27, 2025 · 2 comments
Open

Issues with data model compilation with worker threads #9285

igorlukanin opened this issue Feb 27, 2025 · 2 comments
Assignees
Labels
data modeling question The issue is a question. Please use Stack Overflow for questions.

Comments

@igorlukanin
Copy link
Member

igorlukanin commented Feb 27, 2025

Recently, we've improved the data model compilation by moving critical parts of the code to worker threads.

It can be enabled by upgrading to v1.2.13+ and setting the CUBEJS_TRANSPILATION_WORKER_THREADS environment variable to true. See #9188 for details.

If you encounter any issues or see any changes/improvements to your deployments, please report here.

@igorlukanin igorlukanin added data modeling question The issue is a question. Please use Stack Overflow for questions. labels Feb 27, 2025
@igorlukanin igorlukanin self-assigned this Feb 27, 2025
@marianore-muttdata
Copy link
Contributor

Hey there!
Just wondering, if we have something like 20 data models and 100 tenants, should we be able to see a different at all in request delays?
Do you have insights on the performance difference in scenarios with "smaller" data models?

@igorlukanin
Copy link
Member Author

The smaller your data model is, the smaller the effect of this change is as well. Also, if your data model is compiled just once and not recompiled, then you'd to notice any difference at all.

The only case when the improvements should be evident is:

  1. You have a large data model
  2. It is frequently (re)compiled

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data modeling question The issue is a question. Please use Stack Overflow for questions.
Projects
None yet
Development

No branches or pull requests

2 participants