Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug in manualRemove_dataset_HumanEvalComm_model_deepseek-coder-6.7b-instruct-finetuned-02212025_topn_1_temperature_1.0.log_1 #46

Open
jie-jw-wu opened this issue Mar 13, 2025 · 0 comments
Assignees

Comments

@jie-jw-wu
Copy link
Owner

jie-jw-wu commented Mar 13, 2025

there is a bug in manualRemove_dataset_HumanEvalComm_model_deepseek-coder-6.7b-instruct-finetuned-02212025_topn_1_temperature_1.0.log_1 and other deepseek model and codellama model, that after first run is executed, there will be two or more additional runs added.
Although current it does not affect eval results (confirmed), but we need to investigate why additional runs are added and these runs are not similar to the first runs.

One hypothesis is that results are added to log_1 in an unexpected way when running jobs outside phase 1

@jie-jw-wu jie-jw-wu self-assigned this Mar 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant