You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently, the Run Example test is executed nightly, meaning newly added or modified .pdl files in a pull request are not automatically validated before merging. As a result, developers can introduce changes to .pdl programs and merge their PRs without verifying that Run Example still functions correctly. This leads to frequent failures and inconsistencies, which negatively impact the reliability of Run Example and reflect poorly on the main page.
To improve stability, we should ensure that Run Example runs as part of the PR checks, preventing unverified changes from being merged.
Describe the solution you'd like
A "Run Changed PDL Examples" test should be executed each time a PR is opened. This pipeline should detect modifications to .pdl programs in the PR and execute a Pytest run for each of those files, ensuring that 0.result exists and is validated for that file.
Here's more criterias to consider:
Certain files should be skipped. In that case, 0.result doesn't exist
Certain files could result in runtime and parse errors. We need to ensure that this is reflected in test_example_runs.py
Certain files require inputs. Again, we need to ensure that this is captured in test_example_run.py
The check shouldn't take long for each PR
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Currently, the Run Example test is executed nightly, meaning newly added or modified
.pdl
files in a pull request are not automatically validated before merging. As a result, developers can introduce changes to.pdl
programs and merge their PRs without verifying that Run Example still functions correctly. This leads to frequent failures and inconsistencies, which negatively impact the reliability of Run Example and reflect poorly on the main page.To improve stability, we should ensure that Run Example runs as part of the PR checks, preventing unverified changes from being merged.
Describe the solution you'd like
A "Run Changed PDL Examples" test should be executed each time a PR is opened. This pipeline should detect modifications to
.pdl
programs in the PR and execute a Pytest run for each of those files, ensuring that0.result
exists and is validated for that file.Here's more criterias to consider:
0.result
doesn't existtest_example_runs.py
test_example_run.py
The text was updated successfully, but these errors were encountered: