Fix formatting in quiet mode #116
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The clar understands both "-v" and "-q". Both of these arguments effectively increase or decrease verbosity, but right now their respective effects are somewhat different from one another. While "-v" makes us print more information, "-q" only skips over successful tests without modifying the verbosity itself. This behaviour is somewhat inconsistent and confusing, and the resulting implementation is more complex than it really ought to be. This is demonstrated by the fact that we have several bugs when it comes to the verbosity, all of which seem to be consequences of this complexity. Refactor the code so that we only have a single "verbosity". "-v" increases verbosity, whereas "-q" decreases verbosity. When verbosity is negative, we start to reduce our verbosity, whereas positive values will start to increase our verbosity. This refactoring is a first step towards improving quiet output over the next couple of commits.
The "-q" option is supposed to decrease verbosity. Funny enough though the end result is that we have _increased_ verbosity because we now end up printing every failing test twice: once via `clar_report_errors()`, and once via `clar_report_all()` via `clar_print_clap_shutdown()`. Fix this by dropping the call to `clar_report_errors()` so that failing tests will only be printed on shutdown.
Both `clar_print_ontest()` and `clar_print_onsuite()` are only called when we aren't asked to be quiet. This results in buggy behaviour though, where we for example don't print tests at all with the TAP format, which is invalid. Refactor the code so that the format implementations themselves can decide whether or not to skip writing output based on verbosity. For now the behaviour remains the same, but we will iterate on the respective formats over the next couple of commits.
The TAP format we print when asked to be quiet is broken because we don't the "ok/not ok" line for any of the individual tests. This does not follow the specification and would thus confuse parsers. Fix the issue by printing tests even when asked to be quiet. To have the "-q" flag still have an effect we instead skip the reason why the test has failed.
When asked to be quiet we don't print the status for each of the individual tests (the "...FF..F" indicator). But we still end up printing the legend explaining what each of the indicator means as well as how many test suites we have loaded, both of which feel unnecessarily verbose. Stop printing this information in quiet mode. The end result is that we only end up printing failures, but nothing else.
|
Merging, as this has been open for review for a couple weeks by now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The quiet mode activated via "-q" is quite broken:
This PR fixes all of those issues.