util: add tokens to parseArgs

Offer additional meta-data for building
custom and additional behaviour on
top of parseArgs.

PR-URL: nodejs#43459
Reviewed-By: Ben Coe <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>

Author: shadowspawn

doc/api/util.md

@@ -1028,6 +1028,11 @@ equality.
 
 <!-- YAML
 added: v18.3.0
+changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/43459
+    description: add support for returning detailed parse information
+                 using `tokens` in input `config` and returned properties.
 -->
 
 > Stability: 1 - Experimental
@@ -1044,18 +1049,24 @@ added: v18.3.0
       times. If `true`, all values will be collected in an array. If
       `false`, values for the option are last-wins. **Default:** `false`.
     * `short` {string} A single character alias for the option.
-  * `strict`: {boolean} Should an error be thrown when unknown arguments
+  * `strict` {boolean} Should an error be thrown when unknown arguments
     are encountered, or when arguments are passed that do not match the
     `type` configured in `options`.
     **Default:** `true`.
-  * `allowPositionals`: {boolean} Whether this command accepts positional
+  * `allowPositionals` {boolean} Whether this command accepts positional
     arguments.
     **Default:** `false` if `strict` is `true`, otherwise `true`.
+  * `tokens` {boolean} Return the parsed tokens. This is useful for extending
+    the built-in behavior, from adding additional checks through to reprocessing
+    the tokens in different ways.
+    **Default:** `false`.
 
 * Returns: {Object} The parsed command line arguments:
   * `values` {Object} A mapping of parsed option names with their {string}
     or {boolean} values.
   * `positionals` {string\[]} Positional arguments.
+  * `tokens` {Object\[] | undefined} See [parseArgs tokens](#parseargs-tokens)
+    section. Only returned if `config` includes `tokens: true`.
 
 Provides a higher level API for command-line argument parsing than interacting
 with `process.argv` directly. Takes a specification for the expected arguments
@@ -1104,6 +1115,114 @@ console.log(values, positionals);
 `util.parseArgs` is experimental and behavior may change. Join the
 conversation in [pkgjs/parseargs][] to contribute to the design.
 
+### `parseArgs` `tokens`
+
+Detailed parse information is available for adding custom behaviours by
+specifying `tokens: true` in the configuration.
+The returned tokens have properties describing:
+
+* all tokens
+  * `kind` {string} One of 'option', 'positional', or 'option-terminator'.
+  * `index` {number} Index of element in `args` containing token. So the
+    source argument for a token is `args[token.index]`.
+* option tokens
+  * `name` {string} Long name of option.
+  * `rawName` {string} How option used in args, like `-f` of `--foo`.
+  * `value` {string | undefined} Option value specified in args.
+    Undefined for boolean options.
+  * `inlineValue` {boolean | undefined} Whether option value specified inline,
+    like `--foo=bar`.
+* positional tokens
+  * `value` {string} The value of the positional argument in args (i.e. `args[index]`).
+* option-terminator token
+
+The returned tokens are in the order encountered in the input args. Options
+that appear more than once in args produce a token for each use. Short option
+groups like `-xy` expand to a token for each option. So `-xxx` produces
+three tokens.
+
+For example to use the returned tokens to add support for a negated option
+like `--no-color`, the tokens can be reprocessed to change the value stored
+for the negated option.
+
+```mjs
+import { parseArgs } from 'node:util';
+
+const options = {
+  'color': { type: 'boolean' },
+  'no-color': { type: 'boolean' },
+  'logfile': { type: 'string' },
+  'no-logfile': { type: 'boolean' },
+};
+const { values, tokens } = parseArgs({ options, tokens: true });
+
+// Reprocess the option tokens and overwrite the returned values.
+tokens
+  .filter((token) => token.kind === 'option')
+  .forEach((token) => {
+    if (token.name.startsWith('no-')) {
+      // Store foo:false for --no-foo
+      const positiveName = token.name.slice(3);
+      values[positiveName] = false;
+      delete values[token.name];
+    } else {
+      // Resave value so last one wins if both --foo and --no-foo.
+      values[token.name] = token.value ?? true;
+    }
+  });
+
+const color = values.color;
+const logfile = values.logfile ?? 'default.log';
+
+console.log({ logfile, color });
+```
+
+```cjs
+const { parseArgs } = require('node:util');
+
+const options = {
+  'color': { type: 'boolean' },
+  'no-color': { type: 'boolean' },
+  'logfile': { type: 'string' },
+  'no-logfile': { type: 'boolean' },
+};
+const { values, tokens } = parseArgs({ options, tokens: true });
+
+// Reprocess the option tokens and overwrite the returned values.
+tokens
+  .filter((token) => token.kind === 'option')
+  .forEach((token) => {
+    if (token.name.startsWith('no-')) {
+      // Store foo:false for --no-foo
+      const positiveName = token.name.slice(3);
+      values[positiveName] = false;
+      delete values[token.name];
+    } else {
+      // Resave value so last one wins if both --foo and --no-foo.
+      values[token.name] = token.value ?? true;
+    }
+  });
+
+const color = values.color;
+const logfile = values.logfile ?? 'default.log';
+
+console.log({ logfile, color });
+```
+
+Example usage showing negated options, and when an option is used
+multiple ways then last one wins.
+
+```console
+$ node negate.js
+{ logfile: 'default.log', color: undefined }
+$ node negate.js --no-logfile --no-color
+{ logfile: false, color: false }
+$ node negate.js --logfile=test.log --color
+{ logfile: 'test.log', color: true }
+$ node negate.js --no-logfile --logfile=test.log --color --no-color
+{ logfile: 'test.log', color: false }
+```
+
 ## `util.promisify(original)`
 
 <!-- YAML