[FLINK-37649][datagen] Datagen connector cannot set length for collection type

[reswqa]

What is the purpose of the change

Fix the bug that datagen connector cannot set length for collection type.

Brief change log

• Add lenOption to the legitimate set of options.

Verifying this change

Introduce a unit test case to cover this.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no

[flinkbot]

CI report:

• 3125387 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[davidradl]

I wonder if we should add the test case mentioned in the Jira to the datagen docs as an example WDYT?

[reswqa]

I wonder if we should add the test case mentioned in the Jira to the datagen docs as an example WDYT?

@davidradl The documentation indeed is not very friendly especially when it does not mention some of the supported features(e.g. fields.xxx.element/key/value.yyy for nested type). I'm inclined that we should rewrite/refactor it, but there's not much bandwidth for me at the moment . If you'd like to update it, I'd be happy to review :).

[davidradl]

@reswqa thanks I can improve the docs. I have raised a Jira . Please could you assign me the Jira, thanks :-)

[codenohup]

LGTM