[protobuf] add support for Protobuf v3

[kubacech]

Hello,
I want to ask in what state is adding support for protobufs v3? V3 is released for some time now and in docs is a mention, that there is plan to add support for it.

Thanks,
Jakub

[cowtowncoder]

@kubacech I don't think anyone is working on protobuf v3 support at this point. I don't have time myself, but can help if anyone is interested. I am not sure how easy/possible it will be support both v2 and v3 with same package; I hope that is plausible. If not we'd need a separate format module (which would be ok too).

[wujimin]

i have a try to add basic map support
currently map key can be Integer/Long/String

can i raise a PR for this?

[cowtowncoder]

@wujimin Since this is not related to question of protobuf 3 support, could you please add it either to an existing issue that is related to Map support (I think there is one), or, if none exists, file a new one?

[wujimin]

very sorry for the late reply.

after add map feature (huaweicse#1)
i had tried to add "Any" feature, but found too difficult to do this.
in fact, my map feature solution is not so good too.

after a performance test, i found, compare to protoStuff, jackson solution is not so good

so i had gave up jackson solution, and will write new serializer/deserializer based on jackson and protoStuff

[cowtowncoder]

@wujimin I would be interested in performance part, as Jackson backend should have performance comparable to protostuff (when binding to POJOs). Not necessarily faster, but similar. So mostly interested in kind of test you tried since maybe I could find things to improve here.

Thank you for your update, even if you are not planning to use Jackson backend. And good luck with development using protostuff, that is a good library too.

[wujimin]

i have removed jackson test part. (maybe my new map feature cause a bad performance, :))
and my new solution just only finished most serializers(scalars/collection/map/message/any, not include enum/oneOf yet.), have not start deserializers

after all features finished, i will do more end to end performance test, include jackson
and then update to this issue.

[cowtowncoder]

@wujimin Thanks!

[wujimin]

hi, i have finished most features

PR in my project: apache/servicecomb-java-chassis#930
there are performance data in the page.

performance test code: apache/servicecomb-java-chassis@4ab91bc

[vincentfree]

Any progress with support for V3?

[cowtowncoder]

Usually when there is progress there are notes here so I assume no progress unfortunately.

[jedvardsson]

Proto3 syntax have been available in an official Protobuf release since 2016. I can’t find any note that the syntax is not official. They’ve added back optional but other than that it has been quite stable, right? Is anyone using proto2 nowadays.

So, what would the efforts be for implementing a Jackson module for it. Any benefits of waiting for Jackson 3?

[cowtowncoder]

The only blocker here is the lack of resources -- someone to write the code, test.
Support need not wait until Jackson 3; nor is there anything specific within Jackson 3 (master branch) to make it easier to support.

So if there is someone with an itch and time, it'd be great to get support added. But I do not personally have time to work on this, unfortunately. As much as I would like to see it happen. I try to find time help others, however, when they produce something.

So: The usual OSS Credo applies here -- Contributions Welcome!
(new module, or update to this one if backwards compatible, either way is fine)

[jedvardsson]

I appreciate the effort you put into Jackson. I’ll see if there was a something I can do to help. I’ll need to study the existing code to understand the scope.

[cowtowncoder]

Thank you @jedvardsson (and apologies for slow follow up). Please let me know if you need any help!
And any findings, suggestions might be helpful too -- perhaps there are incremental things that could be added.
If I remember correctly, even with ProtoV2 there are some things (like convention to support Maps) that could use improvements.