Skip to content

Add combo analysis to core analysis plugins #19449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stesteau opened this issue Jul 15, 2016 · 3 comments
Closed

Add combo analysis to core analysis plugins #19449

stesteau opened this issue Jul 15, 2016 · 3 comments

Comments

@stesteau
Copy link

The combo analysis plugin is not maintained anymore and it's quite usefull and necessery for multilingual indexes. This plugin should be moved to core and maintened by elastic.

https://github.com/yakaz/elasticsearch-analysis-combo
yakaz/elasticsearch-analysis-combo#21

Thank you

@clintongormley
Copy link
Contributor

This was already proposed in #1169 and we decided not to offer this plugin as it is a much better approach to use multiple fields with different analyzers instead of a single field with a mish-mash of tokens.

@stesteau
Copy link
Author

Could you reference some literature?

@clintongormley
Copy link
Contributor

I can't, but I can tell you that Lucene's analysis streams are designed to operate on a single field, with a single analyzer. Mixing up analyzers is likely to give you broken offsets and positions which could prevent indexing or result in run time exceptions with eg the FVH.

On top of that, you're messing with term statistics by creating duplicate terms for tokens which analyse identically, so the relevance calculation will be poor.

Lastly, you don't actually save much space compared to having a field per analyzer. And even if you do today, things may well change tomorrow as Lucene adds optimisations for its primary use case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants