Skip to content

Enable sort optimization on float and half_float #126342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

mayya-sharipova
Copy link
Contributor

@mayya-sharipova mayya-sharipova commented Apr 4, 2025

Before we enabled sort optimization on long, double and date types,
but left other types for follow-up.

This enables sort optimization on float and half_float types.

Optimizations on INT, BYTE, SHORT are left for follow-up,
because they need more work: we currently use SORT.LONG
type for all integer types and this doesn't allow to use
optimization.

@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Apr 4, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@elasticsearchmachine
Copy link
Collaborator

Hi @mayya-sharipova, I've created a changelog YAML for you.

@mayya-sharipova
Copy link
Contributor Author

mayya-sharipova commented Apr 4, 2025

Tested on rally nyc_taxis track, where we index tip_amount field as float and half_float.

Results:

  • baseline (current main branch)
  • contender (this PR)
Metric Task Baseline Contender Diff Unit Diff %
90th percentile service time desc_sort_tip_amount_float 1195.78 11.9369 -1183.85 ms -99.00%
90th percentile service time asc_sort_tip_amount_float 1180.06 2.42422 -1177.63 ms -99.79%
90th percentile service time asc_sort_tip_amount_half_float 2589.36 2.20325 -2587.16 ms -99.91%
90th percentile service time asc_sort_tip_amount_half_float 2589.36 2.20325 -2587.16 ms -99.91%
Full results for baseline
|                                                         Metric |                            Task |            Value |   Unit |
|---------------------------------------------------------------:|--------------------------------:|-----------------:|-------:|
|                     Cumulative indexing time of primary shards |                                 |     85.6696      |    min |
|             Min cumulative indexing time across primary shards |                                 |     85.6696      |    min |
|          Median cumulative indexing time across primary shards |                                 |     85.6696      |    min |
|             Max cumulative indexing time across primary shards |                                 |     85.6696      |    min |
|            Cumulative indexing throttle time of primary shards |                                 |      0           |    min |
|    Min cumulative indexing throttle time across primary shards |                                 |      0           |    min |
| Median cumulative indexing throttle time across primary shards |                                 |      0           |    min |
|    Max cumulative indexing throttle time across primary shards |                                 |      0           |    min |
|                        Cumulative merge time of primary shards |                                 |     61.2744      |    min |
|                       Cumulative merge count of primary shards |                                 |     73           |        |
|                Min cumulative merge time across primary shards |                                 |     61.2744      |    min |
|             Median cumulative merge time across primary shards |                                 |     61.2744      |    min |
|                Max cumulative merge time across primary shards |                                 |     61.2744      |    min |
|               Cumulative merge throttle time of primary shards |                                 |     32.8222      |    min |
|       Min cumulative merge throttle time across primary shards |                                 |     32.8222      |    min |
|    Median cumulative merge throttle time across primary shards |                                 |     32.8222      |    min |
|       Max cumulative merge throttle time across primary shards |                                 |     32.8222      |    min |
|                      Cumulative refresh time of primary shards |                                 |      1.97777     |    min |
|                     Cumulative refresh count of primary shards |                                 |     69           |        |
|              Min cumulative refresh time across primary shards |                                 |      1.97777     |    min |
|           Median cumulative refresh time across primary shards |                                 |      1.97777     |    min |
|              Max cumulative refresh time across primary shards |                                 |      1.97777     |    min |
|                        Cumulative flush time of primary shards |                                 |      2.84968     |    min |
|                       Cumulative flush count of primary shards |                                 |     32           |        |
|                Min cumulative flush time across primary shards |                                 |      2.84968     |    min |
|             Median cumulative flush time across primary shards |                                 |      2.84968     |    min |
|                Max cumulative flush time across primary shards |                                 |      2.84968     |    min |
|                                        Total Young Gen GC time |                                 |     43.38        |      s |
|                                       Total Young Gen GC count |                                 |    423           |        |
|                                          Total Old Gen GC time |                                 |      0           |      s |
|                                         Total Old Gen GC count |                                 |      0           |        |
|                                                   Dataset size |                                 |     26.1902      |     GB |
|                                                     Store size |                                 |     26.1902      |     GB |
|                                                  Translog size |                                 |      5.12227e-08 |     GB |
|                                         Heap used for segments |                                 |      0           |     MB |
|                                       Heap used for doc values |                                 |      0           |     MB |
|                                            Heap used for terms |                                 |      0           |     MB |
|                                            Heap used for norms |                                 |      0           |     MB |
|                                           Heap used for points |                                 |      0           |     MB |
|                                    Heap used for stored fields |                                 |      0           |     MB |
|                                                  Segment count |                                 |     32           |        |
|                                    Total Ingest Pipeline count |                                 |      0           |        |
|                                     Total Ingest Pipeline time |                                 |      0           |      s |
|                                   Total Ingest Pipeline failed |                                 |      0           |        |
|                                                 Min Throughput |                           index | 196786           | docs/s |
|                                                Mean Throughput |                           index | 200442           | docs/s |
|                                              Median Throughput |                           index | 199493           | docs/s |
|                                                 Max Throughput |                           index | 205806           | docs/s |
|                                        50th percentile latency |                           index |    338.342       |     ms |
|                                        90th percentile latency |                           index |    467.953       |     ms |
|                                        99th percentile latency |                           index |   2003.41        |     ms |
|                                      99.9th percentile latency |                           index |   5223.63        |     ms |
|                                     99.99th percentile latency |                           index |   6114.89        |     ms |
|                                       100th percentile latency |                           index |   6510.01        |     ms |
|                                   50th percentile service time |                           index |    338.342       |     ms |
|                                   90th percentile service time |                           index |    467.953       |     ms |
|                                   99th percentile service time |                           index |   2003.41        |     ms |
|                                 99.9th percentile service time |                           index |   5223.63        |     ms |
|                                99.99th percentile service time |                           index |   6114.89        |     ms |
|                                  100th percentile service time |                           index |   6510.01        |     ms |
|                                                     error rate |                           index |      0.01        |      % |
|                                                 Min Throughput |               default_no_target |    285.11        |  ops/s |
|                                                Mean Throughput |               default_no_target |    285.11        |  ops/s |
|                                              Median Throughput |               default_no_target |    285.11        |  ops/s |
|                                                 Max Throughput |               default_no_target |    285.11        |  ops/s |
|                                        50th percentile latency |               default_no_target |      1.95542     |     ms |
|                                        90th percentile latency |               default_no_target |      2.40826     |     ms |
|                                        99th percentile latency |               default_no_target |      2.67739     |     ms |
|                                       100th percentile latency |               default_no_target |      5.47121     |     ms |
|                                   50th percentile service time |               default_no_target |      1.95542     |     ms |
|                                   90th percentile service time |               default_no_target |      2.40826     |     ms |
|                                   99th percentile service time |               default_no_target |      2.67739     |     ms |
|                                  100th percentile service time |               default_no_target |      5.47121     |     ms |
|                                                     error rate |               default_no_target |      0           |      % |
|                                                 Min Throughput |                         default |      3.02        |  ops/s |
|                                                Mean Throughput |                         default |      3.03        |  ops/s |
|                                              Median Throughput |                         default |      3.03        |  ops/s |
|                                                 Max Throughput |                         default |      3.06        |  ops/s |
|                                        50th percentile latency |                         default |     14.1325      |     ms |
|                                        90th percentile latency |                         default |     17.3301      |     ms |
|                                        99th percentile latency |                         default |     19.7128      |     ms |
|                                       100th percentile latency |                         default |     20.8316      |     ms |
|                                   50th percentile service time |                         default |     12.164       |     ms |
|                                   90th percentile service time |                         default |     15.2736      |     ms |
|                                   99th percentile service time |                         default |     18.3746      |     ms |
|                                  100th percentile service time |                         default |     19.4802      |     ms |
|                                                     error rate |                         default |      0           |      % |
|                                                 Min Throughput |                      default_1k |      3.02        |  ops/s |
|                                                Mean Throughput |                      default_1k |      3.03        |  ops/s |
|                                              Median Throughput |                      default_1k |      3.03        |  ops/s |
|                                                 Max Throughput |                      default_1k |      3.05        |  ops/s |
|                                        50th percentile latency |                      default_1k |     22.7621      |     ms |
|                                        90th percentile latency |                      default_1k |     24.814       |     ms |
|                                        99th percentile latency |                      default_1k |     32.4233      |     ms |
|                                       100th percentile latency |                      default_1k |     56.992       |     ms |
|                                   50th percentile service time |                      default_1k |     20.5098      |     ms |
|                                   90th percentile service time |                      default_1k |     22.5856      |     ms |
|                                   99th percentile service time |                      default_1k |     26.0072      |     ms |
|                                  100th percentile service time |                      default_1k |     29.9932      |     ms |
|                                                     error rate |                      default_1k |      0           |      % |
|                                                 Min Throughput |                           range |     49.86        |  ops/s |
|                                                Mean Throughput |                           range |     49.86        |  ops/s |
|                                              Median Throughput |                           range |     49.86        |  ops/s |
|                                                 Max Throughput |                           range |     49.86        |  ops/s |
|                                        50th percentile latency |                           range |     15.6114      |     ms |
|                                        90th percentile latency |                           range |     17.125       |     ms |
|                                        99th percentile latency |                           range |     21.7086      |     ms |
|                                       100th percentile latency |                           range |     22.0313      |     ms |
|                                   50th percentile service time |                           range |     15.6114      |     ms |
|                                   90th percentile service time |                           range |     17.125       |     ms |
|                                   99th percentile service time |                           range |     21.7086      |     ms |
|                                  100th percentile service time |                           range |     22.0313      |     ms |
|                                                     error rate |                           range |      0           |      % |
|                                                 Min Throughput |      desc_sort_tip_amount_float |      0.77        |  ops/s |
|                                                Mean Throughput |      desc_sort_tip_amount_float |      0.81        |  ops/s |
|                                              Median Throughput |      desc_sort_tip_amount_float |      0.82        |  ops/s |
|                                                 Max Throughput |      desc_sort_tip_amount_float |      0.83        |  ops/s |
|                                        50th percentile latency |      desc_sort_tip_amount_float |   1184.63        |     ms |
|                                        90th percentile latency |      desc_sort_tip_amount_float |   1195.78        |     ms |
|                                       100th percentile latency |      desc_sort_tip_amount_float |   1203.2         |     ms |
|                                   50th percentile service time |      desc_sort_tip_amount_float |   1184.63        |     ms |
|                                   90th percentile service time |      desc_sort_tip_amount_float |   1195.78        |     ms |
|                                  100th percentile service time |      desc_sort_tip_amount_float |   1203.2         |     ms |
|                                                     error rate |      desc_sort_tip_amount_float |      0           |      % |
|                                                 Min Throughput |       asc_sort_tip_amount_float |      0.85        |  ops/s |
|                                                Mean Throughput |       asc_sort_tip_amount_float |      0.85        |  ops/s |
|                                              Median Throughput |       asc_sort_tip_amount_float |      0.85        |  ops/s |
|                                                 Max Throughput |       asc_sort_tip_amount_float |      0.85        |  ops/s |
|                                        50th percentile latency |       asc_sort_tip_amount_float |   1166.21        |     ms |
|                                        90th percentile latency |       asc_sort_tip_amount_float |   1180.06        |     ms |
|                                       100th percentile latency |       asc_sort_tip_amount_float |   1194.18        |     ms |
|                                   50th percentile service time |       asc_sort_tip_amount_float |   1166.21        |     ms |
|                                   90th percentile service time |       asc_sort_tip_amount_float |   1180.06        |     ms |
|                                  100th percentile service time |       asc_sort_tip_amount_float |   1194.18        |     ms |
|                                                     error rate |       asc_sort_tip_amount_float |      0           |      % |
|                                                 Min Throughput | desc_sort_tip_amount_half_float |      0.37        |  ops/s |
|                                                Mean Throughput | desc_sort_tip_amount_half_float |      0.38        |  ops/s |
|                                              Median Throughput | desc_sort_tip_amount_half_float |      0.38        |  ops/s |
|                                                 Max Throughput | desc_sort_tip_amount_half_float |      0.39        |  ops/s |
|                                        50th percentile latency | desc_sort_tip_amount_half_float |   2565.13        |     ms |
|                                        90th percentile latency | desc_sort_tip_amount_half_float |   2575.25        |     ms |
|                                       100th percentile latency | desc_sort_tip_amount_half_float |   2613.72        |     ms |
|                                   50th percentile service time | desc_sort_tip_amount_half_float |   2565.13        |     ms |
|                                   90th percentile service time | desc_sort_tip_amount_half_float |   2575.25        |     ms |
|                                  100th percentile service time | desc_sort_tip_amount_half_float |   2613.72        |     ms |
|                                                     error rate | desc_sort_tip_amount_half_float |      0           |      % |
|                                                 Min Throughput |  asc_sort_tip_amount_half_float |      0.39        |  ops/s |
|                                                Mean Throughput |  asc_sort_tip_amount_half_float |      0.39        |  ops/s |
|                                              Median Throughput |  asc_sort_tip_amount_half_float |      0.39        |  ops/s |
|                                                 Max Throughput |  asc_sort_tip_amount_half_float |      0.39        |  ops/s |
|                                        50th percentile latency |  asc_sort_tip_amount_half_float |   2569.29        |     ms |
|                                        90th percentile latency |  asc_sort_tip_amount_half_float |   2589.36        |     ms |
|                                       100th percentile latency |  asc_sort_tip_amount_half_float |   2622.1         |     ms |
|                                   50th percentile service time |  asc_sort_tip_amount_half_float |   2569.29        |     ms |
|                                   90th percentile service time |  asc_sort_tip_amount_half_float |   2589.36        |     ms |
|                                  100th percentile service time |  asc_sort_tip_amount_half_float |   2622.1         |     ms |
|                                                     error rate |  asc_sort_tip_amount_half_float |      0           |      % |
|                                                 Min Throughput |             distance_amount_agg |      0.88        |  ops/s |
|                                                Mean Throughput |             distance_amount_agg |      0.89        |  ops/s |
|                                              Median Throughput |             distance_amount_agg |      0.89        |  ops/s |
|                                                 Max Throughput |             distance_amount_agg |      0.89        |  ops/s |
|                                        50th percentile latency |             distance_amount_agg |   1105.46        |     ms |
|                                        90th percentile latency |             distance_amount_agg |   1176.85        |     ms |
|                                       100th percentile latency |             distance_amount_agg |   1287.86        |     ms |
|                                   50th percentile service time |             distance_amount_agg |   1105.46        |     ms |
|                                   90th percentile service time |             distance_amount_agg |   1176.85        |     ms |
|                                  100th percentile service time |             distance_amount_agg |   1287.86        |     ms |
|                                                     error rate |             distance_amount_agg |      0           |      % |
|                                                 Min Throughput |                   autohisto_agg |     12.54        |  ops/s |
|                                                Mean Throughput |                   autohisto_agg |     13.23        |  ops/s |
|                                              Median Throughput |                   autohisto_agg |     13.31        |  ops/s |
|                                                 Max Throughput |                   autohisto_agg |     13.72        |  ops/s |
|                                        50th percentile latency |                   autohisto_agg |     66.8652      |     ms |
|                                        90th percentile latency |                   autohisto_agg |     67.3299      |     ms |
|                                        99th percentile latency |                   autohisto_agg |     69.9935      |     ms |
|                                       100th percentile latency |                   autohisto_agg |     70.5453      |     ms |
|                                   50th percentile service time |                   autohisto_agg |     66.8652      |     ms |
|                                   90th percentile service time |                   autohisto_agg |     67.3299      |     ms |
|                                   99th percentile service time |                   autohisto_agg |     69.9935      |     ms |
|                                  100th percentile service time |                   autohisto_agg |     70.5453      |     ms |
|                                                     error rate |                   autohisto_agg |      0           |      % |
|                                                 Min Throughput |              date_histogram_agg |    239.68        |  ops/s |
|                                                Mean Throughput |              date_histogram_agg |    240.83        |  ops/s |
|                                              Median Throughput |              date_histogram_agg |    240.83        |  ops/s |
|                                                 Max Throughput |              date_histogram_agg |    241.99        |  ops/s |
|                                        50th percentile latency |              date_histogram_agg |      3.4941      |     ms |
|                                        90th percentile latency |              date_histogram_agg |      5.62715     |     ms |
|                                        99th percentile latency |              date_histogram_agg |     11.4177      |     ms |
|                                       100th percentile latency |              date_histogram_agg |    119.41        |     ms |
|                                   50th percentile service time |              date_histogram_agg |      3.4941      |     ms |
|                                   90th percentile service time |              date_histogram_agg |      5.62715     |     ms |
|                                   99th percentile service time |              date_histogram_agg |     11.4177      |     ms |
|                                  100th percentile service time |              date_histogram_agg |    119.41        |     ms |
|                                                     error rate |              date_histogram_agg |      0           |      % |
Full results for candidate
|                                                         Metric |                            Task |            Value |   Unit |
|---------------------------------------------------------------:|--------------------------------:|-----------------:|-------:|
|                     Cumulative indexing time of primary shards |                                 |     83.5571      |    min |
|             Min cumulative indexing time across primary shards |                                 |     83.5571      |    min |
|          Median cumulative indexing time across primary shards |                                 |     83.5571      |    min |
|             Max cumulative indexing time across primary shards |                                 |     83.5571      |    min |
|            Cumulative indexing throttle time of primary shards |                                 |      0           |    min |
|    Min cumulative indexing throttle time across primary shards |                                 |      0           |    min |
| Median cumulative indexing throttle time across primary shards |                                 |      0           |    min |
|    Max cumulative indexing throttle time across primary shards |                                 |      0           |    min |
|                        Cumulative merge time of primary shards |                                 |     55.4796      |    min |
|                       Cumulative merge count of primary shards |                                 |     76           |        |
|                Min cumulative merge time across primary shards |                                 |     55.4796      |    min |
|             Median cumulative merge time across primary shards |                                 |     55.4796      |    min |
|                Max cumulative merge time across primary shards |                                 |     55.4796      |    min |
|               Cumulative merge throttle time of primary shards |                                 |     29.3164      |    min |
|       Min cumulative merge throttle time across primary shards |                                 |     29.3164      |    min |
|    Median cumulative merge throttle time across primary shards |                                 |     29.3164      |    min |
|       Max cumulative merge throttle time across primary shards |                                 |     29.3164      |    min |
|                      Cumulative refresh time of primary shards |                                 |      1.6677      |    min |
|                     Cumulative refresh count of primary shards |                                 |     70           |        |
|              Min cumulative refresh time across primary shards |                                 |      1.6677      |    min |
|           Median cumulative refresh time across primary shards |                                 |      1.6677      |    min |
|              Max cumulative refresh time across primary shards |                                 |      1.6677      |    min |
|                        Cumulative flush time of primary shards |                                 |      2.98622     |    min |
|                       Cumulative flush count of primary shards |                                 |     34           |        |
|                Min cumulative flush time across primary shards |                                 |      2.98622     |    min |
|             Median cumulative flush time across primary shards |                                 |      2.98622     |    min |
|                Max cumulative flush time across primary shards |                                 |      2.98622     |    min |
|                                        Total Young Gen GC time |                                 |     38.445       |      s |
|                                       Total Young Gen GC count |                                 |    389           |        |
|                                          Total Old Gen GC time |                                 |      0           |      s |
|                                         Total Old Gen GC count |                                 |      0           |        |
|                                                   Dataset size |                                 |     25.9803      |     GB |
|                                                     Store size |                                 |     25.9803      |     GB |
|                                                  Translog size |                                 |      5.12227e-08 |     GB |
|                                         Heap used for segments |                                 |      0           |     MB |
|                                       Heap used for doc values |                                 |      0           |     MB |
|                                            Heap used for terms |                                 |      0           |     MB |
|                                            Heap used for norms |                                 |      0           |     MB |
|                                           Heap used for points |                                 |      0           |     MB |
|                                    Heap used for stored fields |                                 |      0           |     MB |
|                                                  Segment count |                                 |     35           |        |
|                                    Total Ingest Pipeline count |                                 |      0           |        |
|                                     Total Ingest Pipeline time |                                 |      0           |      s |
|                                   Total Ingest Pipeline failed |                                 |      0           |        |
|                                                 Min Throughput |                           index | 201096           | docs/s |
|                                                Mean Throughput |                           index | 209905           | docs/s |
|                                              Median Throughput |                           index | 207515           | docs/s |
|                                                 Max Throughput |                           index | 228610           | docs/s |
|                                        50th percentile latency |                           index |    340.992       |     ms |
|                                        90th percentile latency |                           index |    478.772       |     ms |
|                                        99th percentile latency |                           index |   1929.45        |     ms |
|                                      99.9th percentile latency |                           index |   5484.74        |     ms |
|                                     99.99th percentile latency |                           index |   7569.14        |     ms |
|                                       100th percentile latency |                           index |   7844.92        |     ms |
|                                   50th percentile service time |                           index |    340.992       |     ms |
|                                   90th percentile service time |                           index |    478.772       |     ms |
|                                   99th percentile service time |                           index |   1929.45        |     ms |
|                                 99.9th percentile service time |                           index |   5484.74        |     ms |
|                                99.99th percentile service time |                           index |   7569.14        |     ms |
|                                  100th percentile service time |                           index |   7844.92        |     ms |
|                                                     error rate |                           index |      0.01        |      % |
|                                                 Min Throughput |               default_no_target |    319.98        |  ops/s |
|                                                Mean Throughput |               default_no_target |    319.98        |  ops/s |
|                                              Median Throughput |               default_no_target |    319.98        |  ops/s |
|                                                 Max Throughput |               default_no_target |    319.98        |  ops/s |
|                                        50th percentile latency |               default_no_target |      1.79496     |     ms |
|                                        90th percentile latency |               default_no_target |      2.2277      |     ms |
|                                        99th percentile latency |               default_no_target |      2.73558     |     ms |
|                                       100th percentile latency |               default_no_target |      6.80229     |     ms |
|                                   50th percentile service time |               default_no_target |      1.79496     |     ms |
|                                   90th percentile service time |               default_no_target |      2.2277      |     ms |
|                                   99th percentile service time |               default_no_target |      2.73558     |     ms |
|                                  100th percentile service time |               default_no_target |      6.80229     |     ms |
|                                                     error rate |               default_no_target |      0           |      % |
|                                                 Min Throughput |                         default |      3.02        |  ops/s |
|                                                Mean Throughput |                         default |      3.03        |  ops/s |
|                                              Median Throughput |                         default |      3.03        |  ops/s |
|                                                 Max Throughput |                         default |      3.06        |  ops/s |
|                                        50th percentile latency |                         default |     11.9339      |     ms |
|                                        90th percentile latency |                         default |     14.1389      |     ms |
|                                        99th percentile latency |                         default |     16.1274      |     ms |
|                                       100th percentile latency |                         default |     18.8175      |     ms |
|                                   50th percentile service time |                         default |     10.0321      |     ms |
|                                   90th percentile service time |                         default |     11.9062      |     ms |
|                                   99th percentile service time |                         default |     14.1333      |     ms |
|                                  100th percentile service time |                         default |     16.4925      |     ms |
|                                                     error rate |                         default |      0           |      % |
|                                                 Min Throughput |                      default_1k |      3.02        |  ops/s |
|                                                Mean Throughput |                      default_1k |      3.03        |  ops/s |
|                                              Median Throughput |                      default_1k |      3.03        |  ops/s |
|                                                 Max Throughput |                      default_1k |      3.05        |  ops/s |
|                                        50th percentile latency |                      default_1k |     19.3333      |     ms |
|                                        90th percentile latency |                      default_1k |     21.428       |     ms |
|                                        99th percentile latency |                      default_1k |     22.6266      |     ms |
|                                       100th percentile latency |                      default_1k |     42.182       |     ms |
|                                   50th percentile service time |                      default_1k |     17.1181      |     ms |
|                                   90th percentile service time |                      default_1k |     19.2999      |     ms |
|                                   99th percentile service time |                      default_1k |     19.93        |     ms |
|                                  100th percentile service time |                      default_1k |     20.4645      |     ms |
|                                                     error rate |                      default_1k |      0           |      % |
|                                                 Min Throughput |                           range |     54.13        |  ops/s |
|                                                Mean Throughput |                           range |     54.13        |  ops/s |
|                                              Median Throughput |                           range |     54.13        |  ops/s |
|                                                 Max Throughput |                           range |     54.13        |  ops/s |
|                                        50th percentile latency |                           range |     15.0456      |     ms |
|                                        90th percentile latency |                           range |     17.1059      |     ms |
|                                        99th percentile latency |                           range |     18.237       |     ms |
|                                       100th percentile latency |                           range |     18.2661      |     ms |
|                                   50th percentile service time |                           range |     15.0456      |     ms |
|                                   90th percentile service time |                           range |     17.1059      |     ms |
|                                   99th percentile service time |                           range |     18.237       |     ms |
|                                  100th percentile service time |                           range |     18.2661      |     ms |
|                                                     error rate |                           range |      0           |      % |
|                                                 Min Throughput |      desc_sort_tip_amount_float |     46.78        |  ops/s |
|                                                Mean Throughput |      desc_sort_tip_amount_float |     46.78        |  ops/s |
|                                              Median Throughput |      desc_sort_tip_amount_float |     46.78        |  ops/s |
|                                                 Max Throughput |      desc_sort_tip_amount_float |     46.78        |  ops/s |
|                                        50th percentile latency |      desc_sort_tip_amount_float |     11.31        |     ms |
|                                        90th percentile latency |      desc_sort_tip_amount_float |     11.9369      |     ms |
|                                       100th percentile latency |      desc_sort_tip_amount_float |     12.7691      |     ms |
|                                   50th percentile service time |      desc_sort_tip_amount_float |     11.31        |     ms |
|                                   90th percentile service time |      desc_sort_tip_amount_float |     11.9369      |     ms |
|                                  100th percentile service time |      desc_sort_tip_amount_float |     12.7691      |     ms |
|                                                     error rate |      desc_sort_tip_amount_float |      0           |      % |
|                                                 Min Throughput |       asc_sort_tip_amount_float |    248.45        |  ops/s |
|                                                Mean Throughput |       asc_sort_tip_amount_float |    248.45        |  ops/s |
|                                              Median Throughput |       asc_sort_tip_amount_float |    248.45        |  ops/s |
|                                                 Max Throughput |       asc_sort_tip_amount_float |    248.45        |  ops/s |
|                                        50th percentile latency |       asc_sort_tip_amount_float |      2.05563     |     ms |
|                                        90th percentile latency |       asc_sort_tip_amount_float |      2.42422     |     ms |
|                                       100th percentile latency |       asc_sort_tip_amount_float |      3.50667     |     ms |
|                                   50th percentile service time |       asc_sort_tip_amount_float |      2.05563     |     ms |
|                                   90th percentile service time |       asc_sort_tip_amount_float |      2.42422     |     ms |
|                                  100th percentile service time |       asc_sort_tip_amount_float |      3.50667     |     ms |
|                                                     error rate |       asc_sort_tip_amount_float |      0           |      % |
|                                                 Min Throughput | desc_sort_tip_amount_half_float |     59.87        |  ops/s |
|                                                Mean Throughput | desc_sort_tip_amount_half_float |     59.87        |  ops/s |
|                                              Median Throughput | desc_sort_tip_amount_half_float |     59.87        |  ops/s |
|                                                 Max Throughput | desc_sort_tip_amount_half_float |     59.87        |  ops/s |
|                                        50th percentile latency | desc_sort_tip_amount_half_float |     10.8056      |     ms |
|                                        90th percentile latency | desc_sort_tip_amount_half_float |     12.7851      |     ms |
|                                       100th percentile latency | desc_sort_tip_amount_half_float |     17.7602      |     ms |
|                                   50th percentile service time | desc_sort_tip_amount_half_float |     10.8056      |     ms |
|                                   90th percentile service time | desc_sort_tip_amount_half_float |     12.7851      |     ms |
|                                  100th percentile service time | desc_sort_tip_amount_half_float |     17.7602      |     ms |
|                                                     error rate | desc_sort_tip_amount_half_float |      0           |      % |
|                                                 Min Throughput |  asc_sort_tip_amount_half_float |    290.69        |  ops/s |
|                                                Mean Throughput |  asc_sort_tip_amount_half_float |    290.69        |  ops/s |
|                                              Median Throughput |  asc_sort_tip_amount_half_float |    290.69        |  ops/s |
|                                                 Max Throughput |  asc_sort_tip_amount_half_float |    290.69        |  ops/s |
|                                        50th percentile latency |  asc_sort_tip_amount_half_float |      1.97798     |     ms |
|                                        90th percentile latency |  asc_sort_tip_amount_half_float |      2.20325     |     ms |
|                                       100th percentile latency |  asc_sort_tip_amount_half_float |      2.3505      |     ms |
|                                   50th percentile service time |  asc_sort_tip_amount_half_float |      1.97798     |     ms |
|                                   90th percentile service time |  asc_sort_tip_amount_half_float |      2.20325     |     ms |
|                                  100th percentile service time |  asc_sort_tip_amount_half_float |      2.3505      |     ms |
|                                                     error rate |  asc_sort_tip_amount_half_float |      0           |      % |
|                                                 Min Throughput |             distance_amount_agg |      0.95        |  ops/s |
|                                                Mean Throughput |             distance_amount_agg |      0.95        |  ops/s |
|                                              Median Throughput |             distance_amount_agg |      0.95        |  ops/s |
|                                                 Max Throughput |             distance_amount_agg |      0.96        |  ops/s |
|                                        50th percentile latency |             distance_amount_agg |   1032.4         |     ms |
|                                        90th percentile latency |             distance_amount_agg |   1056.36        |     ms |
|                                       100th percentile latency |             distance_amount_agg |   1077.32        |     ms |
|                                   50th percentile service time |             distance_amount_agg |   1032.4         |     ms |
|                                   90th percentile service time |             distance_amount_agg |   1056.36        |     ms |
|                                  100th percentile service time |             distance_amount_agg |   1077.32        |     ms |
|                                                     error rate |             distance_amount_agg |      0           |      % |
|                                                 Min Throughput |                   autohisto_agg |     12.13        |  ops/s |
|                                                Mean Throughput |                   autohisto_agg |     13.04        |  ops/s |
|                                              Median Throughput |                   autohisto_agg |     13.16        |  ops/s |
|                                                 Max Throughput |                   autohisto_agg |     13.64        |  ops/s |
|                                        50th percentile latency |                   autohisto_agg |     66.1679      |     ms |
|                                        90th percentile latency |                   autohisto_agg |     70.9618      |     ms |
|                                        99th percentile latency |                   autohisto_agg |     78.2489      |     ms |
|                                       100th percentile latency |                   autohisto_agg |     81.4795      |     ms |
|                                   50th percentile service time |                   autohisto_agg |     66.1679      |     ms |
|                                   90th percentile service time |                   autohisto_agg |     70.9618      |     ms |
|                                   99th percentile service time |                   autohisto_agg |     78.2489      |     ms |
|                                  100th percentile service time |                   autohisto_agg |     81.4795      |     ms |
|                                                     error rate |                   autohisto_agg |      0           |      % |
|                                                 Min Throughput |              date_histogram_agg |    165.42        |  ops/s |
|                                                Mean Throughput |              date_histogram_agg |    168.77        |  ops/s |
|                                              Median Throughput |              date_histogram_agg |    168.77        |  ops/s |
|                                                 Max Throughput |              date_histogram_agg |    172.13        |  ops/s |
|                                        50th percentile latency |              date_histogram_agg |      4.43994     |     ms |
|                                        90th percentile latency |              date_histogram_agg |      7.24175     |     ms |
|                                        99th percentile latency |              date_histogram_agg |     11.2178      |     ms |
|                                       100th percentile latency |              date_histogram_agg |     19.3952      |     ms |
|                                   50th percentile service time |              date_histogram_agg |      4.43994     |     ms |
|                                   90th percentile service time |              date_histogram_agg |      7.24175     |     ms |
|                                   99th percentile service time |              date_histogram_agg |     11.2178      |     ms |
|                                  100th percentile service time |              date_histogram_agg |     19.3952      |     ms |
|                                                     error rate |              date_histogram_agg |      0           |      % |

@mayya-sharipova mayya-sharipova requested a review from jimczi April 7, 2025 12:53
@@ -199,7 +200,8 @@ private XFieldComparatorSource comparatorSource(
Nested nested
) {
return switch (targetNumericType) {
case HALF_FLOAT, FLOAT -> new FloatValuesComparatorSource(this, missingValue, sortMode, nested);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more a question than feedback. Since HALF_FLOAT is no longer handled by FloatValuesComparatorSource are there missing unit tests that are needed. I noticed FloatValuesComparatorSource is referenced by FloatNestedSortingTests. Does it make sense to have a HalfFloatNestedSortingTests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@john-wagster Thanks for the feedback, addressed in ee021d7

Copy link
Contributor

@john-wagster john-wagster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right off everything lgtm

@@ -524,6 +524,9 @@ public void testSimpleSorts() throws Exception {
.startObject("float_value")
.field("type", "float")
.endObject()
.startObject("half_float_value")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my initial concern was if this integration test would ever be multi-node. This way we have multiple data nodes and thus might sort again on the coordinator. But it looks like this is randomly multiple data nodes. So I think we are good on coverage there.

Copy link
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @mayya-sharipova

Copy link
Contributor

@ChrisHegarty ChrisHegarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mayya-sharipova mayya-sharipova added the auto-backport Automatically create backport pull requests when merged label Apr 9, 2025
@mayya-sharipova mayya-sharipova merged commit d82886f into elastic:main Apr 9, 2025
17 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 126342

@mayya-sharipova mayya-sharipova deleted the sort_other_numeric_types branch April 9, 2025 13:29
elasticsearchmachine pushed a commit that referenced this pull request Apr 9, 2025
* Enable sort optimization on float and half_float (#126342)

Before we enabled sort optimization on long, double and date types,
but left other types for follow-up.

This enables sort optimization on float and half_float types.

Optimizations on INT, BYTE, SHORT are left for follow-up,
because they need more work: we currently use SORT.LONG
type for all integer types and this doesn't allow to use
optimization.

Backport for #126342

* Modification
mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this pull request Apr 9, 2025
Before we enabled sort optimization on long, double and date types,
but left other types for follow-up.

This enables sort optimization on float and half_float types.

Optimizations on INT, BYTE, SHORT are left for follow-up,
because they need more work: we currently use SORT.LONG
type for all integer types and this doesn't allow to use
optimization.
elasticsearchmachine pushed a commit that referenced this pull request Apr 9, 2025
Before we enabled sort optimization on long, double and date types,
but left other types for follow-up.

This enables sort optimization on float and half_float types.

Optimizations on INT, BYTE, SHORT are left for follow-up,
because they need more work: we currently use SORT.LONG
type for all integer types and this doesn't allow to use
optimization.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged backport pending >enhancement :Search Relevance/Search Catch all for Search Relevance Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v8.19.0 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants