Bidirectional text rendering (RTL)

[elsassph]

New renderers

New text texture renderers with opt-in bidirectional layout capability. This allows rendering right-to-left scripts (Arabic, Hebrew) and combination of left-to-right.

• Text renderers were converted to TypeScript,
• General code of renderers was largely unchanged, so there are no metrics changes for instance and the new renderers should be a drop-in upgrade,
• TextTextureAdvancedRenderer now extends the TextTextureRenderer, allowing duplication to be removed,
• Biggest change is around text layout: tokenization, wrapping and styling,
• Text rendering tokenizer can now be overridden for instance to better support Chinese or Thai (not included by default),
• By default bidirectional text isn't supported; it has to be enabled.

The API was designed for tree-shaking and performance: if you don't need bidi support, it won't be included and shouldn't be slower than before.

New implementation passes existing tests, and accuracy is confirmed by extra Mocha tests and new Playwright tests doing pixel comparison between real HTML rendering and the new renderer:

Usage

// RTL support
import { TextTexture, TextTokenizer } from '@lightningjs/core';
import { getBidiTokenizer } from '@lightningjs/core/bidiTokenizer';
TextTokenizer.setCustomTokenizer(getBidiTokenizer());
TextTexture.forceAdvancedRenderer = true; // only advanced renderer supports bidi layout

// Chinese/Thai support
import { TextTokenizer } from '@lightningjs/core';
import { loadDefaultSimplifiedChineseParser } from 'budoux';

const getSimplifiedChineseTokenizer = (): TextTokenizer.ITextTokenizerFunction => {
  const parser = loadDefaultSimplifiedChineseParser();
  return (text) => [{ tokens: parser.parse(text) }];
};

TextTokenizer.setCustomTokenizer(getSimplifiedChineseTokenizer(), true); 
// This Chinese tokenizer is very efficient but doesn't correctly tokenize English, so the second 
// `true` parameter hints `TextTokenizer` to handle 100% English text with the default tokenizer.

TODO

• Fix (slightly) incorrect truncation of embedded RTL text inside LTR.

[anand-patel-ap]

Hi Team,
When can we expect this PR to be reviewed and merged?

[elsassph]

@anand-patel-ap I found some edge cases so I'm working on hardening the implementation and adding unit tests. You may need to build it yourself to try.

[anand-patel-ap]

Sure, While building i got few mirnor issues. PFA:

[rdkcmf-jenkins]

Coverity detected 3 issues; a quality concern.