Support text-shaping

[InterGalacticCold]

I have some text written in Bengali in this upcoming.md file. In the screenshot that markdown file is opened with Ecode and Notepads. Both have utf-8 selected as their encoder, but Ecode fails to render the text. Please fix it and let me know if there's more information you need.
Thanks a lot for making such a great application.

[SpartanJ]

Hi!
I just realized there was an actual bug for setting fonts fallback.
Although, this is a font issue, basically in order to be able to render those characters (they are called glyphs) we need to have loaded a font that supports those glyphs. In your case you'll need to set a bengali fallback font, for example I just tried using as fallback font Noto Serif Bengali Regular and worked. But, first, I had to fix fallback fonts (it was not loading the user-set fallback font but the default one even if the user changed it). So in order to test this you'll need to download the latest nightly build.

Go to Settings -> Window -> Fallback Font... and select the downloaded font, then you'll need to restart ecode and that should be it, now you can open any file with bengali text.

In the future I'll try to simplify the process to easily download / get the required fonts. But for now this is enough.

[InterGalacticCold]

@SpartanJ it's somewhat rendering this time. The characters are rendering properly but their position and form is messed up.
I am writing below the first three lines of your screenshot. These is how they are supposed to look:

1. Hello: হ্যালো / নমস্কার
2. Good morning: সুপ্রভাত
3. Good night: শুভ রাত্রি

I can't find an easy way to describe you the problem. One thing you need to know that in Bengali, we have symbols for vowels and characters sometimes change to a different shape/form depending on their position in a word. In this case the position of the symbols are not correct and the characters aren't changing their form/shape where they're supposed to.

I've also tried using two different fonts and all of them have the same issue.

[SpartanJ]

Oh, sorry, I did not realise that Bengali required text-shaping (mentioned here).
Some languages require text-shaping for the reasons you described. ecode still does not support text-shaping but there's a partial implementation that can be activated as an optional parameter before opening the editor, if you run: ecode --text-shaper (it's hidden because it's really not ready) it will start ecode with text-shaping capabilities. I think that for your particular case you could use it although I don't recommend it for every day use, since you'll most surely encounter problems (I did't finish for example cursor navigation, it will behave in a wonky manner). Right to left writting languages are still not working but I think Bengali seems to work somewhat OK, I really have difficulties recognising it

So in short: it's not fully supported, it's a work in progress, and there's plenty of work to be done.

I'll use this issue to keep track of the progress (also will rename the issue to make it more general).

I'd really like to have this working properly but it's a lot of work and honestly not easy nor fun either. Text shaping is a very CPU heavy operation and using it with the current implementation reduces the general text-rendering performance beyond the 50%, to avoid that it's required to do some heavy caching that goes against the current architecture (which is very straightforward and very very fast).

[InterGalacticCold]

@SpartanJ in the screenshot of your last comment, it is working properly. So I think you're there, I don't know how far.

[SpartanJ]

Cool, I just enabled this --text-shaper option in the nightly builds just in case you want to try it. I'll continue working on this issue soon.

[InterGalacticCold]

I've tried the nightly with text-shaper, it seems to work just fine. I am not noticing any problems. You also mentioned concern about CPU usage, which I found to be totally normal.