[fontscan] Honor fontconfig substitutions

[auroraanna]

I am having issues with japanese characters in a program (supersonic) that uses Fyne which uses this project to render text. they are not rendered using the system default font for them which is Noto Sans CJK but another font which makes them way more pixelated, small and less legible for me.

Tracklist of album "supercell" in supersonic:

One of the song titles in neovim:

Choosing Noto Sans CJK as the application font in the supersonic settings renders japanese characters nicely but makes all the latin characters have a weird wide serif font which is also not desirable.

Actually since taking these screenshots, the program now does not display japanese characters at all anymore while other programs have no issue doing so with the system's font for this (set through font fallbacks).

I get that application developers might want to use specific fonts but in this case it is obviously necessary to use (system) fallbacks as fonts do not typically cover all unicode characters.

[benoitkugler]

Hi,

Thank you for the report.

Could you precise your OS ? Also, what do you mean by "system default font" ? Could you elaborate on how it is defined ?

[auroraanna]

I'm using the linux distribution NixOS. fontconfig is used on linux distributions to determine default fonts. You can specifiy defaults for sans serif, serif, monospace and emoji. I expect this to be similar on other operating systems.

There probably exist cross-platform go libraries which abstract over these OS specific implementations on how to get the default fonts.

[benoitkugler]

Do you have a custom fontconfig file or do you use the config coming with NixOS ?

We have a "custom" implementation of fontconfig, but we try to follow the default one, so there migth be hope here :)

[benoitkugler]

Also, for convenience, could you copy paste a textual form of the string showing poor font selection ?

[auroraanna]

Also, for convenience, could you copy paste a textual form of the string showing poor font selection ?

e.g. "ワールドイズマイン"

[auroraanna]

Do you have a custom fontconfig file or do you use the config coming with NixOS ?

We have a "custom" implementation of fontconfig, but we try to follow the default one, so there migth be hope here :)

I am not using the default fontconfig on NixOS but my fontconfig is very unlikely to contain any syntax errors or invalid configuration.

[hajimehoshi]

I'm not sure there is always an OS specific font-fallback rule available. By the way, a browser like Chrome has its own fallback rule (e.g. https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/fonts/win/font_fallback_win.cc;l=120;drc=46845d063a7f0a61f4fe479c7d735ba19fcaca8f).

[benoitkugler]

Do you have a custom fontconfig file or do you use the config coming with NixOS ?
We have a "custom" implementation of fontconfig, but we try to follow the default one, so there migth be hope here :)

I am not using the default fontconfig on NixOS but my fontconfig is very unlikely to contain any syntax errors or invalid configuration.

Thank you for the clarification. Currently, we do not support custom fontconfig configuration. (We only emulate the default one.)
We might want to support the subset of the fontconfig rules dealing with font substitution.

[benoitkugler]

Also, for future reference, could you share the relevant part of your custom fontconfig configuration ?

[auroraanna]

i'm not sure if we're talking about the same thing…

what i thought fontconfig was is that you configure you preferred fonts for sansserif, serif, monospace, but that part is just one of the many files in /etc/fonts/conf.d. it contains many XML elements like this one:

<alias binding="same">
  <family>serif</family>
  <prefer>
    <family>Roboto Slab</family>
    <family>Noto Serif CJK JP</family>
  </prefer>
</alias>

do you mean that you don't support this kind of configuration? i would think that there is no default configuration for the preferred fonts from the fontconfig developers.

as for the other files in /etc/fonts, there's a lot of generated files which seem to just do a lot of classification like "all these fonts are monospace, and these others are serif" or telling fontconfig where fonts are installed.

my /etc/fonts/fonts.conf seems to be directly from the fontconfig developers if that is what you mean with fontconfig configuration and that you don't support a custom one…

[benoitkugler]

I was rather imprecise, sorry. I would like to know if you have written yourself a config file, or if you only use the ones coming with NixOS.

Besides, what I've been calling "default configuration" is actually the configuration installed by default on Ubuntu (copied here). I guess we could add some rules if they make sense as sensible default.

[auroraanna]

a nixos system with fontconfig enabled will have files similar to those from ubuntu installed. by changing the nixos config, e.g. setting the preference for serif to Font X, then Font Y, these files can be changed indirectly.

as far as i can tell, combining these files (while only ever picking one file for each prefix number?) yields a config which dictates a certain font preference/fallback chain for each font type (e.g. system-ui, sans-serif, serif, emoji, math). i am guessing that the fontconfig library does this.

so if an application developer using this library and sets a font for some text, if a character in that text is not contained in the font, the next font from fontconfig shall be used.

so i don't think you need to care about how my fontconfig looks or if it is "custom".

[benoitkugler]

a nixos system with fontconfig enabled will have files similar to those from ubuntu installed. by changing the nixos config, e.g. setting the preference for serif to Font X, then Font Y, these files can be changed indirectly.

as far as i can tell, combining these files (while only ever picking one file for each prefix number?) yields a config which dictates a certain font preference/fallback chain for each font type (e.g. system-ui, sans-serif, serif, emoji, math). i am guessing that the fontconfig library does this.

so if an application developer using this library and sets a font for some text, if a character in that text is not contained in the font, the next font from fontconfig shall be used.

so i don't think you need to care about how my fontconfig looks or if it is "custom".

Yes, it's true that fontconfig supports any setup, a default one or with custom files.
But it is not clear that we want do support the full scope of the C version, because it is quite involved. It would be easier to focus on a set of default config files and extract the relevant substitutions. It it what we have done with the Ubuntu default config.
Of course, this workaround does not work if you write your self additional files, hence my question.

The best solution would be to fully support fontconfig substitutions rules.

My question was also about fetching test references: something like "with these config files, I get this font" so that we could check our implementation