Re: support C/C++ identifiers named with non-ASCII characters

Joseph Myers
2018-05-21T20:27:00Z

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

On Mon, 21 May 2018, Paul.Koning@dell.com wrote:

> I don't know what the C/C++ standards say about non-ASCII identifiers.
> I assume they are stated to be Unicode, and presumably specific Unicode

They are defined in terms of ISO 10646 (and so concepts from Unicode that
don't appear in ISO 10646, such as normalization forms, are not relevant
to them).

See C11 Annex D, which is also aligned with C++11 and later (older
standard versions had generally more restrictive sets of allowed
characters, different for C and C++, based on TR 10176).

> Yet another issue: for many characters, there are multiple ways to
> represent them in Unicode. For example, Ã¼ (latin small letter u with
> dieresis) can be coded as the single Unicode character 0xfc, or as the
> pair 0x0308 0x75 (combining dieresis, latin small letter u). These are

(The letter goes before the combining mark in that case, not after. Thus
such combining marks are not generally permitted at the start of
identifiers.)

> supposed to be synonymous; when doing string matches, you'd want them to

They are *not* synonymous in C or C++. (GCC has -Wnormalized= options to
warn about identifiers not in an appropriate normalization form, with
-Wnormalized=nfc as the default.)

GCC always generates UTF-8 in its .s output for such identifiers, which
gas then transfers straight through to its output (thus, UTF-8 ELF
symbols). The generic ELF ABI is silent on the encoding of such symbols
(it just says "External C symbols have the same names in C and object
files' symbol tables.").

--
Joseph S. Myers
joseph@codesourcery.com