Re: support C/C++ identifiers named with non-ASCII characters

[Joseph Myers <joseph@codesourcery.com>]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

On Mon, 21 May 2018, Paul.Koning@dell.com wrote:

> I don't know what the C/C++ standards say about non-ASCII identifiers.  
> I assume they are stated to be Unicode, and presumably specific Unicode 

They are defined in terms of ISO 10646 (and so concepts from Unicode that 
don't appear in ISO 10646, such as normalization forms, are not relevant 
to them).

See C11 Annex D, which is also aligned with C++11 and later (older 
standard versions had generally more restrictive sets of allowed 
characters, different for C and C++, based on TR 10176).

> Yet another issue: for many characters, there are multiple ways to 
> represent them in Unicode.  For example, ü (latin small letter u with 
> dieresis) can be coded as the single Unicode character 0xfc, or as the 
> pair 0x0308 0x75 (combining dieresis, latin small letter u).  These are 

(The letter goes before the combining mark in that case, not after.  Thus 
such combining marks are not generally permitted at the start of 
identifiers.)

> supposed to be synonymous; when doing string matches, you'd want them to 

They are *not* synonymous in C or C++.  (GCC has -Wnormalized= options to 
warn about identifiers not in an appropriate normalization form, with 
-Wnormalized=nfc as the default.)


GCC always generates UTF-8 in its .s output for such identifiers, which 
gas then transfers straight through to its output (thus, UTF-8 ELF 
symbols).  The generic ELF ABI is silent on the encoding of such symbols 
(it just says "External C symbols have the same names in C and object 
files' symbol tables.").

-- 
Joseph S. Myers
joseph@codesourcery.com