Re: support C/C++ identifiers named with non-ASCII characters

Simon Marchi
2018-05-21T14:03:52Z

On 2018-05-21 04:52 AM, å¼µä¿ŠèŠ wrote:
> Hello, team.
>
> This patch fixes the bug at
> https://sourceware.org/bugzilla/show_bug.cgi?id=22973 .
>
> Here is how to test the patch:
>
> Step 1. If you are using Clang or any other C compilers that have
> implemented
> support for Unicode identifiers, then create a C file with the
> following
> content:
>
> int main(int åƒé‡, char* åƒ[])
> {
> struct é›†
> {
> int æ•¸[3];
> } é›† = {100, 200, 300};
> int åº = 2;
> return 0;
> }
>
> Or if you are using GCC, create a C file with the following content as a
> workaround(GCC still doesn't actually support Unicode identifiers in
> 2018, which
> is a pity):
>
> int main(int \u53C3\u91CF, char* \u53C3[])
> {
> struct \u96C6
> {
> int \u6578[3];
> } \u96C6 = {100, 200, 300};
> int \u5E8F = 2;
> return 0;
> }
>
> Step 2. Compile the C file.
>
> Step 3. Run GDB for the compiled executable, add a breakpoint in "return 0".
>
> Step 4. Run until the breakpoint.
>
> Step 5. Test the following commands to see if they work:
> p åƒé‡
> p åƒ
> p é›†
> p é›†.æ•¸
> p é›†.æ•¸[åº]
>
> Thanks for your review.
>

Hi Zhang,

Thanks for the patch, I tested it quickly, it seems to work as expected.

Could you please write a small test case in testsuite/gdb.base with the example
you gave, so we make sure this doesn't get broken later? If you can write it
in such a way that both clang and gcc understand it would be better, because
most people run the testuite using gcc to compile test programs.

I am not a specialist in lexing and parsing C, so can you explain quickly why
you think this is a good solution? Quickly, I understand that you change the
identifier recognition algorithm to a blacklist of characters rather than
a whitelist, so bytes that are not recognized (such as those that compose
the utf-8 encoded characters) are not rejected.

Given unlimited time, would the right solution be to use a lib to parse the
string as utf-8, and reject strings that are not valid utf-8?

Here are some not and formatting comments:

> +static bool is_identifier_separator (char);

You don't have to forward declare the function if it's not necessary.

> + /* '<' should not be a token separator, because it can be an open angle
> + bracket followed by a nested template identifier in C++. */

Please use two spaces after the final period (...C++. */).

> + if (is_identifier_separator(c))

Please use a space before the parentheses:

is_identifier_separator (c)

> + for (c = tokstart[namelen]; !is_identifier_separator(c);)

Here too.

Thanks!

Simon