RE: [PATCH] Fix crash when target-wide-charset doesn't match wchar_t size

[Aditya Kamath <Aditya.Kamath1@ibm.com>]

[-- Attachment #1: Type: text/plain, Size: 10779 bytes --]


Hi Simon,

> Can you provide the full backtrace where the crash happens?

Yeah sure. Thanks once again for asking.

So I ran this

/tmp/wchar64 is 64 bit binary of wchar.c in the test suite.

# gdb/gdb /tmp/wchar64
GNU gdb (GDB) 19.0.50.20261015-git
Copyright (C) 2026 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "powerpc64-ibm-aix7.2.0.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.

+------------------------------------------------------------------------------+
| Find the GDB manual online at:                                               |
| http://www.gnu.org/software/gdb/documentation/.                              |
| For help, type "help".                                                       |
| Type "apropos word" to search for commands related to "word".                |
+------------------------------------------------------------------------------+

Reading symbols from /tmp/wchar64...
(gdb) b wchar.c:44
Breakpoint 1 at 0x1000091c: file binutils-gdb/gdb/testsuite/gdb.base/wchar.c, line 44.
(gdb) r
Breakpoint 1, main () at binutils-gdb/gdb/testsuite/gdb.base/wchar.c:44
44    do_nothing (&narrow); /* START */
(gdb) set target-wide-charset UTF-16
(gdb) p single


Fatal signal: Segmentation fault
----- Backtrace -----
0x100f19b6b gdb_internal_backtrace_1
        /current_gdb/binutils-gdb/gdb/bt-utils.c:122
0x100f19c5b _Z22gdb_internal_backtracev
        /current_gdb/binutils-gdb/gdb/bt-utils.c:173
0x1004fbe2b handle_fatal_signal
        /current_gdb/binutils-gdb/gdb/event-top.c:1008
0x1004fc0d7 handle_sigsegv
        /current_gdb/binutils-gdb/gdb/event-top.c:1085
0x4fdf ???
---------------------
A fatal error internal to GDB has been detected, further
debugging is not possible.  GDB will now terminate.

This is a bug, please report it.  For instructions, see:
<https://www.gnu.org/software/gdb/bugs/>.

Segmentation fault         (core dumped) gdb/gdb /tmp/wchar64

If I take the core dump and analyse we get,

# gdb/gdb -c core gdb/gdb
Core was generated by `gdb'.
Program terminated with signal SIGILL, Illegal instruction.
#0  0x0a00000015c580a8 in ?? ()
warning: File "/current_gdb/binutils-gdb/gdb/gdb-gdb.gdb" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
        add-auto-load-safe-path /current_gdb/binutils-gdb/gdb/gdb-gdb.gdb
line to your configuration file "/.config/gdb/gdbinit".
To completely disable this security protection add
        set auto-load safe-path /
line to your configuration file "/.config/gdb/gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
        info "(gdb)Auto-loading safe path"
(gdb) bt
#0  0x0a00000015c580a8 in ?? ()
#1  0x0000000100f197b0 in sig_write (msg=0x1101489e0 "0x100f19b6b ") at bt-utils.c:48
#2  0x0000000100f199f4 in libbacktrace_print (data=0x0, pc=4310801259,
    filename=0xa0000000038f620 <error: Cannot access memory at address 0xa0000000038f620>, lineno=122,
    function=0xa000000041e4631 <error: Cannot access memory at address 0xa000000041e4631>) at bt-utils.c:94
#3  0x0000000100f27b98 in dwarf_lookup_pc (state=0xa00000000000000, ddata=0xa00000017895288, pc=4310801259,
    callback=@0x1100af528: 0x100f19998 <libbacktrace_print(void*, uintptr_t, char const*, int, char const*)>,
    error_callback=@0x1100af510: 0x100f198bc <libbacktrace_error(void*, char const*, int)>, data=0x0, found=0x110148c14) at dwarf.c:3896
#4  0x0000000100f27c54 in dwarf_fileline (state=0xa00000000000000, pc=4310801259,
    callback=@0x1100af528: 0x100f19998 <libbacktrace_print(void*, uintptr_t, char const*, int, char const*)>,
    error_callback=@0x1100af510: 0x100f198bc <libbacktrace_error(void*, char const*, int)>, data=0x0) at dwarf.c:3918
#5  0x0000000100f1b570 in backtrace_pcinfo (state=0xa00000000000000, pc=4310801259,
    callback=@0x1100af528: 0x100f19998 <libbacktrace_print(void*, uintptr_t, char const*, int, char const*)>,
    error_callback=@0x1100af510: 0x100f198bc <libbacktrace_error(void*, char const*, int)>, data=0x0) at fileline.c:362
#6  0x0000000100f1ad60 in unwind (context=0x110149210, vdata=0x1101497e8) at backtrace.c:91
#7  0x09000000000cbb04 in _Unwind_Backtrace (trace=0x101497e8, trace_argument=0x0) at ../../../../gcc-13.3.0/libgcc/unwind.inc:330
#8  0x2244422810149780 in ?? ()
#9  0x0000000100f19b6c in gdb_internal_backtrace_1 () at bt-utils.c:122
#10 0x0000000100f19c5c in gdb_internal_backtrace () at bt-utils.c:173
#11 0x00000001004fbe2c in handle_fatal_signal (sig=11) at event-top.c:1008
#12 0x00000001004fc0d8 in handle_sigsegv (sig=11) at event-top.c:1085
#13 <signal handler called>
#14 0x0900000001a42454 in qsort () from /opt/freeware/lib/libiconv.a(libiconv.so.2)
#15 0x0900000001a60024 in qsort () from /opt/freeware/lib/libiconv.a(libiconv.so.2)
#16 0x0900000001a689d4 in libiconv_set_relocation_prefix () from /opt/freeware/lib/libiconv.a(libiconv.so.2)
#17 0x00000001001c0738 in gdb_iconv (utf_flag=<error reading variable: Cannot access memory at address 0x678>,
    inbuf=<error reading variable: Cannot access memory at address 0x680>,
    inbytesleft=<error reading variable: Cannot access memory at address 0x688>,
    outbuf=<error reading variable: Cannot access memory at address 0x690>,
    outbytesleft=<error reading variable: Cannot access memory at address 0x698>) at charset.c:204
#18 0x00000001001c1c14 in wchar_iterator::iterate (this=0xfffffffffffe080, out_result=0xfffffffffffe128, out_chars=0xfffffffffffe110,
    ptr=0xfffffffffffe118, len=0xfffffffffffe120) at charset.c:630
#19 0x0000000100a861b8 in wchar_printer::print (this=0xfffffffffffe1b0, c=48879, stream=0x11044bdd0) at char-print.c:194
#20 0x00000001000ba110 in generic_emit_char (c=48879, type=0x11049b648, stream=0x11044bdd0, encoding=0x11019fbb0 "UTF-16BE") at valprint.c:2100
#21 0x00000001007e5204 in language_defn::printchar (this=0x11011d360 <global_index_cache+448>, c=48879, type=0x11049b648, stream=0x11044bdd0)
    at c-lang.c:163
#22 0x00000001007e9530 in c_value_print_int (val=0x11045e370, stream=0x11044bdd0, options=...) at c-valprint.c:414
#23 0x00000001007e96f0 in c_value_print_inner (val=0x11045e370, stream=0x11044bdd0, recurse=0, options=...) at c-valprint.c:446
#24 0x000000010034fdc0 in language_defn::value_print_inner (this=0x11011d360 <global_index_cache+448>, val=0x11045e370, stream=0x11044bdd0,
    recurse=0, options=...) at language.c:659
#25 0x00000001000b6d28 in common_val_print (value=0x11045e370, stream=0x11044bdd0, recurse=0, options=...,
    language=0x11011d360 <global_index_cache+448>) at valprint.c:1114
#26 0x00000001007e9da4 in c_value_print (val=0x11045e370, stream=0x11044bdd0, options=...) at c-valprint.c:589
#27 0x000000010034fc6c in language_defn::value_print (this=0x11011d360 <global_index_cache+448>, val=0x11045e370, stream=0x11044bdd0, options=...)
    at language.c:641
#28 0x00000001000b74a0 in value_print (val=0x11045e370, stream=0x11044bdd0, options=...) at valprint.c:1225
#29 0x00000001007eb6d4 in print_formatted (val=0x11045e370, size=0, options=..., stream=0x11044bdd0) at printcmd.c:333
#30 0x00000001007eea0c in print_value (val=0x11045e370, opts=...) at printcmd.c:1302
--Type <RET> for more, q to quit, c to continue without paging--
#31 0x00000001007ef214 in print_command_1 (args=0xffffffffffff7b4 "single", voidprint=1) at printcmd.c:1422
#32 0x00000001007ef4f8 in print_command (exp=0xffffffffffff7b4 "single", from_tty=0) at printcmd.c:1448
#33 0x000000010039dcf4 in do_simple_func (args=0xffffffffffff7b4 "single", from_tty=0, c=0x110214f10) at cli/cli-decode.c:86
#34 0x00000001003a7930 in cmd_func (cmd=0x110214f10, args=0xffffffffffff7b4 "single", from_tty=0) at cli/cli-decode.c:2810
#35 0x00000001003888ec in execute_command (p=0xffffffffffff7b9 "e", from_tty=0) at top.c:632
#36 0x00000001008f7bc0 in catch_command_errors (command=@0x1100a3348: 0x100388224 <execute_command(char const*, int)>,
    arg=0xffffffffffff7b2 "p single", from_tty=0, do_bp_actions=true) at main.c:508
#37 0x00000001008f7f98 in execute_cmdargs (cmdarg_vec=0xffffffffffff200, file_type=CMDARG_FILE, cmd_type=CMDARG_COMMAND, ret=0xffffffffffff230)
    at main.c:607
#38 0x00000001008f9980 in captured_main_1 (context=0xffffffffffff550) at main.c:1340
#39 0x00000001008f9cd8 in captured_main (context=0xffffffffffff550) at main.c:1363
#40 0x00000001008f9e10 in gdb_main (args=0xffffffffffff550) at main.c:1392
#41 0x0000000100000aa8 in main (argc=11, argv=0xffffffffffff5f0) at gdb.c:38


You see something happened at 13 and 14 itself.

Then at frame 17 inside libiconv with garbage args
Then frame 18 this=0xfffffffffffe080  = CORRUPT (stack pointer)
Likewise 19
Then #20  generic_emit_char           — calls wchar_printer::print(c=0xbeef, …)
Then #21  language_defn::printchar
Then #22  c_value_print_int          — wchar_t, encoding resolved to "UTF-16BE”
.
.
.
Then #31  print_command_1 ("single")

In frame 18 itself the data is off.

The wchar_printer object sits at 0xfffffffffffe1b0 (frame #19) and the wchar_iterator iter embedded inside it is at 0xfffffffffffe080 — these are valid-looking stack addresses on 64-bit AIX (AIX stack grows down from 0xffffffffffffffff). But by the time iterate() is entered, m_bytes has wrapped to a huge number and m_input is pointing to unmapped memory, so iconv() inside libiconv crashes in its internal qsort.

See #20  generic_emit_char  encoding = "UTF-16BE"

set_be_le_names did find UTF-16BE in the charset enum (from the set target-wide-charset UTF-16 → set_be_le_names lookup). So, the encoding passed to wchar_iterator is "UTF-16BE". The iterator is constructed with:

source encoding: "UTF-16BE" (2 bytes per unit)
m_width = 4 — because on AIX wchar_t is 4 bytes wide
That means iconv_open("UTF-32BE", "UTF-16BE") is fine, but then iterate() passes m_bytes = 4 (one wchar_t) and strides by m_width = 4 , when iconv consumes only 2 bytes of a UTF-16BE code unit, the m_input / m_bytes accounting in charset.c:630–650 walks off the end of the 4-byte buffer, corrupting the stack and causing the wrap-around values visible in the backtrace.

This is how I arrived. Let me know if I analysed something wrong.

Regards,
Aditya.

[-- Attachment #2: Type: text/html, Size: 34148 bytes --]