Re: [PATCHv2] gdb/tui: use init_extended_color where possible

[Simon Marchi <simark@simark.ca>]

On 8/5/26 7:50 AM, Andrew Burgess wrote:
> In v2:
> 
>   - Split the scaling of GDB's RGB value out from the init_color and
>     init_extended_color calls.  This removes some code duplication and
>     allows the comment to sit closer to the code in question.
> 
>   - Rebase to current HEAD and retest.
> 
> ---
> 
> After commit:
> 
>   commit fbe7f20a0f098ca03913452b29f50f0dc8568f77
>   Date:   Sat May 9 23:27:43 2026 +0200
> 
>     gdb/tui: fix unexpected reuse of color pairs
> 
> which converted GDB to use init_extended_pair where possible, I
> realised we could also make use of init_extended_color.
> 
> The motivation for using init_extended_color is slightly less than
> init_extended_pair.  Assuming the terminal supports it the standard
> init_color API supports up to SHRT_MAX (32767) different colors,
> switching to init_extended_color removes the SHRT_MAX limit on color
> indices, allowing us to support the full range of COLORS.
> 
> But the cost of making this change is minimal, we already track the
> color indices as an `int` within the global COLOR_MAP, so it's mostly
> just a case of calling init_extended_color where needed.
> 
> We only use init_extended_color when both that function and
> init_extended_pair is available.  The fallback to init_extended_pair
> is init_pair, which expects the color indices to be shorts.  If we are
> using the init_pair fallback then using init_extended_color is
> pointless.
> 
> In reality init_extended_pair and init_extended_color were both added
> in ncurses 6.1, so should both be available together.
> 
> There is one additional change in here.  Assuming that a terminal does
> support more than SHRT_MAX colours, but for some reason GDB is
> compiled with a version of the curses library that doesn't support
> init_extended_color, then it is possible that in `get_color` the value
> of NEXT could end up above SHRT_MAX, in which case the `init_color`
> call will truncate the value of NEXT to a short and we will end up
> redefining an earlier color index.  To avoid this unlikely case I've
> added a compare against SHRT_MAX.
> 
> The init_extended_color path doesn't have this risk as COLORS is an
> `int` and NEXT is passed as an `int` on this path so there is no risk
> of truncation.

Seems fine to me, thanks.  Hopefully one day we can just assume that the
extended functions are present.

Approved-By: Simon Marchi <simon.marchi@efficios.com>

Simon