Shadow stack backtrace command name

Shadow stack backtrace command name

[Schimpe, Christina via Gdb]

Hi all, 

I am writing to you to collect feedback for the name of a new command, we would
like to introduce.  The command shall be used to print the shadow stack backtrace.

A shadow stack is a second stack for a program introduced in the Intel (R)
Control-Flow Enforcement Technology (CET).  The shadow stack is used for
control transfer operations to store the return addresses.

This is an example command name and output for the shadow stack backtrace:
~~~~
(gdb) info shadow-stack bt
   Address            Symbol
#0 0x0000000000401131 call1
#1 0x0000000000401145 main
#2 0x00007ffff7c3fe70 __libc_start_call_main
#3 0x00007ffff7c3ff20 __libc_start_main_impl
(gdb) set print symbol-filename on
(gdb) info shadow-stack bt
   Address            Symbol
#0 0x0000000000401131 call1 at amd64-shstk.c:51
#1 0x0000000000401145 main at amd64-shstk.c:56
#2 0x00007ffff7c3fe70 __libc_start_call_main
#3 0x00007ffff7c3ff20 __libc_start_main_impl
(gdb) help info shadow-stack bt
info shadow-stack backtrace, info shadow-stack bt
Print the entire backtrace of shadow stack,
or the innermost [COUNT | -COUNT] addresses for the current process.
To print the source filename and line number in the backtrace,
the "symbol-filename" option of the print command should be toggled on.
(See "show print symbol-filename")
~~~

It is configurable using "print symbol-filename" and COUNT.
The command can be called by the following names:
- "info shadow-stack bt", "info shadow-stack backtrace"

From my perspective, the command name has the following pros and cons:
(+) Easy to understand by just looking at the command name.
(-) Rather long syntax

We also considered other command names such as 

- "info cet bt", "info cet backtrace"
(+) Short syntax possible
(-) Not so easy to understand by just looking at the command name. I miss the
name "shadow stack".

- "info shstk bt", "info shstk backtrace"
(+) Short syntax possible
(-) "shstk" ist not an official abbreviation (in contrast to "cet"). "shstk" is
mostly used by the linux kernel and might not be known by the user.

- "info shstk", "info shadow-stack"
(+) short syntax possible
(-) Without "backtrace" in the name, it might not be so easy to understand.

Having in mind that that the shadow stack is not only a x86-specific feature
but can be seen as a generic concept we also considered that it could be
part of the existing backtrace command, e.g.:
- "bt -shadow"
(+) Short syntax
(+/-) Most of the settings of the bt command don't apply to the shadow
stack (frame arguments and info). This might cause confusion.

For this option, it might make sense to introduce a new setting for the bt
command which is for shadow stack only, e.g. "-symbol-filename [on|off]".

What are your thoughts on this topic? Any feedback and new ideas are welcome.

Best Regards,
Christina
Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: Shadow stack backtrace command name

[Guinevere Larsen via Gdb]

On 20/12/2023 10:42, Schimpe, Christina via Gdb wrote:
> Hi all,
>
> I am writing to you to collect feedback for the name of a new command, we would
> like to introduce.  The command shall be used to print the shadow stack backtrace.
>
> A shadow stack is a second stack for a program introduced in the Intel (R)
> Control-Flow Enforcement Technology (CET).  The shadow stack is used for
> control transfer operations to store the return addresses.
>
> This is an example command name and output for the shadow stack backtrace:
> ~~~~
> (gdb) info shadow-stack bt
>     Address            Symbol
> #0 0x0000000000401131 call1
> #1 0x0000000000401145 main
> #2 0x00007ffff7c3fe70 __libc_start_call_main
> #3 0x00007ffff7c3ff20 __libc_start_main_impl
> (gdb) set print symbol-filename on
> (gdb) info shadow-stack bt
>     Address            Symbol
> #0 0x0000000000401131 call1 at amd64-shstk.c:51
> #1 0x0000000000401145 main at amd64-shstk.c:56
> #2 0x00007ffff7c3fe70 __libc_start_call_main
> #3 0x00007ffff7c3ff20 __libc_start_main_impl
> (gdb) help info shadow-stack bt
> info shadow-stack backtrace, info shadow-stack bt
> Print the entire backtrace of shadow stack,
> or the innermost [COUNT | -COUNT] addresses for the current process.
> To print the source filename and line number in the backtrace,
> the "symbol-filename" option of the print command should be toggled on.
> (See "show print symbol-filename")
> ~~~

Hi!

My first thought about this output is that I don't think most users are 
interested in what goes on below the main function. I think it would be 
nice to suppress that output (I personally would like that to be the 
default, but as long as its an option I'm happy).

As for names, I don't see a need for starting with "info". In my 
opinion, just calling it "shadow-stack backtrace" is fine. Users could 
then set their own abbreviations, like shstk or cet if they like.

As for if the name should have 'backtrace', I think it would be good 
that if actual command has it, but then adding an abbreviation to just 
"shadow-stack" if backtrace is the only command associated with it, but 
I can see how that could be confusing for end users if we add a new 
command after one or more releases, so I'm not married to this idea.

Some more thoughts below

>
> It is configurable using "print symbol-filename" and COUNT.
> The command can be called by the following names:
> - "info shadow-stack bt", "info shadow-stack backtrace"
>
>  From my perspective, the command name has the following pros and cons:
> (+) Easy to understand by just looking at the command name.
> (-) Rather long syntax
>
> We also considered other command names such as
>
> - "info cet bt", "info cet backtrace"
> (+) Short syntax possible
> (-) Not so easy to understand by just looking at the command name. I miss the
> name "shadow stack".
I don't like this one very much, because searching CET online won't give 
you much info, so a user would need to look at the help text to then 
figure out keywords to search to learn more.
>
> - "info shstk bt", "info shstk backtrace"
> (+) Short syntax possible
> (-) "shstk" ist not an official abbreviation (in contrast to "cet"). "shstk" is
> mostly used by the linux kernel and might not be known by the user.
I like this better because it is much easier to find what is going on, 
but as you said, users may not be familiar, so I still lean towards 
spelling out shadow-stack and letting users make their own abbreviations.
>
> - "info shstk", "info shadow-stack"
> (+) short syntax possible
> (-) Without "backtrace" in the name, it might not be so easy to understand.
>
> Having in mind that that the shadow stack is not only a x86-specific feature
> but can be seen as a generic concept we also considered that it could be
> part of the existing backtrace command, e.g.:
> - "bt -shadow"
> (+) Short syntax
> (+/-) Most of the settings of the bt command don't apply to the shadow
> stack (frame arguments and info). This might cause confusion.
I started really disliking this option, but the more I think about it, 
the more it sounds like a neutral one to me.
sure, the output is much more basic, and many options/commands wont 
apply, but if the regular backtrace is failing due to some stack 
corruption or something, this feels like a more natural approach to 
solving that issue, which I imagine would be the most common use case 
for most users. That said, this is all speculations pulled from thin 
air, so feel free to ignore this last part.

-- 
Cheers,
Guinevere Larsen
She/Her/Hers

>
> For this option, it might make sense to introduce a new setting for the bt
> command which is for shadow stack only, e.g. "-symbol-filename [on|off]".
>
> What are your thoughts on this topic? Any feedback and new ideas are welcome.
>
> Best Regards,
> Christina
> Intel Deutschland GmbH
> Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
> Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
> Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva
> Chairperson of the Supervisory Board: Nicole Lau
> Registered Office: Munich
> Commercial Register: Amtsgericht Muenchen HRB 186928
>

Re: Shadow stack backtrace command name

[Luis Machado via Gdb]

Hi,

On 12/20/23 09:42, Schimpe, Christina via Gdb wrote:
> Hi all, 
> 
> I am writing to you to collect feedback for the name of a new command, we would
> like to introduce.  The command shall be used to print the shadow stack backtrace.
> 
> A shadow stack is a second stack for a program introduced in the Intel (R)
> Control-Flow Enforcement Technology (CET).  The shadow stack is used for
> control transfer operations to store the return addresses.
> 
> This is an example command name and output for the shadow stack backtrace:
> ~~~~
> (gdb) info shadow-stack bt
>    Address            Symbol
> #0 0x0000000000401131 call1
> #1 0x0000000000401145 main
> #2 0x00007ffff7c3fe70 __libc_start_call_main
> #3 0x00007ffff7c3ff20 __libc_start_main_impl
> (gdb) set print symbol-filename on
> (gdb) info shadow-stack bt
>    Address            Symbol
> #0 0x0000000000401131 call1 at amd64-shstk.c:51
> #1 0x0000000000401145 main at amd64-shstk.c:56
> #2 0x00007ffff7c3fe70 __libc_start_call_main
> #3 0x00007ffff7c3ff20 __libc_start_main_impl
> (gdb) help info shadow-stack bt
> info shadow-stack backtrace, info shadow-stack bt
> Print the entire backtrace of shadow stack,
> or the innermost [COUNT | -COUNT] addresses for the current process.
> To print the source filename and line number in the backtrace,
> the "symbol-filename" option of the print command should be toggled on.
> (See "show print symbol-filename")
> ~~~
> 
> It is configurable using "print symbol-filename" and COUNT.
> The command can be called by the following names:
> - "info shadow-stack bt", "info shadow-stack backtrace"
> 
>>From my perspective, the command name has the following pros and cons:
> (+) Easy to understand by just looking at the command name.
> (-) Rather long syntax
> 
> We also considered other command names such as 
> 
> - "info cet bt", "info cet backtrace"
> (+) Short syntax possible
> (-) Not so easy to understand by just looking at the command name. I miss the
> name "shadow stack".
> 
> - "info shstk bt", "info shstk backtrace"
> (+) Short syntax possible
> (-) "shstk" ist not an official abbreviation (in contrast to "cet"). "shstk" is
> mostly used by the linux kernel and might not be known by the user.
> 
> - "info shstk", "info shadow-stack"
> (+) short syntax possible
> (-) Without "backtrace" in the name, it might not be so easy to understand.
> 
> Having in mind that that the shadow stack is not only a x86-specific feature
> but can be seen as a generic concept we also considered that it could be
> part of the existing backtrace command, e.g.:
> - "bt -shadow"
> (+) Short syntax
> (+/-) Most of the settings of the bt command don't apply to the shadow
> stack (frame arguments and info). This might cause confusion.
> 
> For this option, it might make sense to introduce a new setting for the bt
> command which is for shadow stack only, e.g. "-symbol-filename [on|off]".
> 
> What are your thoughts on this topic? Any feedback and new ideas are welcome.

I like the option of reusing whatever is possible to reuse from the current backtrace command, so "bt -shadow" seems like a sensible option.

It doesn't seem to me like this command will be used a lot. I expect it will be useful only when we catch a fault due to a corrupt stack trace, so
putting it within the more general "backtrace" option would accomplish that.

With that said, depending on how shadow stack support is implemented in gdb, I expect gdb will automatically validate the stack trace against the
shadow stack (maybe on a fault), and complain if they go out of sync. Does that sound reasonable? Maybe even display where the flow veered off course.

AArch64 will have a counterpart of this, with the Guarded Control Stack (GCS) feature, so the more generic we make this, the better.

RE: Shadow stack backtrace command name

[Schimpe, Christina via Gdb]

Thanks a lot for you review. Please see my feedback to your comments below.

> My first thought about this output is that I don't think most users are interested in
> what goes on below the main function. I think it would be nice to suppress that
> output (I personally would like that to be the default, but as long as its an option
> I'm happy).

For the normal bt command there are the options -past-main and -past-entry. We
could consider adding them to the new command for shadow stack bt as well.
However, for the shadow-stack it's a bit different as the stack size depends on the
enablement of shstk.

> As for names, I don't see a need for starting with "info". In my opinion, just calling
> it "shadow-stack backtrace" is fine. Users could then set their own abbreviations,
> like shstk or cet if they like.

One reason for me to have the "info" is to make the command easier to find. If the 
user would type "help info" the command would be listed  (among others).

Christina
Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

RE: Shadow stack backtrace command name

[Schimpe, Christina via Gdb]

Hi, 

Thanks a lot for your feedback. Please find my answers to your comments below.

> > Having in mind that that the shadow stack is not only a x86-specific
> > feature but can be seen as a generic concept we also considered that
> > it could be part of the existing backtrace command, e.g.:
> > - "bt -shadow"
> > (+) Short syntax
> > (+/-) Most of the settings of the bt command don't apply to the shadow
> > stack (frame arguments and info). This might cause confusion.
> >
> > For this option, it might make sense to introduce a new setting for
> > the bt command which is for shadow stack only, e.g. "-symbol-filename
> [on|off]".
> >
> > What are your thoughts on this topic? Any feedback and new ideas are
> welcome.
> 
> I like the option of reusing whatever is possible to reuse from the current
> backtrace command, so "bt -shadow" seems like a sensible option.
> 
> It doesn't seem to me like this command will be used a lot. I expect it will be
> useful only when we catch a fault due to a corrupt stack trace, so putting it within
> the more general "backtrace" option would accomplish that.
> 
> With that said, depending on how shadow stack support is implemented in gdb, I
> expect gdb will automatically validate the stack trace against the shadow stack
> (maybe on a fault), and complain if they go out of sync. Does that sound
> reasonable? Maybe even display where the flow veered off course.

No, we don't plan to validate the stack trace in GDB, as we don't see much
additional value for the user.
In case of a CET violation the user will see a SEGV with CP specific 
si_code = 10 (SEGV_CPERR). Printing siginfo will help to find out the reason for SEGV.
Inspecting the shadow stack and normal bt will show where the traces got out of sync.

> AArch64 will have a counterpart of this, with the Guarded Control Stack (GCS)
> feature, so the more generic we make this, the better.

Would the option's name "-shadow" be suitable for the GCS? I find it difficult to come
up with a more generic name that would cover both.

BR,
Christina
Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: Shadow stack backtrace command name

[Luis Machado via Gdb]

Hi,

On 12/20/23 15:35, Schimpe, Christina wrote:
> Hi, 
> 
> Thanks a lot for your feedback. Please find my answers to your comments below.
> 
>>> Having in mind that that the shadow stack is not only a x86-specific
>>> feature but can be seen as a generic concept we also considered that
>>> it could be part of the existing backtrace command, e.g.:
>>> - "bt -shadow"
>>> (+) Short syntax
>>> (+/-) Most of the settings of the bt command don't apply to the shadow
>>> stack (frame arguments and info). This might cause confusion.
>>>
>>> For this option, it might make sense to introduce a new setting for
>>> the bt command which is for shadow stack only, e.g. "-symbol-filename
>> [on|off]".
>>>
>>> What are your thoughts on this topic? Any feedback and new ideas are
>> welcome.
>>
>> I like the option of reusing whatever is possible to reuse from the current
>> backtrace command, so "bt -shadow" seems like a sensible option.
>>
>> It doesn't seem to me like this command will be used a lot. I expect it will be
>> useful only when we catch a fault due to a corrupt stack trace, so putting it within
>> the more general "backtrace" option would accomplish that.
>>
>> With that said, depending on how shadow stack support is implemented in gdb, I
>> expect gdb will automatically validate the stack trace against the shadow stack
>> (maybe on a fault), and complain if they go out of sync. Does that sound
>> reasonable? Maybe even display where the flow veered off course.
> 
> No, we don't plan to validate the stack trace in GDB, as we don't see much
> additional value for the user.
> In case of a CET violation the user will see a SEGV with CP specific 
> si_code = 10 (SEGV_CPERR). Printing siginfo will help to find out the reason for SEGV.
> Inspecting the shadow stack and normal bt will show where the traces got out of sync.

Thanks for clarifying it. My understanding is that aarch64 will also use SEGV_CPERR for
the GCS faults, so it will be in sync with CET regarding reporting.

> 
>> AArch64 will have a counterpart of this, with the Guarded Control Stack (GCS)
>> feature, so the more generic we make this, the better.
> 
> Would the option's name "-shadow" be suitable for the GCS? I find it difficult to come
> up with a more generic name that would cover both.

From this entry [1], looks like the term shadow stack is a reasonably generic way to refer
to this feature. So I think it is a suitable name, and would work for GCS as well.

[1] https://en.wikipedia.org/wiki/Shadow_stack

Re: Shadow stack backtrace command name

[Thiago Jung Bauermann via Gdb]

Hello,

Luis Machado <luis.machado@arm.com> writes:

> Hi,
>
> On 12/20/23 15:35, Schimpe, Christina wrote:
>> Hi, 
>> 
>> Thanks a lot for your feedback. Please find my answers to your comments below.
>> 
>>>> Having in mind that that the shadow stack is not only a x86-specific
>>>> feature but can be seen as a generic concept we also considered that
>>>> it could be part of the existing backtrace command, e.g.:
>>>> - "bt -shadow"
>>>> (+) Short syntax
>>>> (+/-) Most of the settings of the bt command don't apply to the shadow
>>>> stack (frame arguments and info). This might cause confusion.
>>>>
>>>> For this option, it might make sense to introduce a new setting for
>>>> the bt command which is for shadow stack only, e.g. "-symbol-filename
>>> [on|off]".
>>>>
>>>> What are your thoughts on this topic? Any feedback and new ideas are
>>> welcome.
>>>
>>> I like the option of reusing whatever is possible to reuse from the current
>>> backtrace command, so "bt -shadow" seems like a sensible option.

I like it too.

>>> It doesn't seem to me like this command will be used a lot. I expect it will be
>>> useful only when we catch a fault due to a corrupt stack trace, so putting it within
>>> the more general "backtrace" option would accomplish that.
>>>
>>> With that said, depending on how shadow stack support is implemented in gdb, I
>>> expect gdb will automatically validate the stack trace against the shadow stack
>>> (maybe on a fault), and complain if they go out of sync. Does that sound
>>> reasonable? Maybe even display where the flow veered off course.
>> 
>> No, we don't plan to validate the stack trace in GDB, as we don't see much
>> additional value for the user.
>> In case of a CET violation the user will see a SEGV with CP specific 
>> si_code = 10 (SEGV_CPERR). Printing siginfo will help to find out the reason for SEGV.
>> Inspecting the shadow stack and normal bt will show where the traces got out of sync.
>
> Thanks for clarifying it. My understanding is that aarch64 will also use SEGV_CPERR for
> the GCS faults, so it will be in sync with CET regarding reporting.

Yes, that is true.

>>> AArch64 will have a counterpart of this, with the Guarded Control Stack (GCS)
>>> feature, so the more generic we make this, the better.
>> 
>> Would the option's name "-shadow" be suitable for the GCS? I find it difficult to come
>> up with a more generic name that would cover both.
>
> From this entry [1], looks like the term shadow stack is a reasonably generic way to refer
> to this feature. So I think it is a suitable name, and would work for GCS as well.
>
> [1] https://en.wikipedia.org/wiki/Shadow_stack

Yes, the Linux kernel patch series adding support for GCS¹ plugs into
generic code that uses the "shadow stack" terminology. E.g., AArch64
programs will call "prctl(PR_SET_SHADOW_STACK_STATUS, PR_SHADOW_STACK_ENABLE)"
to enable GCS, "map_shadow_stack()" to map it, etc.

So that name will already be familiar to programmers debugging an
app using GCS.

PS: FWIW, I'm in the middle of adding GCS support in GDB.

-- 
Thiago

¹ https://lore.kernel.org/linux-arm-kernel/20231122-arm64-gcs-v7-0-201c483bd775@kernel.org/

Shadow stack command to host related subcommands (was Re: Shadow stack backtrace command name)

[Thiago Jung Bauermann via Gdb]

Hello again,

This isn't about backtrace, but still related to your proposal I think.

"Schimpe, Christina via Gdb" <gdb@sourceware.org> writes:

> It is configurable using "print symbol-filename" and COUNT.
> The command can be called by the following names:
> - "info shadow-stack bt", "info shadow-stack backtrace"
>
> From my perspective, the command name has the following pros and cons:
> (+) Easy to understand by just looking at the command name.
> (-) Rather long syntax

As I mentioned yesterday, I like the "bt -shadow" proposal but I'd also
like to have an "info shadow-stack" command (or just "shadow-stack" as
suggested by Guinevere) to hang a couple of GCS subcommands onto, e.g.:

info shadow-stack enabled
info shadow-stack locked

which would get/set the GCS features that are enabled or locked. These
correspond to a couple of entries in the structure returned by ptrace's
NT_ARM_GCS regset in Linux:

struct user_gcs
{
  uint64_t features_enabled;
  uint64_t features_locked;
  uint64_t gcspr_el0;
};

Initially I thought of exposing them as pseudo-registers, but this way
seems more straightforward.

What do you think? Would this be useful for Intel CET as well?

-- 
Thiago

Re: Shadow stack backtrace command name

[Tom Tromey]

>>>>> Schimpe, Christina via Gdb <gdb@sourceware.org> writes:

> A shadow stack is a second stack for a program introduced in the Intel (R)
> Control-Flow Enforcement Technology (CET).  The shadow stack is used for
> control transfer operations to store the return addresses.

One question I had is when, as a gdb user, would I want to see this
information?

> It is configurable using "print symbol-filename" and COUNT.
> The command can be called by the following names:
> - "info shadow-stack bt", "info shadow-stack backtrace"

Like others in the thread, I'm -1 on "info" as a prefix.
I liked "bt -shadow", but I was also wondering if the information should
just be integrated into the ordinary backtrace when available... that's
why I'm wondering when I'd want to see this.

Tom

Re: Shadow stack backtrace command name

[Thiago Jung Bauermann via Gdb]

Tom Tromey <tom@tromey.com> writes:

>>>>>> Schimpe, Christina via Gdb <gdb@sourceware.org> writes:
>
>> A shadow stack is a second stack for a program introduced in the Intel (R)
>> Control-Flow Enforcement Technology (CET).  The shadow stack is used for
>> control transfer operations to store the return addresses.
>
> One question I had is when, as a gdb user, would I want to see this
> information?

I think the most common shadow stack error a GDB user would encounter
would be when the inferior is returning from a function and gets a
SIGSEGV because the return address is wrong (e.g., because a buffer
overflow wrote over it).

There are other possibilities, for example a program can create
different shadow stacks and switch between them (e.g., when it
implements userspace-level threading) so some error could happen during
that process. E.g., in AArch64's Guarded Control Stacks, there needs to
be a special "cap" value at the end of the incoming stack and a SIGSEGV
is generated if that's not the case. In this case I think the user would
want to be able to direct the shadow stack backtrace command to print
the backtrace of that other stack, instead of the currently active one.

Another example would be trying to write to a mapped shadow stack that
is read-only. That also causes a SIGSEGV. Though not sure if the shadow
stack backtrace is relevant in this scenario.

>> It is configurable using "print symbol-filename" and COUNT.
>> The command can be called by the following names:
>> - "info shadow-stack bt", "info shadow-stack backtrace"
>
> Like others in the thread, I'm -1 on "info" as a prefix.
> I liked "bt -shadow", but I was also wondering if the information should
> just be integrated into the ordinary backtrace when available... that's
> why I'm wondering when I'd want to see this.

In my first example, it would be useful if the regular backtrace output
noted where it differs from the shadow stack. Though I think a way to
see the shadow stack backtrace by itself would still be useful.

-- 
Thiago

RE: Shadow stack command to host related subcommands (was Re: Shadow stack backtrace command name)

[Schimpe, Christina via Gdb]

Hi Thiago,

For "info shadow-stack enabled":
We will expose the shadow stack pointer as register, so the user can
check if it is available. 
It is only available in case shadow stack is enabled for a program. 
Based on that, I don't see the need to have an extra command
for shadow stack enablement state, I think.

In case the user runs "bt -shadow" and shadow-stack is not enabled,
we could also provide a useful command output.

For "info shadow-stack locked":
We don't have a separate note for such a feature a but similar information
would be available using "/proc/$PID/status", which the user could check
himself:
https://github.com/torvalds/linux/commit/0ee44885fe9cf19eb3870947c8f3c275017e48a7

So I am not sure if the two commands would make much sense for our case.

Does that sound reasonable to you?

Christina

> -----Original Message-----
> From: Thiago Jung Bauermann <thiago.bauermann@linaro.org>
> Sent: Thursday, December 21, 2023 11:27 PM
> To: Schimpe, Christina <christina.schimpe@intel.com>
> Cc: gdb@sourceware.org
> Subject: Shadow stack command to host related subcommands (was Re:
> Shadow stack backtrace command name)
> 
> 
> Hello again,
> 
> This isn't about backtrace, but still related to your proposal I think.
> 
> "Schimpe, Christina via Gdb" <gdb@sourceware.org> writes:
> 
> > It is configurable using "print symbol-filename" and COUNT.
> > The command can be called by the following names:
> > - "info shadow-stack bt", "info shadow-stack backtrace"
> >
> > From my perspective, the command name has the following pros and cons:
> > (+) Easy to understand by just looking at the command name.
> > (-) Rather long syntax
> 
> As I mentioned yesterday, I like the "bt -shadow" proposal but I'd also like to
> have an "info shadow-stack" command (or just "shadow-stack" as suggested by
> Guinevere) to hang a couple of GCS subcommands onto, e.g.:
> 
> info shadow-stack enabled
> info shadow-stack locked
> 
> which would get/set the GCS features that are enabled or locked. These
> correspond to a couple of entries in the structure returned by ptrace's
> NT_ARM_GCS regset in Linux:
> 
> struct user_gcs
> {
>   uint64_t features_enabled;
>   uint64_t features_locked;
>   uint64_t gcspr_el0;
> };
> 
> Initially I thought of exposing them as pseudo-registers, but this way seems more
> straightforward.
> 
> What do you think? Would this be useful for Intel CET as well?
> 
> --
> Thiago
Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

RE: Shadow stack backtrace command name

[Schimpe, Christina via Gdb]

Hi all, 

Thanks a lot for your feedback and ideas. Please see my comments below.

> >> A shadow stack is a second stack for a program introduced in the
> >> Intel (R) Control-Flow Enforcement Technology (CET).  The shadow
> >> stack is used for control transfer operations to store the return addresses.
> >
> > One question I had is when, as a gdb user, would I want to see this
> > information?

> I think the most common shadow stack error a GDB user would encounter
> would be when the inferior is returning from a function and gets a SIGSEGV
> because the return address is wrong (e.g., because a buffer overflow wrote over
> it).
> 
> There are other possibilities, for example a program can create different shadow
> stacks and switch between them (e.g., when it implements userspace-level
> threading) so some error could happen during that process. E.g., in AArch64's
> Guarded Control Stacks, there needs to be a special "cap" value at the end of the
> incoming stack and a SIGSEGV is generated if that's not the case. In this case I
> think the user would want to be able to direct the shadow stack backtrace
> command to print the backtrace of that other stack, instead of the currently
> active one.

Yes, I agree here. Common use cases would be debugging of stack corruption
or more complex scenarios that cause a control protection exception, as Thiago
points out.

> Another example would be trying to write to a mapped shadow stack that is
> read-only. That also causes a SIGSEGV. Though not sure if the shadow stack
> backtrace is relevant in this scenario.
> 
> >> It is configurable using "print symbol-filename" and COUNT.
> >> The command can be called by the following names:
> >> - "info shadow-stack bt", "info shadow-stack backtrace"
> >
> > Like others in the thread, I'm -1 on "info" as a prefix.
> > I liked "bt -shadow", but I was also wondering if the information
> > should just be integrated into the ordinary backtrace when
> > available... that's why I'm wondering when I'd want to see this.
> 
> In my first example, it would be useful if the regular backtrace output noted
> where it differs from the shadow stack. Though I think a way to see the shadow
> stack backtrace by itself would still be useful.

I like the idea to combine the outputs as the traces can be easily compared by
the user in this way. 

However, based on the use cases that I am aware of, I am not sure if the user
wants to always see the shadow stack bt in the ordinary bt output (if shadow
stack is enabled).

In addition, I wonder how the output would look like. Dependent on the source
code of the debugged program, the frames of the ordinary bt command can be
very long already. 

Christina

Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: Shadow stack backtrace command name

[Florian Weimer via Gdb]

* Christina via Gdb Schimpe:

> However, based on the use cases that I am aware of, I am not sure if
> the user wants to always see the shadow stack bt in the ordinary bt
> output (if shadow stack is enabled).

Based on my experiments, Linux currently does not push the instruction
pointer onto the shadow stack if code is interrupted by a signal.  It
still works because the return mechanism is different.  This would be a
very visible difference between ordinary backtraces and shadow stack
based backtraces.

As far as I understand it, the kernel could change, and it may still be
early enough to make this change.

By the way, is there a way to tell if a process is in shadow stack mode
using upstream GDB today?  Preferably something that does not rely on
glibc internals?

Thanks,
Florian

RE: Shadow stack backtrace command name

[Schimpe, Christina via Gdb]

> * Christina via Gdb Schimpe:
> 
> > However, based on the use cases that I am aware of, I am not sure if
> > the user wants to always see the shadow stack bt in the ordinary bt
> > output (if shadow stack is enabled).
> 
> Based on my experiments, Linux currently does not push the instruction
> pointer onto the shadow stack if code is interrupted by a signal.  It still works
> because the return mechanism is different.  This would be a very visible
> difference between ordinary backtraces and shadow stack based backtraces.
> As far as I understand it, the kernel could change, and it may still be early
> enough to make this change.

Could you explain a bit why and what you think the kernel will change ?

Just to be sure that I understand correctly: 
Do you think that this different display for the ordinary and shadow stack bt in
case of signals is one argument more for displaying the stacks together?
How would this look like?

> By the way, is there a way to tell if a process is in shadow stack mode using
> upstream GDB today?  Preferably something that does not rely on glibc
> internals?

You can run "info proc status" and check for "shstk" in x86_Thread_features, see
https://docs.kernel.org/arch/x86/shstk.html.

Christina
Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Sean Fennelly, Jeffrey Schneiderman, Tiffany Doon Silva
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: Shadow stack backtrace command name

[Florian Weimer via Gdb]

* Christina Schimpe:

>> * Christina via Gdb Schimpe:
>> 
>> > However, based on the use cases that I am aware of, I am not sure if
>> > the user wants to always see the shadow stack bt in the ordinary bt
>> > output (if shadow stack is enabled).
>> 
>> Based on my experiments, Linux currently does not push the instruction
>> pointer onto the shadow stack if code is interrupted by a signal.  It still works
>> because the return mechanism is different.  This would be a very visible
>> difference between ordinary backtraces and shadow stack based backtraces.
>> As far as I understand it, the kernel could change, and it may still be early
>> enough to make this change.
>
> Could you explain a bit why and what you think the kernel will change ?

I could imagine that an additional address is pushed onto the shadow
stack when a signal is delivered, so that we can get a full backtrace
across signal returns.

> Just to be sure that I understand correctly: 

> Do you think that this different display for the ordinary and shadow
> stack bt in case of signals is one argument more for displaying the
> stacks together?  How would this look like?

Currently, it's not possible to use shadow stacks for backtraces because
you won't be able to print the location after <signal handler called>
line.  At least that's what I encountered when I tried to use shadow
stack for implementing the glibc backtrace function.

> You can run "info proc status" and check for "shstk" in
> x86_Thread_features, see https://docs.kernel.org/arch/x86/shstk.html.

Thanks, that's a bit better than this (for SHSTK enabled):

(gdb) print (*(tcbhead_t *)$fs_base)->feature_1
$1 = 2

Florian

RE: Shadow stack backtrace command name

[Schimpe, Christina via Gdb]

> >> * Christina via Gdb Schimpe:
> >>
> >> > However, based on the use cases that I am aware of, I am not sure
> >> > if the user wants to always see the shadow stack bt in the ordinary
> >> > bt output (if shadow stack is enabled).
> >>
> >> Based on my experiments, Linux currently does not push the
> >> instruction pointer onto the shadow stack if code is interrupted by a
> >> signal.  It still works because the return mechanism is different.
> >> This would be a very visible difference between ordinary backtraces and
> shadow stack based backtraces.
> >> As far as I understand it, the kernel could change, and it may still
> >> be early enough to make this change.
> >
> > Could you explain a bit why and what you think the kernel will change ?
> 
> I could imagine that an additional address is pushed onto the shadow stack
> when a signal is delivered, so that we can get a full backtrace across signal
> returns.
> 
> > Just to be sure that I understand correctly:
> 
> > Do you think that this different display for the ordinary and shadow
> > stack bt in case of signals is one argument more for displaying the
> > stacks together?  How would this look like?
> 
> Currently, it's not possible to use shadow stacks for backtraces because you
> won't be able to print the location after <signal handler called> line.  At least
> that's what I encountered when I tried to use shadow stack for implementing
> the glibc backtrace function.

We are currently working on the implementation of the shadow stack backtrace.
This is how we would print a shadow stack backtrace for signals:

~~~
(gdb) bt shadow
#0  0x00007ffff7c54d90 in __restore_rt from /lib64/libc.so.6
#1  0x80007ffff79fffd8
#2  0x00007ffff7c54ce6 in __GI_raise at ../sysdeps/posix/raise.c:27
#3  0x000000000040115d in main at /tmp/amd64-shadow-stack-signal.c:32
[...]
~~~

The elements on the shadow stack are following the description of the linux 
kernel for signals:
"When a signal happens, the old pre-signal state is pushed on the stack.
 When shadow stack is enabled, the shadow stack specific state is pushed 
onto the shadow stack. Today this is only the old SSP (shadow stack pointer),
 pushed in a special format with bit 63 set."
(https://docs.kernel.org/arch/x86/shstk.html)

Frame 1 contains the old SSP with bit 63 set.

This would be the corresponding ordinary stack:
~~~
(gdb) bt
#0  handler (signo=10) at /tmp/amd64-shadow-stack-signal.c:25
#1  <signal handler called>
#2  __pthread_kill_implementation ([...]) at pthread_kill.c:44
#3  0x00007ffff7ca15f3 in __pthread_kill_internal (signo=10, threadid=<optimized out>) at pthread_kill.c:78
#4  0x00007ffff7c54ce6 in __GI_raise (sig=10) at ../sysdeps/posix/raise.c:26
#5  0x000000000040115d in main () at /tmp/amd64-shadow-stack-signal.c:31
~~~
Do you see much value in combining the outputs? 

Christina
Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Sean Fennelly, Jeffrey Schneiderman, Tiffany Doon Silva
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: Shadow stack backtrace command name

[Florian Weimer via Gdb]

* Christina Schimpe:

> We are currently working on the implementation of the shadow stack backtrace.
> This is how we would print a shadow stack backtrace for signals:
>
> ~~~
> (gdb) bt shadow
> #0  0x00007ffff7c54d90 in __restore_rt from /lib64/libc.so.6
> #1  0x80007ffff79fffd8
> #2  0x00007ffff7c54ce6 in __GI_raise at ../sysdeps/posix/raise.c:27
> #3  0x000000000040115d in main at /tmp/amd64-shadow-stack-signal.c:32
> [...]
> ~~~

> This would be the corresponding ordinary stack:
> ~~~
> (gdb) bt
> #0  handler (signo=10) at /tmp/amd64-shadow-stack-signal.c:25
> #1  <signal handler called>
> #2  __pthread_kill_implementation ([...]) at pthread_kill.c:44
> #3  0x00007ffff7ca15f3 in __pthread_kill_internal (signo=10, threadid=<optimized out>) at pthread_kill.c:78
> #4  0x00007ffff7c54ce6 in __GI_raise (sig=10) at ../sysdeps/posix/raise.c:26
> #5  0x000000000040115d in main () at /tmp/amd64-shadow-stack-signal.c:31
> ~~~
> Do you see much value in combining the outputs?

The difference is that the shadow stack backtrace does not contain the
interrupted instruction, so frame #2 in the traditional backtrace.  This
is more important for CPU-generated signals such as division by zero or
invalid memory access, where you really want to see the fault address in
the backtrace.

> The elements on the shadow stack are following the description of the linux 
> kernel for signals:
> "When a signal happens, the old pre-signal state is pushed on the stack.
>  When shadow stack is enabled, the shadow stack specific state is pushed 
> onto the shadow stack. Today this is only the old SSP (shadow stack pointer),
>  pushed in a special format with bit 63 set."
> (https://docs.kernel.org/arch/x86/shstk.html)
>
> Frame 1 contains the old SSP with bit 63 set.

I would like the kernel to push the address of the interrupted
instruction as well, potentially with additional flag markup.  Or maybe
it's so early that we don't need it.  The signal return path would have
pop it off the stack and not validate it because I think it's an
expected use case to redirect execution from a signal handler by
patching the signal context.  Alternatively, the kernel could push the
address of the signal context, which might be even more useful.  I think
it would be useful if the shadow stack contained all the data needed to
implement the glibc backtrace function, not because glibc is important,
but because it seems to be a good indicator what programmers expect from
a backtrace.

We also need to figure out how this interacts with LAM.  Does the CPU
push tagged addresses onto the shadow stack?  It could impact the type
of tagging the kernel can use for its own special addresses.

Thanks,
Florian

RE: Shadow stack backtrace command name

[Schimpe, Christina via Gdb]

> * Christina Schimpe:
> 
> > We are currently working on the implementation of the shadow stack
> backtrace.
> > This is how we would print a shadow stack backtrace for signals:
> >
> > ~~~
> > (gdb) bt shadow
> > #0  0x00007ffff7c54d90 in __restore_rt from /lib64/libc.so.6
> > #1  0x80007ffff79fffd8
> > #2  0x00007ffff7c54ce6 in __GI_raise at ../sysdeps/posix/raise.c:27
> > #3  0x000000000040115d in main at /tmp/amd64-shadow-stack-signal.c:32
> > [...]
> > ~~~
> 
> > This would be the corresponding ordinary stack:
> > ~~~
> > (gdb) bt
> > #0  handler (signo=10) at /tmp/amd64-shadow-stack-signal.c:25
> > #1  <signal handler called>
> > #2  __pthread_kill_implementation ([...]) at pthread_kill.c:44
> > #3  0x00007ffff7ca15f3 in __pthread_kill_internal (signo=10,
> > threadid=<optimized out>) at pthread_kill.c:78
> > #4  0x00007ffff7c54ce6 in __GI_raise (sig=10) at
> > ../sysdeps/posix/raise.c:26
> > #5  0x000000000040115d in main () at
> > /tmp/amd64-shadow-stack-signal.c:31
> > ~~~
> > Do you see much value in combining the outputs?
> 
> The difference is that the shadow stack backtrace does not contain the
> interrupted instruction, so frame #2 in the traditional backtrace.  This is more
> important for CPU-generated signals such as division by zero or invalid
> memory access, where you really want to see the fault address in the
> backtrace.
> 
> > The elements on the shadow stack are following the description of the
> > linux kernel for signals:
> > "When a signal happens, the old pre-signal state is pushed on the stack.
> >  When shadow stack is enabled, the shadow stack specific state is
> > pushed onto the shadow stack. Today this is only the old SSP (shadow
> > stack pointer),  pushed in a special format with bit 63 set."
> > (https://docs.kernel.org/arch/x86/shstk.html)
> >
> > Frame 1 contains the old SSP with bit 63 set.
> 
> I would like the kernel to push the address of the interrupted instruction as
> well, potentially with additional flag markup.  Or maybe it's so early that we
> don't need it.  The signal return path would have pop it off the stack and not
> validate it because I think it's an expected use case to redirect execution from
> a signal handler by patching the signal context.  Alternatively, the kernel
> could push the address of the signal context, which might be even more
> useful.  I think it would be useful if the shadow stack contained all the data
> needed to implement the glibc backtrace function, not because glibc is
> important, but because it seems to be a good indicator what programmers
> expect from a backtrace.
> 
> We also need to figure out how this interacts with LAM.  Does the CPU push
> tagged addresses onto the shadow stack?  

No, I don't think so. For LAM only tagged data pointers are supported. 

Christina

Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Sean Fennelly, Jeffrey Schneiderman, Tiffany Doon Silva
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928