[RFA 3/5 v2] Darwin: set startup-with-shell to off on Sierra and later.

[RFA 3/5 v2] Darwin: set startup-with-shell to off on Sierra and later.

[Xavier Roirand]

On Mac OS X Sierra and later, the shell is not allowed to be
debug so add a check and disable startup with shell in that
case. This disabling is done temporary before forking
inferior and restored after the fork.

gdb/ChangeLog:

        * darwin-nat.c (should_disable_startup_with_shell):
        New function.
        (darwin_nat_target::create_inferior): Add call.

Change-Id: Ie4d9090f65fdf2e83ecf7a0f9d0647fb1c27cdcc
---
 gdb/darwin-nat.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/gdb/darwin-nat.c b/gdb/darwin-nat.c
index be80163d22e..d23706d79fd 100644
--- a/gdb/darwin-nat.c
+++ b/gdb/darwin-nat.c
@@ -1854,15 +1854,47 @@ darwin_execvp (const char *file, char * const argv[], char * const env[])
   posix_spawnp (NULL, argv[0], NULL, &attr, argv, env);
 }
 
+/* Read kernel version, and return false on Sierra or later.  */
+
+static int
+should_disable_startup_with_shell ()
+{
+  char str[16];
+  size_t sz = sizeof (str);
+  int ret;
+
+  ret = sysctlbyname ("kern.osrelease", str, &sz, NULL, 0);
+  if (ret == 0 && sz < sizeof (str))
+    {
+      unsigned long ver = strtoul (str, NULL, 10);
+      if (ver >= 16)
+        return TRUE;
+    }
+  return FALSE;
+}
+
 void
 darwin_nat_target::create_inferior (const char *exec_file,
 				    const std::string &allargs,
 				    char **env, int from_tty)
 {
+  gdb::optional<scoped_restore_tmpl<int>> restore_startup_with_shell;
+  int startup_shell_disabled = 0;
+
+  if (startup_with_shell && should_disable_startup_with_shell ())
+    {
+      warning (_("startup-with-shell not supported on this macOS version, disabling it."));
+      restore_startup_with_shell.emplace (&startup_with_shell, 0);
+      startup_shell_disabled = 1;
+    }
+
   /* Do the hard work.  */
   fork_inferior (exec_file, allargs, env, darwin_ptrace_me,
 		 darwin_ptrace_him, darwin_pre_ptrace, NULL,
 		 darwin_execvp);
+
+  if (startup_shell_disabled)
+    restore_startup_with_shell.emplace (&startup_with_shell, 1);
 }
 \f
 
-- 
2.15.2 (Apple Git-101.1)

Re: [RFA 3/5 v2] Darwin: set startup-with-shell to off on Sierra and later.

[Simon Marchi]

Thanks for the update!

CCing Tom, because he had a similar patch that was unfortunately not 
looked at yet... so maybe he has an opinion about this.

On 2018-09-09 11:17, Xavier Roirand wrote:
> On Mac OS X Sierra and later, the shell is not allowed to be
> debug so add a check and disable startup with shell in that
> case. This disabling is done temporary before forking
> inferior and restored after the fork.
> 
> gdb/ChangeLog:
> 
>         * darwin-nat.c (should_disable_startup_with_shell):
>         New function.
>         (darwin_nat_target::create_inferior): Add call.
> 
> Change-Id: Ie4d9090f65fdf2e83ecf7a0f9d0647fb1c27cdcc
> ---
>  gdb/darwin-nat.c | 32 ++++++++++++++++++++++++++++++++
>  1 file changed, 32 insertions(+)
> 
> diff --git a/gdb/darwin-nat.c b/gdb/darwin-nat.c
> index be80163d22e..d23706d79fd 100644
> --- a/gdb/darwin-nat.c
> +++ b/gdb/darwin-nat.c
> @@ -1854,15 +1854,47 @@ darwin_execvp (const char *file, char * const
> argv[], char * const env[])
>    posix_spawnp (NULL, argv[0], NULL, &attr, argv, env);
>  }
> 
> +/* Read kernel version, and return false on Sierra or later.  */

I think this is the other way around (return trun on Sierra or later).

> +
> +static int
> +should_disable_startup_with_shell ()
> +{
> +  char str[16];
> +  size_t sz = sizeof (str);
> +  int ret;
> +
> +  ret = sysctlbyname ("kern.osrelease", str, &sz, NULL, 0);
> +  if (ret == 0 && sz < sizeof (str))
> +    {
> +      unsigned long ver = strtoul (str, NULL, 10);
> +      if (ver >= 16)
> +        return TRUE;
> +    }
> +  return FALSE;
> +}
> +
>  void
>  darwin_nat_target::create_inferior (const char *exec_file,
>  				    const std::string &allargs,
>  				    char **env, int from_tty)
>  {
> +  gdb::optional<scoped_restore_tmpl<int>> restore_startup_with_shell;
> +  int startup_shell_disabled = 0;
> +
> +  if (startup_with_shell && should_disable_startup_with_shell ())
> +    {
> +      warning (_("startup-with-shell not supported on this macOS
> version, disabling it."));
> +      restore_startup_with_shell.emplace (&startup_with_shell, 0);
> +      startup_shell_disabled = 1;
> +    }
> +
>    /* Do the hard work.  */
>    fork_inferior (exec_file, allargs, env, darwin_ptrace_me,
>  		 darwin_ptrace_him, darwin_pre_ptrace, NULL,
>  		 darwin_execvp);
> +
> +  if (startup_shell_disabled)
> +    restore_startup_with_shell.emplace (&startup_with_shell, 1);

You shouldn't need this last bit, nor the startup_shell_disabled 
variable.  When it gets destroyed, the restore_startup_with_shell object 
takes care of restoring the value of the startup_with_shell variable to 
the value it had when it was constructed (the first emplace call).

In other words, this call:

   restore_startup_with_shell.emplace (&startup_with_shell, 0);

means "save the current value of startup_with_shell, then put 0 in it".

When the destructor of restore_startup_with_shell is called, it restores 
the saved value.

With that fixed, it would LGTM, although I did not try it because I 
don't have access to a Mac at the moment.

Simon

Re: [RFA 3/5 v2] Darwin: set startup-with-shell to off on Sierra and later.

[Xavier Roirand]

Le 9/9/18 à 1:04 PM, Simon Marchi a écrit :
> Thanks for the update!
> 
> CCing Tom, because he had a similar patch that was unfortunately not 
> looked at yet... so maybe he has an opinion about this.
> 
> On 2018-09-09 11:17, Xavier Roirand wrote:
>> On Mac OS X Sierra and later, the shell is not allowed to be
>> debug so add a check and disable startup with shell in that
>> case. This disabling is done temporary before forking
>> inferior and restored after the fork.
>>
>> gdb/ChangeLog:
>>
>>         * darwin-nat.c (should_disable_startup_with_shell):
>>         New function.
>>         (darwin_nat_target::create_inferior): Add call.
>>
>> Change-Id: Ie4d9090f65fdf2e83ecf7a0f9d0647fb1c27cdcc
>> ---
>>  gdb/darwin-nat.c | 32 ++++++++++++++++++++++++++++++++
>>  1 file changed, 32 insertions(+)
>>
>> diff --git a/gdb/darwin-nat.c b/gdb/darwin-nat.c
>> index be80163d22e..d23706d79fd 100644
>> --- a/gdb/darwin-nat.c
>> +++ b/gdb/darwin-nat.c
>> @@ -1854,15 +1854,47 @@ darwin_execvp (const char *file, char * const
>> argv[], char * const env[])
>>    posix_spawnp (NULL, argv[0], NULL, &attr, argv, env);
>>  }
>>
>> +/* Read kernel version, and return false on Sierra or later.  */
> 
> I think this is the other way around (return trun on Sierra or later).
> 
>> +
>> +static int
>> +should_disable_startup_with_shell ()
>> +{
>> +  char str[16];
>> +  size_t sz = sizeof (str);
>> +  int ret;
>> +
>> +  ret = sysctlbyname ("kern.osrelease", str, &sz, NULL, 0);
>> +  if (ret == 0 && sz < sizeof (str))
>> +    {
>> +      unsigned long ver = strtoul (str, NULL, 10);
>> +      if (ver >= 16)
>> +        return TRUE;
>> +    }
>> +  return FALSE;
>> +}
>> +
>>  void
>>  darwin_nat_target::create_inferior (const char *exec_file,
>>                      const std::string &allargs,
>>                      char **env, int from_tty)
>>  {
>> +  gdb::optional<scoped_restore_tmpl<int>> restore_startup_with_shell;
>> +  int startup_shell_disabled = 0;
>> +
>> +  if (startup_with_shell && should_disable_startup_with_shell ())
>> +    {
>> +      warning (_("startup-with-shell not supported on this macOS
>> version, disabling it."));
>> +      restore_startup_with_shell.emplace (&startup_with_shell, 0);
>> +      startup_shell_disabled = 1;
>> +    }
>> +
>>    /* Do the hard work.  */
>>    fork_inferior (exec_file, allargs, env, darwin_ptrace_me,
>>           darwin_ptrace_him, darwin_pre_ptrace, NULL,
>>           darwin_execvp);
>> +
>> +  if (startup_shell_disabled)
>> +    restore_startup_with_shell.emplace (&startup_with_shell, 1);
> 
> You shouldn't need this last bit, nor the startup_shell_disabled 
> variable.  When it gets destroyed, the restore_startup_with_shell object 
> takes care of restoring the value of the startup_with_shell variable to 
> the value it had when it was constructed (the first emplace call).
> 
> In other words, this call:
> 
>    restore_startup_with_shell.emplace (&startup_with_shell, 0);
> 
> means "save the current value of startup_with_shell, then put 0 in it".
> 
> When the destructor of restore_startup_with_shell is called, it restores 
> the saved value.
> 
> With that fixed, it would LGTM, although I did not try it because I 
> don't have access to a Mac at the moment.
> 
> Simon

Thanks, I'll fix this.