Python API for supplying thread information?

Python API for supplying thread information?

[Freddie Chopin]

Hello!

The idea I'm asking about is related to the Pretty Printing API in
Python, which allows one to extend behaviour of GDB completely on
"client side" - there's absolutely no need to modify any component of
the toolchain to print your own types in any way you like.

It would be really great if something like this existed for _supplying_
thread information to GDB. I'm not talking about the Python API to
_query_ GDB about threads.
( https://sourceware.org/gdb/onlinedocs/gdb/Threads-In-Python.html ).

Such feature may seem completely unimportant if you work with PC only,
as there are just a few "old" operating systems and they are all
handled perfectly fine by GDB and the toolchains, so there's no need to
add/extend/modify anything here.

But the situation looks completely different on deeply embedded
platforms and I'm talking about microcontrollers here (like chips with
ARM Cortex-M3 core). There are hundreds of RTOSes for that platforms
and each is slightly different, so it's not possible to handle them all
(or even some part of them) with some common tool in a generic way.

Currently for RTOSes running on ARM the "solution" is provided by
OpenOCD project. OpenOCD is a "GDB server" which communicates with the
embedded target via JTAG interface and allows debugging such devices
with GDB. Some time ago this project got support for a few popular
RTOSes, so that when GDB queries OpenOCD for threads it supplies the
info read from the target memory. Currently OpenOCD supports just a few
(8) RTOSes.

Although adding such support to OpenOCD is not "rocket science" it
requires some effort and poses some challenges.

1. OpenOCD has slow release cycle - usually one can expect a release no
more than once a year. With that in mind even if support for particular
RTOS is added to OpenOCD, some platforms would have to wait years to
actually get that. For example "stable" Debian has OpenOCD that is 3
years old, while "backports", "testing" and "unstable" have release
from 2015 with some patches.

2. Once support is added to OpenOCD, the layout and naming of some RTOS
structures has to be "frozen", as any change in that will break that
support (the code to handle RTOS in OpenOCD is using internal knowledge
about ABI of the RTOS). Obviously when these parameters change, they
can (and should!) be updated in OpenOCD, but see item 1 above - some
users would have to wait years to see that. Then it causes problems
when you want to debug some new and old projects - they may need very
specific versions of tools to work correctly. This may not be such a
big deal for projects which actively participate with OpenOCD team, but
some code for RTOS handling was written by people unrelated to the
team/company of the RTOS, so it's not possible to expect them to keep
everything up to date. Do note that such handling code can be made more
generic by not hardcoding any offsets/addresses/etc., but it has to
hardcode something anyway (at least the names of some objects, the
assumptions of how a system arranges the threads, the stacking of
registers)...

3. Above requirement is a big problem in case of developing your own
RTOS, where you need good debugging tools to catch all the possible
errors, but due to early stage of development the concept of freezing
the layout/naming/configs is not very tempting.

4. The code to handle RTOS in OpenOCD is kind of a "middle man" - it
has to query both GDB and target to get the whole picture. GDB has the
info about symbols and their addresses, OpenOCD has access to the
target. You need both to actually get something useful, as you have to
read and interpret data from the addresses where some specific symbols
were placed by linker. Then OpenOCD fills the info about existing
threads and when GDB wants to know something about particular thread it
asks OpenOCD to get that from the memory.

For an example how such RTOS support code looks in OpenOCD, see here:
http://repo.or.cz/openocd.git/blob/HEAD:/src/rtos/ChibiOS.c
or more generally the files in this folder:
http://repo.or.cz/openocd.git/tree/HEAD:/src/rtos

Yesterday while writing my own pretty printer, I got an idea that this
RTOS support could be done just like "pretty printers" are done. User
would load his/her own Python script which would do the same thing
OpenOCD is doing right now. As this would again be completely "client
side", there would be no problem of incompatible versions of slow rate
of releases. Such Python support script could easily be made part of
the RTOS project or be automatically generated (to match current
configuration).

Would that be even possible to implement in GDB? Maybe something like
that is possible even now, but not documented?

Thanks for any info!

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Liviu Ionescu]

> On 22 Mar 2017, at 11:43, Freddie Chopin <freddie_chopin@op.pl> wrote:
> 
> The idea I'm asking about is related to the Pretty Printing API in
> Python, ...
> Would that be even possible to implement in GDB? Maybe something like
> that is possible even now, but not documented?

even if this is possible, it does not cover all debugging tools (increasing in popularity right now, at least for FreeRTOS) that might need to introspect the RTOS, for example Eclipse plug-ins to display the status of semaphores, queues, memory allocators, etc.

regards,

Liviu

Re: [OpenOCD-devel] Python API for supplying thread information?

[Freddie Chopin]

On Wed, 2017-03-22 at 13:01 +0200, Liviu Ionescu wrote:
> even if this is possible, it does not cover all debugging tools
> (increasing in popularity right now, at least for FreeRTOS) that
> might need to introspect the RTOS, for example Eclipse plug-ins to
> display the status of semaphores, queues, memory allocators, etc.

The tools you mentioned clearly don't use OpenOCD's RTOS support or
GDB's thread info for that anyway, as neither provides any info about
things you mentioned (semaphores, queues, allocators, ...). So that
hypothetical change that I suggested neither prevents nor enables them
to do that. And if these tools do that now, they do that by directly
accessing target memory via GDB, so no special support for anything
like that is required.

Regards,
FCh

Re: Python API for supplying thread information?

[Phil Muldoon]

On 22/03/17 09:43, Freddie Chopin wrote:
> Hello!
> 
> The idea I'm asking about is related to the Pretty Printing API in
> Python, which allows one to extend behaviour of GDB completely on
> "client side" - there's absolutely no need to modify any component of
> the toolchain to print your own types in any way you like.
> 
> It would be really great if something like this existed for _supplying_
> thread information to GDB. I'm not talking about the Python API to
> _query_ GDB about threads.
> ( https://sourceware.org/gdb/onlinedocs/gdb/Threads-In-Python.html ).

Is it possible? Sure. But without looking at it in any real depth it
would be a major undertaking. GDB does not have the concept of
"Providers", it assembles state from many different sources: kernel,
glibc, libthread_db etc. This state is internally structured and
relevant to GDB and is not really structured to receive arbitrary
input from an API at the moment. It could, but I think it would
require major surgery.

Currently the Scripting API is almost entirely focused with providing
information and not really designed to take information. You have
information providers APIs and data decorators APIs.

My advice would be to undertake a study to estimate the work needed.

Cheers

Phil

Re: Python API for supplying thread information?

[Phil Muldoon]

On 22/03/17 09:43, Freddie Chopin wrote:
> Hello!
> 
> The idea I'm asking about is related to the Pretty Printing API in
> Python, which allows one to extend behaviour of GDB completely on
> "client side" - there's absolutely no need to modify any component of
> the toolchain to print your own types in any way you like.
> 
> It would be really great if something like this existed for _supplying_
> thread information to GDB. I'm not talking about the Python API to
> _query_ GDB about threads.
> ( https://sourceware.org/gdb/onlinedocs/gdb/Threads-In-Python.html ).

(Resending to include correct CC's/To's)

Is it possible? Sure. But without looking at it in any real depth it
would be a major undertaking. GDB does not have the concept of
"Providers", it assembles state from many different sources: kernel,
glibc, libthread_db etc. This state is internally structured and
relevant to GDB and is not really structured to receive arbitrary
input from an API at the moment. It could, but I think it would
require major surgery.

Currently the Scripting API is almost entirely focused with providing
information and not really designed to take information. You have
information providers APIs and data decorators APIs.

My advice would be to undertake a study to estimate the work needed.

Cheers

Phil

Re: Python API for supplying thread information?

[Freddie Chopin]

Hello Phil!

On Wed, 2017-03-22 at 15:37 +0000, Phil Muldoon wrote:
> Is it possible? Sure. But without looking at it in any real depth it
> would be a major undertaking. GDB does not have the concept of
> "Providers", it assembles state from many different sources: kernel,
> glibc, libthread_db etc.

Please note that for deeply embedded target these concepts don't apply
- there's no standard "kernel" (usually there's also no separation
between application and kernel), glibc is too huge and none of the
"standard" facilities. From what I've seen 

> This state is internally structured and
> relevant to GDB and is not really structured to receive arbitrary
> input from an API at the moment. It could, but I think it would
> require major surgery.

Well, the info OpenOCD provides to GDB is structured quite nice.
Basically OpenOCD just returns a list of tuples consisting of: thread
name (string), thread extra info (string, in most implementation it's
the info about state) and the thread ID (opaque value). Then it also
informs GDB which of these is currently active. GDB may also request
the list of thread registers, from what I understand by providing
"thread ID" and expecting a string with registers as hex.

All of that is done with GDB packets like "qThreadExtraInfo",
"qfThreadInfo", "qC" and some more.

In this case it would be enough to just get all that data with the
Python script instead of querying GDB server (OpenOCD).

> Currently the Scripting API is almost entirely focused with providing
> information and not really designed to take information. You have
> information providers APIs and data decorators APIs.

I might got something wrong, but wouldn't this be an example of
"information provider API"?

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Pierre-Marie de Rodat]

Hello,

On 03/22/2017 12:01 PM, Liviu Ionescu wrote:
>> The idea I'm asking about is related to the Pretty Printing API in
>> Python, ...
>> Would that be even possible to implement in GDB? Maybe something like
>> that is possible even now, but not documented?
>
> even if this is possible, it does not cover all debugging tools
> (increasing in popularity right now, at least for FreeRTOS) that
> might need to introspect the RTOS, for example Eclipse plug-ins to
> display the status of semaphores, queues, memory allocators, etc.

I remember a talk at the last GNU Cauldron that talked about a project 
to describe way info and the like in a platfrom-independent way: 
https://infinitynotes.org/wiki/Infinity

It’s not a Python API and it’s still work in progress IIUC, but this 
would not be something tied to GDB, so ideally other debugging tools 
could use this info at some point. Would this fit your need?

-- 
Pierre-Marie de Rodat

Re: [OpenOCD-devel] Python API for supplying thread information?

[Freddie Chopin]

On Wed, 2017-03-22 at 18:23 +0100, Pierre-Marie de Rodat wrote:
> I remember a talk at the last GNU Cauldron that talked about a
> project 
> to describe way info and the like in a platfrom-independent way: 
> https://infinitynotes.org/wiki/Infinity
> 
> It’s not a Python API and it’s still work in progress IIUC, but this 
> would not be something tied to GDB, so ideally other debugging tools 
> could use this info at some point. Would this fit your need?

Looks like something targeting PC, so not really suited for a
microcontroller.

Regards,
FCh

Re: Python API for supplying thread information?

[Gareth McMullin]

On Wed, Mar 22, 2017 at 10:43 PM, Freddie Chopin <freddie_chopin@op.pl> wrote:
> Would that be even possible to implement in GDB? Maybe something like
> that is possible even now, but not documented?

There has also been some discussion on this in the Black Magic Probe
user community.
Black Magic Probe is a similar project to OpenOCD, but the GDB server
is embedded in firmware in the JTAG hardware.
https://github.com/blacksphere/blackmagic
It currently has no support for RTOS threads.

I am also of the opinion that this would be better achieved as a
Python extension to GDB.
A while ago I worked on some Python scripts that fake it in GDB, by
overriding the `thread`
and `info threads` CLI commands.
https://github.com/gsmcmullin/gdb_chibios/blob/master/chibios.py
(This is for ChibiOS 2 and is not currently maintained.)  It obviously
doesn't work
with frontends using MI.

I unfortunately don't have the time to work on getting the necessary
extensions into GDB.

Regards,
Gareth

Re: [OpenOCD-devel] Python API for supplying thread information?

[Pierre-Marie de Rodat]

On 03/22/2017 06:52 PM, Freddie Chopin wrote:
> Looks like something targeting PC, so not really suited for a
> microcontroller.

What makes you think that? My understanding of the design is that it's 
centered on adding new metadata sections to ELF binaries and make the 
debugger use this metedata. This looks quite embedded-friendly to me.

-- 
Pierre-Marie de Rodat

Re: Python API for supplying thread information?

[Yao Qi]

Freddie Chopin <freddie_chopin@op.pl> writes:

>> Is it possible? Sure. But without looking at it in any real depth it
>> would be a major undertaking. GDB does not have the concept of
>> "Providers", it assembles state from many different sources: kernel,
>> glibc, libthread_db etc.
>
> Please note that for deeply embedded target these concepts don't apply
> - there's no standard "kernel" (usually there's also no separation
> between application and kernel), glibc is too huge and none of the
> "standard" facilities. From what I've seen 
>

The differences between "deeply embedded target" and PC doesn't matter
here.  As Phil said, GDB has to assemble state from different sources,
including OpenOCD and others.

>> This state is internally structured and
>> relevant to GDB and is not really structured to receive arbitrary
>> input from an API at the moment. It could, but I think it would
>> require major surgery.
>
> Well, the info OpenOCD provides to GDB is structured quite nice.
> Basically OpenOCD just returns a list of tuples consisting of: thread
> name (string), thread extra info (string, in most implementation it's
> the info about state) and the thread ID (opaque value). Then it also
> informs GDB which of these is currently active. GDB may also request
> the list of thread registers, from what I understand by providing
> "thread ID" and expecting a string with registers as hex.
>
> All of that is done with GDB packets like "qThreadExtraInfo",
> "qfThreadInfo", "qC" and some more.
>
> In this case it would be enough to just get all that data with the
> Python script instead of querying GDB server (OpenOCD).

OpenOCD can provide thread information because it embedded the RTOS
internal knowledge into it, as you mentioned in challenge #2 in your
first mail.  IMO, it is a pain.  In order to get rid of this pain
completely, python script is needed to *provide* thread information to
GDB by parsing some RTOS's data structures, so that OpenOCD don't need
these C code to get OS-specific thread information at all.  Providing
thread info isn't just printing thread name and state in "info
threads".  GDB internally needs thread information too.

>
>> Currently the Scripting API is almost entirely focused with providing
>> information and not really designed to take information. You have
>> information providers APIs and data decorators APIs.
>
> I might got something wrong, but wouldn't this be an example of
> "information provider API"?

I agree with Phil.  Some python APIs are needed to provide thread
information to GDB, but they are missing in GDB now.  Note that we are
working Linux kernel awareness debugging in GDB, in which we get Linux
specific information, for example, kernel threads, via C code.  We
thought about doing it in python two years ago, but decide to implement
it C first, and then, add needed python interfaces in GDB and move the
code to python if possible.

-- 
Yao (齐尧)

Re: [OpenOCD-devel] Python API for supplying thread information?

[Duane Ellis]

Hi - I’ve been following this thread

A while ago - I proposed a very different architecture that would solve a lot of this.

Fundamentally, there are two processes:   (A) The GDB process and (B) the GDBSERVER process.
When debugging an application that runs under a true OS (such as linux) this works great.

But - when debugging anything bare metal, and that includes things like the Linux Kernel, and just about any RTOS in any micro-controller things are very different.

The problem is these things are two processes that cannot share knowledge of each other, 

The current architecture is this:  At the “target remote” - level - GDB sends various packets like Register Read/Write,  Memory Read/Write packet. The problem is you need to write lots of GDB Protocol additions to add features - these extra things are not (and never) required when debugging an application under and Operating system. It is more then just “thread level information” - it extends to hardware registers, MMU state, Cache Information - things that an application debugger never sees and never uses.

In the current architecture it is the job of the GDB SERVER to translate these extra packets in to the actual operation. This makes sense when you are talking about a OS-APP-TARGET (i.e.: Windows, Linux, Mac, Cellphone apps)  The OS solution is written once and is reused over and over again. And there are not that many of of these things. It works great for apps because there are very few of these things to write, and you are talking to an OS… not bare metal..

In the bare metal case, there are countless GDB SERVERS - and every one has to duplicate the process, the features and the packets. It is even worse when we talk about the RTOS support, every GDBSERVER solution has to write its own version of the RTOS support.  Don’t forget about MMU level inspection, or Cache Debugging. These are things an APPLICATION level debugger never sees - a bare metal debugger sees these things all the time (of course depending on the chip, i.e.: ARMV8 kernel debug, vrs CortexM3 debug).

Bottom line: I believe the current gdb server solution is not well suited for some complex bare metal things - and that is what this thread is about.

What I proposed a while back was this:

What if we 

1) introduced a “JTAG/SWD” protocol instead.  For ARM (Cortex)- bare metal - all operations there are a series of DAP operations. i.e.: Dap Read/Dap Write.

2) For JTAG based chips (i.e.: earlier ARM 9 chips, Mips chips, etc) there are a number of JTAG level protocols (XILINX for example has a SOCKET based solution)

3) For SWD (cortex) based chips - all SWD operations can be ‘downgraded to JTAG’ - by a shim layer if needed.

4) Then - write a *NEW* target layer in GDB that uses this new protocol and effectively implement the memory read/write and register read/write operations in python, the bottom layer of that python implementation would send JTAG or SWD commands over the wire instead of “memory read” and “register read/write” packets.

The advantage I see is this:

    GDB can be come very aware of the target MMU, for example - GDB can walk through all of the ARMv8 Hypervisor and

    All of this is in one process - not spread across a protocol.

    You have a simple common (and rich) script language (i.e.: Python) that lets one write some major scripts once - for all bare metal targets.

    Flash programing operations (very common in micro-controllers) are written once - in Python - for all targets.

    All of this is “write once” and it works - in one common way at the Python layer in GDB.

-Duane.

Re: Python API for supplying thread information?

[Freddie Chopin]

On Thu, 2017-03-23 at 11:34 +0000, Yao Qi wrote:Providing
> thread info isn't just printing thread name and state in "info
> threads".  GDB internally needs thread information too.

But that's all GDB is getting from OpenOCD - thread name (string),
extra info (string) and stack frame (registers).

Unless I'm missing something, which is of course possible (;

Regards,
FCh

Re: Python API for supplying thread information?

[Freddie Chopin]

On Thu, 2017-03-23 at 10:50 +1300, Gareth McMullin wrote:
> I am also of the opinion that this would be better achieved as a
> Python extension to GDB.
> A while ago I worked on some Python scripts that fake it in GDB, by
> overriding the `thread`
> and `info threads` CLI commands.
> https://github.com/gsmcmullin/gdb_chibios/blob/master/chibios.py
> (This is for ChibiOS 2 and is not currently maintained.)  It
> obviously
> doesn't work
> with frontends using MI.

I had a crazy idea yesterday, that it should (maybe) be possible to
implement a python "bridge" between GDB and OpenOCD. This script would
open sockets for both ends and generally be transparent to the traffic.
But it could intercept all the GDB packets about threads. This way this
logic could be implemented completely in the script. This seems
promising, but I'm not sure it would be possible to create such bridged
connection between GDB and OpenOCD while intercepting some of the
traffic...

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Freddie Chopin]

On Thu, 2017-03-23 at 10:23 +0100, Pierre-Marie de Rodat wrote:
> What makes you think that? My understanding of the design is that
> it's 
> centered on adding new metadata sections to ELF binaries and make
> the 
> debugger use this metedata. This looks quite embedded-friendly to me.
> 

Maybe I got the wrong impression, sorry. But the problem is that this
project doesn't seem very active... Generally fragmenting the effort
(yet another library with yet another standard and so on...) doesn't
seem like a very good idea to me, that's why in my opinion a solution
implemented completely in GDB would be better suited and would have a
higher chance of "survival" and "adoption". But I might got the wrong
impression again.

However me and Liviu had a discussion about describing RTOS structure
in a generic way and I'm still pretty certain that this is generally
not possible in a generic and agnostic way. In the end it would become
either extremely complex or you'd have to implement some kind of
scripting/code to actually deal with that.

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Freddie Chopin]

On Wed, 2017-03-22 at 18:51 +0100, Daniel Krebs wrote:
> Hi Freddie,
> 
> first of all, I feel the same way as you do. I'm one of those guys
> who
> have undergone the endeavour to add RTOS support to OpenOCD (in my
> case
> RIOT OS). All the problems (slow OpenOCD release cycle, hard-coded
> offsets/desynchronized information description, messy implementations
> in
> OpenOCD) you described are true, I only want to add another
> perspective.
> While trying to add support for RIOT, there was kind of a chicken-egg
> problem: without patches being merged in RIOT, you cannot test the
> OpenOCD implementation as well as the other way around when merging
> OpenOCD patches. Additionally, I've found it quite difficult to find
> people reviewing those patches on the OpenOCD side (it is not merged
> yet, I lost track of it). However, I totally understand that. Most of
> use are doing this in our free-time and so on, so I'd say this
> approach
> doesn't scale so well anyway.

Yes, that would be another issue with implementing such RTOS support.
It gets pretty hard when you have to modify both the RTOS and OpenOCD,
while _NOT_ being the developer of that RTOS.

I forgot about that, as I have the "luxury" of being the developer of
my own RTOS ( http://distortos.org/ ), so it's not a problem for me to
add any support for such integration with OpenOCD and/or GDB.
Especially because I see that as a thing of great importance!

> I have some experience with GDB Python scripting (there are also
> "new"
> OSes on x86), but I ran into the same problem as you did. To my
> knowledge (as of Dec 2016), there's no way to tell GDB about thread
> structures. That's why I ended up working around that by switching
> contexts myself. If it's of any help or interest to you, have a look
> at [1].
> 
> I'm not sure if it's possible for OpenOCD to add such support. It
> would
> make more sense to do that upstream in GDB IMHO, but it's the right
> direction. The upside of such an implemention is definetly that every
> project can manage these scripts in-tree and in sync with their
> development, with no dependency (as in providing patches) on OpenOCD
> anymore.

Exactly my thought - in that case the whole problem of "frozen" ABI is
just gone!

> Sadly, I cannot provide support on that matter at the moment because
> I'm
> lacking a use-case. Just wanted to add my 2 cents :)

Thanks for the input!

> Cheers,
> Daniel
> 
> 
> [1] https://github.com/RWTH-OS/HermitCore/tree/devel/usr/gdb

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Liviu Ionescu]

> On 23 Mar 2017, at 18:50, Freddie Chopin <freddie_chopin@op.pl> wrote:
> 
> ... However me and Liviu had a discussion about describing RTOS structure
> in a generic way and I'm still pretty certain that this is generally
> not possible in a generic and agnostic way. In the end it would become
> either extremely complex or you'd have to implement some kind of
> scripting/code to actually deal with that.

as I already mentioned, my next version of the DRTM library will use a compiled binary JSON, so everything that can be described in a JSON will be perfectly acceptable.

are JSONs generic enough? I would say they are.

will I make the DRTM library 'absolutely generic and agnostic' from the very beginning? definitely not realistic, I'll first define the data types and memory structures that I need for my µOS++.

will this be expandable with more memory types? definitely yes! at the end, the number of ways a list of threads is kept may be large, at the limit each RTOS may invent a different scheme, but the number is still finite. ;-)


my assumption is that relatively low and manageable, so it should be easier to add a new definition to an existing framework that is already fully functional, than to redo an implementation completely from scratch. and once you do it, the result should be directly available to all servers that use the DRTM library (OpenOCD, J-Link, QEMU being on my TODO list).


regards,

Liviu

Re: [OpenOCD-devel] Python API for supplying thread information?

[Freddie Chopin]

On Thu, 2017-03-23 at 12:38 -0500, Steven Stallion wrote:
> 
> On Thu, Mar 23, 2017 at 11:56 AM, Freddie Chopin <freddie_chopin@op.p
> l> wrote:
> > Yes, that would be another issue with implementing such RTOS
> > support.
> > It gets pretty hard when you have to modify both the RTOS and
> > OpenOCD,
> > while _NOT_ being the developer of that RTOS.
> > 
> > I forgot about that, as I have the "luxury" of being the developer
> > of
> > my own RTOS ( http://distortos.org/ ), so it's not a problem for me
> > to
> > add any support for such integration with OpenOCD and/or GDB.
> > Especially because I see that as a thing of great importance!
> > 
> 
> Why do you need to modify your RTOS? If this is a case where you need
> additional information at runtime, take a look under contrib at the
> RTOS helpers for FreeRTOS and uC/OS-III. This should give you an idea
> of how you can supplement the image under test with information you
> need for transient information like struct offsets. You have
> unfettered access to memory, so there's nothing holding you back or
> requires you to alter the RTOS source code.

It's not that I "need" to. I was just referring to the chicken-egg
problem mentioned earlier - if you are a "third party" (not OpenOCD
developer, not the developer of the RTOS in question), then just adding
these structures with RTOS mapping to the RTOS repository may pose a
problem. I know that these can be put into contrib/ folder of OpenOCD,
but I'm pretty certain that it's much better if such structs are part
of the main RTOS repository, which allows them to be maintained and "in
sync".

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Freddie Chopin]

On Thu, 2017-03-23 at 11:46 -0500, Steven Stallion wrote:
> There's a bit more to it than that, but that's what users see most
> often. It's important to point out that GDB's remote target support
> isn't the best; it's not on par with a local target by any means. It
> wasn't until recently that thread names could be assigned/sign for
> remote targets (7.11.2 IIRC). To the OP, if you're interested in
> extending behavior, I'd suggest starting with a small PoC. I think
> you'll get a better idea of the issues at hand once you start
> exploring the GDB source.

This would be a problem, as I'm not really a good desktop programmer
(I'm working with microcontrollers only), I also know almost nothing
about GDB interals. That's why I was hoping that something like that is
actually possible now (directly in GDB or maybe with some evil
trickery)...

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Freddie Chopin]

Hi Duane!

On Thu, 2017-03-23 at 07:33 -0700, Duane Ellis wrote:
> Bottom line: I believe the current gdb server solution is not well
> suited for some complex bare metal things - and that is what this
> thread is about.
> 
> What I proposed a while back was this:
> [...]

Wouldn't that actually result in OpenOCD's functionality being merged
into GDB? It's an interesting idea, but I think it would be the hardest
one to actually implement, given all the issues with JTAG interfaces
and their configurations... Unless I misread your proposal? Maybe you
suggest that GDB should not interact with OpenOCD as with a "GDB
server" but communicate using some special interface designed just for
that situation?

Regards,
FCh

Re: [OpenOCD-devel] Python API for supplying thread information?

[Gary Benson]

Hi,

I'm the guy who did the Infinity talk at Cauldron...

Freddie Chopin wrote:
> On Thu, 2017-03-23 at 10:23 +0100, Pierre-Marie de Rodat wrote:
> > What makes you think that? My understanding of the design is that
> > it's centered on adding new metadata sections to ELF binaries and
> > make the debugger use this metedata. This looks quite
> > embedded-friendly to me.
> 
> Maybe I got the wrong impression, sorry. But the problem is that
> this project doesn't seem very active... Generally fragmenting the
> effort (yet another library with yet another standard and so on...)
> doesn't seem like a very good idea to me, that's why in my opinion a
> solution implemented completely in GDB would be better suited and
> would have a higher chance of "survival" and "adoption". But I might
> got the wrong impression again.
> 
> However me and Liviu had a discussion about describing RTOS
> structure in a generic way and I'm still pretty certain that this is
> generally not possible in a generic and agnostic way. In the end it
> would become either extremely complex or you'd have to implement
> some kind of scripting/code to actually deal with that.

Infinity is active, but it's just me, and I've had some distractions
that've kept me from making progress of late.

I think it could do what you're asking.  I'm not super-familiar with
embedded systems but I assume you're using GDB talking to something
via the remote protocol?  Here's how I see it working:

* There's some information you need from the remote.  I'll use the
  example that you have a numeric thread ID and you want that
  thread's priority, which is also a number.

* You write an Infinity function, called thread::priority_for_ID or
  something.  That function gets compiled by the Infinity note
  compiler and lives in the executable.  Possibly in your case it
  would be stripped from the binary the embedded system sees, but
  present in the binary GDB sees.  [Note this is future work: Infinity
  functions are always unstripped at present.]  The function you wrote
  is basically things like "read this register, add this other thing
  to it, read this memory".

* GDB loads the binary, it has the Infinity functions.

* Your pretty printer is able to get the information it requires
  by calling the Infinity function you wrote via a Python API in
  GDB.  [Note again, this part is future work, there's no such API
  at present.]

* GDB executes the Infinity function you wrote, deferring to the
  remote for register and memory reads using existing remote protocol
  features.

This can all be done without adding extra features to the remote--the
Infinity function is executed by GDB.

It's not something that's weeks away from release, but if you want to
lend a hand? ;)

Cheers,
Gary
  
-- 
https://gbenson.net/