From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 66396 invoked by alias); 1 Feb 2016 18:19:45 -0000 Mailing-List: contact gdb-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-owner@sourceware.org Received: (qmail 66372 invoked by uid 89); 1 Feb 2016 18:19:44 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.2 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=engagement, beneath, UD:binutils-gdb.git, Phil X-HELO: mail-wm0-f46.google.com Received: from mail-wm0-f46.google.com (HELO mail-wm0-f46.google.com) (74.125.82.46) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Mon, 01 Feb 2016 18:19:41 +0000 Received: by mail-wm0-f46.google.com with SMTP id l66so84314646wml.0 for ; Mon, 01 Feb 2016 10:19:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=RxDCPrBUUzPUdBUF/Unr8dxIFyGpVvOIFlX1pViIZ5I=; b=BoMcueGNXcVNlFSaf2JYdTXA5zQzl2GRB4LeI0h4OvwgUwpP4LYi42qsjxmnfiZjmK 30/YypUfW15t9GNuIxCqTGVj4ZSQlXkT63TcmcZFWSHVsnLQrQnuWohjK/AVdlpK7Omx O0lYdvD2slirW7K+Kx5yvFE1OJWrqYPiKjJQ9URSDAuWPAP38t6/KkRX2loLEMnApNMW 2YVCQ4hpKDoZfALFqRfkyzzAjQBnOIzb83RZeRIZujGUjvEpWm8oBEmqjy+v73URo09+ Fffz9U9hLhXUF5jE4i5qNoZXcgxhVqVJJGEsQ12NjLIAeZN1VIORgHQnHjHbOgpyNyE9 +9MA== X-Gm-Message-State: AG10YOReqYETYlFU2p0CiHz054RYC3EXVlROROjkiXefVvQSdHpRzi+De7P7Mz2unxXwCgNw X-Received: by 10.194.5.36 with SMTP id p4mr8767898wjp.167.1454350778560; Mon, 01 Feb 2016 10:19:38 -0800 (PST) Received: from [192.168.0.31] (cpc87017-aztw30-2-0-cust65.18-1.cable.virginm.net. [92.232.232.66]) by smtp.gmail.com with ESMTPSA id t195sm12732639wme.13.2016.02.01.10.19.37 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 01 Feb 2016 10:19:37 -0800 (PST) Subject: Re: [RFC] Target Layer Python Interface To: Phil Muldoon , gdb@sourceware.org References: <56AB5C11.1020800@linaro.org> <56AE2674.5060101@redhat.com> Cc: Yao Qi , Peter Griffin , Jan Kiszka , Lee Jones , Jeff Mahoney , Ales Novak From: Kieran Bingham X-Enigmail-Draft-Status: N1110 Message-ID: <56AFA1B8.6060402@linaro.org> Date: Mon, 01 Feb 2016 18:19:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <56AE2674.5060101@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SW-Source: 2016-02/txt/msg00000.txt.bz2 Hi Phil, On 31/01/16 15:21, Phil Muldoon wrote: > On 29/01/16 12:33, Kieran Bingham wrote: >> While investigating how to get Python to define the threads presented by > the application layer (in my instance the linux kernel) I found that I > could create threads with a call from Python, (by writing a hook through > the inferior object) > > A snippet would be cool to see what you are doing here, if possible. My initial test was simply a method to generate PID's and add them to test what would happen. Commit viewable @ : https://git.linaro.org/people/kieran.bingham/binutils-gdb.git/commitdiff/0e2692cdc6d195810ed61ac96ada91a715f94990 In testing this, I quickly discovered that the thread list was updated as soon as I then called "info threads" which cleared down the list. Thus, I then went to look at creating a gdb.Target object to mirror ST's implementations, and provide the ability to call into the kernels python iterators to generate the list instead. So far I have created a gdb.Target python object representation which has a call to register the layer from a python subclass (following the implementation style of gdb.Command, where calling super.__init__ performs the registration) An early version of (work in progress) gdb.Target is available here: https://git.linaro.org/people/kieran.bingham/binutils-gdb.git/commitdiff/77ca621b2d2fcdc9376221833c9dbae557881090 And of course usual caveats of this branch will be rebased very soon :) > >> However, whilst these threads were added, they were immediately removed > by the next call to (target).to_update_thread_list() > > I presume you mean GDB's accounting of them, not the actual threads? Aha, of course :) - Only GDB's 'view' of the threads are removed. Not actual Kernel threads. This brings me on to needing to implement .to_update_thread_list() et al. I believe if we can expose this functionality through to Python, the implementation will live in the kernel - and be common to the targets, and maintain kernel specific versions in the correct place. The kernel. > >> This has led me to believe I should create a Python Target layer > interface, which I have started, and have been able to register a > target, and provide Python bindings so that (currently _to_thread_name) > calls into the python layer object, and returns a new name. > > My intention is that this can be extended to provide the whole set of > target operations allowing me to implement much of the Linux Kernel > Debugger interface in Python. > > At this point, there's just not enough information to form an opinion > of this being a good thing or a bad thing. Do you have an interface in > mind? An API? The API, I would expect to match that of the Target API operations. I would expect a one-to-one mapping of (required) operation names to perform functions. For instance, to support thread listings, I would expect something along the lines of: .to_update_thread_list .to_pid_to_str .to_extra_thread_info .to_thread_alive .to_fetch_registers ... And having seen Jeff's work today, we could utilise Jeff's py-regcache object quite effectively > >>> Through this layer, we can even tackle some of the issues with memory > address spaces, by adding appropriate hooks to adjust MMU context, or > interpret page tables through the relevant target_ops hooks. > > Interesting! > It appears, that the guys at SUSE are already looking at SLAB table walks, and I wonder if we could introduce this into our layers and provide a common architecture for both kdump/core files, and live targets. The STLinuxLKD implementation for interacting with Userspace is to manipulate the MMU, which Jan has noted has issues as it changes the state of the target you are debugging. However if we can identify the data locations to read by walking tables, then this could be a route to providing a much more enriched data retrieval >> Some of the interface can even hopefully be autogenerated by a > make-target-delegate equivalent > > Presume you mean the Python C code/interface? Yes, I suspect some of the functionality to implement will be very repeatable throughout each of the operation call implementations. However, the more I look into it - the more I see each function is likely to need very specific bindings, as it is not simple passing from c function to c function. Perhaps we can factor out commonality as we go - and try to keep as DRY as possible, but I suspect it will be an iterative implementation process. >> Before I head too much further, I wanted to ask the opinions of the > community as to whether this is an acceptable interface to implement, or > if it opens up too much of the GDB internals, to external dependencies. > > This is something that is always a balancing act with Python > GDB. There are, as you allude too, some parts of GDB that are not > suitable to be extended to the Python layer, and some parts that while > theoretically OK, need a little work to make safe and accessible in a > Pythonic way. I was discussing this with Jan this yesterday morning. We don't want a fragile plugin implementation causing undesirable effects in the rest of the GDB program space. The layers would have to be able to gracefully handle failures in the python functions, without catastrophic crashing of gdb. The layer would also need to be able to enable/disable which is perhaps as simple as poping it off the target stack..., or perhaps more likely having an enable flag in the gdb.Target object, which would effectively make the entire layer delegate to beneath. > I can't comment without more details though. My initial reaction > though is yeah, this sounds useful and exciting. Perfect :) > >> We could of course tackle this by providing a versioned API which would > allow adaptation in the Python objects, but I thought it was a big > enough question that it should be posed to a wider audience (wider than > just me!) > > The versioned API would also need some documenting first. Of course. > I ask a few questions here that might provide you with some > additional work! Sorry ;) But I find this an interesting proposal. Ask away! I'm keen to have engagement. I'm considering putting submitting a proposal for a presentation on the topic at ELC to follow on from Peter's presentation in Dublin, and hopefully find out what use cases people would *actually* expect from this. I think we already can see that we will need to make this layer work with a large group of users. I hope that it will be common across remote(jtag/serial) targets, including QEmu/KVM stubs, and with the work from the guys at SUSE we should be able to cater this for kdump cores too. I think the importance is to make sure everything works well together. > > Cheers > > Phil > > Regards Kieran