Re: GDB/XMI (XML Machine Interface)

Nick NoSpam
2004-08-11T08:51:40Z

First I'd like to commend you on the initiative to do something about
the MI problems. I too have fiddled w/ a frontend and aun into a slew
of problems w/ MI2 (some of which made their way into the bug tracker
months ago w/out any attention). I also have a great deal of experience
w/ XML (have used it for years and continue to do so at my job).

Now for the real honesty: I don't think an XML interface is going to
fix the real problem.

Disclaimer: I'm not an expert at GDB (it's internals nor usage) or GDB
MI. The following is my candid opinion--it's too late (technically
early) to be tactful.

The real problem is that GDB is not a library. It's a fairly convoluted
mess that uses the console as it's primary interface. That is wrong.
All abstractions and extensions (including MI*) work through the
console--essentially automating a user's keystrokes. And there lies the
fundamental problem: all methods of offering more than the console
interface are subject to the pitfalls of a console interface. Michael
Chastain points out a popular one: distinguishing GDB output from
inferior output.
What's worse, GDB has been massaged over time to be a one-size-fits-all
solution (ie. tightly coupled). That's also wrong. The problems of
which are evident when one tries to add or modify significant
functionality.

The time spent to generate a well-defined XML interface won't provide
much more than spending the same time improving the MI2 interface
(perhaps yielding MI3). But neither of these approaches solves the real
problem outlined above.

So I think the best (and proper) thing to do is work on splitting GDB
into a separate, well-defined library. Then, all interfaces (including
the console) are simply clients to this library. At that time, an XML
interface would go further (as it adds language independence and
remoting).

A quick google search tells me I'm not the only one w/ this idea. I
don't necessarily agree w/ the design of Andrew Cagney's libGDB, but the
idea is spot on:
http://sources.redhat.com/gdb/papers/libgdb2/

Regards,
Nick G.

On Tue, 2004-08-10 at 16:14, Bob Rossi wrote:
> Hi,
>
> As most of you know, I have been writing a front end to GDB called CGDB.
> However, for a while now, I have been a little disappointed with the MI
> interface. In my RFC I have described the problems I have with MI and I
> have proposed a new method of communication between GDB and the front
> end.
>
> If there is good feedback on this RFC, I will continue research on it,
> filling in details that I thought would be best to wait until after an
> initial acceptance.
>
> If the RFC is accepted by the community I plan on implementing an
> initial version of the XMI interface in GDB, and programming CGDB to
> sit on top of GDB. This will at least give the community one open
> source front end to GDB and an example to prove that the new
> machine interface works.
>
> I can't wait to hear your opinions/comments.
>
> Thanks,
> Bob Rossi
>
> GDB/XMI (XML Machine Interface)
>
> Robert Rossi
> August 2004
> bob@brasko.net
>
>
> TABLE OF CONTENTS
>
> 1. Introduction
> 1.1 The Problem
> 1.2 The Objective
>
> 2. The GDB/XMI Overview
> 2.1 Passing Information between GDB and the front end
> 2.2 Parsing XMI Commands from GDB's output
> 2.3 Parsing the Inferior's output
> 2.4 Validation
> 2.5 Version Management
> 2.6 Understanding the XML data
>
> 3. GDB/XMI Specification
> 3.1 XMI Types
> 3.1.1 Scalar Types
> 3.1.2 Compound Types (Structs)
> 3.2 A Breakpoint Listing
> 3.2.1 The GDB/MI -break-list Specification
> 3.2.2 GDB/MI Output
> 3.2.3 The GDB/XMI Output
> 3.3 A Backtrace
> 3.3.1 The GDB/MI -stack-list-frames Specification
> 3.3.2 GDB/MI Output
> 3.3.3 The GDB/XMI Output
>
> 4. A GDB/XMI Example
> 4.1 The Inferior's Code
> 4.2 GDB/MI Output
> 4.3 GDB/XMI Output
> 4.4 A Brief Description of the XMI output
> 4.4.1 GDB_DISPLAY_PROMPT Document
> 4.4.2 GDB_WAITING_FOR_COMMAND Document
> 4.4.3 The Front End Transmits a Command
> 4.4.4 GDB_INFERIOR_STARTED Document
> 4.4.5 GDB_INFERIOR_FINISHED Document
>
> 5. Issues to resolve after RFC Initial approval.
> 5.1 If GDB will link to libxml, or just output XML
> 5.2 How the schema will represent each document sent to the front end
> 5.3 Separating GDB output and the Inferior output
> 5.4 If GDB recieves commands in XML
>
> Preface
>
> GDB/XMI is meant to be the successor of GDB/MI. It improves upon the
> idea of GDB/MI and fixes many of the known issues that GDB/MI contains.
>
> 1. Introduction
> 1.1 The Problem
>
> Writing a front end for GDB/MI means that a front end has to be able to
> implement an ad hoc parser, capable of understanding all of the data
> sent from GDB to it. For each version of GDB, functionality might change,
> without ever allowing the GUI to understand what the interface differences
> are in a scientific manner. Any guessing from version to version is a
> best-chance scenario, leaving the front end guessing what functionality
> a CVS snapshot has that has been installed on any particular distribution.
>
> 1.2 The Objective
>
> The objective of GDB/XMI is to create a reliable protocol between
> GDB and the front end controlling GDB. A goal of this project is to
> remove most of the parsing from the front end, and move it to a
> reliable parsing library, capable of validating the data it is
> parsing, and building a tree representation of the data without
> any front end infrastructure. It is believed that the protocol between
> GDB and any front end can be automated in every front end and that it
> is not necessary to have any front end developer worry about the syntax
> or grammar of the actual commands sent between GDB and the front end.
>
> Another goal of this project is to formalize the maintenance and
> version management of the XMI commands, as GDB matures, and modifies the
> XMI commands over time. The front end should be independent of the
> XMI command versions, and should be able to determine what data is
> available to it, without worrying about the version of any particular
> XMI command.
>
> It is believed this can help front end writers in several ways. First,
> there will be a significant reduction of time spent in parsing the
> obscure MI commands, and understanding what the output represents.
> This gives front end developers a greater chance of actually getting
> to 1.0 on any given front end, and allowing for much more time to
> be spent on the look & feel of the front end, not including the much
> needed features that most front ends do not have. Secondly, It will
> reduce the code size of each front end, potentially reducing the number
> of bugs, and allowing for a more reliable set of front ends that sit
> on top of GDB.
>
> 2. The GDB/XMI Overview
>
> GDB/XMI is based mostly on the model of GDB/MI. The major differences
> are the need to pass the information from GDB to the front end via XML
> documents and the ability to give the front end a schema, capable
> of validating an XMI command.
>
> 2.1 Passing Information between GDB and the front end
>
> All information passed between GDB and the front end is done in
> the form of an XML document. Each XMI command, generates an XML
> document, and is sent from GDB to the front end.
>
> 2.2 Parsing XMI Commands from GDB's output
>
> XMI is the protocol between GDB and the front end. It is an XML
> based Markup Language that is intended to be the Machine Interface
> between GDB and any application that would like to communicate with it.
>
> An XML document is started and stopped with escape sequences. These
> sequences tell the front end when an XML document is being sent, and
> when the XML document is done being sent. For each XMI command, GDB
> will output an escape sequence saying that it is ready to transmit
> the XML Document response. Then it will output the XML Document. Finally,
> GDB will transmit another escape sequence, saying that the document
> has been completed. This allows the front end to understand what
> information to pass to the XML parser, and what information is output
> from the inferior.
>
> 2.3 Parsing the Inferior's output
>
> When the GDB sends the XML command stating that the inferior has started,
> the front end can then parse the output of GDB waiting for an escape
> sequence, saying that the inferior has stopped running. All of the
> data captured between these two points are output from the inferior.
> It can be processed real time and sent to the user to update them of what
> the inferior is doing. Any asynchronous commands can of course break the
> stream and output data to the front end, and then notify the front end
> that the inferior has started again. This easily allows the front end
> to understand what data is from GDB and what data is from the inferior.
> The only downside to this design is that escape sequences are needed.
> However, the only way to change that is to allow the inferior's output
> to be on a separate communication link (pipe) than the output of GDB.
> So far, this has not happened. Section 5, point 3 illustrates that it
> would be desirable to split GDB's and the inferior's output up, this
> would cause the protocol between GDB and the front end to not include
> the escape sequences before and after each XMI document sent.
>
> 2.4 Validation
> The validation document (DTD) has to be up to date with the XMI
> command for the particular version of GDB. Also, this document has to
> be in one place. Therefore, I propose, that an XMI command be given to
> output to the front end all of the XMI validation documents (DTD?) or
> pass a parameter to have GDB output the validation document for a
> particular XMI command. This document will prove that the XMI commands
> are valid, and GDB is not misbehaving. Front end writers can report
> bad output from GDB easily, and the development process can self-adjust.
>
> The main goal of this tactic is that GDB is capable of telling front ends
> if the output it is generating is correct. Front ends can use this
> capability, at virtually no extra cost, to determine if the protocol
> between the two processes is incorrect. By allowing GDB to keep the
> Schema's, the DRY principle is honored, and things will not get out of
> date.
>
> 2.5 Version Management
>
> From version to version, XMI commands will be modified. Commands will
> be fully backwards compatible, from version to version, allowing only
> fields to be added, or providing more information than is already
> there. The front ends will not have to be updated in this way,
> allowing existing front ends to always be able to use at least the
> amount of information that they have been programmed to understand.
>
> Since it is not a perfect world, it is not always possible to make
> XMI commands backwards compatible. In this case, the XMI command to
> be replaced will be marked as deprecated and a new XMI command will
> be generated to take it's place. This new command will force the
> front end writer to program the functionality of the new command into
> there program, and from then on, will again have a front end capable
> of parsing the new command with a new GDB, and the old command with
> the old GDB.
>
> This model keeps front ends working with both old and new GDB's, which
> is unfortunately a task that most front ends have to and should deal with.
>
> 2.6 Understanding the XML data
>
> Since all commands are backwards compatible, the tree representation
> for any XMI command can be walked the same way, no matter what version
> of the XMI command is being used. The front end is capable of throwing
> away any data that it does not know how or wish to process, allowing
> it to pick and choose what data is appropriate.
>
> 3. GDB/XMI Specification
>
> 3.1 XMI Types
>
> The XMI types are to be used for validation purposes. They can also
> be used to tell the front end what kind of type a particular element
> is that GDB is sending to the front end.
>
> 3.1.1 Scalar Types
>
> <int8> one-byte singed integer
> <uint8> one-byte unsigned integer
> <int16> two-byte signed integer
> <uint16> two-byte unsigned integer
> <int32> four-byte signed integer
> <uint32> four-byte unsigned integer
> <int64> eight-byte signed integer
> <uint64> eight-byte unsigned integer
> <boolean> 0 (false) or 1 (true)
> <string> string
> <float> single-precision signed floating point number
> <double> double-precision signed floating point number
>
> 3.1.2 Compound Types (Structs)
>
> A "struct" is a compound value in which accessor name is the only
> distinction among member values, and no accessor has the same name
> as any other.
>
> <struct_example>
> <element_one>one</element_one>
> <element_two>two</element_two>
> <element_three>three</element_three>
> </struct_example>
>
> A Schema for such a struct could look like this.
>
> <element name="struct_example">
> <complexType>
> <element name="element_one" type="string"/>
> <element name="element_two" type="string"/>
> <element name="element_three" type="string"/>
> </complexType>
> </struct_example>
>
> 3.2 A Breakpoint Listing
>
> 3.2.1 The GDB/MI -break-list Specification
> The -break-list Command
>
> Displays the list of inserted breakpoints, showing the following fields:
>
> `Number'
> number of the breakpoint
> `Type'
> type of the breakpoint: `breakpoint' or `watchpoint'
> `Disposition'
> should the breakpoint be deleted or disabled when it is hit:
> `keep' or `nokeep' `Enabled' is the breakpoint enabled or no:
> `y' or `n'
> `Address'
> memory location at which the breakpoint is set
> `What'
> logical location of the breakpoint, expressed by function name,
> file name, line number
> `Times'
> number of times the breakpoint has been hit
>
> If there are no breakpoints or watchpoints, the BreakpointTable body
> field is an empty list.
>
> 3.2.2 GDB/MI Output
>
> -break-list
> ^done,
> BreakpointTable={
> nr_rows="4",nr_cols="6",
> hdr=[
> {width="3",alignment="-1",col_name="number",colhdr="Num"},
> {width="14",alignment="-1",col_name="type",colhdr="Type"},
> {width="4",alignment="-1",col_name="disp",colhdr="Disp"},
> {width="3",alignment="-1",col_name="enabled",colhdr="Enb"},
> {width="10",alignment="-1",col_name="addr",colhdr="Address"},
> {width="40",alignment="2",col_name="what",colhdr="What"}
> ],
> body=[
> bkpt={
> number="1",type="breakpoint",disp="keep",enabled="y",
> addr="0x0804844f",func="main",file="test.c",line="35",times="1"},
> bkpt={
> number="2",type="breakpoint",disp="keep",enabled="y",
> addr="0x080483ef",func="short_func",file="test.c",line="13",times="0"},
> bkpt={
> number="3",type="breakpoint",disp="keep",enabled="y",
> addr="0x0804851d",func="main",file="test.c",line="61",times="0"},
> bkpt={
> number="4",type="hw watchpoint",disp="keep",enabled="y",addr="",
> what="argc",times="0"}
> ]
> }
>
> 3.2.3 The GDB/XMI Output
>
> <?xml version="1.0"?>
> <breakpoints>
> <breakpoint>
> <number>1</number>
> <type>breakpoint</type>
> <disp>keep</disp>
> <enabled>1</enabled>
> <addr>0x0804844f</addr>
> <func>main</func>
> <file>test.c</file>
> <line>35</line>
> <times>0</times>
> </breakpoint>
> <breakpoint>
> <number>2</number>
> <type>breakpoint</type>
> <disp>keep</disp>
> <enabled>1</enabled>
> <addr>0x080483ef</addr>
> <func>short_func</func>
> <file>test.c</file>
> <line>13</line>
> <times>0</times>
> </breakpoint>
> <breakpoint>
> <number>3</number>
> <type>breakpoint</type>
> <disp>keep</disp>
> <enabled>1</enabled>
> <addr>0x0804851d</addr>
> <func>main</func>
> <file>test.c</file>
> <line>61</line>
> <times>0</times>
> </breakpoint>
> <watchpoint>
> <number>4</number>
> <type>watchpoint</type>
> <disp>keep</disp>
> <enabled>1</enabled>
> <addr></addr>
> <what>argc</what>
> <times>0</times>
> </watchpoint>
> </breakpoints>
>
> 3.3 A Backtrace
>
> 3.3.1 The GDB/MI -stack-list-frames Specification
>
> The -stack-list-frames Command
>
> Synopsis: -stack-list-frames [ low-frame high-frame ]
>
> List the frames currently on the stack. For each frame it displays
> the following info:
>
> `level'
> The frame number, 0 being the topmost frame,
> i.e. the innermost function.
> `addr'
> The $pc value for that frame.
> `func'
> Function name.
> `file'
> File name of the source file where the function lives.
> `line'
> Line number corresponding to the $pc.
>
> If invoked without arguments, this command prints a backtrace for
> the whole stack. If given two integer arguments, it shows the frames
> whose levels are between the two arguments (inclusive). If the two
> arguments are equal, it shows the single frame at the corresponding level.
>
> 3.3.2 GDB/MI Output
>
> (gdb)
> -stack-list-frames
> ^done,stack=[frame={level="0",addr="0x080483ef",func="short_func",file="test.c",line="13"},frame={level="1",addr="0x080484f5",func="main",file="test.c",line="54"}]
>
> ^done,stack=[frame={level="0",addr="0x080483ef",func="short_func",
> file="test.c",line="13"},frame={level="1",addr="0x080484f5",
> func="main",file="test.c",line="54"}]
>
> 3.3.3 The GDB/XMI Output
>
> <?xml version="1.0"?>
> <stack>
>
> <level>0</level>
> <addr>0x080483ef</addr>
> <func>short_func</func>
> <file>test.c</file>
> <line>13</line>
>
>
> <level>1</level>
> <addr>0x080484f5</addr>
> <func>main</func>
> <file>test.c</file>
> <line>54</line>
>
> </stack>
>
> 4. A GDB/XMI Example
>
> 4.1 The Inferior's Code
> #include <stdio.h>
> int main(int argc, char **argv){
> printf ( "newline\n" e;
> printf ( "nonewline" );
> return 0;
> }
>
> 4.2 GDB/MI Output
> (gdb)
> -exec-run
> ^running
> (gdb)
> newline
> nonewline*stopped,reason="exited-normally"
> (gdb)
>
> 4.3 GDB/XMI Output
> Assuming that the escape sequence is simply \027
> The whitespace is not necessary, and is simply there for presentation.
>
> \027<GDB_DISPLAY_PROMPT>
> (gdb)
> </GDB_DISPLAY_PROMPT>\027
> \027<GDB_WAITING_FOR_COMMAND/>\027
> -exec-run
> \027<GDB_INFERIOR_STARTED/>\027
> newline
> nonewline\027<GDB_INFERIOR_FINISHED/>\027
> \027<GDB_DISPLAY_PROMPT>
> gdb
> </GDB_DISPLAY_PROMPT>\027
> \027<GDB_WAITING_FOR_COMMAND/>\027
>
> 4.4 A Brief Description of the XMI output
>
> 4.4.1 GDB_DISPLAY_PROMPT Document
> The front end receives the document below.
> \027<GDB_DISPLAY_PROMPT>
> (gdb)
> </GDB_DISPLAY_PROMPT>\027
> The front end sends the data to the XML processor, which then tells
> it that the current prompt is "(gdb) ". The front end can choose to
> do whatever it wants with this information, including display it to the
> user, or just ignore it.
>
> 4.4.2 GDB_WAITING_FOR_COMMAND Document
> Next, GDB transmits the document below.
> \027<GDB_WAITING_FOR_COMMAND/>\027
> The front end receives the data and sends it to the XML processor. The
> processor parses the document and the front end can determine that
> the command says that GDB is ready to accept a user command. At this
> point, the front end understands that it is OK to transmit a command to
> GDB.
>
> 4.4.3 The Front End Transmits a Command
> The front end transmits the command "-exec-run".
> Should the front end transmit commands in XML or just
> ordinary text?
>
> 4.4.4 GDB_INFERIOR_STARTED Document
> GDB would send a document telling the front end that the inferior
> has begun processing.
> \027<GDB_INFERIOR_STARTED/>\027
> Everything the front end processes from the pipe from this point
> until the next GDB XMI Document transmission is from the inferior.
> It is considered to be output that the inferior would like to have
> the user see. The front end can process this data realtime, since
> it does not have to be sent to the XML parser for processing.
>
> The data
> newline
> nonewline
> is outputted by the inferior, and the front end processes this data
> realtime, understanding that it is not the output of GDB.
>
> 4.4.5 GDB_INFERIOR_FINISHED Document
> GDB would send a document telling the front end that the inferior
> has finished processing.
> \027<GDB_INFERIOR_FINISHED/>\027
>
> This alerts the front end that GDB is now outputting XMI commands
> again. The front end no longer expects information from the inferior.
> Optionally, GDB could output a XMI command stating that a signal
> has been caught, or some other asynchronous event has occurred. This
> could be something like
> \027<GDB_SIGNAL_CAUGHT>
> <SIGNAL>
> <NAME>SIGINT</NAME>
> <NUMBER>2</NUMBER>
> <COMMENT>Interrupt from keyboard</COMMENT>
> </SIGNAL>
> </GDB_SIGNAL_CAUGHT>\027
> In this way, GDB could alert the front end of any asynchronous event,
> and no special parsing mechanism will have to take place to
> understand such actions.
>
> 5. Issues to resolve after RFC Initial approval.
>
> 5.1 If GDB will link to libxml, or just output XML
> It would be nice if GDB was capable of sending the XML documents by
> creating them via libxml. However, libxml2 is licensed under the
> MIT license. Also, I don't know if libxml2 is as portable as GDB.
>
> 5.2 How the schema will represent each document sent to the front end
> If there will be one schema for each XMI document, or a schema
> that represents every XMI document.
>
> 5.3 Separating GDB output and the Inferior output
> If GDB can output it's data on a different pipe than the
> inferior then the escape sequences would not be needed to tell
> the front end which data is from GDB and which is from the inferior.
>
> This would simply the front end processing.
>
> 5.4 If GDB recieves commands in XML
> If GDB links in libxml2, then it could easy receive commands from the
> front end in XML. This would be a nice feature, although, since the
> parser in GDB is only written once, it doesn't really matter what format
> the commands are that is sent to it.