[PATCH V2] dwarf.exp: Allow generating a stub .debug_line section

[PATCH V2] dwarf.exp: Allow generating a stub .debug_line section

[Petr Machata]

Hello,

this is half ping, half V2 of the patch that I sent last week.  I fixed
a bug in generating the header since V1.

This is what I wrote last week, and it all still holds:

like GDB itself, dwgrep uses dwarf.exp to fabricate Dwarfs for its test
cases.  For one test case, I need to create a stub .debug_line section
and reference files defined therein from DW_AT_decl_file.  Hence this
patch, which implements generating barest minimum of .debug_line.

I suspect this could be useful for GDB as well, git grep showed me some
.exp's creating .debug_line by hand, but I didn't really take a close
look.

There were no regressions on x86_64.

Example of use:

	Dwarf::assemble "foo.s" {
	    build_id 0102030405060708

	    cu {is_64 0 version 4 addr_size 8} {
		DW_TAG_compile_unit {
		    {MACRO_AT_stmt_list {
			{include "foo"}
			{file_name "foo.c" 1}
		    }}
		} {
		    DW_TAG_subprogram {
			# We can now reference the source file.
			{DW_AT_decl_file 1 DW_FORM_data1}
		    }
		}
	    }
	}

I do not have commit rights, so if this patch is acceptable, I will need
someone to commit this for me.

Let me know what you think.

Thanks,

--8<------------------------------------------------------------------

Signed-off-by: Petr Machata <pmachata@redhat.com>
---
 gdb/testsuite/ChangeLog     |   6 +++
 gdb/testsuite/lib/dwarf.exp | 112 +++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 117 insertions(+), 1 deletion(-)

diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index 1e36b45..47fc450 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,9 @@
+2015-02-17  Petr Machata  <pmachata@redhat.com>
+
+	* lib/dwarf.exp (Dwarf::_handle_DW_FORM): Handle DW_FORM_sec_offset.
+	(Dwarf::_handle_macro_at_stmt_list): New function.
+	(Dwarf::_handle_DW_TAG): Handle new macro MACRO_AT_stmt_list.
+
 2015-02-17  Jose E. Marchesi  <jose.marchesi@oracle.com>
 
 	* lib/dtrace.exp: New file.
diff --git a/gdb/testsuite/lib/dwarf.exp b/gdb/testsuite/lib/dwarf.exp
index 8ea7a0f..bfcccbc 100644
--- a/gdb/testsuite/lib/dwarf.exp
+++ b/gdb/testsuite/lib/dwarf.exp
@@ -205,6 +205,19 @@ proc function_range { func src } {
 #  - MACRO_AT_func { FUNC FILE }
 #  It is substituted by DW_AT_name with FUNC and MACRO_AT_range.
 #
+#  - MACRO_AT_stmt_list { LIST_OF_DIRECTIVES }
+#  It is substituted by DW_AT_stmt_list with value referencing a
+#  newly-created .debug_line section with data as described by the
+#  directives.  Individual directives can be:
+#
+#    - {include NAME}
+#      defines a new include directory
+#
+#    - {file_name NAME DIRIDX}
+#      defines a new file with NAME, defined in the directory
+#      specified by DIRIDX-th include directive (numbered from 1, with
+#      0 meaning current directory, as specified in Dwarf standard).
+#
 # If FORM is given, it should name a DW_FORM_ constant.
 # This can either be the short form, like 'DW_FORM_addr', or a
 # shortened version, like 'addr'.  If the form is given, VALUE
@@ -422,6 +435,11 @@ namespace eval Dwarf {
 		_op .${size}byte $value
 	    }
 
+	    DW_FORM_sec_offset {
+		variable _cu_offset_size
+		_op .${_cu_offset_size}byte $value
+	    }
+
 	    DW_FORM_ref1 -
 	    DW_FORM_flag -
 	    DW_FORM_data1 {
@@ -494,7 +512,6 @@ namespace eval Dwarf {
 
 	    DW_FORM_ref2 -
 	    DW_FORM_indirect -
-	    DW_FORM_sec_offset -
 	    DW_FORM_exprloc -
 
 	    DW_FORM_GNU_addr_index -
@@ -586,6 +603,97 @@ namespace eval Dwarf {
 	_handle_macro_at_range $attr_value
     }
 
+    # Handle macro attribute MACRO_AT_stmt_list.
+
+    proc _handle_macro_at_stmt_list { attr_value } {
+	variable _cu_count
+
+	set dirlist [list ""]
+	set filelist {}
+	proc include { dirname } {
+	    upvar 1 dirlist dirlist
+	    lappend dirlist $dirname
+	}
+	proc file_name { filename diridx } {
+	    upvar 1 dirlist dirlist
+	    if {![expr $diridx >= 0 && $diridx < [llength $dirlist]]} {
+		error [concat "Directory index out of bounds: $diridx." \
+		       "Use \"include\" to define directories."]
+	    }
+
+	    upvar 1 filelist filelist
+	    lappend filelist [list $filename $diridx 0 0]
+	}
+
+	foreach cmd $attr_value {
+	    {*}$cmd
+	}
+
+	set unit_label [_compute_label "line${_cu_count}"]
+	set unit_len_label [_compute_label "line${_cu_count}_start"]
+	set unit_end_label [_compute_label "line${_cu_count}_end"]
+	set header_len_label [_compute_label "line${_cu_count}_header_start"]
+	set header_end_label [_compute_label "line${_cu_count}_header_end"]
+
+	_defer_output ".debug_line" {
+	    define_label $unit_label
+	    _op .4byte "$unit_end_label - $unit_len_label" "unit_length"
+	    define_label $unit_len_label
+	    _op .2byte 0x2 version
+	    _op .4byte "$header_end_label - $header_len_label" "header_length"
+	    define_label $header_len_label
+	    _op .byte 1 "minimum_instruction_length"
+	    _op .byte 0 "default_is_stmt"
+	    _op .byte 1 "line_base"
+	    _op .byte 1 "line_range"
+	    _op .byte 1 "opcode_base"
+	    # Since we emit opcode_base==1, we skip
+	    # standard_opcode_length table altogether.
+	}
+
+	# Ignore the first dummy entry in DIRLIST when emitting.
+	foreach dirname [lrange $dirlist 1 end] {
+	    _defer_output ".debug_line" {
+		_op .ascii [_quote $dirname] "include"
+	    }
+	}
+
+	_defer_output ".debug_line" {
+	    _op .byte 0 "Terminator."
+	}
+
+	foreach entry $filelist {
+	    set fn [lindex $entry 0]
+	    set diridx [lindex $entry 1]
+	    set mtime [lindex $entry 2]
+	    set length [lindex $entry 3]
+
+	    set dir [lindex $dirlist $diridx]
+	    set full_fn [file join $dir $fn]
+
+	    _defer_output ".debug_line" {
+		_op .ascii [_quote $fn] $full_fn
+		_op .sleb128 $diridx
+		_op .sleb128 $mtime
+		_op .sleb128 $length
+	    }
+	}
+
+	variable _cu_version
+	if {[expr $_cu_version > 3]} {
+	    set form DW_FORM_sec_offset
+	} else {
+	    set form DW_FORM_data4
+	}
+	_handle_attribute DW_AT_stmt_list "$unit_label" $form
+
+	_defer_output ".debug_line" {
+	    _op .byte 0 "Terminator."
+	    define_label $header_end_label
+	    define_label $unit_end_label
+	}
+    }
+
     proc _handle_DW_TAG {tag_name {attrs {}} {children {}}} {
 	variable _abbrev_section
 	variable _abbrev_num
@@ -612,6 +720,8 @@ namespace eval Dwarf {
 		_handle_macro_at_func $attr_value
 	    } elseif { [string equal "MACRO_AT_range" $attr_name] } {
 		_handle_macro_at_range $attr_value
+	    } elseif { [string equal "MACRO_AT_stmt_list" $attr_name] } {
+		_handle_macro_at_stmt_list $attr_value
 	    } else {
 		if {[llength $attr] > 2} {
 		    set attr_form [lindex $attr 2]
-- 
2.1.0

Re: [PATCH V2] dwarf.exp: Allow generating a stub .debug_line section

[Yao Qi]

On 02/23/2015 02:46 PM, Petr Machata wrote:
> This is what I wrote last week, and it all still holds:
> 
> like GDB itself, dwgrep uses dwarf.exp to fabricate Dwarfs for its test
> cases.  For one test case, I need to create a stub .debug_line section
> and reference files defined therein from DW_AT_decl_file.  Hence this
> patch, which implements generating barest minimum of .debug_line.
> 
> I suspect this could be useful for GDB as well, git grep showed me some
> .exp's creating .debug_line by hand, but I didn't really take a close
> look.

Yeah, it is useful to GDB.

> 
> There were no regressions on x86_64.
> 

but the new macro you added isn't used by any test cases.

> Example of use:
> 
> 	Dwarf::assemble "foo.s" {
> 	    build_id 0102030405060708
> 
> 	    cu {is_64 0 version 4 addr_size 8} {
> 		DW_TAG_compile_unit {
> 		    {MACRO_AT_stmt_list {

Is there any reason you decide to implement this attribute as a macro
attribute?  DW_AT_stmt_list attribute itself is quite simple, whose
value is a section offset to the .debug_line section, so we don't need
to bother macro attribute, which doesn't exist in dwarf spec but are
used to generate some dwarf attributes.

I'd like DW_AT_stmt_list implemented in the same way as other
attributes, for example,

  cu {is_64 0 version 4 addr_size 8} {
    compile_unit {
      { stmt_list $label1 sec_offset}
    }
    ...
    label1: line_number_info {
        line_number_program_header {
        }
        line_number_program {
        }
    }
  }

what do you think?

> 			{include "foo"}
> 			{file_name "foo.c" 1}
> 		    }}
> 		} {
> 		    DW_TAG_subprogram {
> 			# We can now reference the source file.
> 			{DW_AT_decl_file 1 DW_FORM_data1}
> 		    }
> 		}
> 	    }
> 	}
> 
> I do not have commit rights, so if this patch is acceptable, I will need
> someone to commit this for me.

Why don't you apply for an account for you?  It is easy to do so
https://sourceware.org/cgi-bin/pdw/ps_form.cgi I (yao.qi@arm.com) could
be the people to approve your request.

> @@ -422,6 +435,11 @@ namespace eval Dwarf {
>  		_op .${size}byte $value
>  	    }
>  
> +	    DW_FORM_sec_offset {
> +		variable _cu_offset_size
> +		_op .${_cu_offset_size}byte $value
> +	    }
> +
>  	    DW_FORM_ref1 -
>  	    DW_FORM_flag -
>  	    DW_FORM_data1 {
> @@ -494,7 +512,6 @@ namespace eval Dwarf {
>  
>  	    DW_FORM_ref2 -
>  	    DW_FORM_indirect -
> -	    DW_FORM_sec_offset -
>  	    DW_FORM_exprloc -

This part looks right to me, and DW_FORM_sec_offset can be used by
other attributes too, such as DW_AT_ranges.

-- 
Yao

Re: [PATCH V2] dwarf.exp: Allow generating a stub .debug_line section

[Petr Machata]

Yao Qi <qiyaoltc@gmail.com> writes:

>> There were no regressions on x86_64.
>
> but the new macro you added isn't used by any test cases.

Yep, that helps in not introducing regressions :) But you never know.

> I'd like DW_AT_stmt_list implemented in the same way as other
> attributes [...]

Makes sense.  I'll try to figure out how to put this together and get
back to you later in the week.

> Why don't you apply for an account for you?

I applied now.

Thank you,
Petr

Re: [PATCH V2] dwarf.exp: Allow generating a stub .debug_line section

[Yao Qi]

Petr Machata <pmachata@redhat.com> writes:

>
>> Why don't you apply for an account for you?
>
> I applied now.

Your account should be approved.

-- 
Yao (齐尧)

[PATCH][V2] dwarf.exp: Allow generating a stub .debug_line section

[Petr Machata]

Hi there,

this is second iteration of my .debug_line support for dwarf.exp DWARF
assembler.  Last time around I implemented it as a macro, and the
request was to make it a first-class section.  This is now implemented.

As before, only minimal support for include and file name lists is
added, so that DW_AT_decl_file references elsewhere in DWARF file work.

Thoughts?

Thanks,
Petr

-- 8< ----------------------------------------------------------------

Example of use:

	Dwarf::assemble "foo.s" {
	    build_id 0102030405060708

	    declare_labels L;
	    cu {is_64 0 version 4 addr_size 8} {
		DW_TAG_compile_unit {
		    {DW_AT_stmt_list $L DW_FORM_sec_offset}
		} {
		    DW_TAG_subprogram {
			# We can now reference the source file.
			{DW_AT_decl_file 1 DW_FORM_data1}
		    }
		}
	    }

	    lines {is_64 0 version 2 addr_size 8} L {
		include_dir "foo"
		include_dir "bar"
		file_name "foo.c" 1
		file_name "bar.c" 1
		file_name "baz.c" 2
	    }
	}

Signed-off-by: Petr Machata <pmachata@redhat.com>
---
 gdb/testsuite/ChangeLog     |   7 +++
 gdb/testsuite/lib/dwarf.exp | 143 +++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 149 insertions(+), 1 deletion(-)

diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index 31f01a0..73796ae 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,10 @@
+2015-03-27  Petr Machata  <pmachata@redhat.com>
+
+	* lib/dwarf.exp (Dwarf::_handle_DW_FORM): Handle DW_FORM_sec_offset.
+	(Dwarf::_line_count, Dwarf::_line_saw_file): New variables.
+	(Dwarf::assemble): Initialize them.
+	(Dwarf::lines): New function.
+
 2015-03-26  Jon Turney  <jon.turney@dronecode.org.uk>
 
 	* lib/gdb.exp (gdb_target_symbol_prefix_flags): Don't set
diff --git a/gdb/testsuite/lib/dwarf.exp b/gdb/testsuite/lib/dwarf.exp
index 19963c4..878c650 100644
--- a/gdb/testsuite/lib/dwarf.exp
+++ b/gdb/testsuite/lib/dwarf.exp
@@ -297,6 +297,12 @@ namespace eval Dwarf {
     # value is the label for that string.
     variable _strings
 
+    # Current .debug_line unit count.
+    variable _line_count
+
+    # Whether a file_name entry was seen.
+    variable _line_saw_file
+
     proc _process_one_constant {name value} {
 	variable _constants
 	variable _AT
@@ -427,6 +433,11 @@ namespace eval Dwarf {
 		_op .${size}byte $value
 	    }
 
+	    DW_FORM_sec_offset {
+		variable _cu_offset_size
+		_op .${_cu_offset_size}byte $value
+	    }
+
 	    DW_FORM_ref1 -
 	    DW_FORM_flag -
 	    DW_FORM_data1 {
@@ -499,7 +510,6 @@ namespace eval Dwarf {
 
 	    DW_FORM_ref2 -
 	    DW_FORM_indirect -
-	    DW_FORM_sec_offset -
 	    DW_FORM_exprloc -
 
 	    DW_FORM_GNU_addr_index -
@@ -1099,6 +1109,132 @@ namespace eval Dwarf {
 	define_label $end_label
     }
 
+    # Emit a DWARF .debug_line unit.
+    # OPTIONS is a list with an even number of elements containing
+    # option-name and option-value pairs.
+    # Current options are:
+    # is_64 0|1    - boolean indicating if you want to emit 64-bit DWARF
+    #                default = 0 (32-bit)
+    # version n    - DWARF version number to emit
+    #                default = 4
+    # addr_size n  - the size of addresses, 32, 64, or default
+    #                default = default
+    #
+    # LABEL is the label of the current unit (which is probably
+    # referenced by a DW_AT_stmt_list), or "" if there is no such
+    # label.
+    #
+    # BODY is Tcl code that emits the parts which make up the body of
+    # the line unit.  It is evaluated in the caller's context.  The
+    # following commands are available for the BODY section:
+    #
+    #   include_dir "dirname" -- adds a new include directory
+    #
+    #   file_name "file.c" idx -- adds a new file name.  IDX is a
+    #   1-based index referencing an include directory or 0 for
+    #   current directory.
+
+    proc lines {options label body} {
+	variable _line_count
+	variable _line_saw_file
+
+	# Establish the defaults.
+	set is_64 0
+	set _unit_version 4
+	set _unit_addr_size default
+
+	foreach { name value } $options {
+	    switch -exact -- $name {
+		is_64 { set is_64 $value }
+		version { set _unit_version $value }
+		addr_size { set _unit_addr_size $value }
+		default { error "unknown option $name" }
+	    }
+	}
+	if {$_unit_addr_size == "default"} {
+	    if {[is_64_target]} {
+		set _unit_addr_size 8
+	    } else {
+		set _unit_addr_size 4
+	    }
+	}
+
+	set unit_num [incr _line_count]
+
+	set section ".debug_line"
+	_section $section
+
+	if { "$label" != "" } {
+	    $label:
+	}
+
+	set unit_len_label [_compute_label "line${_line_count}_start"]
+	set unit_end_label [_compute_label "line${_line_count}_end"]
+	set header_len_label [_compute_label "line${_line_count}_header_start"]
+	set header_end_label [_compute_label "line${_line_count}_header_end"]
+
+	if {$is_64} {
+	    _op .4byte 0xffffffff
+	    _op .8byte "$unit_end_label - $unit_len_label" "unit_length"
+	} else {
+	    _op .4byte "$unit_end_label - $unit_len_label" "unit_length"
+	}
+
+	define_label $unit_len_label
+
+	_op .2byte $_unit_version version
+
+	if {$is_64} {
+	    _op .8byte "$header_end_label - $header_len_label" "header_length"
+	} else {
+	    _op .4byte "$header_end_label - $header_len_label" "header_length"
+	}
+
+	define_label $header_len_label
+
+	_op .byte 1 "minimum_instruction_length"
+	_op .byte 0 "default_is_stmt"
+	_op .byte 1 "line_base"
+	_op .byte 1 "line_range"
+	_op .byte 1 "opcode_base"
+	# Since we emit opcode_base==1, we skip
+	# standard_opcode_length table altogether.
+
+	proc include_dir {dirname} {
+	    _op .ascii [_quote $dirname]
+	}
+
+	proc file_name {filename diridx} {
+	    variable _line_saw_file
+	    if "! $_line_saw_file" {
+		# Terminate the dir list.
+		_op .byte 0 "Terminator."
+		set _line_saw_file 1
+	    }
+
+	    _op .ascii [_quote $filename]
+	    _op .sleb128 $diridx
+	    _op .sleb128 0 "mtime"
+	    _op .sleb128 0 "length"
+	}
+
+	uplevel $body
+
+	rename include_dir ""
+	rename file_name ""
+
+	# Terminate dir list if we saw no files.
+	if "! $_line_saw_file" {
+	    _op .byte 0 "Terminator."
+	}
+
+	# Terminate the file list.
+	_op .byte 0 "Terminator."
+
+	define_label $header_end_label
+	define_label $unit_end_label
+    }
+
     proc _empty_array {name} {
 	upvar $name the_array
 
@@ -1178,6 +1314,8 @@ namespace eval Dwarf {
 	variable _label_num
 	variable _strings
 	variable _cu_count
+	variable _line_count
+	variable _line_saw_file
 
 	if {!$_initialized} {
 	    _read_constants
@@ -1191,6 +1329,9 @@ namespace eval Dwarf {
 	set _label_num 0
 	_empty_array _strings
 
+	set _line_count 0
+	set _line_saw_file 0
+
 	# Not "uplevel" here, because we want to evaluate in this
 	# namespace.  This is somewhat bad because it means we can't
 	# readily refer to outer variables.
-- 
2.1.0

Re: [PATCH][V2] dwarf.exp: Allow generating a stub .debug_line section

[Doug Evans]

Petr Machata writes:
 > Hi there,
 > 
 > this is second iteration of my .debug_line support for dwarf.exp DWARF
 > assembler.  Last time around I implemented it as a macro, and the
 > request was to make it a first-class section.  This is now implemented.
 > 
 > As before, only minimal support for include and file name lists is
 > added, so that DW_AT_decl_file references elsewhere in DWARF file work.
 > 
 > Thoughts?
 > 
 > Thanks,
 > Petr

Hi.

Just one nit.
Otherwise LGTM.

 > +	if { "$label" != "" } {
 > +	    $label:
 > +	}

Should this be:

	if { "$label" != "" } {
	    _emit "$label:"
	}

?

Re: [PATCH][V2] dwarf.exp: Allow generating a stub .debug_line section

[Petr Machata]

Doug Evans <dje@google.com> writes:

>  > +	if { "$label" != "" } {
>  > +	    $label:
>  > +	}
>
> Should this be:
>
> 	if { "$label" != "" } {
> 	    _emit "$label:"
> 	}

No, it's calling the function named "$label:".  To define the label that
the user supplied.  Would this make it clearer?

--- a/gdb/testsuite/lib/dwarf.exp
+++ b/gdb/testsuite/lib/dwarf.exp
@@ -1165,6 +1165,7 @@ namespace eval Dwarf {
 	_section $section
 
 	if { "$label" != "" } {
+	    # Define the user-provided label at this point.
 	    $label:
 	}
 

Thanks,
Petr

Re: [PATCH][V2] dwarf.exp: Allow generating a stub .debug_line section

[Doug Evans]

On Mon, Mar 30, 2015 at 12:48 PM, Petr Machata <pmachata@redhat.com> wrote:
> Doug Evans <dje@google.com> writes:
>
>>  > +  if { "$label" != "" } {
>>  > +      $label:
>>  > +  }
>>
>> Should this be:
>>
>>       if { "$label" != "" } {
>>           _emit "$label:"
>>       }
>
> No, it's calling the function named "$label:".  To define the label that
> the user supplied.  Would this make it clearer?
>
> --- a/gdb/testsuite/lib/dwarf.exp
> +++ b/gdb/testsuite/lib/dwarf.exp
> @@ -1165,6 +1165,7 @@ namespace eval Dwarf {
>         _section $section
>
>         if { "$label" != "" } {
> +           # Define the user-provided label at this point.
>             $label:
>         }
>
>
> Thanks,
> Petr

That helps enough.
Thanks.

Re: [PATCH][V2] dwarf.exp: Allow generating a stub .debug_line section

[Petr Machata]

Doug Evans <dje@google.com> writes:

> Just one nit.
> Otherwise LGTM.

Pushed with that clarifying comment.

Thanks,
Petr