[PATCH] Fixup internal variables' endian (again)

[Andrew STUBBS <andrew.stubbs@st.com>]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

Hi all,

This is a rework of a patch I submitted last autumn. I have updated it 
to fit with the new convenience variable preservation and rethought the 
requirements.

The patch fixes up the endian of all integer and pointer internal 
variables and leaves all other types alone.

My understanding is that an internal variable cannot contain anything 
other than a built-in type. Any other value is merely a memory 
reference. Indeed, value_of_internalvar() goes out of it's way to ensure 
that values loaded from memory are never saved long term. The result is 
that there is no point in attempting to do anything with the endian of 
these values because they will always be of the same endian as the 
target/program being debugged.

The upshot is that, although it is possible to contrive an example [1] 
that causes gdb to print values in the wrong endian, in the normal case, 
in which the user does not change the endian except upon connection to 
the target, the values will always be presented as they expect. 
Including internal variables written prior to connection.

[1] Once connected to the target, write data in endian A, create a 
variable referencing that data, type 'set endian B' and the print the 
data - it comes out reversed.

If I have something wrong here please point it out.

Thanks

Andrew Stubbs

[-- Attachment #2: endian.patch --]
[-- Type: text/plain, Size: 3380 bytes --]

2006-03-03  Andrew Stubbs  <andrew.stubbs@st.com>

	* value.h (struct internalvar): Add field 'endian'.
	* value.c (lookup_internalvar): Initialise endian.
	(value_of_internalvar): Flip the endian of built-in types if required.
	(set_internalvar): Set the endian.
	(show_convenience): Access the value through value_of_internalvar().


Index: src/gdb/value.c
===================================================================
--- src.orig/gdb/value.c	2006-03-03 17:16:03.000000000 +0000
+++ src/gdb/value.c	2006-03-03 17:16:20.000000000 +0000
@@ -755,6 +755,7 @@ lookup_internalvar (char *name)
   var = (struct internalvar *) xmalloc (sizeof (struct internalvar));
   var->name = concat (name, (char *)NULL);
   var->value = allocate_value (builtin_type_void);
+  var->endian = TARGET_BYTE_ORDER;
   release_value (var->value);
   var->next = internalvars;
   internalvars = var;
@@ -765,12 +766,49 @@ struct value *
 value_of_internalvar (struct internalvar *var)
 {
   struct value *val;
+  int i, j;
+  gdb_byte temp;
+  gdb_byte *array;
 
   val = value_copy (var->value);
   if (value_lazy (val))
     value_fetch_lazy (val);
   VALUE_LVAL (val) = lval_internalvar;
   VALUE_INTERNALVAR (val) = var;
+
+  /* Values are always stored in the target's byte order.  When connected to a
+     target this will most likely always be correct, so there's normally no
+     need to worry about it.
+
+     However, internal variables can be set up before the target endian is
+     known and so may become out of date.  Fix it up before anybody sees.
+
+     Since internal variables cannot hold complex types, such as structs,
+     unions, arrays and strings - those are always held in target memory and
+     the variable just holds a reference, there is no need to worry about
+     those either.
+
+     Floating point values vary differently across endianness - many targets
+     just keep them the same.  If they do no work correctly on your host/target
+     then add support as required here.  */
+  if (var->endian != TARGET_BYTE_ORDER)
+    {
+      array = value_contents_raw (val);
+      switch (TYPE_CODE (value_type (val)))
+        {
+        case TYPE_CODE_INT:
+        case TYPE_CODE_PTR:
+          /* Reverse the bytes.  */
+          for (i=0,j=TYPE_LENGTH (value_enclosing_type (val))-1; i<j; i++,j--)
+	    {
+	      temp = array[j];
+	      array[j] = array[i];
+	      array[i] = temp;
+	    }
+	  break;
+	}
+    }
+
   return val;
 }
 
@@ -809,6 +847,7 @@ set_internalvar (struct internalvar *var
      long.  */
   xfree (var->value);
   var->value = newval;
+  var->endian = TARGET_BYTE_ORDER;
   release_value (newval);
   /* End code which must not call error().  */
 }
@@ -877,7 +916,7 @@ show_convenience (char *ignore, int from
 	  varseen = 1;
 	}
       printf_filtered (("$%s = "), var->name);
-      value_print (var->value, gdb_stdout, 0, Val_pretty_default);
+      value_print (value_of_internalvar (var), gdb_stdout, 0, Val_pretty_default);
       printf_filtered (("\n"));
     }
   if (!varseen)
Index: src/gdb/value.h
===================================================================
--- src.orig/gdb/value.h	2006-02-20 18:39:55.000000000 +0000
+++ src/gdb/value.h	2006-03-03 15:11:42.000000000 +0000
@@ -245,6 +245,7 @@ struct internalvar
   struct internalvar *next;
   char *name;
   struct value *value;
+  int endian;
 };
 
 \f