64-bit (>4GB) inferior data types rules; TYPE_LENGTH: unsigned -> ULONGEST

[Jan Kratochvil <jan.kratochvil@redhat.com>]

Hello,

original problem - GCC produces working >4GB data types code while GDB cannot
handle them:
	[PATCH] Expand bitpos to LONGEST to allow access to large offsets within a struct
	http://sourceware.org/ml/gdb-patches/2012-02/msg00403.html

There should be checked in soon thanks to Siddhesh Poyarekar an extension of
TYPE_LENGTH from 'unsigned int' to ULONGEST:
	[PATCH 0/4] bitpos expansion summary reloaded
	http://sourceware.org/ml/gdb-patches/2012-09/msg00628.html

Anything checked-in since the following commit would be great to no longer
break this goal to handle it correctly:
	http://sourceware.org/ml/gdb-cvs/2012-09/msg00161.html
	commit cec90d9d386f57f116e114c50e4287281420f531
	Author: siddhesh <siddhesh>
	Date:   Thu Sep 27 10:39:58 2012 +0000
		* amd64-tdep.c (amd64_return_value): Revert previous change
		that used TYPE_LENGTH directly.
	[...]
		* vax-tdep.c (vax_return_value): Likewise.

So far GDB was freely using 'int' for any inferior sizes.  This is no longer
compatible, use always wide enough types:

(1) signed vs. unsigned types do not matter for inferior data types sizes.
    Such as LONGEST vs. ULONGEST or ssize_t vs. size_t.

    GDB will never successfully allocate more than 2GB on 32-bit host (size_t
    max larger than ssize_t).  And it does not make sense to consider anyhow
    sizes above 9 quintillion (ULONGEST max larger than LONGEST on any host or
    size_t larger than ssize_t on 64-bit hosts).

(2) Use the smallest valid type, therefore use size_t/ssize_t for objects
    present in host GDB memory - not LONGEST/ULONGEST.  On 32-bit hosts it
    cheaper and also IMO the code is more readable with appropriate types.
    BTW for size_t/ssize_t format string is "%z".

    Also use CORE_ADDR where appropriate, GDB was also using sometimes
    incorrectly LONGEST/ULONGEST instead.
    BTW for CORE_ADDR format string is "%s", paddress (gdbarch, X).

    BTW for LONGEST/ULONGEST format string
    is "%s", plongest (X) or "%s", pulongest (X) accordingly.

    Never use long / unsigned long (it is host platform dependent).

(3) For allocating inferior data in host GDB memory, therefore assigning
    LONGEST/ULONGEST (type of TYPE_LENGTH) to size_t/ssize_t variable:
    One needs to manually check the value fits in.
    Call ulongest_fits_host_or_error for that.

    It will ensure the size <= SIZE_MAX / 8 so even small additions after the
    check will not overflow the type before it gets passed to xmalloc.

(4) Copying >2GB inferior data to host memory is probably unreal.  In reality
    GDB should copy the data on-demand.  But that is a future patch outside of
    the scope of this patchset.
    
    IIUC such patch will be at least easier to write when the code already
    knows the valid 64-bit size of the inferior object.  Still it is currently
    more correct to "lock up" GDB (interruptible by CTRL-C) than to respond to
    user with invalid/shortened data without printing any error.

(5) Patched GDB expects only these inferior types may have >2GB size:
    	TYPE_CODE_ARRAY, TYPE_CODE_STRUCT, TYPE_CODE_UNION
    And of course where applicable:
    	TYPE_CODE_TYPEDEF
    While a compiler may create
    	 <1><57>: Abbrev Number: 3 (DW_TAG_base_type)
    	    <58>   DW_AT_byte_size   : 0x123456789
    	    <59>   DW_AT_encoding    : 5        (signed)
    	    <5a>   DW_AT_name        : veryhugeint
    it is OK GDB will not internally handle it correctly, not giving any error.
    
    There could be a new patch to reject such types in dwarf2read.c on read-in.

(6) If a function is called only for small inferior types (scalars) then it is
    OK to have there / keep there 'int' type for the inferior type size.
    Callers have to ensure array/struct/union type cannot be passed there.

    If a function does
    	gdb_assert (length < 16);
    then sure it should not happen - it is an assert.  But still one needs to
    keep the 'length' parameter as LONGEST/ULONGEST so that the assert makes
    sense.


Thanks,
Jan