Re: Final(?) patch to update libtool in GCC and src trees

Charles Wilson
2007-04-12T06:36:57Z

[-- Attachment #1: Type: text/plain, Size: 8057 bytes --]

Charles Wilson wrote:
> Steve Ellcey wrote:
>>> I appreciate the problem, but Cygwin can be a bit 'special' when it
>>> comes to
>>> libtool; if it hasn't been done yet, can you possibly bear to hold
>>> off a bit
>>> while I give it some hasty testing? (By 'a bit', I mean no more than
>>> say
>>> 'overnight'). Sorry to be a pain :-/
>>
>> I can wait, especially since no one has given me the go ahead yet. :-)
>> I know of no cygwin testing that has been done.
>
> I'll rip that failed experiment out of my tree and try again with just
> Steve's patches; however I'd imagine Dave Korn will have results before
> I do.

Well, I ran into some trouble, related to the new libtool. I'm building
in a combined tree, with binutils and gcc, but not winsup or newlib.
(configuration args at end of post). All of the binutils stuff built
fine (non-shared, natch). The error is this, when I get into building
libgcc (stage1):

configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details.
make[2]: *** [configure-stage1-target-libgcc] Error 1
make[1]: *** [stage1-bubble] Error 2
make: *** [bootstrap] Error 2

In _build/i686-pc-cygwin/libgcc, config.log shows:

configure:2587: /usr/local/src/gcc/_build/./gcc/xgcc
-B/usr/local/src/gcc/_build
/./gcc/ -B/opt/i686-pc-cygwin/bin/ -B/opt/i686-pc-cygwin/lib/ -isystem
/opt/i686
-pc-cygwin/include -isystem /opt/i686-pc-cygwin/sys-include
-L/usr/local/src/gcc
/_build/./ld -c -O2 -g -O2 conftest.c >&5
/bin/sh: /usr/local/src/gcc/_build/./gcc/as: No such file or directory

Taking a look in _build/./gcc, ls -l (for selected files) shows:

-rwxr-xr-x 2 cwilson None 5632 Apr 11 03:57 as.exe*
-rwxr-xr-x 2 cwilson None 5632 Apr 11 03:58 collect-ld.exe*
-rwxr-xr-x 2 cwilson None 5632 Apr 11 03:55 nm.exe*

These .exe files are all hardlinks (I'm using cygwin on NTFS, where
hardlinks work) to *wrapper* executables (here's the relevant section
from stdout during the build):

creating as
ln ../gas/as-new.exe as.exe
echo timestamp > stamp-as
creating collect-ld
ln ../ld/ld-new.exe collect-ld.exe
echo timestamp > stamp-collect-ld
creating nm
ln ../binutils/nm-new.exe nm.exe
echo timestamp > stamp-nm

This is wrong. libtool-ToT (and libtool-1.5) on cygwin and mingw, when
shared libraries are "in the mix", use the following scheme:

(1) a wrapper script (./foo) is used in the "normal" libtool way to set
$PATH so that just-built .dll's can be located by the wrapped
executable. It will invoke the actual executable (./.libs/foo.exe)

(2) a wrapper executable (./foo.exe) whose sole purpose is to ensure
that 'make' is happy. New automake-derived Makefile.in's (and thus,
Makefile's) have program targets like 'foo$(EXEEXT):'. If only 'foo'
exists, then 'make' thinks that the target wasn't built, and continually
relinks executables...over and over and over. So, we arrange that
libtool produces a wrapper executable in . with the correct name, so
that 'make' will know that the target has been satisfied. The wrapper
executable then runs the wrapper script via 'execv("$SHELL",newargz);'
where newargz is a copy of all args to the wrapper script, with [0] set
to the path ${thisdir}/foo (e.g. the wrapper /script/).

(3) ./.libs/foo.exe is the "actual" executable.

Thus, on mingw/cygwin, the "top-level" wrapper is ./foo.exe which calls
./foo which calls ./.libs/foo.exe

On other platforms, the "top-level" wrapper is ./foo (a script), which
calls .libs/foo (an executable).

But there's still a problem (and this may affect other platforms;
*should* do so, I think -- so I'm confused by Steve's success here;
Steve: are you building the src/ stuff separately from the gcc stuff, or
are you using a combined tree like I am? I think the issue I am seeing
will affect all platforms, but only if doing a combined build AND when
--disable-shared is NOT explicitly passed during configure).

Current code in gcc/Makefile.in is

stamp-as: $(ORIGINAL_AS_FOR_TARGET)
@echo creating as; \
case "$(ORIGINAL_AS_FOR_TARGET)" in \
./as) ;; \
../*) \
rm -f as$(exeext); \
echo $(LN) $< as$(exeext); \
$(LN) $< as$(exeext) || cp $< as$(exeext) ;; \
*) \
rm -f as; \
echo '#!$(SHELL)' > as; \
echo 'exec $(ORIGINAL_AS_FOR_TARGET) "$$@"' >> as ; \
chmod +x as ;; \
esac
echo timestamp > $@

The source code for the wrapper executable (if you're curious, it is
embedded in ltmain.sh), says:

This wrapper executable should never be moved out of the build
directory. If it is, it will not operate correctly.

Similarly, wrapper scripts say the following:

# This wrapper script should never be moved out of the build directory.
# If it is, it will not operate correctly.

They can be exec'ed via a relative PATH, but you can't just copy (or
hardlink) the wrapper to a different directory, and invoke that
copy/hardlink -- because it will then look in the wrong place for
.libs/<actual-executable> (non-cygwin) or ./<wrapper-script>
(cygwin/mingw). You *can* create a symlink to the wrapper /script/, and
it will work properly (it walks its own symlink path back to the "real"
script, before computing the directory in which to look for the target
executable). However, the wrapper /executable/ does not have this
ability -- so symlinking to the wrapper executable is a no-go.

However, we can't use symlinks anyway, because (according to
gcc/Makefile.in):

# We need hard links so that directories can be shuffled
# during toplevel bootstrap.

So, I'm not really sure how this worked, even on "normal"
platforms...unless Steve wasn't doing a top-level (combined-tree) bootstrap.

=======

It's a little unorthodox (and assumes a lot about libtool's
implementation) but we COULD:

(1) hardlink/copy ${ORIGINAL_AS_FOR_TARGET--without-$exeext} .
(2) hardlink/copy ${ORIGINAL_AS_FOR_TARGET--with_$exeext} .
(3) mkdir .libs
(4) hardlink/copy ${ORIGINAL_AS_FOR_TARGET--with-/.libs/-inserted} .libs/

all with appropriate existence checks. And since (e.g) ld-new gets
renamed to collect-ld, you'd have to rename (both) wrappers, similarly
-- but NOT rename the .libs/ inhabitant. (This is because the
executable wrapper inspects argv[0] to figure out how to invoke the
script wrapper. The script wrapper doesn't care what its own name is,
it just "knows" what its target in .libs/ *should* be named.)

But what a mess. I've attached a preliminary patch for gcc/Makefile.in.
It works for native/cygwin -- that is, it gets me past THAT problem --
but some more testing for other platforms and crosses is definitely in
order...

Obviously this patch depends on Steve's new libtool stuff.

2007-04-11 Charles Wilson <...>

* gcc/Makefile.in [stamp-as]: handle libtool wrappers when
copying binutils applications during combined-tree build.
[stamp-collect-ld]: ditto
[stamp-nm]: ditto

--
Chuck

configure options:
${RELSRCDIR}/configure \
--prefix=/opt \
--exec-prefix=/opt \
--sysconfdir=/opt/etc \
--libdir=/opt/lib \
--libexecdir=/opt/lib \
--mandir=/opt/share/man \
--infodir=/opt/share/info \
--with-datarootdir=/opt/share \
--enable-languages=c,c++,objc,fortran \
--with-gcc \
--enable-nls \
--without-included-gettext \
--enable-version-specific-runtime-libs \
--without-x \
--with-system-zlib \
--enable-threads=posix \
--disable-win32-registry \
--disable-sjlj-exceptions \ [*]
--enable-libstdcxx-debug \
--enable-cxx-flags='-fno-function-sections -fno-data-sections' \
--enable-libgomp \
--with-arch=i486 \
--with-tune=i686 \
--disable-werror

[*] oh yeah, I've also got Danny Smith's dwarf2 exceptions patch
(20061203) in my tree.

And that's my next roadblock: the dwarf2 patch adds crtbegin.o/crtend.o
to cygwin's build, and I get:
../../combined_src/gcc/crtstuff.c:445:2: error: #error "What are you
doing with crtstuff.c, then?"

Sigh.

[-- Attachment #2: gcc-Makefile.in-handle-libtool-wrappers-csw20070411.patch --]
[-- Type: text/plain, Size: 3343 bytes --]

Index: Makefile.in
===================================================================
--- Makefile.in (revision 123710)
+++ Makefile.in (working copy)
@@ -1478,8 +1478,31 @@
./as) ;; \
../*) \
rm -f as$(exeext); \
+ if [ -e as ]; then \
+ rm -f as ;\
+ fi ;\
+ if [ -e .libs/as$(exeext) ]; then \
+ rm -f .libs/as$(exeext) ;\
+ fi ;\
echo $(LN) $< as$(exeext); \
- $(LN) $< as$(exeext) || cp $< as$(exeext) ;; \
+ $(LN) $< as$(exeext) || cp $< as$(exeext) ;\
+ if [ -n "$(exeext)" ]; then \
+ if [ -e $(<:%$(exeext)=%) ]; then \
+ echo $(LN) $(<:%$(exeext)=%) as ;\
+ $(LN) $(<:%$(exeext)=%) as ||\
+ cp $(<:%$(exeext)=%) as ;\
+ fi ;\
+ fi ;\
+ if [ -e $(<D)/.libs ]; then \
+ if [ ! -e .libs ]; then \
+ mkdir .libs ;\
+ fi ;\
+ if [ -e $(<D)/.libs/$(<F) ]; then \
+ echo $(LN) $(<D)/.libs/$(<F) .libs/$(<F);\
+ $(LN) $(<D)/.libs/$(<F) .libs/$(<F) ||\
+ cp $(<D)/.libs/$(<F) .libs/$(<F) ;\
+ fi ;\
+ fi ;; \
*) \
rm -f as; \
echo '#!$(SHELL)' > as; \
@@ -1494,8 +1517,31 @@
./collect-ld) ;; \
../*) \
rm -f collect-ld$(exeext); \
+ if [ -e collect-ld ]; then \
+ rm -f collect-ld ;\
+ fi ;\
+ if [ -e .libs/collect-ld$(exeext) ]; then \
+ rm -f .libs/collect-ld$(exeext) ;\
+ fi ;\
echo $(LN) $< collect-ld$(exeext); \
- $(LN) $< collect-ld$(exeext) || cp $< collect-ld$(exeext) ;; \
+ $(LN) $< collect-ld$(exeext) || cp $< collect-ld$(exeext) ;\
+ if [ -n "$(exeext)" ]; then \
+ if [ -e $(<:%$(exeext)=%) ]; then \
+ echo $(LN) $(<:%$(exeext)=%) collect-ld ;\
+ $(LN) $(<:%$(exeext)=%) collect-ld ||\
+ cp $(<:%$(exeext)=%) collect-ld ;\
+ fi ;\
+ fi ;\
+ if [ -e $(<D)/.libs ]; then \
+ if [ ! -e .libs ]; then \
+ mkdir .libs ;\
+ fi ;\
+ if [ -e $(<D)/.libs/$(<F) ]; then \
+ echo $(LN) $(<D)/.libs/$(<F) .libs/$(<F);\
+ $(LN) $(<D)/.libs/$(<F) .libs/$(<F) ||\
+ cp $(<D)/.libs/$(<F) .libs/$(<F) ;\
+ fi ;\
+ fi ;; \
*) \
rm -f collect-ld$(exeext); \
echo '#!$(SHELL)' > collect-ld; \
@@ -1510,8 +1556,31 @@
./nm) ;; \
../*) \
rm -f nm$(exeext); \
+ if [ -e nm ]; then \
+ rm -f nm ;\
+ fi ;\
+ if [ -e .libs/nm$(exeext) ]; then \
+ rm -f .libs/nm$(exeext) ;\
+ fi ;\
echo $(LN) $< nm$(exeext); \
- $(LN) $< nm$(exeext) || cp $< nm$(exeext) ;; \
+ $(LN) $< nm$(exeext) || cp $< nm$(exeext) ;\
+ if [ -n "$(exeext)" ]; then \
+ if [ -e $(<:%$(exeext)=%) ]; then \
+ echo $(LN) $(<:%$(exeext)=%) nm ;\
+ $(LN) $(<:%$(exeext)=%) nm ||\
+ cp $(<:%$(exeext)=%) nm ;\
+ fi ;\
+ fi ;\
+ if [ -e $(<D)/.libs ]; then \
+ if [ ! -e .libs ]; then \
+ mkdir .libs ;\
+ fi ;\
+ if [ -e $(<D)/.libs/$(<F) ]; then \
+ echo $(LN) $(<D)/.libs/$(<F) .libs/$(<F);\
+ $(LN) $(<D)/.libs/$(<F) .libs/$(<F) ||\
+ cp $(<D)/.libs/$(<F) .libs/$(<F) ;\
+ fi ;\
+ fi ;; \
*) \
rm -f nm$(exeext); \
echo '#!$(SHELL)' > nm; \