From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id eMChAjxkBGYYhhgAWB0awg (envelope-from ) for ; Wed, 27 Mar 2024 14:23:56 -0400 Authentication-Results: simark.ca; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=q4YkyJpT; dkim-atps=neutral Received: by simark.ca (Postfix, from userid 112) id 057F71E0C0; Wed, 27 Mar 2024 14:23:56 -0400 (EDT) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id E49C11E030 for ; Wed, 27 Mar 2024 14:23:53 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 762A2385E454 for ; Wed, 27 Mar 2024 18:23:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 762A2385E454 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1711563833; bh=8imOb8kBbkV5v8I/WBRCTQFaVropbB2W4KSh8FOlZjo=; h=References:In-Reply-To:Date:Subject:To:Cc:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=q4YkyJpT4OD7IXunVTPJnv9X8Ybw7LoqTepnpr127erd1DWcX09QFQZvlITC85jxS gS08uZoTASD+Tt7yJDSI1ZlTxRafeo2sgC+9OI1b1UU/JTEewdEygVqqptAMYGS7JF hqdfp9tZBO0gXCkaL4AeU9rnm+J4UPTM3H2hEtT0= Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by sourceware.org (Postfix) with ESMTPS id 2146D3858C98 for ; Wed, 27 Mar 2024 18:23:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2146D3858C98 ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2146D3858C98 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1711563790; cv=none; b=BqKlyneDTOWyzJszK1tg26jerSPrWsz6zf7N15Ga5ydaglkaEltmLLKQEOEnAqX6m2a7TkfTTmUjfX6Wbm1xvI+nkVtB0jWK5+zRhr9pUTb8AC2wHWme36EbJumZoM5IpAC5nHi+OsSVeKkHaCLFieOg+LHcyyxKtFFgc+Xq4zY= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1711563790; c=relaxed/simple; bh=/2je2jgqoggenpkIJvidhEiCAfkFjkR099ZHsqyL3KA=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=P+dTUU3uXGoqgyFk+fu6ZFqKp1R9q70pdiVpGYy3FBOKe02vizQEfO/aXvWE715glUpVva5UqX+8FZ5gZZHbudHs6fFkoVV1h20Ad+O/qQnJzwoKC5DiZuHR6NASQrFNsIiloQwWDOKiDs3+DBQ6r7OjNvI8M9HX2zN+j7vRG5I= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-a466fc8fcccso17014566b.1 for ; Wed, 27 Mar 2024 11:23:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711563787; x=1712168587; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8imOb8kBbkV5v8I/WBRCTQFaVropbB2W4KSh8FOlZjo=; b=pzjhCPU/kl8TNyh5a2wrZfP+x3l4BwQxrmEFSe9FxePVhUE9LLb21GrfaVOYBI8LZe sgZPkngVaKeXDW0RwNh70I/kQT/2HeT/w1rNerjfc3YjuFd/gkYOdPySjObcyYBSVDOt M9XYreoQ/IySCvysEwxqvszCgTKN0oLspGkhTaiu00ptT4eY2NVhg7ZUQXpwreYkoOcK cPodnrgtV8Zvl99VWIpJwwr9NWKWdLtdreFNTgNpW76A7zI4EYWPbpZ0YKqI6Yw+uTxJ 5qUPP9pkrzSn23st+AusuSssQHQhapIfFPanZGSUGTCa+sS+J2c6Z/+sWzmZXs9H3RZU bQVg== X-Forwarded-Encrypted: i=1; AJvYcCXqSB78KKWaK1zEovpHfk8jre4+4Os1HBqFmv1Q1pXUp3Fg8dQ/X/4goutJxJ/bl8ZKCjBKtfK2ABEhxvHa9NF/7Qo= X-Gm-Message-State: AOJu0YwCBuw+vqh+ue9b9xftgyTsEx7lHjk01D3l1X7gN4wy4UkuUmgG OTkMbd47KbH996sWVtHX0t3fmMLOf1yIulkbWJ9O5n4aJ8NKX3lMIRtMq0U371bx6lkRuEEkuNK cbqpolWpBLX1v34QZvc387gdkSgGRCnYgErIaTw== X-Google-Smtp-Source: AGHT+IGhpHaZd4tza3YZsx9CNboBpG0Sy0xEXuWT0fJwReRsWLO0CEyhbgPCnsk+BhhSR51K6yTpWdket467HZPK3Z0= X-Received: by 2002:a17:906:80c7:b0:a47:4162:766e with SMTP id a7-20020a17090680c700b00a474162766emr237141ejx.22.1711563786739; Wed, 27 Mar 2024 11:23:06 -0700 (PDT) MIME-Version: 1.0 References: <20240313080237.1143034-1-christophe.lyon@linaro.org> <1eb529f2-3842-4090-a8e2-f713a28f2394@simark.ca> <33f72ef7-f990-437d-8fc4-dba08d7db24b@simark.ca> In-Reply-To: Date: Wed, 27 Mar 2024 19:22:57 +0100 Message-ID: Subject: Re: [RFC] add regenerate Makefile target To: Simon Marchi Cc: binutils@sourceware.org, gdb@sourceware.org, gcc@gcc.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Christophe Lyon via Gdb Reply-To: Christophe Lyon Errors-To: gdb-bounces+public-inbox=simark.ca@sourceware.org Sender: "Gdb" Hi! On Mon, 25 Mar 2024 at 15:19, Christophe Lyon wrote: > > On Thu, 21 Mar 2024 at 15:32, Christophe Lyon > wrote: > > > > On Wed, 20 Mar 2024 at 16:34, Simon Marchi wrote: > > > > > > On 3/18/24 13:25, Christophe Lyon wrote: > > > > Well the rule to regenerate Makefile.in (eg in in opcodes/) is a bit > > > > more complex > > > > than just calling automake. IIUC it calls automake --foreign it any of > > > > *.m4 file from $(am__configure_deps) that is newer than Makefile.in > > > > (with an early exit in the loop), does nothing if Makefile.am or > > > > doc/local.mk are newer than Makefile.in, and then calls 'automake > > > > --foreign Makefile' > > > > > > The rules looks complex because they've been generated by automake, this > > > Makefile.in is not written by hand. And I guess automake has put > > > `--foreign` there because foreign is used in Makefile.am: > > Yes, I know :-) > > > > > > > > AUTOMAKE_OPTIONS = foreign no-dist > > > > > > But a simple call so `automake -f` (or `autoreconf -f`) just works, as > > > automake picks up the foreign option from AUTOMAKE_OPTIONS, so a human > > > or an external script who wants to regenerate things would probably just > > > use that. > > > > Indeed. I guess my concern is: if some change happens to > > Makefile.am/Makefile.in which would imply that 'autoreconf -f' would > > not work, how do we make sure autoregen.py (or whatever script) is > > updated accordingly? Or maybe whatever change is made to > > Makefile.am/Makefile.in, 'autoreconf -f' is supposed to handle it > > without additional flag? > > > I think I've just noticed a variant of this: if you look at > opcodes/Makefile.in, you can see that aclocal.m4 depends on > configure.ac (among others). So if configure.ac is updated, a > maintainer-mode rule in Makefile.in will call aclocal and regenerate > aclocal.m4. > > However, autoregen.py calls aclocal only if configure.ac contains > AC_CONFIG_MACRO_DIRS, which is not the case here. > > That's probably a bug in opcode/configure.ac, but still the current > Makefile.in machinery would update aclocal.m4 as needed when > autoregen.py will not. > > I haven't audited all configure.ac but there are probably other > occurrences of this. > As another follow-up on this topic, while working on a tentative GCC patch to implement this, I realized an obvious issue: all target libraries configure steps depend on 'all-gcc' (of course, we need a compiler to build the libs...) So they idea of doing roughly: - configure --enable-maintainer-mode - make regenerate -j1 (to avoid current race conditions in maintainer-mode) - make all -jXXX means that the regenerate step will trigger the configure step for all host and target subdirs as needed, and configuring target-libs requires building 'all-gcc', which would happen at -j1 ! sigh :-) Looks like we should handle binutils, gdb, and gcc differently for the sake of precommit CI. Thanks, Christophe > Christophe > > > > > > > > The bot I want to put in place would regenerate things as they are > > > > supposed to be, then build and run the testsuite to make sure that > > > > what is supposed to be committed would work (if the committer > > > > regenerates everything correctly) > > > > > > For your job, would it be fine to just force-regenerate everything and > > > ignore timestamps (just like the buildbot's autoregen job wants to do)? > > > It would waste a few cycles, but it would be much simpler. > > > > > Yes, that would achieve the purpose: be able to handle as many patches > > as possible in precommit-CI. > > And as described earlier, for binutils this currently means: > > autoregen > > confgure --enable-maintainer-mode > > make all (with a low -j value otherwise we have random build failures) > > and my proposal to workaround the problem with -j is to do > > make all-bfd all-libiberty regenerate -j1 > > make all -j XXX > > > > Another possibility would be a policy change in how patches are > > submitted, to require that they contain all the autogenerated files. > > > > > > > Simon