Notes on conversion to bugzilla

Notes on conversion to bugzilla

[Tom Tromey]

A couple days ago I hacked up Dan Berlin's gnats->bugzilla converter
for gdb.  This went well.  I started with the converter on my F8 box,
which is the one that comes with bugzilla 3.0.4.

I have set it up to CC all bugs to gdb-prs, and I had the converter
add a comment saying "[Converted from Gnats PR NNN]" to the start of
each bug report.  This should be enough to find old bug reports.

The appended patch is what I think is needed for the gdb web pages.

I don't know whether the output I have is directly insertable into the
existing sourceware bugzilla.  I didn't look at the sourceware
database scheme (I don't even know how to).  Dan, what would you like
from me in order to do this?  I can send the SQL, the hacked
gnatsparse.py, ...

I disabled the file-content "magic" system for categorizing
attachments.  The code caused some Python error and I didn't want to
deal.  If you think this matters, let me know and I will figure it
out.

I looked at the commit hooks in CVSROOT but they all seem to still be
using Gnats -- even the ones for binutils et al.  From a glance on
sourceware it looks like perhaps just using log_accum_bugzillified is
enough, but I have not tested this.  Dan, do you know?


After Dan vets the data, what next?  I think:

* Shut down access to Gnats
* Temporarily hack CVSROOT to avoid commits going to gnats
* Redo the conversion (takes a couple minutes max)
* Import the data
* Fix CVSROOT to use bugzilla
* Check in the web page change

Did I miss something?

If anyone else wants the SQL, or the conversion script, just ask.

Tom

Index: bugs/index.html
===================================================================
RCS file: /cvs/gdb/htdocs/bugs/index.html,v
retrieving revision 1.36
diff -u -r1.36 index.html
--- bugs/index.html	7 Aug 2007 23:48:49 -0000	1.36
+++ bugs/index.html	10 Jul 2008 18:16:48 -0000
@@ -52,7 +52,7 @@
 Please report bugs!<p>
 
 GDB has a <a
-href="http://sourceware.org/cgi-bin/gnatsweb.pl?database=gdb&user=guest&password=guest&cmd=login">Bug
+href="http://sourceware.org/bugzilla/">Bug
 Database</a>.  It is used to track bugs (Problem Reports or PRs) and
 enhancements (Change Requests or CRs).  In addition to problems
 encountered when running GDB, bugs include: errors or missing
@@ -73,7 +73,7 @@
 distribution.<p>
 
 Before submitting a new PR/CR, try <a
-href="http://sourceware.org/cgi-bin/gnatsweb.pl?database=gdb&user=guest&password=guest&cmd=query">browsing</a>
+href="http://sourceware.org/bugzilla/">browsing</a>
 the database to see if the problem has already been reported or even
 fixed.<p>
 
Index: mailing-lists/index.html
===================================================================
RCS file: /cvs/gdb/htdocs/mailing-lists/index.html,v
retrieving revision 1.36
diff -u -r1.36 index.html
--- mailing-lists/index.html	4 May 2008 23:19:13 -0000	1.36
+++ mailing-lists/index.html	10 Jul 2008 18:16:48 -0000
@@ -73,7 +73,7 @@
 discussion should go through this list.</dd>
 
 <dt><a href="http://sourceware.org/ml/gdb-prs/" name="gdb-prs">
-<b>gdb-prs</b></a></dt> <dd> Mailing list for discussing bug reports.
+<b>gdb-prs</b></a></dt> <dd> A read-only mailing list for bug reports.
 See the <a href="../../gdb/bugs/">Bug Database</a> for information on how to
 submit a bug.</dd>
 

Re: Notes on conversion to bugzilla

[Tom Tromey]

>>>>> "Tom" == Tom Tromey <tromey@redhat.com> writes:

Ping...

Tom> The appended patch is what I think is needed for the gdb web pages.

Could a gdb maintainer respond to this?  The patch is quite small.

Tom> I don't know whether the output I have is directly insertable into the
Tom> existing sourceware bugzilla.  I didn't look at the sourceware
Tom> database scheme (I don't even know how to).  Dan, what would you like
Tom> from me in order to do this?  I can send the SQL, the hacked
Tom> gnatsparse.py, ...

Tom> From a glance on sourceware it looks like perhaps just using
Tom> log_accum_bugzillified is enough, but I have not tested this.
Tom> Dan, do you know?

Dan?

Tom

Re: Notes on conversion to bugzilla

[Daniel Jacobowitz]

On Tue, Jul 22, 2008 at 11:23:10AM -0600, Tom Tromey wrote:
> Tom> The appended patch is what I think is needed for the gdb web pages.
> 
> Could a gdb maintainer respond to this?  The patch is quite small.

The patch is fine.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Notes on conversion to bugzilla

[Daniel Berlin]

On Tue, Jul 22, 2008 at 10:23 AM, Tom Tromey <tromey@redhat.com> wrote:
>>>>>> "Tom" == Tom Tromey <tromey@redhat.com> writes:
>
> Ping...


Sorry, this got shoved into my gdb label, which i only read a small time.

>
> Tom> The appended patch is what I think is needed for the gdb web pages.
>
> Could a gdb maintainer respond to this?  The patch is quite small.
>
> Tom> I don't know whether the output I have is directly insertable into the
> Tom> existing sourceware bugzilla.  I didn't look at the sourceware
> Tom> database scheme (I don't even know how to).  Dan, what would you like
> Tom> from me in order to do this?  I can send the SQL, the hacked
> Tom> gnatsparse.py, ...

This is tricky.
So, the output of gnatsparse matches what bugzilla's schema looked
like a few versions ago.
Bugzilla includes auto-upgrade scripts so you could take an old format
database and convert the schema
However, this won't work well with the already-existing databases.

You will have to update the script to output something that is the
same as the current schema.

If you look in my homedir on sourceware, there are a bunch of perl
scripts that contain the mysql password, and with that, you can dump
the current schema.

Once gnatsparse outputs things in the current schema, we just stop the
sourceware bugzilla for a few minutes, tell gnatsparse to start
numbering bugs at the last bug in the database (the bug number is
unique across all products), let it run, and then import that into
sql.
>
> Tom> From a glance on sourceware it looks like perhaps just using
> Tom> log_accum_bugzillified is enough, but I have not tested this.
> Tom> Dan, do you know?
>
> Dan?
Yes, this would work fine.


>
> Tom
>

Re: Notes on conversion to bugzilla

[Tom Tromey]

>>>>> "Dan" == Daniel Berlin <dberlin@dberlin.org> writes:

Dan> Sorry, this got shoved into my gdb label, which i only read a
Dan> small time.

No problem.

Dan> You will have to update the script to output something that is the
Dan> same as the current schema.

BTW there's a reference for various versions of bugzilla schema here:

    http://www.ravenbrook.com/project/p4dti/tool/cgi/bugzilla-schema/

I dumped the schema from the database, though, and updated the script,
using my newly-acquired minimal knowledge of SQL.

There were some field name changes, and I also changed some inserts to
look up IDs in other tables.  So, we end up with things like:

insert into bugs(
  bug_id, assigned_to, bug_severity, priority, bug_status, creation_ts, delta_ts
,
  short_desc,
  reporter, version,
  product_id, component_id, resolution, target_milestone, qa_contact,
  gccbuild, gcctarget, gcchost, keywords
  ) values (
4, 10, 'critical', 'P2', 'CLOSED', '2000-11-01 18:08:00', '20020430091342',
'Attempt to initialize Variable Array',
17, '5.0',
(select id from products where products.name = 'gdb'), (select id from components where components.name = 'gdb'), 'FIXED', '---', 0,
'', '', '', 0
);

Dan> Once gnatsparse outputs things in the current schema, we just stop the
Dan> sourceware bugzilla for a few minutes, tell gnatsparse to start
Dan> numbering bugs at the last bug in the database (the bug number is
Dan> unique across all products), let it run, and then import that into
Dan> sql.

Sounds good to me.  Is there an easy way to test it first?
I can send you the output, the script, the diff between the pristine
script and what I came up with, ... whatever you like.

I wasn't sure what to do about already existing entries in the
'profiles' table.  I don't know enough SQL to know whether the inserts
generated by this script will be a problem :-/

Tom> From a glance on sourceware it looks like perhaps just using
Tom> log_accum_bugzillified is enough, but I have not tested this.

Dan> Yes, this would work fine.

Great.  I've appended a patch to CVSROOT/loginfo.
I only updated the gdb bits; I will do the rest later after pinging
the various lists.

Tom

*** loginfo	26 Mar 2008 18:16:35 -0600	1.34
--- loginfo	25 Jul 2008 13:14:51 -0600	
***************
*** 41,56 ****
  ^/?src/cgen	(/usr/sourceware/bin/log_accum -T cgen -G cgen -C src -m cgen-cvs@sourceware.org -s %{sVv})
  ^/?src/sid	(/usr/sourceware/bin/log_accum -T sid -G sid -C src -m sid-cvs@sourceware.org -s %{sVv})
  
! ^/?src/gdb	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/mmalloc	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/readline	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/sim	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/utils	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/tcl	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/itcl	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/libgui	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/tix	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/tk	(/usr/sourceware/bin/log_accum -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
  
  ^/?src/winsup	(QMAILHOST=cygwin.com /usr/sourceware/bin/log_accum -T cygwin -G cygwin -C src -m cygwin-cvs@cygwin.com -s %{sVv})
  ^/?src/cygwin.*	(QMAILHOST=cygwin.com /usr/sourceware/bin/log_accum -T cygwin -G cygwin -C src -m cygwin-cvs@cygwin.com -s %{sVv})
--- 41,56 ----
  ^/?src/cgen	(/usr/sourceware/bin/log_accum -T cgen -G cgen -C src -m cgen-cvs@sourceware.org -s %{sVv})
  ^/?src/sid	(/usr/sourceware/bin/log_accum -T sid -G sid -C src -m sid-cvs@sourceware.org -s %{sVv})
  
! ^/?src/gdb	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/mmalloc	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/readline	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/sim	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/utils	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/tcl	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/itcl	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/libgui	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/tix	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
! ^/?src/tk	(/usr/sourceware/bin/log_accum_bugzillified -T gdb -G gdb -C src -m gdb-cvs@sourceware.org -s %{sVv})
  
  ^/?src/winsup	(QMAILHOST=cygwin.com /usr/sourceware/bin/log_accum -T cygwin -G cygwin -C src -m cygwin-cvs@cygwin.com -s %{sVv})
  ^/?src/cygwin.*	(QMAILHOST=cygwin.com /usr/sourceware/bin/log_accum -T cygwin -G cygwin -C src -m cygwin-cvs@cygwin.com -s %{sVv})

Re: Notes on conversion to bugzilla

[Tom Tromey]

>>>>> "Tom" == Tom Tromey <tromey@redhat.com> writes:

Tom> * Shut down access to Gnats
Tom> * Temporarily hack CVSROOT to avoid commits going to gnats
Tom> * Redo the conversion (takes a couple minutes max)
Tom> * Import the data
Tom> * Fix CVSROOT to use bugzilla
Tom> * Check in the web page change

Today I set up a local mysql database, imported sourceware's bugzilla
contents, and tested an import of the gdb gnats database.  After some
hacking, this worked.  I also set up a local bugzilla and tested
it... I looked at a few PRs and everything seems good.

So, I think I am ready to do the actual switch.

If you have any last requests, now would be a good time to mention
them.

Tom