Re: multi-thread debugging not working

Re: multi-thread debugging not working

[Lucy Zhang]

----- Original Message ----- 
From: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
To: "Dan Kegel" <dkegel@ixiacom.com>; <gdb@source.redhat.com>
Sent: Friday, July 16, 2038 7:05 PM
Subject: Re: multi-thread debugging not working


> Really?
> I'm running on Linux Redhat
> Version 2.2
> Machine i686
> 
> the core dump is converted and has the VMA's for the thread stacks. GDB
> still won't see it?
> 
> ----- Original Message -----
> From: "Dan Kegel" <dkegel@ixiacom.com>
> To: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
> Sent: Tuesday, July 16, 2002 6:10 PM
> Subject: Re: multi-thread debugging not working
> 
> 
> > What operating system?  Linux by default doesn't do multithreaded core
> dumps.
> >
> 

Re: multi-thread debugging not working

[Daniel Jacobowitz]

On Tue, Jul 16, 2002 at 07:07:09PM -0700, Lucy Zhang wrote:
> 
> ----- Original Message ----- 
> From: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
> To: "Dan Kegel" <dkegel@ixiacom.com>; <gdb@source.redhat.com>
> Sent: Friday, July 16, 2038 7:05 PM
> Subject: Re: multi-thread debugging not working
> 
> 
> > Really?
> > I'm running on Linux Redhat
> > Version 2.2
> > Machine i686
> > 
> > the core dump is converted and has the VMA's for the thread stacks. GDB
> > still won't see it?

It has the VMA for every stack.  It doesn't have their registers,
however.

> > 
> > ----- Original Message -----
> > From: "Dan Kegel" <dkegel@ixiacom.com>
> > To: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
> > Sent: Tuesday, July 16, 2002 6:10 PM
> > Subject: Re: multi-thread debugging not working
> > 
> > 
> > > What operating system?  Linux by default doesn't do multithreaded core
> > dumps.
> > >
> > 
> 
> 

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: multi-thread debugging not working

[Lucy Zhang]

I see. For the note sections, are you talking about the prstatus, prpsinfo,
and taskstruct structs? I have the data to fill in the prstatus and prpsinfo
structs, but I don't know about taskstruct. What does GDB use the taskstruct
for? Can I do without it?

Thanks,
Lucy


----- Original Message -----
From: "Daniel Jacobowitz" <drow@mvista.com>
To: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
Cc: <gdb@sources.redhat.com>
Sent: Tuesday, July 16, 2002 7:08 PM
Subject: Re: multi-thread debugging not working


> On Tue, Jul 16, 2002 at 07:07:09PM -0700, Lucy Zhang wrote:
> >
> > ----- Original Message -----
> > From: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
> > To: "Dan Kegel" <dkegel@ixiacom.com>; <gdb@source.redhat.com>
> > Sent: Friday, July 16, 2038 7:05 PM
> > Subject: Re: multi-thread debugging not working
> >
> >
> > > Really?
> > > I'm running on Linux Redhat
> > > Version 2.2
> > > Machine i686
> > >
> > > the core dump is converted and has the VMA's for the thread stacks.
GDB
> > > still won't see it?
>
> It has the VMA for every stack.  It doesn't have their registers,
> however.
>
> > >
> > > ----- Original Message -----
> > > From: "Dan Kegel" <dkegel@ixiacom.com>
> > > To: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
> > > Sent: Tuesday, July 16, 2002 6:10 PM
> > > Subject: Re: multi-thread debugging not working
> > >
> > >
> > > > What operating system?  Linux by default doesn't do multithreaded
core
> > > dumps.
> > > >
> > >
> >
> >
>
> --
> Daniel Jacobowitz                           Carnegie Mellon University
> MontaVista Software                         Debian GNU/Linux Developer

problem loading libpthread.so

[Lucy Zhang]

Hi,

I used GDB to debug an ELF core dump which was converted from a different
formatted dump. However when I run GDB I get the following error:
This GDB was configured as "i386-redhat-linux"...
Core was generated by `vlio_test8'.
Program terminated with signal 4, Illegal instruction.
Reading symbols from /lib/i686/libpthread.so.0...done.

warning: Unable to set global thread event mask: generic error
Segmentation fault (core dumped)


So I tried to debug GDB using GDB and got the following results:

/work/lucy/lucy1_0524/src/user/elf > gdb gdb -c core
GNU gdb 5.0rh-5 Red Hat Linux 7.1
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...(no debugging symbols
found)...
Core was generated by `gdb -f vlio_test8.020715.042017 -c
elf.020715.042017'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libncurses.so.5...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/libncurses.so.5
Reading symbols from /lib/i686/libm.so.6...done.
Loaded symbols for /lib/i686/libm.so.6
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/i686/libc.so.6...done.
Loaded symbols for /lib/i686/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libthread_db.so.1...done.
Loaded symbols for /lib/libthread_db.so.1
#0 td_ta_thr_iter (ta=0x82f7468, callback=0x80e91f0 <thread_db_init+3456>,
cbdata_p=0x0, state=TD_THR_ANY_STATE, ti_pri=-20, ti_sigmask_p=0x0,
ti_user_flags=4294967295) at td_ta_thr_iter.c:28
28 td_ta_thr_iter.c: No such file or directory.
in td_ta_thr_iter.c

(gdb) backtrace
#0 td_ta_thr_iter (ta=0x82f7468, callback=0x80e91f0 <thread_db_init+3456>,
cbdata_p=0x0, state=TD_THR_ANY_STATE, ti_pri=-20, ti_sigmask_p=0x0,
ti_user_flags=4294967295) at td_ta_thr_iter.c:28
#1 0x080e9291 in thread_db_init () at eval.c:41
#2 0x080e8a31 in thread_db_init () at eval.c:41
#3 0x0808845d in symbol_file_add () at eval.c:41
#4 0x080a66c8 in free_so () at eval.c:41
#5 0x080c8f1b in catch_errors () at eval.c:41
#6 0x080a69d9 in solib_add () at eval.c:41
#7 0x080e6698 in default_check_format () at eval.c:41
#8 0x080c8f1b in catch_errors () at eval.c:41
#9 0x080e6996 in default_check_format () at eval.c:41
#10 0x080ba128 in core_file_command () at eval.c:41
#11 0x080c8fa3 in catch_errors () at eval.c:41
#12 0x080c8f1b in catch_errors () at eval.c:41
#13 0x080c8ff0 in catch_command_errors () at eval.c:41
#14 0x0806f1c9 in _start () at eval.c:41
#15 0x080c8f1b in catch_errors () at eval.c:41
#16 0x0806f3b7 in main () at eval.c:41
#17 0x400ac177 in __libc_start_main (main=0x806f390 <main>, argc=5,
ubp_av=0xbffffb4c, init=0x806de0c <_init>, fini=0x818555c <_fini>,
rtld_fini=0x4000e184 <_dl_fini>, stack_end=0xbffffb3c)
at ../sysdeps/generic/libc-start.c:129

Does anyone know what may have caused the seg fault. I'm not very familiar
with GDB at all. So if anyone has any guess or intuitions about the cause of
this problem please share.

Thank you very much,
Lucy

Re: problem loading libpthread.so

[Daniel Jacobowitz]

On Tue, Jul 16, 2002 at 01:17:05PM -0700, Lucy Zhang wrote:
> Hi,
> 
> I used GDB to debug an ELF core dump which was converted from a different
> formatted dump. However when I run GDB I get the following error:
> This GDB was configured as "i386-redhat-linux"...
> Core was generated by `vlio_test8'.
> Program terminated with signal 4, Illegal instruction.
> Reading symbols from /lib/i686/libpthread.so.0...done.
> 
> warning: Unable to set global thread event mask: generic error
> Segmentation fault (core dumped)


> Does anyone know what may have caused the seg fault. I'm not very familiar
> with GDB at all. So if anyone has any guess or intuitions about the cause of
> this problem please share.

You need to get a newer version of GDB.  Check 5.2, from
http://sources.redhat.com/gdb/, and see if that works better.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: multi-thread debugging not working

[Lucy Zhang]

Hi,


Yes, the newer version of GDB solved the previous problem.

Now I have a new problem with not being able to debug my multi-threaded
program. I'm debugging using an ELF core dump that was coverted from another
format. The program i'm debugging has about 20 threads. It may be because
the ELF file i'm creating is missing certain information. Right now I'm at a
loss as to what could be missing which GDB might be looking for in order to
see the multiple threads. Does anyone have any ideas on how GDB recognize
threads from an ELF core dump?

Below is the output from GDB:
----------------------------------------
/work/lucy/gdb-5.2/gdb > ./gdb
GNU gdb 5.2
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
Setting up the environment for debugging gdb.
.gdbinit:5: Error in sourced command file:
No symbol table is loaded. Use the "file" command.
(gdb) file /work/lucy/lucy1_0524/src/user/elf/vlio_test8.020715.042017
Reading symbols from
/work/lucy/lucy1_0524/src/user/elf/vlio_test8.020715.042017...done.
(gdb) core-file /work/lucy/lucy1_0524/src/user/elf/elf.020715.042017
Core was generated by `vlio_test8'.
Program terminated with signal 4, Illegal instruction.
Reading symbols from /lib/i686/libpthread.so.0...done.
Loaded symbols for /lib/i686/libpthread.so.0
Reading symbols from /lib/i686/libm.so.6...done.
Loaded symbols for /lib/i686/libm.so.6
Reading symbols from /usr/lib/libpopt.so.0...done.
Loaded symbols for /usr/lib/libpopt.so.0
Reading symbols from /lib/i686/libc.so.6...done.
Loaded symbols for /lib/i686/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
#0 0x40091aa5 in sigset (sig=-1073747472, disp=0x20) at
../sysdeps/posix/sigset.c:69
69 ../sysdeps/posix/sigset.c: No such file or directory.
in ../sysdeps/posix/sigset.c

(gdb) info threads
* 1 process 1491 0x40091aa5 in sigset (sig=-1073747472, disp=0x20)
at ../sysdeps/posix/sigset.c:69
(gdb) thread 0
Thread ID 0 not known.
(gdb) thread 2
Thread ID 2 not known.
(gdb) thread 3
Thread ID 3 not known.
(gdb)

Thanks,
Lucy


----- Original Message -----
From: "Daniel Jacobowitz" <drow@mvista.com>
To: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
Cc: <gdb@sources.redhat.com>
Sent: Tuesday, July 16, 2002 1:23 PM
Subject: Re: problem loading libpthread.so


> On Tue, Jul 16, 2002 at 01:17:05PM -0700, Lucy Zhang wrote:
> > Hi,
> >
> > I used GDB to debug an ELF core dump which was converted from a
different
> > formatted dump. However when I run GDB I get the following error:
> > This GDB was configured as "i386-redhat-linux"...
> > Core was generated by `vlio_test8'.
> > Program terminated with signal 4, Illegal instruction.
> > Reading symbols from /lib/i686/libpthread.so.0...done.
> >
> > warning: Unable to set global thread event mask: generic error
> > Segmentation fault (core dumped)
>
>
> > Does anyone know what may have caused the seg fault. I'm not very
familiar
> > with GDB at all. So if anyone has any guess or intuitions about the
cause of
> > this problem please share.
>
> You need to get a newer version of GDB.  Check 5.2, from
> http://sources.redhat.com/gdb/, and see if that works better.
>
> --
> Daniel Jacobowitz                           Carnegie Mellon University
> MontaVista Software                         Debian GNU/Linux Developer

Re: multi-thread debugging not working

[Daniel Jacobowitz]

On Tue, Jul 16, 2002 at 04:55:57PM -0700, Lucy Zhang wrote:
> Hi,
> 
> 
> Yes, the newer version of GDB solved the previous problem.
> 
> Now I have a new problem with not being able to debug my multi-threaded
> program. I'm debugging using an ELF core dump that was coverted from another
> format. The program i'm debugging has about 20 threads. It may be because
> the ELF file i'm creating is missing certain information. Right now I'm at a
> loss as to what could be missing which GDB might be looking for in order to
> see the multiple threads. Does anyone have any ideas on how GDB recognize
> threads from an ELF core dump?

There should be a PT_NOTE section (or two or three of them) for every
single thread, containing its registers.  Most GNU/Linux systems at
least do not dump adequate information; you'll have to investigate your
dump format/conversion.



> 
> Below is the output from GDB:
> ----------------------------------------
> /work/lucy/gdb-5.2/gdb > ./gdb
> GNU gdb 5.2
> Copyright 2002 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for details.
> This GDB was configured as "i686-pc-linux-gnu".
> Setting up the environment for debugging gdb.
> .gdbinit:5: Error in sourced command file:
> No symbol table is loaded. Use the "file" command.
> (gdb) file /work/lucy/lucy1_0524/src/user/elf/vlio_test8.020715.042017
> Reading symbols from
> /work/lucy/lucy1_0524/src/user/elf/vlio_test8.020715.042017...done.
> (gdb) core-file /work/lucy/lucy1_0524/src/user/elf/elf.020715.042017
> Core was generated by `vlio_test8'.
> Program terminated with signal 4, Illegal instruction.
> Reading symbols from /lib/i686/libpthread.so.0...done.
> Loaded symbols for /lib/i686/libpthread.so.0
> Reading symbols from /lib/i686/libm.so.6...done.
> Loaded symbols for /lib/i686/libm.so.6
> Reading symbols from /usr/lib/libpopt.so.0...done.
> Loaded symbols for /usr/lib/libpopt.so.0
> Reading symbols from /lib/i686/libc.so.6...done.
> Loaded symbols for /lib/i686/libc.so.6
> Reading symbols from /lib/ld-linux.so.2...done.
> Loaded symbols for /lib/ld-linux.so.2
> #0 0x40091aa5 in sigset (sig=-1073747472, disp=0x20) at
> ../sysdeps/posix/sigset.c:69
> 69 ../sysdeps/posix/sigset.c: No such file or directory.
> in ../sysdeps/posix/sigset.c
> 
> (gdb) info threads
> * 1 process 1491 0x40091aa5 in sigset (sig=-1073747472, disp=0x20)
> at ../sysdeps/posix/sigset.c:69
> (gdb) thread 0
> Thread ID 0 not known.
> (gdb) thread 2
> Thread ID 2 not known.
> (gdb) thread 3
> Thread ID 3 not known.
> (gdb)
> 
> Thanks,
> Lucy
> 
> 
> ----- Original Message -----
> From: "Daniel Jacobowitz" <drow@mvista.com>
> To: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
> Cc: <gdb@sources.redhat.com>
> Sent: Tuesday, July 16, 2002 1:23 PM
> Subject: Re: problem loading libpthread.so
> 
> 
> > On Tue, Jul 16, 2002 at 01:17:05PM -0700, Lucy Zhang wrote:
> > > Hi,
> > >
> > > I used GDB to debug an ELF core dump which was converted from a
> different
> > > formatted dump. However when I run GDB I get the following error:
> > > This GDB was configured as "i386-redhat-linux"...
> > > Core was generated by `vlio_test8'.
> > > Program terminated with signal 4, Illegal instruction.
> > > Reading symbols from /lib/i686/libpthread.so.0...done.
> > >
> > > warning: Unable to set global thread event mask: generic error
> > > Segmentation fault (core dumped)
> >
> >
> > > Does anyone know what may have caused the seg fault. I'm not very
> familiar
> > > with GDB at all. So if anyone has any guess or intuitions about the
> cause of
> > > this problem please share.
> >
> > You need to get a newer version of GDB.  Check 5.2, from
> > http://sources.redhat.com/gdb/, and see if that works better.
> >
> > --
> > Daniel Jacobowitz                           Carnegie Mellon University
> > MontaVista Software                         Debian GNU/Linux Developer
> 
> 

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: multi-thread debugging not working

[Jim Blandy]

"Lucy Zhang" <lucyz@uclink4.berkeley.edu> writes:
> Yes, the newer version of GDB solved the previous problem.
> 
> Now I have a new problem with not being able to debug my multi-threaded
> program. I'm debugging using an ELF core dump that was coverted from another
> format. The program i'm debugging has about 20 threads. It may be because
> the ELF file i'm creating is missing certain information. Right now I'm at a
> loss as to what could be missing which GDB might be looking for in order to
> see the multiple threads. Does anyone have any ideas on how GDB recognize
> threads from an ELF core dump?

Try running the command `readelf -n' on your core dump.  In a
multi-threaded core dump, you should see something like this:

$ readelf -n core.9940

Notes at offset 0x00000254 with length 0x00000d4c:
 Owner         Data size       Description
 CORE          0x0000007c      NT_PRPSINFO (prpsinfo structure)
 CORE          0x00000090      NT_PRSTATUS (prstatus structure)
 CORE          0x0000006c      NT_FPREGSET (floating point registers)
 LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
 CORE          0x00000090      NT_PRSTATUS (prstatus structure)
 CORE          0x0000006c      NT_FPREGSET (floating point registers)
 LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
 CORE          0x00000090      NT_PRSTATUS (prstatus structure)
 CORE          0x0000006c      NT_FPREGSET (floating point registers)
 LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
 CORE          0x00000090      NT_PRSTATUS (prstatus structure)
 CORE          0x0000006c      NT_FPREGSET (floating point registers)
 LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)

This program had four threads in it; note that there's one prstatus
structure, one user_xfpregs structure, and one floating point register
structure for each thread.

If you only see one of each kind of register note, then there's only
one thread in the core file.

Re: multi-thread debugging not working

[Lucy Zhang]

Great thank you. One more question though. Right now I only have one NOTE
segment which contain one of each types of note (prstatus, prpsinfo,
taskstruct). When I add the new note structs for each thread, should I add
in into the same NOTE segment after my current single thread structs? Which
is what readelf -n seems to be suggesting.
OR should I create a new NOTE segment, and add it in there like the
following? Which way does GDB prefer?

Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
NOTE 0x001454 0x00000000 0x00000000 0x00170 0x00000 0
NOTE 0x0015c4 0x00000000 0x00000000 0x001b0 0x00000 0  //add a second NOTE
segment???
LOAD 0x001774 0x40b9e000 0x00000000 0x1bd000 0x1bd000 RW 0x1000
LOAD 0x1be774 0x40d5b000 0x00000000 0xc17000 0xc17000 RW 0x1000
LOAD 0xdd5774 0x41972000 0x00000000 0xbd000 0xbd000 RW 0x1000

Notes at offset 0x00001454 with length 0x00000170: //notes for original
single thread
Owner Data size Description
CORE 0x00000090 NT_PRSTATUS (prstatus structure)
CORE 0x0000007c NT_PRPSINFO (prpsinfo structure)
CORE 0x00000034 NT_TASKSTRUCT (task structure)

Notes at offset 0x000015c4 with length 0x000001b0:  //notes for threads i
will add??
Owner Data size Description
CORE 0x00000090 NT_PRSTATUS (prstatus structure)
CORE 0x00000090 NT_PRSTATUS (prstatus structure)
CORE 0x00000090 NT_PRSTATUS (prstatus structure)


Thanks,
Lucy


----- Original Message -----
From: "Jim Blandy" <jimb@redhat.com>
To: "Lucy Zhang" <lucyz@uclink4.berkeley.edu>
Cc: <gdb@sources.redhat.com>
Sent: Wednesday, July 17, 2002 3:07 PM
Subject: Re: multi-thread debugging not working


>
> "Lucy Zhang" <lucyz@uclink4.berkeley.edu> writes:
> > Yes, the newer version of GDB solved the previous problem.
> >
> > Now I have a new problem with not being able to debug my multi-threaded
> > program. I'm debugging using an ELF core dump that was coverted from
another
> > format. The program i'm debugging has about 20 threads. It may be
because
> > the ELF file i'm creating is missing certain information. Right now I'm
at a
> > loss as to what could be missing which GDB might be looking for in order
to
> > see the multiple threads. Does anyone have any ideas on how GDB
recognize
> > threads from an ELF core dump?
>
> Try running the command `readelf -n' on your core dump.  In a
> multi-threaded core dump, you should see something like this:
>
> $ readelf -n core.9940
>
> Notes at offset 0x00000254 with length 0x00000d4c:
>  Owner         Data size       Description
>  CORE          0x0000007c      NT_PRPSINFO (prpsinfo structure)
>  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
>  CORE          0x0000006c      NT_FPREGSET (floating point registers)
>  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
>  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
>  CORE          0x0000006c      NT_FPREGSET (floating point registers)
>  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
>  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
>  CORE          0x0000006c      NT_FPREGSET (floating point registers)
>  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
>  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
>  CORE          0x0000006c      NT_FPREGSET (floating point registers)
>  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
>
> This program had four threads in it; note that there's one prstatus
> structure, one user_xfpregs structure, and one floating point register
> structure for each thread.
>
> If you only see one of each kind of register note, then there's only
> one thread in the core file.

Re: multi-thread debugging not working

[Jim Blandy]

"Lucy Zhang" <lucyz@uclink4.berkeley.edu> writes:
> Great thank you. One more question though. Right now I only have one NOTE
> segment which contain one of each types of note (prstatus, prpsinfo,
> taskstruct). When I add the new note structs for each thread, should I add
> in into the same NOTE segment after my current single thread structs? Which
> is what readelf -n seems to be suggesting.
> OR should I create a new NOTE segment, and add it in there like the
> following? Which way does GDB prefer?

I don't think GDB will care.  The core file I've seen has them all in
one NOTE segment.

If you end up trying to debug this, some starting points: the code
which actually parses the NOTE segment is elfcore_read_notes, in
bfd/elf.c.  It turns the prstatus notes into `.reg/PID' sections.  The
GDB code which then consumes those sections is add_to_thread_list in
corelow.c.