OSLab
Operating Systems Lab Resources

1. Download socat for mac using the instrcutions on the site http://socat.darwinports.com/

1. compile the kernel with all those required patches (mentioned above) make sure that you do make install; do the necessary changes in your grub-file:

1. if you havent figured out how to share the source-code between two virtual machines ... no sweat, you can still start by copying the vmlinux (note that its not vmlinuz) file from the source machine to gdb-VM

[@

@]

[@

@]

[@

@]

[@

@]

Connecting to KGDB using VMWare Fusion on Mac OS X

The process is a little long but effective:

1. Download socat for mac using the instrcutions on the site http://socat.darwinports.com/ ... I have one version ... if you want I can send you that

1. compile the kernel with all those required patches.... make sure that you do make install ... so that you have the latest copy of vmlinux (note that its not the same as vmlinuz) ... do the necessary changes in your grub-file ... wait ... dont reboot ...

1. install another virtual machine with linux on it (if you have not already done it); this will be the machine on which you will run gdb. Lets name it gdb-VM
2. if you havent figured out how to share the source-code between two virtual machines ... no sweat, you can still start by copying the vmlinux file the source machine to the 2nd virtual machine (but frankly, you should find out some solution to share the source-code folder, its much neater).
3. shutdown both the machines
4. find the ".vmx" file of the source virtual machine .... for e.g. in my case the VMs name is "Other Linux 2.6.x Kernel 3" ... so for me the file is like this "/Users/upendra/Documents/Virtual\ Machines/Other\ Linux\ 2.6.x\ Kernel\ 3/Other\ Linux\ 2.6.x\ Kernel\ 3.vmx" ... phew .. thats something ...
5. open it using vi (because I like vi for such simple work) ...
6. make the following changes (or add these lines if they are not there) ... make sure that specify the correct path for the serial0.fileName... the example has the path of my configuration

serial0.present = "TRUE" serial0.fileType = "pipe" serial0.yieldOnMsrRead = "TRUE" serial0.startConnected = "TRUE" serial0.fileName = "/Users/upendra/tmp/com_1"

1. start both the VMs
2. run the following command on your mac

socat -d -d /Users/upendra/tmp/com_1 TCP4-LISTEN:8888

1. go to your client gdb-VM and run this

gdb vmlinux target remote <ipaddress of your machine>:8888

so for example, on my machine I run the following command and I think exactly same command should work on your machine also gdb vmlinux target remote 192.168.44.1:8888

... your are done ... happy kernel debugging ....

1. in gdb, type add-symbol-file /path/to/module/object.o 0xADDRESS which will load the symbols.

Debugging modules under KGDB

1. get the address of the running module by running 'cat /proc/modules' on the target machine.
2. in gdb, type add-symbol-file /path/to/module/object.o 0xADDRESS which will load the symbols.

• establishing a serial connection between gdb and a virtual machine running under Windows or Linux

Connecting to VMWare KGDB, Linux host

1. Power down your virtual machine and click "Edit Virtual Machine Settings". Add a serial port, select "Connect to named pipe", and provide the pipe name "/tmp/com_1". (note that this isn't really a named pipe, but a unix-domain socket) Click "advanced settings" and set "yield CPU on poll". You can also get to this setting from the device list once you finish adding the port.
2. GDB can't connect directly to a unix-domain socket, so you'll need to use the socat ("socket cat") utility. It's installed on the elab machines, or you can install it from FC5 RPM or CentOS 4 RPM.
3. If the socket created by VMWare is /tmp/com_1, then invoke socat as: socat -d -d /tmp/com_1 PTY:
4. Find the line in the output identifying the PTY that was created:

   -bash-3.00$ socat -d -d /tmp/com_2 PTY:
   2006/10/17 10:44:46 socat[20383] N successfully connected via 
   2006/10/17 10:44:46 socat[20383] N PTY is '''/dev/pts/7'''
   2006/10/17 10:44:46 socat[20383] N starting data transfer loop ... 

5. Start gdb on the vmlinux executable, and connect to the target:

   -bash-3.00$ gdb vmlinux.kgdb 
   GNU gdb Red Hat Linux (6.3.0.0-1.96rh)
     [...]
   (gdb) target remote /dev/pts/7
   Remote debugging using /dev/pts/7
   0xc012c138 in .text.lock.kgdbstub () at kgdbstub.c:1007
   1007    kgdbstub.c: No such file or directory.
           in kgdbstub.c
   warning: shared library handler failed to enable breakpoint
   (gdb) 

And now you're up and running.

Connecting to KGDB, Windows Hhost

(:title Using KGDB with VMWare:)

KGDB is a set of kernel patches that allows a second machine running a debugger (gdb) to control the target machine over a serial port. This page describes:

• compiling and running KGDB-enabled kernels, for versions 2.4 and 2.6.
• establishing a serial connection between gdb and a virtual machine running under Windows or Linux

1. Download the kgdb 1.9 kernel patch for 2.4.23 from: linux-2.4.23-kgdb-1.9.patch

1. add another lilo entry for the debug kernel, and append the argument "gdb" to get it to stop in the debugger during boot. I added the following lines to my lilo.conf file: [-[@

        append="gdb gdbbaud=115000" @]-]

Note that the default image is the first in the file; the "prompt" and "timeout" arguments will allow you to specify which image to use at boot time. Don't forget to run lilo so it can save the parameters from /etc/lilo.conf. 

1. Boot. It should output about 20 lines and then stop, waiting for the debugger.

Compiling and running a 2.6 kernel with KGDB

1. configure and build your kernel. Under "kernel hacking" enable kgdb, set serial port to "0" (i.e. /dev/ttyS0, or COM1 - the default is ttyS1), and enable kernel debug info.

Setting up KGDB under VMWare, Windows Hhost

1. Add 2 virtual serial ports to your virtual machine. Configure the second one to connect to a named pipe named \\.\pipe\com_2. Under advanced settings, check the box for "yield CPU on poll"
2. Download and install the Named Pipe TCP Proxy utility: nptp_setup.zip
3. Start the proxy, and establish a connector between some port on the local machine (e.g. 8111) and \\.\pipe\com_2. Check the box to enable non-local systems access.

No, you need to use socat, which allows you to connect ...

1. add another lilo entry for the debug kernel, and append the argument "gdb" to get it to stop in the debugger during boot. My lilo.conf file looks like this: [-[@

        append="gdb gdbbaud=115000" @]-]

Compiling and running a 2.4 kernel with KGDB

1. Download the kgdb 1.9 kernel patch for 2.4.23 from here: linux-2.4.23-kgdb-1.9.patch
2. Apply the patch:

     cd /usr/src/linux-2.4.23
     patch -p1 < /tmp/linux-2.4.23-kgdb-1.9.patch 

3. make menuconfig, and under "Kernel Hacking" enable "KGDB" and "KGDB Thread Analysis"
4. build your kernel. (since it's 2.4, that means make dep and make bzImage)
5. copy the kernel (bzImage) to the target virtual machine, or alternately copy vmlinux, the kernel executable, to the debugger machine.
6. add another lilo entry for the debug kernel, and append the argument "gdb" to get it to stop in the debugger during boot. My lilo.conf file looks like this:

   lba32
   install=text
   boot=/dev/hda
   prompt
   timeout=100
   image=/vmlinuz
           label=CRUX
           root=/dev/hda2
           read-only
   image=/vmlinuz.kgdb
           label=KGDB
           root=/dev/hda2
           read-only
           append="gdb gdbbaud=115000" 

By default the 2.4 KGDB will use /dev/ttyS0, or COM1.

Setting up KGDB under VMWare, Windows Hhost

1. get the kgdb patches for kernel 2.6.15 here: http://kgdb.linsyssoft.com/downloads/kgdb-2/linux-2.6.15.5-kgdb-2.4.tar.bz2
2. untar them and then apply them:

     cd linux-2.6.15.6
     patch -b -p1 < ../linux-2.6.15.5-kgdb-2.4/core-lite.patch
     patch -b -p1 < ../ ... core.patch
     patch -b -p1 < ../ ... i386-lite.patch
                            i386.patch
                            8250.patch 

3. configure and build your kernel. Under "kernel hacking" enable kgdb, with serial port set to "1" (i.e. /dev/ttyS1, or COM2), and enable kernel debug info.
4. Add 2 virtual serial ports to your virtual machine. Configure the second one to connect to a named pipe named \\.\pipe\com_2. Under advanced settings, check the box for "yield CPU on poll"
5. Download and install the Named Pipe TCP Proxy utility: nptp_setup.zip
6. Start the proxy, and establish a connector between some port on the local machine (e.g. 8111) and \\.\pipe\com_2. Check the box to enable non-local systems access.
7. Add the command "kgdbwait" to the kernel command line for the kgdb kernel, and boot it. After printing "Booting the kernel" it should hang.
8. In another virtual machine run 'gdb vmlinux'. (you'll have to work out how to share the source tree between the two VMs - clearly you can't use NFS from the VM that you're debugging)
9. type target remote ip_addr:8111 where ip_addr is the windows host IP address that this virtual machine sees. (e.g. on my machine 192.168.149.1, and the VM gets address 192.168.149.128)

KGDB should connect, and away you go.

Caveats - note that I've had a lot of trouble with the DHCP server in VMWare if I'm running multiple machines at once, as it tends to give the same address to each one. You may wish to hardcode the ip address, route, and DNS resolver instead on each VM.

KGDB for kernel 2.4

KGDB for Linux host

I haven't tested it yet, but you can create a named pipe for a serial port - make it something like /tmp/com_1, and then in gdb you should be able to just use the command target remote /tmp/com_1.