KERNEL TRAP(How to READ)

[Larry McVoy]

In article <5543 at brazos.Rice.edu> Rob.Montjoy at uc.edu (Robert C. Montjoy) writes:
>Can anyone  till me how to decode thes messages.  The kernel is panicing
>about 3 or 4 times a week the addresses in the trace are always
>indentical. ALso,it always gives the "BAD TRAP" message before dieing.
>
>Mar  4 22:52:39 vlsilab vmunix: NFS server babbage.ece.uc.edu ok
>Mar  4 23:06:54 vlsilab vmunix: BAD TRAP
>  [panic messages deleted]

OK, try this.  You have to be running the same kernel binary as was
running at the time of the crash.  If you can't do that, you need a copy
of the binary and a core file (usually in /var/crash/`hostname`/vmunix.xxx
and .../vmcore.xxx).

Say something like 

# adb /vmunix /dev/kmem

or 

# adb vmunix.0 vmcore.0

The you start walking backwards like so (look at the stack trace above to
follow along):

0xf806149c?ia
function1+0x100
f801fb74?ia
function2+0x80
f801f988?ia
function3+0x100

This will give you a symbolic stack trace back.  I have a couple of shell
scripts that do this automagically for the last panic on a system, these
work (or used to work) on sun3/sun4 running 4.1.  I'm passing them on for
reference purposes only.

--- sun4 gettrace ----
#!/bin/sh

# $0 [-v] [vmunixfile [messagesfile]]

if [ "$1" = "-v" ]
then    VERBOSE=ON; shift
else    VERBOSE=OFF
fi

if [ $# -ge 1 ]
then    VM=$1; shift
else    VM=/vmunix
fi

if [ $# -ge 1 ]
then    MSGS=$1; shift
else    MSGS=/var/adm/messages
fi

sed -e 's/.*vmunix: //' -e 's/pc=0x/PC= /' -e 's/,//' \
        < $MSGS \
        | egrep 'Called|PC=' > /tmp/trace$$
cat - <<EOF | ed /tmp/trace$$ > /dev/null
$
?PC=?
1,.-1d
w
q
EOF
if [ $VERBOSE = ON ]
then    echo STACK TRACEBACK for $VM on `date`
        echo ''
        echo ADDRESSES + ARGS
        echo ''
        cat /tmp/trace$$
        echo ''
        echo SYMBOLIC TRACEBACK
        echo ''
fi
sed -e 's/Called from \([0-9a-fA-F]*\).*/\1\?ia/p' \
    -e 's/PC= \([0-9a-fA-F]*\).*/\1\?ia/p' < /tmp/trace$$ | adb $VM -
/bin/rm -f /tmp/trace$$

---- sun3 gettrace ----
#!/bin/sh

if [ "$1" = "-v" ]
then VERBOSE=ON; shift
else VERBOSE=OFF
fi

if [ $# -eq 1 ]
then VM=$1
else VM=/vmunix
fi

sed -e 's/.*vmunix: //' -e 's/pc 0x/PC=/' -e 's/,//' \
        < /usr/adm/messages \
        | egrep 'pid|D0-D7|A0-A7|Called|PC=' > /tmp/trace$$
cat - <<EOF | ed /tmp/trace$$ > /dev/null
$
?PC=?
1,.-2d
w
q
EOF
if [ $VERBOSE = ON ]
then    echo STACK TRACEBACK for $VM on `date`
        echo ''
        echo ADDRESSES + ARGS
        echo ''
        cat /tmp/trace$$
        echo ''
        echo SYMBOLIC TRACEBACK
        echo ''
fi
egrep -v 'pid|D0-D7|A0-A7' < /tmp/trace$$ | \
sed -e 's/Called from \([0-9a-fA-F]*\).*/\1\?ia/p' \
    -e 's/.*PC=\([0-9a-fA-F]*\).*/\1\?ia/p' \
| adb $VM -
/bin/rm -f /tmp/trace$$

[[Ed's Note: not placed in archives since they were explicitly posted
"for reference only" -bdg]]

What I say is my opinion.  I am not paid to speak for Sun, I'm paid to hack.
    Besides, I frequently read news when I'm drjhgunghc, err, um, drunk.
Larry McVoy, Sun Microsystems     (415) 336-7627       ...!sun!lm or lm at sun.com