Monster Finder to haunt your disk...

Wed Oct 31 18:47:15 AEST 1990

Since this is the season to find monsters, the script below will, when
put in your crontab, comb out your local disk and report back to you
when it finds NEW monster user files, relative to an OLD monster file.

The problem with a simple find statement is that setting it to print
all files larger than X blocks contains too much information. You want
to exclude all sorts of file, namely, large systems files, Known large
user files, and files that were already reported as monsters.

The script uses the output from the versions command, munges folded lines,
and recalculates the real sizes of the files (the inst database is incorrect).
This gives you a stop list of known system monsters to exclude from reporting.

You edit the file ExcludeKnownMonsters.sh to put a list of names of user
files to exclude (I don't know why unix is not know to versions as a system
file, but you can put it here anyway). The script will put in the sizes for
you. If the size of a known monster change, it will be reported.

The script looks for a file OldMonsters in the Monster directory. That is
a list of known monsters previous reported. If you want to start with
all new monsters, erase OldMonsters and you will get a list excluding
SystemMonsters and KnownMonsters. Every suceeding night, you will get
a note only if NEW monster sprouts, and you will not be bothered
by OLD monsters. You can always look in the Monster directory for a history
of the monsters growing on your disk.

Should you have a large number of large file, increase the THRESHOLD value.
If you have too many files, fgrep will break as its wordlist will be exceeded.
Note also, the threshold is in disk blocks, not bytes.

The script will report on monsters that change size, not just that they exist.
So an active monster will re-appear as often as its size changes are noted
each night.

Version.mac is some included script that reports on how often and what version
of the script is run. I have put that code in, but you can omit it if
you don't care to log when the script is run.

Please embelish and improve this to solve your disk management problems.
Let your users know you are watching the largest files on your disks.

Let me know if this is useful, or if you make changes to solve your problems.

Dan.

---------------------------MonsterFind.sh-------------------------------------
#! /bin/sh
#
# This version only notifies you if you have NEW monsters, relative to
# the OLD_MONSTERS files.
# Excludes System Monsters as culled from the sgi versions command.

. Version.mac
THRESHOLD=1000
STOPDIR=/usr/local/StopList
DIRNAME=/usr/local/Monsters
OLD_MONSTERS=OldMonsters
SYSTEM_MONSTERS=${STOPDIR}/SystemMonsters
KNOWN_MONSTERS=${STOPDIR}/KnownMonsters
TMP_A=Monster.a
TMP_B=Monster.b
TMP_C=Monster.c

        if [ ! -d $DIRNAME ]
        then
        mkdir $DIRNAME
        fi

        if [ ! -d $STOPDIR ]
        then
        mkdir $STOPDIR
        fi

cd $DIRNAME

FILENAME=`date "+Monsters.%m.%d"`

        if [ -f ${FILENAME} ]
        then
        i=0
        _FILENAME_=${FILENAME}
                while [ -f ${_FILENAME_} ]
                do
                _FILENAME_=${FILENAME}.${i}
                i=`expr ${i} + 1`
                done
        FILENAME=${_FILENAME_}
        fi

#       if [ ! -f ${SYSTEM_MONSTERS} ]
#       then
        ExcludeSystemMonsters.sh ${THRESHOLD} ${SYSTEM_MONSTERS}
#       fi

#       if [ ! -f ${KNOWN_MONSTERS} ]
#       then
        ExcludeKnownMonsters.sh ${THRESHOLD} ${KNOWN_MONSTERS}
#       fi

        if [ ! -f ${OLD_MONSTERS} ]
        then
# this is a new series.
        NEW_RUN=""
        echo 0000 dummy > ${OLD_MONSTERS}
        else
        NEW_RUN="notify"
        fi

cat <<HEREDOC > ${TMP_A}
Subject: New Monsters Found on `hostname` at `date`
$VERSION
Monster threshold is ${THRESHOLD} bytes.
------------------------------------------------------------------------------
HEREDOC
find / -local -size +${THRESHOLD} \( \! -type l \) -print | \
xargs /bin/ls -ds |\
fgrep -v -f ${SYSTEM_MONSTERS} |\
fgrep -v -f ${KNOWN_MONSTERS} |\
fgrep -v -f ${OLD_MONSTERS} |\
sort -rn > ${TMP_B}

MONSTERCOUNT=`wc -l ${TMP_B} |tr -s ' ' | /usr/bin/cut -d' ' -f2`

        if [ $MONSTERCOUNT = 0 ]
        then
        echo "$0 : No new monsters found"
        rm -f ${TMP_A} ${TMP_B} ${TMP_C}
        . End.mac
        exit 0
        fi

cat <<HEREDOC > ${TMP_C}
------------------------------------------------------------------------------
$0 Completed
${MONSTERCOUNT} monsters found
HEREDOC

cat ${TMP_A} ${TMP_B} ${TMP_C} > ${FILENAME}
cat ${TMP_B} >> ${OLD_MONSTERS}
# prune the OLD_MONSTER list to it does not exceed the wordlist size limit for f
   grep
sort -nur ${OLD_MONSTERS} -o ${OLD_MONSTERS}
rm -f ${OLD_MONSTERS}.a

        for i in `cut -d' ' -f2 < ${OLD_MONSTERS}`
        do
                if [ -r $i ]
                then
                ls -ds $i >> ${OLD_MONSTERS}.a
                fi
        done

mv ${OLD_MONSTERS} ${OLD_MONSTERS}.bak
mv ${OLD_MONSTERS}.a ${OLD_MONSTERS}
sort -nur ${OLD_MONSTERS} -o ${OLD_MONSTERS}

rm -f ${TMP_A} ${TMP_B} ${TMP_C} ${OLD_MONSTERS}.bak

        if [ "${NEW_RUN}" = notify ]
        then
        cat ${FILENAME} | mail root
        fi

. End.mac
exit 0

------------------------------ExcludeSystemMonsters.sh------------------------
#! /bin/sh

. Version.mac
THRESHOLD=$1
OUTFILE=$2
JOIN_SPLIT_LINES='
Couldn't execute the program!
'

SELECT_MONSTERS='
 $1 > THRESHOLD {print $2}
'

versions -s long |\
nawk   "${JOIN_SPLIT_LINES}" ROOT="/" |\
nawk "${SELECT_MONSTERS}" THRESHOLD=${THRESHOLD} |\
xargs /bin/ls -s |\
tr -s ' ' |
sort -nr > $OUTFILE

exit 0

-------------------------------ExcludeKnownMonsters.sh------------------------
#! /bin/sh
. Version.mac

MONSTER_NAMES="/unix /usr/lib/SoftPC/MSDOS_BOOT /usr/local/bin/iconsmith"

echo ${MONSTER_NAMES} | \
xargs /bin/ls -s |\
tr -s ' ' |\
nawk '$1 > THRESHOLD' THRESHOLD=$1 > $2
exit 0

------------------Version.mac-------------------------------------------------

BIN="/bin"
UBIN="/usr/bin"
PROGNAME=`${BIN}/basename $0`

for i in `echo "${PATH}" | ${UBIN}/tr ':' ' '`
do
case $i
in
\.*)i=`pwd`;;
esac
if [ -x ${i}/${PROGNAME} ]
then
VERSION="`${BIN}/ls -ls ${i}/${PROGNAME} | ${UBIN}/tr -s ' ' | ${UBIN}/cut -d' '
    -f2,7-12| ${BIN}/sed s=//=/=g `"
VERSION="`echo $VERSION | ${UBIN}/cut -d' ' -f6` Version `echo $VERSION | ${UBIN
   }/cut -d' ' -f1-5`"
break;

fi
done

USAGE_DIR=${HOME}/D.USAGE
if [ ! -d ${USAGE_DIR} ]
then
${BIN}/mkdir ${USAGE_DIR}
fi

${BIN}/date "+${VERSION}%tBEGIN%t%y.%m.%d.%t%T" >> ${USAGE_DIR}/${PROGNAME}
echo ${VERSION}
unset BIN UBIN PROGNAME USAGE_DIR

+-----------------------------------------------------------------------------+
| karron at nyu.edu (mail alias that will always find me)                        |
|                                         Dan Karron                          |
| . . . . . . . . . . . . . .             New York University Medical Center  |
| 560 First Avenue           \ \    Pager <1> (212) 397 9330                  |
| New York, New York 10016    \**\        <2> 10896   <3> <your-number-here>  |
| (212) 340 5210               \**\__________________________________________ |
| Please Note : Soon to move to dan at karron.med.nyu.edu 128.122.135.3  (Nov 1 )|
+-----------------------------------------------------------------------------+