bash
Script: Yum Check Update
Submitted by jmarki on 8 October 2009 - 2:50amI have been using this script to check for updates on my Redhat systems for quite some time. Put this into your cron.daily, and you have a daily nag to update your system. 
#!/bin/bash
#########
## Yum Check Update Script
##
## This script checks for system updates and sends email
## to sysmin team if there are any updates.
##
## Changelog
## ---------
## 24 Oct 2008 (Junhao)
## - Initial commit
##
#########
_CAT="/bin/cat"
_DATE="/bin/date"
_HOSTNAME="/bin/hostname"
_MAILX="/bin/mailx"
_RM="/bin/rm"
_TOUCH="/bin/touch"
_YUM="/usr/bin/yum"
HOSTNAME=`${_HOSTNAME}`
DATESTAMP=`${_DATE} +%Y%b%d-%H:%M:%S`
EMAIL=root
MAILSUB="RHEL Update Available for ${HOSTNAME} on ${DATESTAMP}"
TEMPLOG=/tmp/yum-check-update.tmp
${_TOUCH} ${TEMPLOG}
${_YUM} check-update 1> ${TEMPLOG} 2>&1
if [[ $? != 0 ]]; then
${_CAT} ${TEMPLOG} | ${_MAILX} -s "${MAILSUB}" ${EMAIL}
fi
${_RM} ${TEMPLOG}
- jmarki's blog
- Add new comment
- 216 reads
Script: Check No Missing Files After Reorganisation of Directory Trees
Submitted by jmarki on 1 August 2009 - 4:43pmHere's another script I did when I had to reorganised a folder hierarchy of years of data. Basically to ensure files are not missing, or corrupted.
#!/bin/bash
#########################
#
# checkNoMissingFiles
# ===================
#
# This script checks that no files are missing after folders are reorganised.
# Basic algorithm is to checksum all files in both old and new folders, then
# checking through both lists of checksums to ensure all checksums are present
# in both lists.
#
# Changelog
# =========
#
# 18 Oct 2007 - Junhao
# * Initial commit
#
# 11 Dec 2007 - Junhao
# * Tidied style
# * Fixed bug with spaces in filenames
# * added option to save generated checksums
# * changed md5sum to sha1sum
# * changed checksum to general algorithm
#########################
PATH=/bin:/usr/bin
export PATH
## Program Locations
awk=/bin/awk
cat=/bin/cat
echo=/bin/echo
find=/usr/bin/find
grep=/bin/grep
checksum=/usr/bin/sha1sum
mktemp=/bin/mktemp
rm=/bin/rm
tee="/usr/bin/tee -a"
touch=/bin/touch
## End Program Locations
## Start Script
## Script parameters
f_logFile=/dev/null
d_orgLoc=/dev/null
d_newLoc=/dev/null
v_oldFileName=
v_oldFileChksum=
f_oldChksumLog=
f_newChksumLog=
v_missingFilesCount=0
v_missingFiles=""
v_output=
v_f1flag=1
v_f2flag=1
## End Script parameters
function print_usage () {
${echo} "
$0
Usage: $0 [-L logfile] [-f1 filename] [-f2 filename] [oldDir] [newDir]
or $0 -h
Description: Checks that there are no missing files after reorganising a directory.
Options:
-L logfile (Optional) Path to log file
-h (Optional) This help text
-1 (Optional) Filename to save checksum for old directory
-2 (Optional) Filename to save checksum for new directory
oldDir Location of old directory
newDir Location of new directory
"
}
if [ $# -lt 2 ]; then
print_usage
exit 1
else
while getopts hL:1:2: options; do
case "${options}" in
h) print_usage
exit 1
;;
L) f_logFile=${OPTARG}
;;
1) f_oldChksumLog=${OPTARG}
v_f1flag=0
;;
2) f_newChksumLog=${OPTARG}
v_f2flag=0
;;
*) f_logFile=/dev/null
;;
esac
done
shift $((${OPTIND} - 1))
if [ -d "$1" ]; then
d_orgLoc="$1"
else
${echo} "Error: Original directory does not exist!"
print_usage
exit 1
fi
if [ -d "$2" ]; then
d_newLoc="$2"
else
${echo} "Error: New directory does not exist!"
print_usage
exit 1
fi
if [ -z ${f_oldChksumLog} ]; then
f_oldChksumLog=`${mktemp}`
elif [ -f ${f_oldChksumLog} ]; then
${echo} "Error: File ${f_oldChksumLog} exists! Please give another filename."
exit 2
else
${touch} ${f_oldChksumLog}
if [ ! -f ${f_oldChksumLog} ]; then
${echo} "Error: ${f_oldChksumLog} cannot be created!"
exit 4
fi
fi
if [ -z ${f_newChksumLog} ]; then
f_newChksumLog=`${mktemp}`
elif [ -f ${f_newChksumLog} ]; then
${echo} "Error: File ${f_newChksumLog} exists! Please give another filename."
exit 3
else
${touch} ${f_newChksumLog}
if [ ! -f ${f_newChksumLog} ]; then
${echo} "Error: File ${f_newChksumLog} cannot be created!"
exit 5
fi
fi
fi
${echo} "${find} \"${d_orgLoc}\" -type f -exec ${checksum} \"\\{\\}\" \\;" | ${tee} ${f_logFile}
${find} "${d_orgLoc}" -type f -exec ${checksum} "{}" \; | ${tee} ${f_oldChksumLog}
${echo} "${find} \"${d_newLoc}\" -type f -exec ${checksum} \"\\{\\}\" \\;" | ${tee} ${f_logFile}
${find} "${d_newLoc}" -type f -exec ${checksum} "{}" \; | ${tee} ${f_newChksumLog}
while read -r v_oldFileChksum v_oldFileName; do
if [[ `${grep} ${v_oldFileChksum} ${f_newChksumLog}` ]]; then
v_output="Okay: ${v_oldFileName} -> "
v_output="${v_output} `${grep} \"${v_oldFileChksum}\" \"${f_newChksumLog}\" | ${awk} '{print $2}'`"
else
v_output="ERROR: ${v_oldFileName} is missing"
v_missingFiles="${v_missingFiles} ${v_oldFileName}"
v_missingFilesCount=$((v_missingFilesCount+1))
fi
${echo} "${v_output}" | ${tee} ${f_logFile}
done < ${f_oldChksumLog}
#### cleanup ####
if [ "1" == ${v_f1flag} ]; then
${rm} ${f_oldChksumLog}
fi
if [ "1" == ${v_f2flag} ]; then
${rm} ${f_newChksumLog}
fi
if [ ${v_missingFilesCount} -gt 0 ]; then
${echo} "ERROR: ${v_missingFilesCount} files are missing:" | ${tee} ${f_logFile}
${echo} "ERROR: ${v_missingFiles}" | ${tee} ${f_logFile}
exit 99
else
${echo} "Success: ${v_missingFilesCount} files are missing" | ${tee} ${f_logFile}
exit 0
fi
- 2 comments
- 204 reads
Script: Zimbra Backup Script
Submitted by jmarki on 1 August 2009 - 4:29pmI am currently migrating out of Zimbra to a 3rd-party host. Just for archival, here's my Zimbra backup script. Just run this script using a cronjob every day. There will be a short downtime where Zimbra is shutdown to synchronise the last bit of emails, but that should be okay if you have a backup MX server.
This script creates a working live copy of the Zimbra directory, then shutdown Zimbra to sync the directory. The directory is then passed through star, into a small(er) file.
#!/bin/bash
########
## Zimbra backup script
##
## See
##
## Requires star, rsync, bash, gzip
## Does full backups of /opt/zimbra only. Tries to
## minimise zimbra shutdown time with a live rsync,
## then a offline rsync
##
## Changelog
## ---------
## 15 Feb 2009 (Junhao)
## - BUGFIX: deletes leftover archive.tgz tarball before
## creating new tar
## 21 Oct 2008 (Junhao)
## - added disk size to log
## - added tarball -t test
## 19 Oct 2008 (Junhao)
## - Initial commit
##
########
## config
_AWK=`which awk`
_CAT=`which cat`
_CD=cd #bash builtin
_CHECKSUM=`which sha1sum`
_DATE=`which date`
_DF=`which df`
_DU=`which du`
_ECHO=`which echo`
_MV=`which mv`
_MAIL=`which mail`
_RSYNC=`which rsync`
_RM=`which rm`
_SLEEP=`which sleep`
_SU=`which su`
_TAR=`which star`
_TOUCH=`which touch`
DATESTAMP=`date +%Y%b%d-%T`
RELEASE=`${_SU} - zimbra -c"zmcontrol -v"`
RELEASE=`${_ECHO} ${RELEASE} | ${_AWK} '{ print $2"-"$3"-"$4 }'`
TARBALL=zimbra-${RELEASE}-backup-full-${DATESTAMP}.tgz
LOGFILE=zimbra-${RELEASE}-backup-full-${DATESTAMP}.log
EMAIL=user@domain.co.m
MAILSUB="ZCS Backup Report on ${DATESTAMP}"
BKUPRETRIES=5
RESTARTRETRIES=100
ZIMBRADIR=/opt/zimbra
BASEDIR=/opt/zimbra-backups
WORKDIR=${BASEDIR}/working
LOGDIR=${BASEDIR}/logs
SAVEDIR=${BASEDIR}/saved
LOG=${LOGDIR}/${LOGFILE}
CHKSUMLOG=${SAVEDIR}/checksum
TARLOG=${WORKDIR}/tar.log
LOCK=${BASEDIR}/zimbra-backup.lock
TEMPARCHIVE=archive.tgz
function calc_downtime() {
if [ -z "${STARTTIME3}" ]; then
STARTTIME3=`date +%s`
fi
if [ -z "${ENDTIME}" ]; then
ENDTIME=`date +%s`
fi
TOTAL=$((${ENDTIME} - ${STARTTIME1}))
OFFLINE=$((${STARTTIME3} - ${STARTTIME2}))
${_ECHO} "Time taken: $((${TOTAL} / 3600)) hours $((${TOTAL} % 3600 / 60)) minutes $((${TOTAL} % 3600 % 60)) seconds" >> ${LOG}
${_ECHO} "Zimbra Offline: $((${OFFLINE} / 3600)) hours $((${OFFLINE} % 3600 / 60)) minutes $((${OFFLINE} % 3600 % 60)) seconds" >> ${LOG}
return 0
}
function force_restart() {
for (( i=0; i<=${RESTARTRETRIES} ; i=$(($i+1)) )); do
${_SU} - zimbra -c"zmcontrol stop"
${_SLEEP} 10
${_SU} - zimbra -c"zmcontrol start"
${_SLEEP} 10
${_SU} - zimbra -c"zmcontrol status"
if [[ $? == 0 ]]; then
${_ECHO} "Sucessfully restarted zimbra after $((${i}+1)) tries" >> ${LOG}
return 0
else
${_SLEEP} 10
fi
done
${_ECHO} "Could not restart zimbra after ${RESTARTRETRIES} tries" >> ${LOG}
return 254
}
function lock_set() {
${_TOUCH} ${LOCK}
}
function lock_remove() {
${_RM} ${LOCK}
}
function synchronise() {
for (( i=0; i<=${BKUPRETRIES} ; i=$(($i+1)) )); do
${_RSYNC} -avHK --delete --exclude=*.pid ${ZIMBRADIR} ${WORKDIR}
if [[ $? == 0 || $? == 24 ]]; then
return $?
fi
${_SLEEP} 10
done
return 254
}
function send_mail() {
calc_downtime
${_ECHO} "" >> ${LOG}
${_SU} - zimbra -c"zmcontrol status" >> ${LOG}
${_ECHO} "" >> ${LOG}
${_CAT} ${TARLOG} >> ${LOG}
${_RM} ${TARLOG}
${_CAT} ${LOG} | ${_MAIL} -s "${MAILSUB}" ${EMAIL}
}
function send_error() {
MAILSUB="[Failed] ${MAILSUB}"
send_mail
}
function send_success() {
MAILSUB="[Success] ${MAILSUB}"
send_mail
}
${_TOUCH} ${LOG}
lock_set
${_ECHO} "Server: `${_SU} - zimbra -c"zmhostname"`" >>${LOG}
${_ECHO} "Tarball: ${TARBALL}" >> ${LOG}
${_ECHO} "Logfile: ${LOG}" >> ${LOG}
${_ECHO} "Backup started at ${DATESTAMP}" >> ${LOG}
${_ECHO} "" >> ${LOG}
## Outputs time backup started for logging
STARTTIME1=`${_DATE} +%s`
## Online sync to working directory
${_ECHO} "Starting online rsync" >> ${LOG}
synchronise
if [[ $? == 254 ]]; then
${_ECHO} "Error in creating live copy" >> ${LOG}
send_error
lock_remove
exit 1
fi
STARTTIME2=`${_DATE} +%s`
## Shut down zimbra
${_SU} - zimbra -c"zmcontrol stop"
if [[ $? != 0 ]]; then
${_ECHO} "Error stopping zimbra" >> ${LOG}
${_SU} - zimbra -c"zmcontrol status" >>${LOG}
${_ECHO} "Aborting backup, force restarting zimbra" >> ${LOG}
force_restart
if [[ $? == 255 ]]; then
${_ECHO} "Trying again" >> ${LOG}
force_restart
fi
send_error
lock_remove
exit 2
fi
${_SLEEP} 10
## Offline sync to working directory
${_ECHO} "Starting offline rsync" >>${LOG}
synchronise
if [[ $? == 0 ]]; then
${_ECHO} "Offline rsync completed successfully" >> ${LOG}
elif [[ $? == 24 ]]; then
## some files disappeared, meaning some open process running
## wait a while, rerun rsync, and assume okay
${_SLEEP} 60
synchronise
else
${_ECHO} "Error in creating offline copy" >> ${LOG}
${_ECHO} "Aborting backup, force restarting zimbra" >> ${LOG}
force_restart
send_error
lock_remove
exit 3
fi
## Start zimbra
force_restart
if [[ $? == 254 ]]; then
${_ECHO} "Trying again" >> ${LOG}
force_restart
send_error
lock_remove
exit 4
fi
STARTTIME3=`date +%s`
## Synchronization sucessful, create archive
${_CD} ${WORKDIR}
if [ -f ${TARLOG} ]; then
${_RM} ${TARLOG}
else
${_TOUCH} ${TARLOG}
fi
if [ -f ${TEMPARCHIVE} ]; then
${_RM} ${TEMPARCHIVE}
fi
${_TAR} czf ${TEMPARCHIVE} ./ 1>> ${TARLOG} 2>&1
if [[ $? != 0 && $? != 254 ]]; then
${_ECHO} "Error creating tarball. Error Code: $?" >> ${LOG}
${_ECHO} "Aborting backup" >> ${LOG}
${_ECHO} "Tarball location: ${WORKDIR}/${TEMPARCHIVE}" >> ${LOG}
send_error
lock_remove
exit 5
fi
${_TAR} ztf ${TEMPARCHIVE}
if [[ $? == 0 ]]; then
${_MV} ${TEMPARCHIVE} ${SAVEDIR}/${TARBALL}
${_ECHO} "" >> ${LOG}
${_DU} -sh ${SAVEDIR}/${TARBALL} >> ${LOG}
${_DF} -h ${SAVEDIR}/${TARBALL} >> ${LOG}
else
${_ECHO} "Error validating tarball. Error Code: $?" >> ${LOG}
${_ECHO} "Aborting backup" >> ${LOG}
${_ECHO} "Tarball location: ${WORKDIR}/${TEMPARCHIVE}" >> ${LOG}
send_error
lock_remove
exit 6
fi
## Creating checksum
${_CD} ${SAVEDIR}
CHKSUM=`${_CHECKSUM} ${TARBALL}`
${_ECHO} "${CHKSUM}" >> ${CHKSUMLOG}
${_ECHO} "" >> ${LOG}
${_ECHO} "Checksum using ${_CHECKSUM}">> ${LOG}
${_ECHO} "${CHKSUM}" >> ${LOG}
## Backup done!
ENDTIME=`date +%s`
send_success
lock_remove
exit 0
- Add new comment
- 187 reads
Bash Scripting Tips
Submitted by jmarki on 12 January 2008 - 12:43amI went looking around for bash scripting tips, especially secure coding of bash. Can't find much information, so decided to consolidate whatever I found here.
- Salt string comparisons of variables to increase security
if [[ "a$?" == "a4" ]]; then
- Use the full paths to any binaries, either by hardcoding them into the script or use variable substitution. This prevents the script from executing incorrect/rogue binaries in the path.
/bin/grep "hardcoding the full path" * echo=/bin/echo ${echo} "From bash manpage under EXPANSION: The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion." - Change the environment path at the start of the script to ensure no rouge directories are in the PATH
#!/bin/bash # comments PATH=/bin:/usr/bin
- Write a function to explain the usage of the script
function print_usage () { ${echo} " $0 Usage: $0 [-a opts] [arguments] or $0 -h Description: Something fishy Options: -a opts (Optional) Options -h (Optional) Help arguments Smelly smelly fish " } - Here's a sample code snippet to process script options
if [ $# -lt 2 ]; then print_usage exit 1 else while getopts ha:b: options; do case "${options}" in h) print_usage exit 1 ;; a) flag=${options} ;; b) flag=${options} ;; *) echo "default case, everything else fits here" ;; esac done shift $((${OPTIND} - 1)) - Variables should be enclosed in parenthesis when used, to indicate exactly which variable you are using. Of course, this can prevent an exploit involving longer variable names.
a=erie ab=were if [[ "${a}b" == "erieb" ]]; then
- Add new comment
- 785 reads
