Shell script to run iozone with 200 MB file and process the output:
----------------------------------------------------------------------
#!/bin/sh
#exec > ,.r.log 2>&1
set -x
date=`date +%y%m%d_%H%M`
iozone -a -g 200m > iozone_$date.out
ln -sf iozone_$date.out iozone.out
./iozoneplot -a -p
echo > index.html
for j in *.jpg;do
echo '
' >> index.html
done
Iozoneplot script used above to process iozone data with gnuplot:
-------------------------------------------------------------------
#!/bin/sh
#set -x
# $Id: pplot,v 1.3 2004/05/31 23:10:18 pzi Exp $
usage() {
cat <<-EOF
Process iozone output files to produce gnuplot display or pixmaps.
Usage: $0 [-h] [-p] [ ...]
one of:
write
Write: This test measures the performance of writing a new file. When
a new file is written not only does the data need to be stored but also
the overhead information for keeping track of where the data is located
on the storage media. This overhead is called the metadata It consists
of the directory information, the space allocation and any other data
associated with a file that is not part of the data contained in the
file. It is normal for the initial write performance to be lower than
the performance of re-writing a file due to this overhead information.
rewrite
Re-write: This test measures the performance of writing a file that
already exists. When a file is written that already exists the work
required is less as the metadata already exists. It is normal for the
rewrite performance to be higher than the performance of writing a
new file.
read
Read: This test measures the performance of reading an existing file.
reread
Re-Read: This test measures the performance of reading a file that was
recently read. It is normal for the performance to be higher as the
operating system generally maintains a cache of the data for files that
were recently read. This cache can be used to satisfy reads and improves
the performance.
random-read
Random Read: This test measures the performance of reading a file with
accesses being made to random locations within the file. The performance
of a system under this type of activity can be impacted by several
factors such as: Size of operating systems cache, number of disks,
seek latencies, and others.
random-write
Random Write: This test measures the performance of writing a file
with accesses being made to random locations within the file. Again the
performance of a system under this type of activity can be impacted by
several factors such as: Size of operating systems cache, number of
disks, seek latencies, and others.
bkwd-read
Backwards Read: This test measures the performance of reading a file
backwards. This may seem like a strange way to read a file but in
fact there are applications that do this. MSC Nastran is an example
of an application that reads its files backwards. With MSC Nastran,
these files are very large (Gbytes to Tbytes in size). Although many
operating systems have special features that enable them to read a file
forward more rapidly, there are very few operating systems that detect
and enhance the performance of reading a file backwards.
record-rewrite
Record Rewrite: This test measures the performance of writing and
re-writing a particular spot within a file. This hot spot can have very
interesting behaviors. If the size of the spot is small enough to fit in
the CPU data cache then the performance is very high. If the size of the
spot is bigger than the CPU data cache but still fits in the TLB then
one gets a different level of performance. If the size of the spot is
larger than the CPU data cache and larger than the TLB but still fits in
the operating system cache then one gets another level of performance,
and if the size of the spot is bigger than the operating system cache
then one gets yet another level of performance.
stride-read
Strided Read: This test measures the performance of reading a file with
a strided access behavior. An example would be: Read at offset zero for
a length of 4 Kbytes, then seek 200 Kbytes, and then read for a length
of 4 Kbytes, then seek 200 Kbytes and so on. Here the pattern is to
read 4 Kbytes and then Seek 200 Kbytes and repeat the pattern. This
again is a typical application behavior for applications that have
data structures contained within a file and is accessing a particular
region of the data structure. Most operating systems do not detect this
behavior or implement any techniques to enhance the performance under
this type of access behavior. This access behavior can also sometimes
produce interesting performance anomalies. An example would be if the
applications stride causes a particular disk, in a striped file system,
to become the bottleneck.
fwrite
Fwrite: This test measures the performance of writing a file using the
library function fwrite(). This is a library routine that performs
buffered write operations. The buffer is within the users address
space. If an application were to write in very small size transfers
then the buffered & blocked I/O functionality of fwrite() can enhance
the performance of the application by reducing the number of actual
operating system calls and increasing the size of the transfers when
operating system calls are made. This test is writing a new file so
again the overhead of the metadata is included in the measurement.
frewrite
Frewrite: This test measures the performance of writing a file using
the library function fwrite(). This is a library routine that performs
buffered & blocked write operations. The buffer is within the users
address space. If an application were to write in very small size
transfers then the buffered & blocked I/O functionality of fwrite()
can enhance the performance of the application by reducing the number
of actual operating system calls and increasing the size of the
transfers when operating system calls are made. This test is writing
to an existing file so the performance should be higher as there are no
metadata operations required.
fread
Fread: This test measures the performance of reading a file using the
library function fread(). This is a library routine that performs buffered
& blocked read operations. The buffer is within the users address
space. If an application were to read in very small size transfers
then the buffered & blocked I/O functionality of fread() can enhance
the performance of the application by reducing the number of actual
operating system calls and increasing the size of the transfers when
operating system calls are made.
freread
Freread: This test is the same as fread above except that in this test
the file that is being read was read in the recent past. This should
result in higher performance as the operating system is likely to have
the file data in cache.
EOF
}
doit() {
cat < job.gp
$TERMINAL
$OUTPUT
# Top down:
#set view 0, 0, 1, 1
# Normal:
#set view 60, 30, 1, 1
set view 60, 30, 1.2, 1
# sideways:
#set view 90, 0, 1, 1
#set view 40, 210, 1, 1
set title "$title"
#set grid lt 2 lw 1
#set surface
#set parametric
set xtics
set ytics
#set logscale x 2
#set xrange [5:24]
#set xrange [5:24]
#set logscale y 2
set autoscale z
#set zrange [0:500]
#set xrange [4:16384]
#set xlabel "File size in 2^n KBytes"
#set ylabel "Record size in 2^n Kbytes"
set xlabel "File size - MB"
set ylabel "Record size - MB"
set zlabel "MB/sec"
set data style lines
set contour both
#set dgrid3d 80,80,3
set dgrid3d 40,40,3
#set dgrid3d 20,20,6
splot 'iozone.txt' using (\$1/1024):(\$2/1024):(\$$COL/1024) title "$JOB performance"
$PAUSE
EOF
gnuplot job.gp
cjpeg -quality 95 $JOB.pbm > $JOB.jpg
#rm -f $JOB.pbm $JOB.png
}
# MAIN() STARTS HERE:
#-------------------------------
TERMINAL='set terminal x11'
OUTPUT=
PAUSE='pause -1 "Hit return to continue"'
jobs=
while [ $# -gt 0 ];do
case $1 in
write) jobs="$jobs $1_3" ;;
rewrite) jobs="$jobs $1_4" ;;
read) jobs="$jobs $1_5" ;;
reread) jobs="$jobs $1_6" ;;
random-read) jobs="$jobs $1_5" ;;
random-write) jobs="$jobs $1_8" ;;
bkwd-read) jobs="$jobs $1_9" ;;
record-rewrite) jobs="$jobs $1_10" ;;
stride-read) jobs="$jobs $1_11" ;;
fwrite) jobs="$jobs $1_12" ;;
frewrite) jobs="$jobs $1_13" ;;
fread) jobs="$jobs $1_14" ;;
freread) jobs="$jobs $1_15" ;;
-h) usage; exit ;;
-a) jobs="
write_3
rewrite_4
read_5
reread_6
random-read_7
random-write_8
bkwd-read_9
record-rewrite_10
stride-read_11
fwrite_12
frewrite_13
fread_14
freread_15"
;;
-p)
#TERMINAL='set terminal png small color'
TERMINAL='set terminal pbm small color'
PAUSE=
;;
esac
shift
done
if [ -f iozone.out ];then
grep '^[ \t]*[0-9].*' iozone.out > iozone.txt
fi
if [ -f note ];then
while read line; do
title="$title\n$line"
done < note
fi
for job in $jobs;do
JOB=`echo $job | sed 's/_.*$//'`
COL=`echo $job | sed 's/^.*_//'`
[ X"$PAUSE" = X ] && OUTPUT="set output \"$JOB.pbm\""
echo $JOB
doit
done
rm -f job.gp iozone.txt