Default apache access log file location:
RHEL / Red Hat / CentOS / Fedora Linux Apache access file location - /var/log/httpd/access_log
Debian / Ubuntu Linux Apache access log file location - /var/log/apache2/access.log
FreeBSD Apache access log file location - /var/log/httpd-access.log
To find exact apache log file location, you can use grep command:
# grep CustomLog /usr/local/etc/apache22/httpd.conf
# grep CustomLog /etc/apache2/apache2.conf
# grep CustomLog /etc/httpd/conf/httpd.conf
Sample output:
# a CustomLog directive (see below).
# CustomLog "/var/log/httpd-access.log" common
CustomLog "/var/log/httpd-access.log" combined
These assume you use a standard log format. If not you may have to adjust them to suit your format.
Most viewed pages (top ten).
Code:
awk '{print $7}' /path/to/log |sort |uniq -c |sort -rn
|head -10
Top ten referrers:
Code:
awk '{print $11}' /path/to/log |sort |uniq -c |sort -rn
|head -10
Search logs:
Code:
grep /path/to/log query |awk '{print $8}'|tail -n+5|sort|uniq|tr -d [1]
And here is a script I wrote a long time ago that incorparates these and few others.
Code:
#!/bin/bash
# usage
# ./this_script search pattern log_file
# all hosts with 5 or more matches of the given pattern will be banned
# ./this_script ban_from_log log_file
# all hosts that appear more than 5 times in the given log file will be banned
# ./this_script ban_rfi log_file
# bans all hosts that match the rfi pattern(rfi's and proxy requests)
# ./this_script most_viewed log_file
# shows the top ten viewed pages with the number of views
# ./this_script statuses response log_file
# shows the top ten viewed pages for the given response header e.g. 404, 200 ect
# ./this_script referrers log_file
# shows the top ten referrers and page views for each
#
# author tomfmason
ban_file=/etc/hosts.deny
function ban_ip() {
exists=`grep ${1} $ban_file`
if [ ! "$exists" ]; then
echo "ALL: ${1}" >> $ban_file
fi
}
function search() {
ret=`grep ${1} ${2} |awk '{print $8}'|tail -n+5|sort|uniq|tr -d [1]`
for r in $ret; do
ban_ip $r
done
}
function ban_from_log() {
ret=`awk '{print $8}' ${1}|tail -n+5|sort|uniq|tr -d [1]`
for r in $ret; do
ban_ip $r
done
}
function ban_rfi() {
ret=`awk '{print $1 " " $7}' ${1} |grep -iE '(http|https|ftp)'|awk '{print $1}'|tail -n+5|sort|uniq`
for r in $ret; do
ban_ip $r
done
}
function most_viewed() {
awk '{print $7}' ${1} |sort |uniq -c |sort -rn |head -10
}
function statuses() {
awk '{print $7 " " $9}' ${2} |grep -iE '${1}' |sort | uniq -c | sort -rn |awk '{print $1 " " $2 }' |head -10
}
function referrers() {
awk '{print $11}' ${1} |sort |uniq -c |sort -rn |head -10
}
if type "$1" | grep -qF "$1 is a function"; then "$@"; fi
Note:
As this was written a long time ago and now I would not suggest using hosts.deny to ban an ip. Iptables would be a much better choice. Believe you can fix it yourself ;-)
