【www.bbyears.com--linux】
以前使用虚拟主机的时候，查看网站运行日志，就发现了很多异常的恶意的访问。当时因为自己并没有系统权限没办法对这些行为进行屏蔽。现在有了自己的云主机，前端时间查看日志，又发现了很多恶意访问。正好可以通过最近对shell的学习，来做一个简单的日志分析工具，来屏蔽一些这样的操作。
首先，所谓的分析工具，肯定是建立在人为的分析的基础上的。我们来看一点我的域名运行日志：
78.56.78.115 - - [21/May/2014:16:54:30 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:32 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
42.159.83.42 - - [21/May/2014:16:54:36 +0800] "HEAD /521php.rar HTTP/1.1" 404 0 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:36 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:38 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:41 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:45 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:47 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:50 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:53 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:56 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:54:58 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:00 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
42.159.83.42 - - [21/May/2014:16:55:05 +0800] "HEAD /521php.zip HTTP/1.1" 404 0 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:05 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:07 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:11 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:14 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:17 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:21 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:23 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:25 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:27 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:31 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
42.159.83.42 - - [21/May/2014:16:55:31 +0800] "HEAD /wwwroot.rar HTTP/1.1" 404 0 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:33 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:37 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:39 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:41 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:44 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:50 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:52 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
42.159.83.42 - - [21/May/2014:16:55:56 +0800] "HEAD /wwwroot.zip HTTP/1.1" 404 0 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:57 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:55:59 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:56:01 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:56:03 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
78.56.78.115 - - [21/May/2014:16:56:05 +0800] "POST /wp-login.php HTTP/1.0" 200 3198 "-" "-" -
很明显可以看到有ip在恶意暴力破解我的登录信息，还有一个ip在尝试下载我的网站程序。这样的ip我们对比正常的访问日志会发现，他没有头信息，即你浏览器等等的信息，而且一般访问都是get或者post访问，而这里有head访问。所以我们可以将这样的ip加入防火墙或者nginx的ip黑名单，当然还可以有其他规则，比较访问的频次，404出现的频次，当然有的访问你会发现他访问你的首页，却没有加载你的js，css等，当然这里也有缓存的影响，但是即使缓存也会发送304的确认信息。当然还有很多其他的恶意访问行为。
然后，针对这些规则，我们来写处理逻辑和操作即可，写这个之前，我们先做一个nginx的ip黑名单。我这里没有再使用linux的防火墙，因为防火墙操作不太方便，频繁重启有一定的影响，而nginx的黑名单，是可以平滑重启的，而且是让指定ip显示403错误，但是可以访问的，这样就给人家一个改过自新的机会啊是吧，如果人家访问正常了，就可以将此ip移除黑名单。
配置nginx ip黑名单：
在你的nginx.conf配置文件中，包含一个blocksip.conf;
blocksip.conf 存放黑名单ip；
这样写
deny 1.1.1.2;
deny 1.1.1.1;
deny 1.1.1.4;
deny 1.1.3.1;
deny http://www.111cn.net;
然后我们写个shell脚本，自动添加、移除ip，并平滑重启nginx，并发送邮件通知
editblocksip.sh
代码如下 #!/bin/shfile="/etc/nginx/conf/blocksip.conf"
file2="/etc/nginx/conf/blocksip.bak"
v1=$1
v2=$2
if [ $v1 = "add" ]
then
deny_info=`cat $file | grep $2`
if [ -z "$deny_info" ]
then
`echo "deny $v2;" >> $file`
fi
else if [ $v1 = "del" ]
then
`cat $file | grep -v $2 > $file2`
`cat $file2 > $file`
fi
fi
`/usr/sbin/nginx -s reload`
`cat $file|mail -s "edit blocks list" zhangcunchao@izptec.com zhangcunchao_cn@163.com`
exit 0
使用比较简单
代码如下
#添加
sh editblocksip.sh add 1.1.1.1
#移除
sh editblocksip.sh del 1.1.1.1
最后，我们只需要写个脚本，来分析nginx日志文件，有符合条件的记录，来触发这个shell脚本添加ip黑名单即可
思路是这样的，首先记录日志最大行号，然后每次执行分析脚本，从指定行号读取剩下所有记录（tail -n +5 file），并再次记录最大行号（wc）,然后使用awk，逐行分析记录，并切割字符进行匹配判断，符合条件的，提取ip，加入ip黑名单；
这个分析脚本，可以用crontab 或者 做成守护进程来跑。根据自己网站的运行情况，定义执行频次，几秒，几分钟等等。
这样就实现了一个简单的日志分析工具！