I all time receiving bot spam, so I block bots using this.
I'm very surprized but I still receive emails from Django, that requests pass Nginx.
Amount of email decreased from thousands per hour to tens per hour but from same referrers.
[Django] ERROR (EXTERNAL IP): Invalid HTTP_HOST header: 'www.elong.com'. You may need to add 'www.elong.com' to ALLOWED_HOSTS.
My blacklist looks:
map $http_referer $bad_referer {
hostnames;
default 0;
# Put regexes for undesired referers here
"*.cn" 1;
"~bitauto.com" 1;
"~hunantv.com" 1;
...
"~com.cn" 1;
"~sogou.com" 1;
"~sina.com.cn" 1;
"~weibo.com" 1;
...
"~114la.com" 1;
"~quer.com" 1;
"~elong.com" 1;
"~yninfo.com" 1;
"~news.cn" 1;
"~126.com" 1;
"~pcpop.com" 1;
}
If I make curl everything works:
curl --referer http://www.qunar.com my-host.com
curl: (52) Empty reply from server
curl --referer sina.com.cn my-host.com
curl: (52) Empty reply from server