#1 2011-03-20 5:58 pm

pedigree

uıɐbɐ ʎɐqǝ ɯoɹɟ pɹɐoqʎǝʞ ɐ buıʎnq ɹǝʌǝu ɯ,ı

From: New Zealand

Registered: 2008-04-16

Posts: 7,078

HOW TO: Convert bannedips to htaccess

This can be adapted to do the conversion.   This will, of course, overwrite your htaccess file

#!/bin/bash
cd /tmp
wget http://www.stopforumspam.com/downloads/bannedips.zip
unzip bannedips.zip
rm /tmp/.htaccess
cat bannedips.csv | awk '{print "deny from " $1}' RS="," >> /tmp/.htaccess
cp /tmp/.htaccess /var/www/html

replace the cat line with this if youre using nginx and includes

rm /etc/nginx/blacklist.conf
cat bannedips.csv | awk '{print "deny " $1 ";"}' RS="," >> /etc/nginx/blacklist.conf
killall -HUP nginx

randyoo

Member

Registered: 2012-03-27

Posts: 1

Re: HOW TO: Convert bannedips to htaccess

This yields a 5 MB .htaccess file.

I was looking for a solution to prevent known spammers from using server resources. I'm afraid to even try this on my shared host, considering the size of the file.

If I had my own server, I'd probably put this in a daily cron job, though...

zaphod

Jägermonster

From: USA

Registered: 2008-11-22

Posts: 2,985

Website

Re: HOW TO: Convert bannedips to htaccess

Another way, if you run a php site, and you don't want to use .htaccess for all files.

Load bannedips.csv into a $variable in .php, use substr_count against $variable with the IP address of the connection. Chunk it with die(), or give it a nice short 503 that uses less resources than a full header. This is how ZB Block uses bannedips.csv, it's fast, and very light on resources.

If the IP is not found, then destroy the temp variable, and continue execution.

Zap

P.S. I know this is off topic, but it's another simple way to use bannedips.csv. My apologies if it offended. If you need a code example, holler at me.

---

Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

John Darkhorse

Member

Registered: 2012-02-19

Posts: 319

Re: HOW TO: Convert bannedips to htaccess

#!/bin/bash
cd /tmp
wget http://www.stopforumspam.com/downloads/bannedips.zip
unzip bannedips.zip
rm /tmp/.htaccess
cat bannedips.csv | awk '{print "deny from " $1}' RS="," >> /tmp/.htaccess
cat /tmp/.htaccess >> /var/www/html/.htaccess

The above will not clobber your existing .htaccess, but will add the new content to the bottom of your existing .htaccess file

In order to keep from loading up doubles and triples each time you add IPs from the csv file, it's probably best to keep a local copy of any .htaccess file(s) and rsync it to your server.
This way, you can keep the file relatively small, and without duplicate entries from any future csv files.

Alex Kemp

Moderator

From: Nottingham, England

Registered: 2009-12-02

Posts: 2,436

Website

Re: HOW TO: Convert bannedips to htaccess

(if) you don't want to use .htaccess for all files

It is a *really* bad idea to use .htaccess if it starts to get into large sizes. You need to consider that one of the features of .htaccess is that it is:

• loaded on *every* access
  (that means that, depending on your system load, state of local caching, etc., etc., as an absolute minimum Apache will need to check for whether the last copy in memory has changed on disk or not)

• is checked for in *every* accessible directory within the PATH from root to CWD

• ...and so on

Chunk it with die(), or give it a nice short 503

Neither is a good idea, IMHO.

die() sends a http 200 OK Status - exactly the opposite of what you are probably trying to do
http 503 Status says "server has a temp problem; try again later", which is also probably not what you are trying to do.

"403 Forbidden You B*stard Spammer" is far more likely to be close to what you want to say.

(!Warning! Self promotion coming up!) Have a look at Conteg for an easy way to send whatever HTTP Status that you want.

---

Alex Kemp

IP Aggregation (Help) · Easy Firewall Generator (Help)  · Conteg v0.13.14

OnThePike

Member

From: Long Island, New York

Registered: 2010-12-30

Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

I also changed 503 (problem with me) to 403 (problem with you) on die for that exact reason.

bcworkz

Member

Registered: 2012-02-18

Posts: 182

Re: HOW TO: Convert bannedips to htaccess

Do you guys really think spammers pay attention to the type of error? Not sarcasm, I want to know what you think. I envision them as drive by shooters. They either drop their load, or they can't, and move on. They pay no attention to the aftermath of their activity. I can't believe they cache your IP to try again later on a 503. They'll be back, but not because of a 503.

Someone has suggested serving a 301 permanently moved to [REMOTE_ADDR]. I doubt it would ever work, but the mere thought of a spambot probing itself amuses me greatly.

FWIW, I serve a 403 on blocks as well.

Katana

Member

Registered: 2009-08-18

Posts: 1,886

Re: HOW TO: Convert bannedips to htaccess

Do you guys really think spammers pay attention to the type of error? Not sarcasm, I want to know what you think. I envision them as drive by shooters. They either drop their load, or they can't, and move on. They pay no attention to the aftermath of their activity. I can't believe they cache your IP to try again later on a 503. They'll be back, but not because of a 503.

Someone has suggested serving a 301 permanently moved to [REMOTE_ADDR]. I doubt it would ever work, but the mere thought of a spambot probing itself amuses me greatly.

FWIW, I serve a 403 on blocks as well.

it matters if it's a legit search engine that accidentally encounters it due to an issue in your own code.

---

うるさいうるさいうるさい！

OnThePike

Member

From: Long Island, New York

Registered: 2010-12-30

Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

With respect to my reply, the change is made for any bot or any denied to reflect the actual intention. What they do with it is their choice, but my message should be clear nonetheless. As well, there are human spammers that ignore warnings. Humans will receive and read the boot message and understand they'll have to move on.

zaphod

Jägermonster

From: USA

Registered: 2008-11-22

Posts: 2,985

Website

Re: HOW TO: Convert bannedips to htaccess

Good thinking guys, and that's why (my shameless self promotion coming up) ZB Block does serve x many 403s before switching to 503 (not problem with you, but more like "Go away, I'm too busy to serve you now"). I was just thinking more cheap and dirty.

Just not used to sending 403s without reasons.

Zap

---

Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

OnThePike

Member

From: Long Island, New York

Registered: 2010-12-30

Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

Maybe... serve a number of 400 Bad Request, then 403. Dunno... sounds reasonable.

Katana

Member

Registered: 2009-08-18

Posts: 1,886

Re: HOW TO: Convert bannedips to htaccess

400 is for a malformed request - it should not be used, that is not its purpose.  think more along the lines of having the request cut off mid-way and the webserver doesn't receive half or so of it.

---

うるさいうるさいうるさい！

OnThePike

Member

From: Long Island, New York

Registered: 2010-12-30

Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

Yeah you're right. Probably best to just stick with 403 across the board.

403 - Friendly warning.
403 - Behave or begone.
403 - See ya.

galerio

Member

Registered: 2012-11-25

Posts: 2

Re: HOW TO: Convert bannedips to htaccess

Can you tell me how to implement the php code to block those IPs? I think it could be usefull also for many others. I don't pretend a complete guide for every CMS out there, for sure, and I can only thank you for every little help you could give!
But I'm not a php expert, so if you can give me the php code to insert in my php files I will thank you very much!!!

zaphod

Jägermonster

From: USA

Registered: 2008-11-22

Posts: 2,985

Website

Re: HOW TO: Convert bannedips to htaccess

See the link in my signature.

It will do that, and much much more.

Zap

---

Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

galerio

Member

Registered: 2012-11-25

Posts: 2

Re: HOW TO: Convert bannedips to htaccess

Thanks zaphod, your software is a diamond! It works perfectly, does exactly what I need and my server is now running on low cpu and memory usage!!!
I will write a review about it. And thanks to stopforumspam for the bannedips list!

zaphod

Jägermonster

From: USA

Registered: 2008-11-22

Posts: 2,985

Website

Re: HOW TO: Convert bannedips to htaccess

Thanks for the compliment, but...

Give it some time to work. Usually it needs some minor adjustments, site to site. So I don't want a glowing review, and then find some little compatibility flaw that drives you nuts, and a reversal on the review.

But I do try to support it as best as I can over at http://www.spambotsecurity.com/forum . For help, go there.

Zap

---

Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

Schuppeste

Member

Registered: 2012-06-17

Posts: 3

Re: HOW TO: Convert bannedips to htaccess

The Simple Way,

1. Create Database
2. Create PHP class, check IP Via stopforumspam-Api, create access Accept Cookie, else IP-Ban via SQL
2. Create SQL-Database from Stopforumspam.Zip (Weekly refresh), Ban every ip in Database.
3. Check every ip with Api and Ban via IPTables

Last edited by Schuppeste (2013-02-03 10:34 am)

zaphod

Jägermonster

From: USA

Registered: 2008-11-22

Posts: 2,985

Website

Re: HOW TO: Convert bannedips to htaccess

Simpler:

$variable = @file_get_contents('bannedips.csv');
$ip2 = ',' . $ip . ',';
if (substr_count($variable,$ip2)){die('');}

No muss, no fuss, no SQL databases to detonate via injection, and nasty fast.

Or for more fun, snag my script in my signature link. It will stop bots, hacks, and other nasties too, plus do live lookup at stopforumspam if it doesn't find the IP in the local copy of bannedips.csv.

Zap

P.S. Damn I hate having to do 5 edits due to distraction.

P.P.S. This will only shut off bots accessing .php files where it resides... but, it will save checking every file against a huge list on load (.jpg, .gif, .png, .js, .css, etc. bots don't care about, but if done in .htaccess, all valid users will have to wait.)

---

Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

Schuppeste

Member

Registered: 2012-06-17

Posts: 3

Re: HOW TO: Convert bannedips to htaccess

my solution:

http://pastebin.com/4KWTubqp

pedigree

uıɐbɐ ʎɐqǝ ɯoɹɟ pɹɐoqʎǝʞ ɐ buıʎnq ɹǝʌǝu ɯ,ı

From: New Zealand

Registered: 2008-04-16

Posts: 7,078

Re: HOW TO: Convert bannedips to htaccess

my solution:

http://pastebin.com/4KWTubqp

I should point out that anyone using with on CloudFlare, should use the correct $_SERVER variable and not REMOTE_ADDR

Schuppeste

Member

Registered: 2012-06-17

Posts: 3

Re: HOW TO: Convert bannedips to htaccess

Sry i dont know what you mean peddigree ..

no english

Updated: http://pastebin.com/isHdbba4

Last edited by Schuppeste (2013-02-03 8:28 pm)

NeoFox

Member

From: WI, USA, Earth

Registered: 2013-09-26

Posts: 830

Website

Re: HOW TO: Convert bannedips to htaccess

Can anyone explain how to use this with fluxbb please? I'm not quite making the connection on how to use lol.

Papa Parrot

Member

From: Mexico

Registered: 2011-08-19

Posts: 1,826

Website

Re: HOW TO: Convert bannedips to htaccess

Can anyone explain how to use this with fluxbb please? I'm not quite making the connection on how to use lol.

Not sure if you mean Zbblock, or some of the other code shown, I am using zbblock, zbblock   for a fluxbb forum, it works good, instructions come with the download,  when you upload it to your host, and run the set up, you will get a script, "string", that needs to be added right at the beginning of the config.php  and that is all there is to it.
The instructions are easy to follow,

---

Everything is for the Birds

And now, Papa Parrot and his kids

Flavio93Zena

Member

From: Italy

Registered: 2014-12-24

Posts: 15

Re: HOW TO: Convert bannedips to htaccess

Since I can't see this being treated properly I thought about posting and adapting an old guide that I had made about banning users via .htaccess. This of course if you don't want to ban ALL SFS database but just specific IPs/IP ranges. It's fairly obvious though that if you ban the whole SFS database your system will run slower so personally I went down this path.

As this started to become a very frequently asked question and I was directly concerned about it I decided to make this guide, hopefully someone of the team will approve it

A forum software comes with a built-in ban feature that allows admins to use an IP ban to ban users but this function has several problems that cannot be fixed:

• Each time a user loads a page it has to check his IP to see whether he is allowed or not to access that page;

• It has to check all users connected comparing their IPs to the ones in the banned list;

• Banned users will spam your error log, especially if they are spambots attempting to access the page hundreds of times (100 denied accesses mean 100 errors in the log for ONE spambot);

• This check will cause your system to terribly slow down;

• It can't be changed as creating a table and a check is the only way to ban people through any forum software.

How should I ban users?

You should ban users using your .htaccess file that is stored on your server host or either create it. If you don't know how to create it just create a .txt file, put the code in there and when you are done just rename its extension to ".htaccess". Like: example.txt --> rename --> example.htaccess
Open it and add the provided code, better at its end

What should I add to my .htaccess file?

Thanks to Lou69 I can provide these 2 links as reference:

• http://httpd.apache.org/docs/2.2/mod/mo … html#order

• http://httpd.apache.org/docs/current/mo … html#allow

But as they contain a lot of other stuff I will be explaining the only things that we are looking for.
Server can be setup to either accept or deny access to any IP not included in the list you are going to make. How? With this:

order allow,deny

or

order deny,allow

Basically the former sets up the server to deny any unlisted entries while the latter sets it up to allow any unlisted entries by default. Be careful because choosing between these two is vital as they order the operations being performed by the server itself. What do I mean? I mean that "allow,deny" will be processing all whitelisted IPs and then banned ones, while "deny,allow" will be doing the opposite. Like I said choosing between one of these is crucial and you have to type everything properly or your site might be unavailable to everybody.

Let's make some examples, let's say I want to ban this IP 30.31.32.33 and I want any other IP to be allowed on my website, how do I do that? You can do it in 2 ways:

order allow,deny
deny from 30.31.32.33
allow from all

or either

order deny,allow
deny from 30.31.32.33

As you can see they are slightly different

• The first one has the "allow from all" in the end as the default state of the server is to deny any unlisted entried

• The second one doesn't have that string because as the server is processing the banned entries before the allowed ones typing "allow from all" would unban the banned users ( )

How to ban an IP range?

Let's say I want to ban all users with this IP range 30.31.32.XX where those XX stand for any number. You will have to do this:

order allow,deny
deny from 30.31.32.0/24
allow from all

or either

order deny,allow
deny from 30.31.32.0/24

I will not be explaining the reasons of that 24 as it's really complicated for new users.

Another example, I want to ban this IP range: 30.31.XX.XX where those XX stand for any number. You will have to do this:

order allow,deny
deny from 30.31.0/16
allow from all

or either

order deny,allow
deny from 30.31.0/16

Another example again, I want to ban this IP range: 30.XX.XX.XX where those XX stand for any number. You will have to do this:

order allow,deny
deny from 30.0/8
allow from all

or either

order deny,allow
deny from 30.0/8

You can get more informations from here: http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#IPv4_CIDR_blocks

Do you need an exception to unban an IP within a banned IP range?

Let's say I want this IP range to be banned (30.31.32.XX) but I want this IP (30.31.32.33) to be allowed access. How? (Of course any other user is supposed to be granted access)

order deny,allow
deny from 30.31.32.0/24
allow from 30.31.32.33

You just can't do it with the "order allow,deny" as it would be processing banned users after allowed ones and it would inevitably banning the IP exception as well.

Are you running Cloudflare on your forum?

If you are running Cloudflare on your forum all the above codes will not work properly, you will need to change some things. The "order allow,deny" and "order deny,allow" will work with Cloudflare as well but since Cloudflare is masking all IP addresses as if they were (usually) all 108.162.xxx.xxx then any ban made with the aforementioned method will obviously not work.
How to ban the proper IP addresses then? You need to add this code:

SetEnvIf X-FORWARDED-FOR 1.2.3.4 deniedip

or

SetEnvIf X-FORWARDED-FOR 1.2.3.4 allowedip

These must be used in ALL ban strings.

At the end of it you must also add

deny from env=deniedip

in order to get it to work obviously you can change all of them to

allow from env=allowedip

if you want to restrict access to only a few people.

CIDR notation (0/8, 0/16, 0/24) will work as long as you add the

SetEnvIf X-FORWARDED-FOR

code.

Example 1 (IP 1.2.3.4 is the only banned entry):

SetEnvIf X-FORWARDED-FOR 1.2.3.4 deniedip
order allow,deny
deny from env=deniedip

Or example 2 (IP 1.2.3.4 is the only IP with granted access):

SetEnfIf X-FORWARDED-FOR 1.2.3.4 allowedip
order deny,allow
deny from all
allow from env=allowedip

This part gave me a lot of headaches and in my opinion is the most complicated... I tried to be as clear as possible but if you don't understand it feel free to ask about it
As a reference the Cloudflare part has been adapted from here: http://blog.ergatides.com/2011/09/07/se … other-cdn/ it's not exactly the same thing but the basic inspiration and reference is there.

Do not put whitespaces between "order," and "deny" or the opposite, just type "order allow,deny" or "order deny,allow" and no whitespaces next to the comma! Same thing applies to Cloudflare-related strings.

Credits to (hopefully I will remember all of you): Arantor, a10, Antechinus, br360, emanuele and Shambles thank you all for your many tweaks and help.

Special thanks again to Lou69 for the reference links.

Adapting the BBC of this post was really painful, I hope this will help at least
Best regards,
Flavio.

---

I'm an angel with a shotgun,
fighting 'til the war's won,
I don't care if heaven won't take me back.