You are not logged in.

#1 2011-03-20 5:58 pm

pedigree
uıɐbɐ ʎɐqǝ ɯoɹɟ pɹɐoqʎǝʞ ɐ buıʎnq ɹǝʌǝu ɯ,ı
From: New Zealand
Registered: 2008-04-16
Posts: 7,054

HOW TO: Convert bannedips to htaccess

This can be adapted to do the conversion.   This will, of course, overwrite your htaccess file

#!/bin/bash
cd /tmp
wget http://www.stopforumspam.com/downloads/bannedips.zip
unzip bannedips.zip
rm /tmp/.htaccess
cat bannedips.csv | awk '{print "deny from " $1}' RS="," >> /tmp/.htaccess
cp /tmp/.htaccess /var/www/html

replace the cat line with this if youre using nginx and includes

rm /etc/nginx/blacklist.conf
cat bannedips.csv | awk '{print "deny " $1 ";"}' RS="," >> /etc/nginx/blacklist.conf
killall -HUP nginx

Offline

#2 2012-03-30 4:28 am

randyoo
Member
Registered: 2012-03-27
Posts: 1

Re: HOW TO: Convert bannedips to htaccess

This yields a 5 MB .htaccess file.

I was looking for a solution to prevent known spammers from using server resources. I'm afraid to even try this on my shared host, considering the size of the file.

If I had my own server, I'd probably put this in a daily cron job, though...

Offline

#3 2012-03-30 6:28 am

zaphod
Jägermonster
From: USA
Registered: 2008-11-22
Posts: 2,985
Website

Re: HOW TO: Convert bannedips to htaccess

Another way, if you run a php site, and you don't want to use .htaccess for all files.

Load bannedips.csv into a $variable in .php, use substr_count against $variable with the IP address of the connection. Chunk it with die(), or give it a nice short 503 that uses less resources than a full header. This is how ZB Block uses bannedips.csv, it's fast, and very light on resources.

If the IP is not found, then destroy the temp variable, and continue execution.

Zap smile

P.S. I know this is off topic, but it's another simple way to use bannedips.csv. My apologies if it offended. If you need a code example, holler at me.


Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

Offline

#4 2012-03-30 6:45 am

John Darkhorse
Member
Registered: 2012-02-19
Posts: 319

Re: HOW TO: Convert bannedips to htaccess

#!/bin/bash
cd /tmp
wget http://www.stopforumspam.com/downloads/bannedips.zip
unzip bannedips.zip
rm /tmp/.htaccess
cat bannedips.csv | awk '{print "deny from " $1}' RS="," >> /tmp/.htaccess
cat /tmp/.htaccess >> /var/www/html/.htaccess

The above will not clobber your existing .htaccess, but will add the new content to the bottom of your existing .htaccess file

In order to keep from loading up doubles and triples each time you add IPs from the csv file, it's probably best to keep a local copy of any .htaccess file(s) and rsync it to your server.
This way, you can keep the file relatively small, and without duplicate entries from any future csv files.

Offline

#5 2012-03-30 6:23 pm

Alex Kemp
Moderator
From: Nottingham, England
Registered: 2009-12-02
Posts: 2,420
Website

Re: HOW TO: Convert bannedips to htaccess

zaphod wrote:

(if) you don't want to use .htaccess for all files

It is a *really* bad idea to use .htaccess if it starts to get into large sizes. You need to consider that one of the features of .htaccess is that it is:

  • loaded on *every* access
    (that means that, depending on your system load, state of local caching, etc., etc., as an absolute minimum Apache will need to check for whether the last copy in memory has changed on disk or not)

  • is checked for in *every* accessible directory within the PATH from root to CWD

  • ...and so on

Chunk it with die(), or give it a nice short 503

Neither is a good idea, IMHO.

die() sends a http 200 OK Status - exactly the opposite of what you are probably trying to do
http 503 Status says "server has a temp problem; try again later", which is also probably not what you are trying to do.

"403 Forbidden You B*stard Spammer" is far more likely to be close to what you want to say.

(!Warning! Self promotion coming up!) Have a look at Conteg for an easy way to send whatever HTTP Status that you want.

Offline

#6 2012-03-30 6:27 pm

OnThePike
Member
From: Long Island, New York
Registered: 2010-12-30
Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

I also changed 503 (problem with me) to 403 (problem with you) on die for that exact reason.

Offline

#7 2012-03-30 8:33 pm

bcworkz
Member
Registered: 2012-02-18
Posts: 182

Re: HOW TO: Convert bannedips to htaccess

Do you guys really think spammers pay attention to the type of error? Not sarcasm, I want to know what you think. I envision them as drive by shooters. They either drop their load, or they can't, and move on. They pay no attention to the aftermath of their activity. I can't believe they cache your IP to try again later on a 503. They'll be back, but not because of a 503.

Someone has suggested serving a 301 permanently moved to [REMOTE_ADDR]. I doubt it would ever work, but the mere thought of a spambot probing itself amuses me greatly.

FWIW, I serve a 403 on blocks as well.

Offline

#8 2012-03-30 8:34 pm

Katana
Member
Registered: 2009-08-18
Posts: 1,886

Re: HOW TO: Convert bannedips to htaccess

bcworkz wrote:

Do you guys really think spammers pay attention to the type of error? Not sarcasm, I want to know what you think. I envision them as drive by shooters. They either drop their load, or they can't, and move on. They pay no attention to the aftermath of their activity. I can't believe they cache your IP to try again later on a 503. They'll be back, but not because of a 503.

Someone has suggested serving a 301 permanently moved to [REMOTE_ADDR]. I doubt it would ever work, but the mere thought of a spambot probing itself amuses me greatly.

FWIW, I serve a 403 on blocks as well.

it matters if it's a legit search engine that accidentally encounters it due to an issue in your own code.


うるさいうるさいうるさい!

Offline

#9 2012-03-30 8:40 pm

OnThePike
Member
From: Long Island, New York
Registered: 2010-12-30
Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

With respect to my reply, the change is made for any bot or any denied to reflect the actual intention. What they do with it is their choice, but my message should be clear nonetheless. As well, there are human spammers that ignore warnings. Humans will receive and read the boot message and understand they'll have to move on.

Offline

#10 2012-03-31 7:21 am

zaphod
Jägermonster
From: USA
Registered: 2008-11-22
Posts: 2,985
Website

Re: HOW TO: Convert bannedips to htaccess

Good thinking guys, and that's why (my shameless self promotion coming up) ZB Block does serve x many 403s before switching to 503 (not problem with you, but more like "Go away, I'm too busy to serve you now"). I was just thinking more cheap and dirty.

Just not used to sending 403s without reasons.

Zap smile


Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

Offline

#11 2012-03-31 7:23 am

OnThePike
Member
From: Long Island, New York
Registered: 2010-12-30
Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

Maybe... serve a number of 400 Bad Request, then 403. Dunno... sounds reasonable.

Offline

#12 2012-03-31 1:50 pm

Katana
Member
Registered: 2009-08-18
Posts: 1,886

Re: HOW TO: Convert bannedips to htaccess

400 is for a malformed request - it should not be used, that is not its purpose.  think more along the lines of having the request cut off mid-way and the webserver doesn't receive half or so of it.


うるさいうるさいうるさい!

Offline

#13 2012-03-31 2:05 pm

OnThePike
Member
From: Long Island, New York
Registered: 2010-12-30
Posts: 1,977

Re: HOW TO: Convert bannedips to htaccess

Yeah you're right. Probably best to just stick with 403 across the board.

403 - Friendly warning.
403 - Behave or begone.
403 - See ya.

Offline

#14 2012-12-17 8:00 pm

galerio
Member
Registered: 2012-11-25
Posts: 2

Re: HOW TO: Convert bannedips to htaccess

Can you tell me how to implement the php code to block those IPs? I think it could be usefull also for many others. I don't pretend a complete guide for every CMS out there, for sure, and I can only thank you for every little help you could give!
But I'm not a php expert, so if you can give me the php code to insert in my php files I will thank you very much!!!

Offline

#15 2012-12-17 8:03 pm

zaphod
Jägermonster
From: USA
Registered: 2008-11-22
Posts: 2,985
Website

Re: HOW TO: Convert bannedips to htaccess

See the link in my signature.

It will do that, and much much more.

Zap smile


Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

Offline

#16 2012-12-18 3:47 pm

galerio
Member
Registered: 2012-11-25
Posts: 2

Re: HOW TO: Convert bannedips to htaccess

Thanks zaphod, your software is a diamond! It works perfectly, does exactly what I need and my server is now running on low cpu and memory usage!!!
I will write a review about it. And thanks to stopforumspam for the bannedips list!

Offline

#17 2012-12-20 12:42 am

zaphod
Jägermonster
From: USA
Registered: 2008-11-22
Posts: 2,985
Website

Re: HOW TO: Convert bannedips to htaccess

Thanks for the compliment, but...

Give it some time to work. Usually it needs some minor adjustments, site to site. So I don't want a glowing review, and then find some little compatibility flaw that drives you nuts, and a reversal on the review.

But I do try to support it as best as I can over at http://www.spambotsecurity.com/forum . For help, go there.

Zap smile


Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

Offline

#18 2013-02-03 10:33 am

Schuppeste
Member
Registered: 2012-06-17
Posts: 3

Re: HOW TO: Convert bannedips to htaccess

The Simple Way,

1. Create Database
2. Create PHP class, check IP Via stopforumspam-Api, create access Accept Cookie, else IP-Ban via SQL
2. Create SQL-Database from Stopforumspam.Zip (Weekly refresh), Ban every ip in Database.
3. Check every ip with Api and Ban via IPTables

Last edited by Schuppeste (2013-02-03 10:34 am)

Offline

#19 2013-02-03 11:19 am

zaphod
Jägermonster
From: USA
Registered: 2008-11-22
Posts: 2,985
Website

Re: HOW TO: Convert bannedips to htaccess

Simpler:

$variable = @file_get_contents('bannedips.csv');
$ip2 = ',' . $ip . ',';
if (substr_count($variable,$ip2)){die('');}

No muss, no fuss, no SQL databases to detonate via injection, and nasty fast.

Or for more fun, snag my script in my signature link. It will stop bots, hacks, and other nasties too, plus do live lookup at stopforumspam if it doesn't find the IP in the local copy of bannedips.csv.

Zap smile

P.S. Damn I hate having to do 5 edits due to distraction.

P.P.S. This will only shut off bots accessing .php files where it resides... but, it will save checking every file against a huge list on load (.jpg, .gif, .png, .js, .css, etc. bots don't care about, but if done in .htaccess, all valid users will have to wait.)


Get Protected, Stay Protected...
With ZB Block, GNU/GPL Freeware Anti-Spam/Anti-Hack protection for your php based website.

Little boxes in the server farm, little boxes running php...

Offline

#20 2013-02-03 12:10 pm

Schuppeste
Member
Registered: 2012-06-17
Posts: 3

Re: HOW TO: Convert bannedips to htaccess

Offline

#21 2013-02-03 5:47 pm

pedigree
uıɐbɐ ʎɐqǝ ɯoɹɟ pɹɐoqʎǝʞ ɐ buıʎnq ɹǝʌǝu ɯ,ı
From: New Zealand
Registered: 2008-04-16
Posts: 7,054

Re: HOW TO: Convert bannedips to htaccess

Schuppeste wrote:

I should point out that anyone using with on CloudFlare, should use the correct $_SERVER variable and not REMOTE_ADDR smile

Offline

#22 2013-02-03 6:16 pm

Schuppeste
Member
Registered: 2012-06-17
Posts: 3

Re: HOW TO: Convert bannedips to htaccess

Sry i dont know what you mean peddigree ..

no english wink

Updated: http://pastebin.com/isHdbba4

Last edited by Schuppeste (2013-02-03 8:28 pm)

Offline

#23 2013-12-25 4:41 pm

NeoFox
Member
From: WI, USA, Earth
Registered: 2013-09-26
Posts: 830
Website

Re: HOW TO: Convert bannedips to htaccess

Can anyone explain how to use this with fluxbb please? I'm not quite making the connection on how to use lol. smile

Offline

#24 2013-12-25 4:56 pm

Papa Parrot
Member
From: Mexico
Registered: 2011-08-19
Posts: 1,826
Website

Re: HOW TO: Convert bannedips to htaccess

BlueSage wrote:

Can anyone explain how to use this with fluxbb please? I'm not quite making the connection on how to use lol. smile

Not sure if you mean Zbblock, or some of the other code shown, I am using zbblock, zbblock   for a fluxbb forum, it works good, instructions come with the download,  when you upload it to your host, and run the set up, you will get a script, "string", that needs to be added right at the beginning of the config.php  and that is all there is to it.
The instructions are easy to follow,

Offline

#25 2015-02-15 9:41 pm

Flavio93Zena
Member
From: Italy
Registered: 2014-12-24
Posts: 15

Re: HOW TO: Convert bannedips to htaccess

Since I can't see this being treated properly I thought about posting and adapting an old guide that I had made about banning users via .htaccess. This of course if you don't want to ban ALL SFS database but just specific IPs/IP ranges. It's fairly obvious though that if you ban the whole SFS database your system will run slower so personally I went down this path.

As this started to become a very frequently asked question and I was directly concerned about it I decided to make this guide, hopefully someone of the team will approve it smile

A forum software comes with a built-in ban feature that allows admins to use an IP ban to ban users but this function has several problems that cannot be fixed:

  • Each time a user loads a page it has to check his IP to see whether he is allowed or not to access that page;

  • It has to check all users connected comparing their IPs to the ones in the banned list;

  • Banned users will spam your error log, especially if they are spambots attempting to access the page hundreds of times (100 denied accesses mean 100 errors in the log for ONE spambot);

  • This check will cause your system to terribly slow down;

  • It can't be changed as creating a table and a check is the only way to ban people through any forum software.

How should I ban users?

You should ban users using your .htaccess file that is stored on your server host or either create it. If you don't know how to create it just create a .txt file, put the code in there and when you are done just rename its extension to ".htaccess". wink Like: example.txt --> rename --> example.htaccess
Open it and add the provided code, better at its end wink


What should I add to my .htaccess file?

Thanks to Lou69 I can provide these 2 links as reference:

But as they contain a lot of other stuff I will be explaining the only things that we are looking for.
Server can be setup to either accept or deny access to any IP not included in the list you are going to make. How? With this:

order allow,deny

or

order deny,allow

Basically the former sets up the server to deny any unlisted entries while the latter sets it up to allow any unlisted entries by default. Be careful because choosing between these two is vital as they order the operations being performed by the server itself. What do I mean? I mean that "allow,deny" will be processing all whitelisted IPs and then banned ones, while "deny,allow" will be doing the opposite. Like I said choosing between one of these is crucial and you have to type everything properly or your site might be unavailable to everybody.


Let's make some examples, let's say I want to ban this IP 30.31.32.33 and I want any other IP to be allowed on my website, how do I do that? You can do it in 2 ways:

order allow,deny
deny from 30.31.32.33
allow from all

or either

order deny,allow
deny from 30.31.32.33

As you can see they are slightly different

  • The first one has the "allow from all" in the end as the default state of the server is to deny any unlisted entried

  • The second one doesn't have that string because as the server is processing the banned entries before the allowed ones typing "allow from all" would unban the banned users ( tongue )


How to ban an IP range?

Let's say I want to ban all users with this IP range 30.31.32.XX where those XX stand for any number. You will have to do this:

order allow,deny
deny from 30.31.32.0/24
allow from all

or either

order deny,allow
deny from 30.31.32.0/24

I will not be explaining the reasons of that 24 as it's really complicated for new users.

Another example, I want to ban this IP range: 30.31.XX.XX where those XX stand for any number. You will have to do this:

order allow,deny
deny from 30.31.0/16
allow from all

or either

order deny,allow
deny from 30.31.0/16

Another example again, I want to ban this IP range: 30.XX.XX.XX where those XX stand for any number. You will have to do this:

order allow,deny
deny from 30.0/8
allow from all

or either

order deny,allow
deny from 30.0/8

You can get more informations from here: http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#IPv4_CIDR_blocks


Do you need an exception to unban an IP within a banned IP range?

Let's say I want this IP range to be banned (30.31.32.XX) but I want this IP (30.31.32.33) to be allowed access. How? (Of course any other user is supposed to be granted access)

order deny,allow
deny from 30.31.32.0/24
allow from 30.31.32.33

You just can't do it with the "order allow,deny" as it would be processing banned users after allowed ones and it would inevitably banning the IP exception as well.


Are you running Cloudflare on your forum?

If you are running Cloudflare on your forum all the above codes will not work properly, you will need to change some things. The "order allow,deny" and "order deny,allow" will work with Cloudflare as well but since Cloudflare is masking all IP addresses as if they were (usually) all 108.162.xxx.xxx then any ban made with the aforementioned method will obviously not work.
How to ban the proper IP addresses then? You need to add this code:

SetEnvIf X-FORWARDED-FOR 1.2.3.4 deniedip

or

SetEnvIf X-FORWARDED-FOR 1.2.3.4 allowedip

These must be used in ALL ban strings.

At the end of it you must also add

deny from env=deniedip

in order to get it to work smile obviously you can change all of them to

allow from env=allowedip

if you want to restrict access to only a few people.

CIDR notation (0/8, 0/16, 0/24) will work as long as you add the

SetEnvIf X-FORWARDED-FOR

code.

Example 1 (IP 1.2.3.4 is the only banned entry):

SetEnvIf X-FORWARDED-FOR 1.2.3.4 deniedip
order allow,deny
deny from env=deniedip

Or example 2 (IP 1.2.3.4 is the only IP with granted access):

SetEnfIf X-FORWARDED-FOR 1.2.3.4 allowedip
order deny,allow
deny from all
allow from env=allowedip

This part gave me a lot of headaches and in my opinion is the most complicated... I tried to be as clear as possible but if you don't understand it feel free to ask about it smile
As a reference the Cloudflare part has been adapted from here: http://blog.ergatides.com/2011/09/07/se … other-cdn/ it's not exactly the same thing but the basic inspiration and reference is there.


Do not put whitespaces between "order," and "deny" or the opposite, just type "order allow,deny" or "order deny,allow" and no whitespaces next to the comma! Same thing applies to Cloudflare-related strings.


Credits to (hopefully I will remember all of you): Arantor, a10, Antechinus, br360, emanuele and Shambles thank you all for your many tweaks and help.

Special thanks again to Lou69 for the reference links.

Adapting the BBC of this post was really painful, I hope this will help at least smile
Best regards,
Flavio.


I'm an angel with a shotgun,
fighting 'til the war's won,
I don't care if heaven won't take me back.

Offline

Board footer

Powered by FluxBB

Close
Close