Spambot detector (with the use of this API)

diabolic.bg · 2008-11-28 4:08 pm

file_get_contents is available

http: //example.com/check_spammers/htdocs/check_spammers_plain.php?name=test&email=tom@xxx-search.info&ip=195.24.76.232

I get a blank page...

Last edited by diabolic.bg (2011-09-19 5:49 pm)

diabolic.bg · 2008-11-28 4:31 pm

I have placed all content (config.php, en.php, check_spammers.php and folder spambots) in your htdocs folder and uploaded it.
Now http:// wasteland-bg.com/check_spammers/htdocs/
give me full interface and error:

Incompatable PHP version: 5.0.4

PHP Version 5 or above is required for this site to work

Maybe will be better I use your online Spambot detector manually. I have placed link in my ACP and it works perfectly.

You can see this:
http: // example.com/check_spammers/htdocs/?name=test&email=tom%40xxx-search.info&ip=195.24.76.232&submit=check

Last edited by diabolic.bg (2011-09-19 5:51 pm)

MysteryFCM · 2008-11-29 4:16 pm

One of the problems actually appears to have been my cocking up when moving the files from the development, to the production package.

I've re-packaged the files, so it should work just fine for you now

Bear in mind however, the htdocs and private folder are actually indicative of a web server layout. Since you're using Xammp, you are most likely used to the htdocs folder being called wwwroot??

I'll do an installation instruction file for it when I get time, but in the meantime, move the htdocs and private folder to;

{YOUR_WEBSITE_ROOT\

Then rename the htdocs folder to check_spammers

So the layout then becomes;

{YOUR_WEBSITE_ROOT\check_spammers
{YOUR_WEBSITE_ROOT\private

Ideally, the private folder should be above the website root as there should not be a need for anyone to have direct access to the files/folders in the private folder.

I'm a little confused as to why you're getting the incompatable PHP version error however, but you can resolve this in the meantime by commenting out the PHP version check, as follows;

If(phpversion() > 5){
    //Rest of code
}else{
    echo '<span class="error">'.$phpver_error.'</span>';
}

Becomes;

//If(phpversion() > 5){
    //Rest of code
//}else{
//    echo '<span class="error">'.$phpver_error.'</span>';
//}

MysteryFCM · 2008-11-29 5:06 pm

diabolic.bg wrote:

file_get_contents is available

http://wasteland-bg.com/check_spammers/htdocs/check_spammers_plain.php?name=test&email=tom@xxx-search.info&ip=195.24.76.232

I get a blank page...

Looking at the pages source code shows it's actually trying to display the PHP version error, but can't because it can't find config.php or en.php

Re-download the zip file, then overwrite the existing copy of the htdocs and private folders

diabolic.bg · 2008-11-30 12:57 pm

I did it as you say but while test it I have a registration success with e-mail address or nickname (also with e-mail address + nickname) from http://www.stopforumspam.com/

Now I don't have errors but I don't have a detection too...
I will remove all and don't wasting my time.
Sorry for taking yours!

MysteryFCM · 2008-11-30 9:34 pm

I've updated the zip to have both a simple and full version of this. The usage is;

1. Use Simple if this is NOT going to be used as a standalone website and you just want a drag-drop.

To use this version, just extract the Simple folder from the zip, and copy it to wherever you'd like it.

2. Use the full (htdocs/private) version if this is to be used as a standaline website and/or you are familiar enough with PHP to make the necessary path modifications.

If you need help with any aspect of this, feel free to drop by the forums;

http://temerc.com/forums/viewforum.php?f=71

Note, it would help if;

1. You told me which version you were using
2. What error message (if any) you are receiving
3. If you are using this as part of a forum/guestbook filter (if so, which forum/guestbook etc software, and posting a zip'd copy of the respective file (e.g. usercp_register.php for phpBB2)
4. The user details you used to test this (e.g. username, e-mail and IP)

I'm only human, and have limited PHP knowledge (which is why this is extremely basic), and am prone to error.

MysteryFCM · 2008-12-01 12:46 am

Date: 01-12-2008 (v0.5)

+ Now allows querying Projecthoneypot.org (API key required*)
* Modified Spamhaus query code (now also includes description of the return codes)

* Your projecthoneypot.org API key MUST be placed in the respective var in config.php

http://projecthoneypot.org/httpbl_api.php

Last edited by MysteryFCM (2008-12-01 12:53 am)

diabolic.bg · 2008-12-03 7:21 pm

Sorry MysteryFCM!
I find out any weakness in your online SpamBot Search Tool.
I test IP 87.126.170.5

ProjectHoneyPot:    IP found!
Spamhaus:    IP found!

Spammer identified!

If check ProjectHoneyPot the result is: 87.126.170.5

We don't have data on this IP currently. If you know something, you may leave a comment.

And Spamhaus:
IP Address Lookup

87.126.170.5 is not listed in the SBL

87.126.170.5 is listed in the PBL, in the following records:
PBL180908

87.126.170.5 is listed in the XBL, because it appears in:
CBL

What is this?!? I don't want to drive away my users...

MysteryFCM · 2008-12-03 7:36 pm

http://www.spamhaus.org/pbl/index.lasso

The CBL and PBL (127.0.0.4 and 127.0.0.10) is excluded from the temerc.com check (for check_spammers_plain.php, not the main check_spammers script) due to it's intented use. For details see;

http://www.spamhaus.org/faq/answers.las … BL%20Usage

The IP you queried, whilst being flagged in the PBL by SpamHaus (and thus ignored by the script), is also blacklisted by ProjectHoneyPot due to it's neighbours.

This is actually also one of the reasons I recommend hosting the script locally, instead of using the remote copy - you can customize it to suit your needs

/edit to add ref's;

Global checks (used for independant clarification)
http://temerc.com/Check_Spammers/?name= … .126.170.5

Used by the forums filter
http://temerc.com/Check_Spammers/check_ … .126.170.5

In the case of the latter, check_spammers_plain.php ignores 127.0.0.4 and 127.0.0.10 in the Spamhaus result, to avoid potential F/P's in the SpamHaus results (e.g. IP's that are listed because they shouldn't be sending spam, rather than because they actually are).

Last edited by MysteryFCM (2008-12-03 7:43 pm)

diabolic.bg · 2008-12-03 9:22 pm

MysteryFCM wrote:

is also blacklisted by ProjectHoneyPot due to it's neighbours.

What do you want to say? If you are killer I must go to jail because I'm your neighbour? Hmmm, interesting...
Or maybe I something don't understand? Sorry, my english isn't very good!
P.S. Nothing personally. The dispute is of principle.

MysteryFCM wrote:

This is actually also one of the reasons I recommend hosting the script locally, instead of using the remote copy - you can customize it to suit your needs

As you already know, I had big problems with your script and I refuse local version...

Last edited by diabolic.bg (2008-12-04 12:16 pm)

MysteryFCM · 2008-12-04 12:16 pm

You'd have to dispute the issue with ProjectHoneyPot, not myself ....... it's their database, their rules

diabolic.bg · 2008-12-04 12:18 pm

In this you absolutely right.
Sorry for my irritation!

Last edited by diabolic.bg (2008-12-04 12:20 pm)

MysteryFCM · 2008-12-08 5:07 am

Another update ........

v0.6 08-12-2008

+ Now includes SpamCop results

I've also made a slight change to the "Get the code" link. It now links to the following thread to save a bit of hassle.

http://temerc.com/forums/viewtopic.php?f=71&t=6103

diabolic.bg · 2008-12-08 3:59 pm

66.249.66.196 crawl-66-249-66-196.googlebot.com

http://fspamlist.com/checkspammers/?nam … bmit=check

72.30.78.231 llf531320.crawl.yahoo.net

http://fspamlist.com/checkspammers/?nam … bmit=check

This for me is not serious...

Last edited by diabolic.bg (2008-12-08 4:07 pm)

MysteryFCM · 2008-12-08 5:38 pm

It looks to be ProjectHoneyPot thats flagging them, which is unusual, but fine as bots shouldn't be trying to post anything anyway. However, I added the info on who flagged it, to check_spammers_plain, so you could always check for that and ignore it based on that (or based on the IP's PTR) if you wish.

jtdarlington · 2008-12-09 7:45 pm

MysteryFCM, thanks for this great script. I just implemented it on my phpBB3 forum (with some modifications, see below) and it seems to be working great. I only recently started using SFS and have found it invaluable, and your updates to the originally posted script have made it quite a Swiss army knife.

Just a few things I thought I should point out:

You might want to use caution with some of the DNS blacklists. They work great with blocking e-mail spam, but I'm afraid they're less useful when it comes to forum spam. Take, for example, Spamhaus' PBL. The PBL is intended to block direct-to-MX e-mails coming from dynamic IP ranges assigned to ISPs. IPs in this list should be using the ISP's dedicated mail server rather than sending SMTP messages directly to remote hosts. However, that doesn't really apply for forum spammers. Legitimate forum registrations will likely end up coming from PBL-listed addresses. It's not e-mail coming from that address, but simple HTTP traffic. To continue the example, if I look up my own home cable modem's IP in the PBL, it's listed. That's because I've been given an IP by my ISP, and I should be using their mail server for outgoing mail. But it's that same IP that gets sent to my forum when I register, not my ISP's mail server's address. If I used Check Spammers without modification, I wouldn't be able to register at my own forum!

So I modified my local copy of Check Spammers to only check SFS and FSpamList.com. Since these are (apparently) maintained by real human forum admins, it seems like a safe placed to perform automated checks. The DNS blacklists are a bit too restrictive in my case, and I'd bet they'd generate a lot of false positives for other folks as well. It's not that they don't serve their purpose (I was checking many of these manually myself before using Check Spammers), but there is less overlap between their intended function and the function we're using it for. I use them as a secondary level of verification, on the off chance SFS/FSL returns negative. That's when I dig deeper and may a judgment call only a human should make.

I should also point out that according to the DSBL site the DSBL is no more. You might want to remove it from the checks, as a useless check only slows things down.

MysteryFCM · 2008-12-09 9:09 pm

I've updated the script so it no longer checks the DSBL, cheers for the heads up

http://temerc.com/forums/viewtopic.php? … 7#p3435267

With regards to the DNSBL checks, there unfortunately is alot of open potential for F/P's, especially with the likes of Spamhaus. To allow for this, I modified the results page to include information on which SH list the IP was listed in, such as the PBL, CBL, XBL etc. I will also however, modify the script further so that this check is optional for the _plain script

Last edited by MysteryFCM (2008-12-09 9:12 pm)

jtdarlington · 2008-12-09 9:19 pm

Glad to be of help, even if only a tiny bit. I should point out that Check Spammers has already blocked its first "legit" spammer (How often does that phrase get used?) on my forum, so I'm already a happy customer.

MysteryFCM · 2008-12-09 9:21 pm

hehe nice one

MysteryFCM · 2008-12-10 9:20 pm

Date: 10-12-2008

+ Check DNS Blacklists (spamhaus etc) now optional when using check_spammers_plain.php

To leave out the DNS Blacklists check, just append &dbl=no to the querystring

Example;

http://temerc.com/Check_Spammers/check_spammers_plain.php?ip=190.245.57.79&dbl=no

MysteryFCM · 2008-12-13 5:53 am

v0.10 Date: 13-12-2008

+ Added additional information concerning listings in ProjectHoneyPot

If an IP is found to be listed in PHP, if using the main UI, you will be given detailed information.

If using the check_spammers_plain.php script, you will be provided with it's "simple" listing (e.g. 127.x.x.x)

The 127 address results are;

Octet #1: 127 - static
Octet #2: 0-255 - Number of days the IP was last seen
Octet #3: 0-255 - Threat score (0 = low risk, 255 = high risk)
Octet #4: 0-7 - Visitor Type

Detailed info: http://www.projecthoneypot.org/httpbl_api.php

Example (using main UI):
http://temerc.com/Check_Spammers/?name= … .200.96.80

Example (using check_spammers_plain):
http://temerc.com/Check_Spammers/check_ … .200.96.80

Note the return code in the second example, 127.2.26.5. This means;

Last Seen: 2 days ago
Threat Score: 26/255 (moderately low)
Visitor Type (5): Suspicious & Comment Spammer

Using the main UI, this would be displayed as:

This IP's threat score is [ 26/255 ]. Activity was last seen by this IP [ 2 ] days ago. It has been identified as a [ Suspicious & Comment Spammer ]

MysteryFCM · 2008-12-13 10:07 pm

It's got it's own homepage now

http://support.it-mate.co.uk/?mode=Prod … searchtool

RFiend · 2008-12-14 7:13 am

I'm with jtdarlington... most of these 'spam blacklists' are functionally worthless, and particularly for an automated test. As an example, I MYSELF am 'positively blacklisted' with both Spamhaus and SORBS. I've never sent spam in my life! Most of the rest of the spam databases I'm 'neutrally blacklisted' on, undoubtedly because I'm a real person and not a frigging mail server. Why SORBS and Spamhaus feel the need to list our entire IP range as 'positively blacklisted' utterly escapes me... as a sysadmin on several forums (and mail admin on one of them) that type of idiotic blacklist means I'll *never* use SORBS or Spamhaus to filter our incoming mail.

Folks, be VERY CAREFUL interpreting the results of any of these automated tests of whether someone is a forum spammer. Even the SFS database is less than perfect, since it already includes a handful of proxies. On our forums, we've already had 2 different GENUINE HUMANS register and they registered using proxies that were already blacklisted HERE AT SFS. Just because someone's IP address is in a blacklist means *nothing* since all of you folks are tossing IPs in willy-nilly.

Real humans use proxies as well as the blackhats, and you can't damn someone because they either NEED to or WISH to use a proxy. If they're on a college campus, they probably have to proxy in to get past the campus filters. The same is probably true of a lot of free Internet cafes, since I've seen some that block a lot of sites.

If I get a blacklist hit, I Google it with the following string:
http://www.google.com/search?hl=en&q=pr … %22{IP}%22
where the {IP} part is replaced with the IP in question. If I see anything that looks like a proxy, I let it through. As I'd said, just in the last month we've had 2 people register and log in via proxy. Should I disallow them because they use the same proxy as a spambot? No, that's what the Turing test should help with (the captcha).

Check yourself on the multi-blacklist lookup. Toss your IP address in after this URL fragment:
http://openrbl.org/client/?query=
and then hit the LOOKUP button to the right.

You might be surprised at the results. You certainly will get a lower opinion of the quality of the blacklist once you see yourself listed in red.

The best solution I've seen is reCAPTCHA. That stops the robots, and if you have a help description that tells 'em to hit the GET A NEW CHALLENGE button on reCAPTCHA when the text is unreadable, that helps a lot. More than one fifth of the challenges I've seen on reCAPTCHA are unreadable, and until the general public gets used to it, it's not very friendly with that little issue.

MysteryFCM · 2008-12-14 7:28 am

I actually agree, anything automated is prone to F/P's, and worse still, anything user driven is guaranteed to have many F/P's unless they are checked by an admin prior to inclusion.

This is one of the reasons I added the dbl= option, and the main reason that the script only provides the results, and doesn't do the blocking itself.

Smurf_Minions · 2008-12-14 9:11 am

I agree with that this is not the best solution there is, but it has prove very useful to me as i don't have anymore spammers. I also note that if it doesn't allow to register, he can send a mail to the webmaster for an account (which never happened before). As for CAPTCHA's they are fine, but if someone takes the time, they can be sorted out (maybe sound-based captcha's could prove usefull).

#26 2008-11-28 4:08 pm

Re: Spambot detector (with the use of this API)

#27 2008-11-28 4:31 pm

Re: Spambot detector (with the use of this API)

#28 2008-11-29 4:16 pm

Re: Spambot detector (with the use of this API)

#29 2008-11-29 5:06 pm

Re: Spambot detector (with the use of this API)

#30 2008-11-30 12:57 pm

Re: Spambot detector (with the use of this API)

#31 2008-11-30 9:34 pm

Re: Spambot detector (with the use of this API)

#32 2008-12-01 12:46 am

Re: Spambot detector (with the use of this API)

#33 2008-12-03 7:21 pm

Re: Spambot detector (with the use of this API)

#34 2008-12-03 7:36 pm

Re: Spambot detector (with the use of this API)

#35 2008-12-03 9:22 pm

Re: Spambot detector (with the use of this API)

#36 2008-12-04 12:16 pm

Re: Spambot detector (with the use of this API)

#37 2008-12-04 12:18 pm

Re: Spambot detector (with the use of this API)

#38 2008-12-08 5:07 am

Re: Spambot detector (with the use of this API)

#39 2008-12-08 3:59 pm

Re: Spambot detector (with the use of this API)

#40 2008-12-08 5:38 pm

Re: Spambot detector (with the use of this API)

#41 2008-12-09 7:45 pm

Re: Spambot detector (with the use of this API)

#42 2008-12-09 9:09 pm

Re: Spambot detector (with the use of this API)

#43 2008-12-09 9:19 pm

Re: Spambot detector (with the use of this API)

#44 2008-12-09 9:21 pm

Re: Spambot detector (with the use of this API)

#45 2008-12-10 9:20 pm

Re: Spambot detector (with the use of this API)

#46 2008-12-13 5:53 am

Re: Spambot detector (with the use of this API)

#47 2008-12-13 10:07 pm

Re: Spambot detector (with the use of this API)

#48 2008-12-14 7:13 am

Re: Spambot detector (with the use of this API)

#49 2008-12-14 7:28 am

Re: Spambot detector (with the use of this API)

#50 2008-12-14 9:11 am

Re: Spambot detector (with the use of this API)

Board footer