#1 2017-11-30 10:18 pm

NeoFox

Member

From: WI, USA, Earth

Registered: 2013-09-26

Posts: 830

Website

Trying to lookup and send in one lookup if poss.

So I'm using PHP 7 and get lots and lots of reports every hour and every day. I don't want to cause any additional SFS load if I don't have to, but I would like to check with my honeypots that if an entry doesn't exist then add it, otherwise don't use the server resources to log yet another spam request.

Can anyone help me to do this please? I'm using the following snippet to do such things.

$sfs=@file_get_contents("http://www.stopforumspam.com/add?

Any help would be appreciated. I tried this in the past and spent two weeks doing it but could just never get it right. I'd like to do it in one request/lookup if possible.

Last edited by NeoFox (2017-11-30 10:19 pm)

mn171121

Member

Registered: 2017-11-21

Posts: 3

Re: Trying to lookup and send in one lookup if poss.

You're trying to get data you have on spammers from your honeypots, then add it to SFS using the snippet you posted?

Have you already gotten your honeypot data and you're ready to send it? If so, rather than file_get_contents, use the curl code under "Adding to the Database" on the API usage page.

Do you mean you don't want to send a honeypot spammer entry if it already exists at SFS yet want to do it in one pass? But how will you know whether SFS already has it if you don't first check SFS for that particular spammer? You could download the files SFS makes available but that's extra work too.

ETA: if by one pass you meant a lookup at SFS and then add it if it's not already there within the same script run, rather than within one API call, then you can do that. I have some lookup, but not submit (because I do that by hand), code you can have as examples.

Last edited by mn171121 (2017-11-30 11:37 pm)

NeoFox

Member

From: WI, USA, Earth

Registered: 2013-09-26

Posts: 830

Website

Re: Trying to lookup and send in one lookup if poss.

Right now it's just grabbing the posted information and submitting to SFS. Rather then clogging the database with report numbers 100-599 from an IP I'd like to grab and submit if only it's  new submission to the database.

One call or two, I don't mind but this is what I'm hoping for.

kpatz

Member

Registered: 2008-10-09

Posts: 1,437

Re: Trying to lookup and send in one lookup if poss.

When you do your query I'd check the last-seen date returned by the API and if it's older than a few days, submit it anyway, so that the database knows the IP (or email etc.) is still actively being used for spamming.  More submissions (and newer submissions) raise the confidence score.

Another way to reduce load on the API is to cache your requests: after querying a particular IP, email, or username, cache the results locally for a day or so, that way if that IP hits again, you can just reference the cached entry from earlier instead of re-querying.  When you make a submission, cache that info as well--since you just submitted it, you know the next query on that same info will return a positive result.

Last edited by kpatz (2017-12-01 12:12 am)

---

Spam happens when greed meets stupidity.

sklerder

Member

Registered: 2012-10-11

Posts: 336

Website

Re: Trying to lookup and send in one lookup if poss.

Hello !

otherwise don't use the server resources to log yet another spam request.

Well, this could be a bad idea : If you don't submit, the apparition frequency of this spammer (IP, nickname and/or email) is not updated.
If no member submits when the record is already  present in database, the frequency and confidence are no more use

NeoFox

Member

From: WI, USA, Earth

Registered: 2013-09-26

Posts: 830

Website

Re: Trying to lookup and send in one lookup if poss.

How would I go about querying using the api then based on the result use the $sfs= snippet? I was thinking using php to check the lastseen and if > 12 hours then submit.

EDIT:

http://api.stopforumspam.org/api?ip=46.119.115.174

Pseudo code:
$now = now();
$date1=date_create(last seen date);
$date2=date_create($now);
$diff=date_diff($date1,$date2);
if ($diff > 12 hours) {
 // then $sfs = {{submit}}
}

Big suprise face! Could I use the ?: operators in PHP? Like

$sfs = (something? ) 
? (then submit)
: or die! :P

Edit 2: I guess my biggest thing I don't understand is what is meant in the FAQ guide being it all being POST and GETs? How would I go about grabbing the data from a GET on another server? That blows my mind that we can do that! Lol

Last edited by NeoFox (2017-12-01 11:47 pm)

sklerder

Member

Registered: 2012-10-11

Posts: 336

Website

Re: Trying to lookup and send in one lookup if poss.

I was thinking using php to check the lastseen and if > 12 hours then submit.

Well, my point of view is :
The spammer attempted to post ?
We submit, whatever how much time since its last attempt on the same forum is ...
Else, the confidence information becomes erroneous.
I frequently see bots attempting to register on my little forum many times a day.
Some try only once, others can try up to hundreds times a day.
Would you give, to the one which tried once a day,  the same confidence as to the one which tried hundreds times the day ?

NeoFox

Member

From: WI, USA, Earth

Registered: 2013-09-26

Posts: 830

Website

Re: Trying to lookup and send in one lookup if poss.

So you would rather 300 or so junk posts filling up the db? Maybe need to give partial confidence points for lookups alone?

mn171121

Member

Registered: 2017-11-21

Posts: 3

Re: Trying to lookup and send in one lookup if poss.

BlueSage, here's the script I use to check an IP address. Added date difference to it.

#!/usr/bin/php
<?php

/**
 * check-sfs
 *
 * Check an IP address at Stop Forum Spam.
 *
 */

if ($argc != 2 || in_array($argv[1], ['--help', '-help', '-h', '-?'])) {
?>
Usage:
<?php echo $argv[0]; ?> <IP address>
<?php
    exit;
}

$ch = curl_init();
curl_setopt_array(
    $ch, array(
    CURLOPT_URL =>
"http://api.stopforumspam.org/api?ip={$argv[1]}&json",
    CURLOPT_RETURNTRANSFER => true
));

$jsonout = curl_exec($ch);

curl_close($ch);

$result = json_decode($jsonout, true);
if ($result['success'] == 1) {
    if ($result['ip']['appears'] == 1) {
        echo "$jsonout\n";
        // begin date difference
        $lastseen = date_create($result['ip']['lastseen']);
        $now = date('Y-m-d H:i:s');
        $diff = date_diff($lastseen, date_create($now))->format('%a');
        echo $argv[1], ' was last seen ', $diff, " days ago\n";
        // end date difference
    }
}
?>

chrishirst

Member

From: Blackpool UK

Registered: 2011-01-22

Posts: 49

Website

Re: Trying to lookup and send in one lookup if poss.

... ...  that if an entry doesn't exist then add it, otherwise don't use the server resources to log yet another spam request.

Hopefully you realise that some forum plugins DO use the number of instances from the SFS database to determine whether to block or not, so you deciding not to add data because there is at least one entry may be detrimental to the efficacy of SFS for others. Plus the "Toxic lists" (http://www.stopforumspam.com/downloads) that are generated rely on spammer details being reported more than once, so rather than 'helping' you may well be causing a confidence loss in SFS for other users.

I have a BASH script that I use for bulk uploading a spammer list that uses CURL to upload the data from a CSV file, published in this post

---

Indifference will be the downfall of mankind, .... But hey! Who cares.

sklerder

Member

Registered: 2012-10-11

Posts: 336

Website

Re: Trying to lookup and send in one lookup if poss.

Hi !

+10
Fully agree with you, chrishirst, I'm happy to read I'm not alone ...

Papa Parrot

Member

From: Mexico

Registered: 2011-08-19

Posts: 1,826

Website

Re: Trying to lookup and send in one lookup if poss.

I think it is important submitting them even when they are all ready listed, as well. It shows that they are still active,  if every one started not submitting them, just because there are all ready even 1000 listings, eventually
it would result in a data base full of 1000s of listed spammers, but all old and it would appear they are no longer
active, hoping that makes sense to others, it does to me.  Of course if just 1 individual feels like they shouldn't submit ones that all ready are listed, the effect would be insignificant. On the other hand if it was a indivudual, but with several honeypots, the effect would be more noticeable, it could give the impression those listed spammers are no longer active, of course if they are active , most likely other honey pots, and also manually
submitted entries, would show they are active still.
Obviously, if they really are inactive then there would no longer be submissions made, and it would not show.
  Conclusion, ; by not submitting active spammers and bots, based on the theory that it is filling the data base
with more of the same submissions, would cause a inaccurate data base, giving the impression that active spammers/bots are not active, or only active on some honey pots, but else where no acclivity.
So in my opinion, yes a automated bot registration, should still be submitted, regardless as to how many other submissions have been made.  The same with a manual registration, and that posts spam, it would not make sense to not submit it because it is all ready listed X number of time, every submission shows it is still active,
no submissions give the impression it is not active.

---

Everything is for the Birds

And now, Papa Parrot and his kids

kpatz

Member

Registered: 2008-10-09

Posts: 1,437

Re: Trying to lookup and send in one lookup if poss.

Perhaps the only time a honeypot could (should) skip submitting a spammer is if one spammer (or bot) hammers the site with many pieces of spam at one time using the same credentials; submitting all of them in that case would be somewhat redundant.

But, if the honeypot has never seen this particular data set (IP, email, username) before, or even if it has a day or two ago, it should submit it to keep the confidence score, last-seen and frequency values current.

Same for manual submissions on a regular (non honeypot) forum.  I always submit anything that spams my site, regardless of the existing submissions.  It keeps the data current, and provides evidence as I always submit with it.  I only submit a spammer once even if he spammed multiple times--my custom code grabs the most recent post and any links in the profile and signature to include in evidence.  Of course, if he re-registers with new credentials and spams again (some are stupid enough to do this)... I submit again.

Last edited by kpatz (2017-12-03 12:40 am)

---

Spam happens when greed meets stupidity.

pedigree

uıɐbɐ ʎɐqǝ ɯoɹɟ pɹɐoqʎǝʞ ɐ buıʎnq ɹǝʌǝu ɯ,ı

From: New Zealand

Registered: 2008-04-16

Posts: 7,056

Re: Trying to lookup and send in one lookup if poss.

Dont worry about the tests, just add them, its easier

NeoFox

Member

From: WI, USA, Earth

Registered: 2013-09-26

Posts: 830

Website

Re: Trying to lookup and send in one lookup if poss.

Alright I took everything into consideration and just letting it fly. I got more than ample bandwidth so nothing on my end to be concerned about...I was just worry about creating so many repeat tickets about spam w/ my honeypots for the SFS db.

Last edited by NeoFox (2017-12-04 10:03 pm)

pedigree

uıɐbɐ ʎɐqǝ ɯoɹɟ pɹɐoqʎǝʞ ɐ buıʎnq ɹǝʌǝu ɯ,ı

From: New Zealand

Registered: 2008-04-16

Posts: 7,056

Re: Trying to lookup and send in one lookup if poss.

Don't worry about duplicates, it will reject them if submitted within a minute of each other.  If anything, it gives a more reliable indicator of the volume from any specific source

NeoFox

Member

From: WI, USA, Earth

Registered: 2013-09-26

Posts: 830

Website

Re: Trying to lookup and send in one lookup if poss.

Maybe only accept the first 100 chars or 200? How about that?