You are not logged in.

#1 2019-05-09 10:05 am

Alfie
Member
Registered: 2012-11-08
Posts: 6

Automatic download

Hi!

I have problems with the automatic download of 'bannedips.zip' as described in this post.

In my registration-script I have various methods [regex for notorious spammers, local lists of banned e-mails and IPs, fast submission of the registration form (POST-GET <2 seconds), queries to the APIs of stopforumspam and BotScout]. I maintain a log-file (timestamp, IP, e-mail, IP, name, time for submission [maximum 16 milliseconds!]) of blocked registration-attempts. Instead of manually download  'bannedips.zip' I want to let the spammers do the job once a day.
This is the relevant part of my script:

# $cur_date: current date
# $block_date: date of last block
if ($cur_date > $block_date) { # only in 1st call of the day.
  $ch = curl_init("https://www.stopforumspam.com/downloads/bannedips.zip");
  curl_setopt($ch, CURLOPT_TIMEOUT, 300);
  curl_setopt($ch, CURLOPT_USERAGENT, 'alfie via libcurl');
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $result = curl_exec($ch);
  if ($result === true) {
    $file = fopen($file_zip, "w");
    fwrite($file, $result);
    fclose($file);
    curl_close($ch);
    # download ready; decompress to 'bannedips.csv'
    $zip=new ZipArchive();
    if($zip->open($file_zip) == TRUE) {
      $address=__DIR__;
      $zip->extractTo($address);
      $zip->close();
      unlink($file_zip);
    }
  }
}

I had some debug-variables in my code to check what's going on. As expected, the outer if is true once a day. But: $result is never true and hence, the download not initiated. Any idea what I made wrong?

Offline

#2 2019-05-09 10:17 am

Alfie
Member
Registered: 2012-11-08
Posts: 6

Re: Automatic download

I realized that some variables are not clear in my post. On top should be:

$file_zip   = "data/bannedips.zip";
$file_csv   = "data/bannedips.csv";
$file_block = "data/block.log"; # my log-file
$block_date = date('Y-m-d', filemtime($file_block)); # date of last block
$cur_date   = date('Y-m-d', time());

Offline

#3 2019-05-10 1:42 am

Alex Kemp
Moderator
From: Nottingham, England
Registered: 2009-12-02
Posts: 2,420
Website

Re: Automatic download

You have 2 sets of 'if true' in your code, but the actual code is not identical. Try that.

Offline

#4 2019-05-10 9:55 am

Alfie
Member
Registered: 2012-11-08
Posts: 6

Re: Automatic download

Hi Alex,

I guess you are referring to this post of kpatz? I’ll try. Will take a day…

Offline

#5 2019-05-10 11:03 am

Alex Kemp
Moderator
From: Nottingham, England
Registered: 2009-12-02
Posts: 2,420
Website

Re: Automatic download

I was talking about the discrepancy between these two:

if ($result === true) {
and
if($zip->open($file_zip) == TRUE) {

...but kpatz's code is even better. If $result is TRUE then what's the point? Just go ahead & do the business, you are not bothered about anything else. Otherwise you want to know *why* $result is not TRUE and thus spit out some diagnostics when that occurs. Thus, test for FALSE and diagnose on that, else do the business.

Offline

#6 2019-05-10 1:10 pm

kpatz
Member
Registered: 2008-10-09
Posts: 1,437

Re: Automatic download

$result will not equal true on a successful call to curl_exec.  Instead, it will contain the contents of the download.  It will be false on a failure.

So you need to check for false and handle the error, and if it is NOT false, then execute the code to capture the data to a file.

So, something like:

...
  $result = curl_exec($ch);
  if ($result === false) {
    echo 'Curl error: ' . curl_error($ch);
  }
  else {
    $file = fopen($file_zip, "w");
    fwrite($file, $result);
...

Last edited by kpatz (2019-05-10 1:13 pm)


Spam happens when greed meets stupidity.

Offline

#7 2019-05-10 1:14 pm

Alfie
Member
Registered: 2012-11-08
Posts: 6

Re: Automatic download

Hi kpatz,

THX; realized that in the meantime. Set up sumfink similar to see in which part of the script there might be problems. Can’t wait for the first spammer knocking on my door after midnight…

Offline

#8 2019-05-13 2:21 pm

Alfie
Member
Registered: 2012-11-08
Posts: 6

Re: Automatic download

Hi kpatz,

THX for your help again; downloads are working. However, I failed to get the file unpacked. Tried two pieces of code:

# $file_csv "bannedips.csv" including complete path
# permission of folder 664 (-rw-rw-r--)
# code no. 1
$zip = new ZipArchive();
$address = __DIR__;
$zip -> extractTo($address);
$zip -> close();
# code no. 2
$zip = new ZipArchive();
$zip -> extractTo($file_csv);
$zip -> close();

Any ideas?

Offline

#9 2019-05-13 2:27 pm

Alex Kemp
Moderator
From: Nottingham, England
Registered: 2009-12-02
Posts: 2,420
Website

Re: Automatic download

Why not just use unzip?

UNZIP(1)                    General Commands Manual                   UNZIP(1)

NAME
       unzip - list, test and extract compressed files in a ZIP archive

SYNOPSIS
       unzip  [-Z] [-cflptTuvz[abjnoqsCDKLMUVWX$/:^]] file[.zip] [file(s) ...]
       [-x xfile(s) ...] [-d exdir]

DESCRIPTION
       unzip will list, test, or extract files from a  ZIP  archive,  commonly
       found  on MS-DOS systems.  The default behavior (with no options) is to
       extract into the current directory (and subdirectories  below  it)  all
       files  from  the  specified  ZIP archive.  A companion program, zip(1),
       creates ZIP archives; both programs are compatible with  archives  cre‐
       ated  by  PKWARE's  PKZIP and PKUNZIP for MS-DOS, but in many cases the
       program options or default behaviors differ.

Offline

#10 2019-05-13 2:31 pm

Alfie
Member
Registered: 2012-11-08
Posts: 6

Re: Automatic download

Hi Alex,

I want to get it automatically unpacked on the server, not bannedips.zip > FTP to my machine > unzip > FTP to the server. Lazy.

Offline

Board footer

Powered by FluxBB

Close
Close