#1 2021-09-05 6:26 pm

Gordonjcp

Member

Registered: 2018-11-28

Posts: 1

Big (or, I think it's big) list of queries in one go?

Hi folks, I've been manually using stopforumspam for a good long while now, mostly because I haven't got my backside in gear to write a plugin for [flaskbb](https://github.com/flaskbb/flaskbb) yet.  I'm part of the way there with it, but I'd like to sanitise the user data I already have - much of which is from "clearly spammy" usernames or email addresses, some of which I'm not sure.

So my plan is to write a script to fire each one into the API, possibly using bulk queries, but I don't want to run afoul of any rate limiting or seem to be taking the piss with my usage.  There's probably about 3000 users in total, and I'd ideally only run the job once.

After that the plugin would run on every new user registration, and set them up as either a "normal" user if they have no record on stopforumspam, or a "limited" user if they do.  That would probably be (for my site!) a dozen or so queries a day, most of them spammers.

Would firing 3000 queries at the servers in a short space of time be a problem?  Is there a better way to do this?

Gordon

Maikuolan

Member

From: Perth, Western Australia

Registered: 2011-08-09

Posts: 802

Website

Re: Big (or, I think it's big) list of queries in one go?

I would recommend going to the downloads page, downloading the appropriate file (I guess, it sounds like you're wanting to check against email addresses? if so, then the email addresses file would make sense; or whatever other file you need, depending on whatever it is you're checking against), and then just checking locally against that downloaded file, instead of performing large numbers of remote queries/requests like that. Good thing about doing it all locally, too, is that it won't matter anymore how many times you're using it, since it would be using only your own resources. :-)

pedigree

uıɐbɐ ʎɐqǝ ɯoɹɟ pɹɐoqʎǝʞ ɐ buıʎnq ɹǝʌǝu ɯ,ı

From: New Zealand

Registered: 2008-04-16

Posts: 7,104

Re: Big (or, I think it's big) list of queries in one go?

3,000 as a one off isn't a problem