Site issues

pedigree · 2011-06-15 3:34 pm

Firstly, Im sorry about the 24 hours down time. The server hardware failed rather badly. I managed to get a temporary server online many millions of "thank you"s to snelinternet (www.snelis.com) who pulled out all the stops to setup a new server.

Currently, the site is back on the old box until I can find the time over the weekend to investigate moving the website to the faster server.

This server is also rebuilding the disks and is causing serious issues with mysql disk speeds. A lot of pages will time out and error until the sync is finished. Once its rebuilt, Ill put the search and ipcheck pages back online

Im off to have a beer and draft an email to Leaseweb to praise the guy who fixed the server via Skype.

jimmie 48 · 2011-06-15 4:19 pm

thanks for the work on the site as i have become very dependent on it. no solid wall needed.... if a username even SMELLS of spam, i'm going to ban. multiple accounts???? ban ban ban

Erik · 2011-06-15 4:22 pm

yeah give him a beer or two to (the leaseweb guy) enjoy the beer but don't drink to much or the server crashes again when you get back lol

pedigree · 2011-06-15 4:51 pm

Well, I have another server, much faster, powerful and much better suitable for running the site.

Ill look at how I can secure the box for less than that $160/month that theyre currently asking.

If I can reduce it, then Ill move the main box to it and setup this box as a mirror for issues like this.

Both SnelInternet and Leaseweb guys have been fastasic

OnThePike · 2011-06-15 5:37 pm

Welcome back! Anything I can do, lemme know.

Severity One · 2011-06-15 6:14 pm

The 'add spam data' page complains that my API key is invalid. Is this because the database is still slow? Moreover, will it affect the forum that we're running and which the spammers managed to find in no time?

Thanks,

- Peter

pedigree · 2011-06-15 7:00 pm

Thats right. Databases are one of the few things that are seriously impacted by disk performance and while the disks are rebuilding, disk performance is like that of a really slow flash card. mysql just chokes. I will see if I have enough donations to buy a SSD drive for the server and if there is space for it, just for mysql.

insektenfang · 2011-06-15 7:29 pm

pedigree wrote:

...mysql just chokes.

Error: Unable to connect to MySQL and select database. MySQL reported: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2).

Oh, yes!!!

pedigree · 2011-06-15 9:33 pm

update - The site is now running on the new server, full time.

I need to re-add all the backend scheduled jobs but the front end adding, searching etc, should be working now. please post any issues you might fine.

Tooradin · 2011-06-15 11:01 pm

Sorry to hear of the hardware failure.
I have every confidance that you are doing everything you can to get things working again at your earliest convenience.

I have been taking screenshots of my spammers to add when things have settled down.

Weird thing is I have had 36 spam accounts added since your servers went down to now & I only usualy get a few a week. I have had an unusual amount of activity for a few days before your servers went down.
Our forums are for ingame, in guild members only, so very easy to see a spammer as we have very few real members joining.
I might just shut down our forums for a couple of days as this worked the last time this happened.

I will have a quite a few spammers to add when things are working again, but I will not try for a couple of days to let you clear some backlog.

Thanks for the great site & wish you all success.

CS3Regina · 2011-06-15 11:31 pm

I'm just posting to say thank you! I've only been using this site and the SFS mod for my forum for a little over a week but it's already stopped so many spam accounts it's just mind boggling. My forum is small and I'm the only admin, and this is already saving a lot of time.

popupsteve · 2011-06-16 12:57 am

Yes, THANK YOU for all your hard work. I would think that everyone on this forum is an admin of a site, big or small and we all know the trials and tribulations of running a website. We all appreciate this site and the invaluable service you supply.

Thanks!

pedigree · 2011-06-16 7:10 am

I must admit, it was a hard slog.

Leaseweb did a great bit of work in getting the server back up to a point where I could recover data added since the last backup.

Snel Internet are the real heroes, coming to the rescue with a new server, out of hours and when the boss personally handling it.

crow · 2011-06-16 9:11 am

It went completely mad on the forum I mod on yesterday. It was really hard job in keeping up with it all. I think the spammers knew there were issues with your forum.

The cats away the mice can play.

You do such a great job. Well done.

SteeleButterfly · 2011-06-16 12:29 pm

I'll add my thanks to everyone else's. This database is invaluable for forum owners/admins. And thanks to snelinternet and Leaseweb for their help!

somdcomputerguy · 2011-06-16 12:39 pm

I've only briefly read thru the Twitter posts and these too, so I'm not privvy to all the details, but I do know that much work (time, labor, and hardware) was done to get this free service back on it's feet. Thanks and kudo's to pedigree and all others involved!!

Last edited by somdcomputerguy (2011-06-16 12:42 pm)

DaveTch · 2011-06-16 1:24 pm

I'll add my thanks as well, along with the other Modertors on our forum.

SFS has allowed us to keep the bad guys out for over a year now.

Thanks to Pedigree and all of his team for building and maintaining SFS for the rest of us!

Dave

hawkmoth · 2011-06-16 1:54 pm

Many thanks to all concerned for the hardwork in sorting out all the problems with this brilliant site.

tekhedd · 2011-06-16 6:03 pm

It's a good day to donate!

After last weekend's wash of forum spammers, I hastily added support for http:BL, and it looks like it saved me from at least another 50 registrations during the sfs downtime. I heartily recommend the combination of honeypotproject + stopforumspam.

I was afraid you'd been hacked or DOSed or something. Hardware failure is just business as usual isn't it? (But it still sucks!)

Thanks for being there!

Last edited by tekhedd (2011-06-16 6:49 pm)

Severity One · 2011-06-16 7:25 pm

Yes, before I forget, thanks for all the effort. Times like these make me realise how much we've become dependent on SFS.

By the way, I wouldn't recommend using SSD for MySQL. SSDs have a limited number of times you can write them, so it'd probably be better to have a couple of 10,000 RPM SAS discs, or something of the sort. Not cheap, but SSD isn't cheap either.

Again, thanks!

- Peter

asko · 2011-06-16 7:40 pm

many thanks for your so usefull work !

I'll donate today !

FibroTalk · 2011-06-16 7:59 pm

Glad to see you're back!!
Thanks for everything you do

Katana · 2011-06-17 8:47 pm

Severity One wrote:

Yes, before I forget, thanks for all the effort. Times like these make me realise how much we've become dependent on SFS.
By the way, I wouldn't recommend using SSD for MySQL. SSDs have a limited number of times you can write them, so it'd probably be better to have a couple of 10,000 RPM SAS discs, or something of the sort. Not cheap, but SSD isn't cheap either.
Again, thanks!
- Peter

Oh, really? I thought that applied to only things like flash drives.
Mind providing a source? I'd like to read up on that further.

zaphod · 2011-06-17 9:06 pm

http://en.wikipedia.org/wiki/Flash_memory#Limitations

http://en.wikipedia.org/wiki/Ssd#Lifespan

Personal opinion, for server use, one should look for 5400 (preferable) or 7200 RPM with very large on-drive caches. Most operations of a server, per drive, will be far less than 8 megabytes per second, especially if it's just a few records in a large database. If you halve the spindle speed of the drive, you quadruple the lifespan (on the average, check some models that had 5400 and 10,000 RPM variants, and look at MTBF).

Large caching speeds up access far more than spindle speed, and as a bonus, makes for smaller head seeks, and less head seeks, thus it even improves MTBF.

Zap

Katana · 2011-06-17 9:47 pm

Thanks for the links zap. Wasn't aware of that.

As far as the disk speeds go, wouldn't moving something like the database storage to its own disk also help boost performance? I'm guessing here that if the OS is on one disk, it'd not peg up the read/write queue for the DBMS's HDD since it's not even on the same drive.
...Or does that not matter, as the amount of read/write operations made by the OS and other programs would be too insignificant to notice?

#1 2011-06-15 3:34 pm

Site issues

#2 2011-06-15 4:19 pm

Re: Site issues

#3 2011-06-15 4:22 pm

Re: Site issues

#4 2011-06-15 4:51 pm

Re: Site issues

#5 2011-06-15 5:37 pm

Re: Site issues

#6 2011-06-15 6:14 pm

Re: Site issues

#7 2011-06-15 7:00 pm

Re: Site issues

#8 2011-06-15 7:29 pm

Re: Site issues

#9 2011-06-15 9:33 pm

Re: Site issues

#10 2011-06-15 11:01 pm

Re: Site issues

#11 2011-06-15 11:31 pm

Re: Site issues

#12 2011-06-16 12:57 am

Re: Site issues

#13 2011-06-16 7:10 am

Re: Site issues

#14 2011-06-16 9:11 am

Re: Site issues

#15 2011-06-16 12:29 pm

Re: Site issues

#16 2011-06-16 12:39 pm

Re: Site issues

#17 2011-06-16 1:24 pm

Re: Site issues

#18 2011-06-16 1:54 pm

Re: Site issues

#19 2011-06-16 6:03 pm

Re: Site issues

#20 2011-06-16 7:25 pm

Re: Site issues

#21 2011-06-16 7:40 pm

Re: Site issues

#22 2011-06-16 7:59 pm

Re: Site issues

#23 2011-06-17 8:47 pm

Re: Site issues

#24 2011-06-17 9:06 pm

Re: Site issues

#25 2011-06-17 9:47 pm

Re: Site issues

Board footer