Is cj.com abusing your website?

Alex Kemp · 2011-07-22 11:13 am

Most of the folks reading this website do so because they suffer from spammers. At some point, all of us will have asked "why? why are they doing this to my website?", and the answer, of course, is `money'. As Deep Throat said, "Follow the money".

Websites are after links, so as to rise up the Google rankings. They pay SEO companies to promote their site, and those scumbags pay other scumbags who recruit yet more scumbags who spam your site.

There is yet another method that those same SEO companies use, called referal (sic) spamming. It's not as obvious, and it depends upon website mis-coding to be effective, but it has been getting worse & worse upon my site across the last few months. Briefly, the address of a site is placed into the `referal' header of a browser; the browser is then used to hammer your site.

I'll give some figures from my AWStats reports to show how this has risen since January. A couple of things upfront first:

>> there are zero links on cj.com to my site
>> there are just 3 references on my site to cj.com (including this one from 2004, when they sent me a virus email)
>> cj.com are merely the very worse offender in this regard
>> the highest normal referrer is about 200

The rise & rise of referal spamming : example: cj.com

Date      Referrals
Jan 2011       0
Feb 2011       0
Mar 2011     835
Apr 2011   6,045
May 2011   4,700
Jun 2011  24,752
Jul 2011  26,845

Knut · 2011-07-22 12:21 pm

26,845 referrers from cj.com in July

cj.com is on 64.70.54.78.
What IPs are they using when hitting your site?

What do they get out of it, besides using your server resources and bandwith?

Alex Kemp · 2011-07-22 12:54 pm

Knut wrote:

What do they get out of it, besides using your server resources and bandwith?

Identically the same as normal spammers: backlinks.

CJ have been running a bot since at least 2007 (CJNetworkQuality):
http://forums.modem-help.co.uk/viewtopic.php?t=703

To my prejudiced mind, that sounds like perfect prep for spamming. And, to make the point, I haven't made any use of CJ on my site for 5 years.

What IPs are they using when hitting your site?

I examined all 3 sites below. On each that I looked at it had the same formula: a `vcbot' quoting cj.com as a referer (I've broken the CJ site to prevent it getting a backlink):

63.215.202.234 - - [17/Jul/2011:04:26:44 +0100] "GET /mfcs-U/USR/3C888/Firmware/ HTTP/1.1" 200 7501 "http://www.cj.c0m" "vcbot" In:27746 Out:7501:27pct. "-"

# fgrep -c 'cj.com' *
...
access_log:3878
access_log.1:4476
access_log.2:4042
access_log.3:3611
access_log.4:3380
...
download_log:4490
download_log.1:4815
download_log.2:4060
download_log.3:4214
download_log.4:3948
...
forums.log.1:101
forums.log.2:51
forums.log.3:69
forums.log.4:37

fcollingwood · 2011-07-22 10:08 pm

Alex Kemp wrote:

Knut wrote:
What do they get out of it, besides using your server resources and bandwith?
Identically the same as normal spammers: backlinks.

Only if you expose them

Knut · 2011-07-23 8:53 am

Alex Kemp wrote:

63.215.202.234 - - [17/Jul/2011:04:26:44 +0100] "GET /mfcs-U/USR/3C888/Firmware/ HTTP/1.1" 200 7501 "http://www.cj.c0m" "vcbot" In:27746 Out:7501:27pct. "-"

Thanks for the extra information.

I have checked my log files, but did not find this UA, IP or referrer.

26,845 hits is extreme, so as a precaution I have blocked CJ on my servers.

Alex Kemp · 2011-07-23 4:18 pm

Knut wrote:

26,845 hits is extreme, so as a precaution I have blocked CJ on my servers.

Hah! You have no idea...

Last night, some c*nt from Telstra decided to download my site to their hard-disk (several million pages). Between midnight GMT & 5am he attempted to take 37,495 pages at a max 5 pages / second (that's the equivalent of 194,534 per day). All blocked, of course. He's left his computer running whilst he sleeps (Telstra is in Australia), and I've been receiving regular hourly emails from the cron job that collects the block-report; this is the 4pm email:

rbl.bot_log: Records: 7557 Deleted: 0 Skipped: 0 Warnings: 0

Earlier emails are all just a variation on the above.

zaphod · 2011-07-23 4:27 pm

Here, kill him with this: http://www.spambotsecurity.com/files/PH … -treme.zip

Or drop a few links to http://labs.spambotsecurity.com/darkstar/index.php

Both use pseudo-random generation, for unlimited pages, and, if a page is accessed twice, it shows the same data.

Sure, it will cost your site, or mine some access...
but it will fill their hard disks with junk.

Zap

Alex Kemp · 2011-07-23 11:58 pm

zaphod wrote:

Here, kill him with this...

Thanks for the offer, Zaphod!

The guy's bot finally ran out of URLs to request sometime between 6 & 7pm yesterday (GMT1). Either that, or he woke up, checked it, and realised what a waste of time it all was.

Most times a bot is lucky to get 100 URLs from the site before the routines jump them & stop it; normally just a few score. However, if they've browsed around for a bit before setting the bot loose, then they can have several thousand. This guy got 37,000 503s Friday, and 14 x 8,000 = 112,000 yesterday. That's 150,000 URLs! Crikey. That even tops the corporate guy who tried to take 80,000 a couple of months back.

Yesterday's hits have already been reported to Telstra (including 37,000 lines of report). In 4 hours time they are going to get another report with 100,000+ lines included! That'll please 'em.

zaphod · 2011-07-24 1:49 am

That's the problem with govt. sponsored monopolies... they just don't care, because they don't have to.

Well, they used to think that.

Email is handily killing the US Postal Service, and for that, I am happy. They can raise their rates now to where no one will use them, and, now, I won't care.

Zap

pedigree · 2011-07-24 10:35 am

None on this site yet

Alex Kemp · 2011-07-25 3:45 pm

pedigree wrote:

None on this site yet

Unfortunately no email address, so can only report to RBL, not SFS.

To try to complete this thread:

Original attack:
Sat, 23 Jul 2011 00:21:27 +0100:
IP: 139.168.202.185
rDNS: CPE-139-168-202-185.lns1.way.bigpond.net.au
max: 6 pages / second
total: 137,836 pages
ASN: 1221 ASN-TELSTRA Telstra Pty Ltd

To rub salt into the wound:
(this is a simultaneous twin-IP attack at almost 200 hits/sec; a spam botmaster trying to gather results from earlier spam postings?):
Mon, 25 Jul 2011 00:01:03 +0100:
IPs: 203.35.82.133, 203.35.82.136
rDNS: bcbno.tcif.telstra.com.au, bcano.tcif.telstra.com.au
ASN: 1221 ASN-TELSTRA Telstra Pty Ltd

Knut · 2011-07-26 8:16 am

I have added telstra and a few other IPs from your Network Abuse from Bot Scrapers to my htaccess file.

Thanks for useful information.

fcollingwood · 2011-07-26 9:47 pm

Knut wrote:

I have added telstra and a few other IPs from your Network Abuse from Bot Scrapers to my htaccess file.
Thanks for useful information.

MISTAKE!!!!!

Telstra is the biggest ISP in Australia. If you want to block their customers, you'd have to have a pretty big range blocked.

May as well block all of the US, because there's bound to be some moron on AOL with a compromised PC.

Alex Kemp · 2011-07-26 11:26 pm

fcollingwood wrote:

Knut wrote:
I have added telstra ... from your Network Abuse from Bot Scrapers to my htaccess file.
May as well block all of the US, because there's bound to be some moron on AOL with a compromised PC.

I certainly agree with your basic statement on the folly of blocking the entire Telstra ASN. However, if 203.35.82.133 & 203.35.82.136 are simply "compromised PCs", then the bot-masters have learnt two new wrinkles:

1 joint operations
2 server speeds

(both possible, of course)

They normally act together, and at 175 & 116 max hits / second, these two will bring *your* server grinding to a halt. I couldn't even get in on SSL for some time during the last attack. (Remember, my server is in Germany, and these IPs are located in Australia - what is it like for those closer to their home?) If you check out each link, you will see repetitive dual attacks at high rates, ever since my reporting began. I also recognise these rDNS from before that time, so they have been at it for years.

Telstra completely ignore my many reports. If anyone has upper management-level contacts at Telstra, I would appreciate these reports being brought to their attention. If my experience is any reflection, servers across the world are at danger from these two rogue IPs.

fcollingwood · 2011-07-27 12:34 am

Alex Kemp wrote:

fcollingwood wrote:
May as well block all of the US, because there's bound to be some moron on AOL with a compromised PC.
I certainly agree with your basic statement on the folly of blocking the entire Telstra ASN. However, if 203.35.82.133 & 203.35.82.136 are simply "compromised PCs", then the bot-masters have learnt two new wrinkles:
1 joint operations

not really - ever heard of a botnet?

Alex Kemp wrote:

2 server speeds

Again, ever heard of a botnet?

139.168.202.185 is a Bigpond cable modem. (Bigpond is part of Telstra)

203.35.82.133 & 203.35.82.136 are two of many Telstra proxies used by Telstra & Bigpond customers

So yes, you're cutting off a big chunk of Australia, all because a moron or two have compromised machines - and at the levels you're seeing, it's probably a couple of morons all infected with the same crap (Did I mention botnets already?)

Last edited by fcollingwood (2011-07-27 11:41 am)

zaphod · 2011-07-27 7:05 am

Ya won't catch me cutting off Australia. Too many good mates down there.

Australia has a very special ISP problem, ALL ISPs must go through their state sponsored ISP, Telstra. They haven't had a monopoly busting of their MaBell yet, because their Bell system has been stuffing the pockets of government for way too long.

It's really ******* sad when my mates, the closest thing to Americans not on American soil, have less digital freedom...

... than those in the former USSR.

Zap

Jambalaya · 2011-07-28 7:45 pm

You could try raising this matter on http://forums.whirlpool.net.au/
Plenty of Telstra staff read it as it's the largest IT forum in Australia, there is even a branch for BigPond.

Telstra is now mostly publicly owned, the Australian Government having only a minority shareholding following a series of public stock flotations over the past decade.

Alex Kemp · 2011-08-04 8:32 am

fcollingwood wrote:

203.35.82.133 & 203.35.82.136 are two of many Telstra proxies used by Telstra & Bigpond customers

Same two IPs making the identical attempt at downloading my site this morning:
203.35.82.133
203.35.82.136

11 individual attempts since Dec 2010 on 203.35.82.133, normally the same pattern:

1 start 3am GMT1
2 attempt to download entire site at max 176 hits / second
3 get blocked after 7 hits
4 give up after 10 seconds
5 switch to other IP
6 repeat 1 => 4
7 rinse, repeat

There seem to be four IPs in total in this little group:

203.35.82.133 bcano.tcif.telstra.com.au
203.35.82.136 bcbno.tcif.telstra.com.au
203.35.135.133 bcavo.tcif.telstra.com.au
203.35.135.136 bcbvo.tcif.telstra.com.au

(2 sets of twins!)

Alex Kemp · 2011-08-04 8:59 am

Jambalaya wrote:

You could try raising this matter on http://forums.whirlpool.net.au

I've given your suggestion a go:
http://forums.whirlpool.net.au/forum-re … ?t=1750924

matts321 · 2011-08-05 12:56 am

zaphod wrote:

That's the problem with govt. sponsored monopolies... they just don't care, because they don't have to.

Huh? Telstra was privatised many years ago.

203.35.82.133 & 203.35.82.136 are two of many Telstra proxies used by Telstra & Bigpond customers

Telstra don't use proxies. As pointed out on Whirlpool it's actually the Telstra Enterprise Firewall - http://www.telstraenterprise.com/produc … ewall.aspx

zaphod wrote:

Australia has a very special ISP problem, ALL ISPs must go through their state sponsored ISP, Telstra. They haven't had a monopoly busting of their MaBell yet, because their Bell system has been stuffing the pockets of government for way too long.
(

No they don't? Telstra is an ISP like any other. Some ISPs choose to wholesale off Telstra, some choose to use their own equipment. It's no different to the USA.

zaphod · 2011-08-05 7:39 am

Ma Bell, AT&T in the USA was "private" and traded on the NYSE as "T", yet until the trust busting of the 80s, the government looked the other way, as long as AT&T would provide them with longlines for a decent price while they screwed the home customer to the wall. It was nasty, I could go into details, but the breaking up of Ma Bell was the best thing that ever happened to communications, and technology. DSL would only be a pipe dream if they had stayed together, and guessing that 14,400 baud would be the top dial-up limit.

It was real evil.

A friend of mine who lives down in Melbourne has tried several ISPs down tTelstra sets the rules on byte limits per users, and the Tier 2 & 3 ISPhere (oddly enough named Matt too), and has done traceroutes out of all of them. Even though the ISPs are all different companies, the trace has always seemed to go through Telstra before getting under the water. Have also heard that s have to play by their rules.

I am not saying that ISPs lease equipment from Telstra, but the international lines all seemed to be owned by them.

Zap

Alex Kemp · 2011-08-05 7:04 pm

matts321 wrote:

Telstra don't use proxies. As pointed out on Whirlpool it's actually the Telstra Enterprise Firewall - http://www.telstraenterprise.com/produc … ewall.aspx

Hi matts321, welcome to SFS.

It's acting as a proxy, in that it is hiding the actual IPs that are operating behind it.

From the posts & Whims I've received (some from guys working on the Firewall, and now blocked from access to my & other sites) they self-describe it as a "corporate proxy", and heavily monitored (a `Whim' is the Whirlpool equivalent of a PM, whilst a `Herring' is the means to report a post). My response would be that yes, it is a corporate proxy, and yes, it may well be 'heavily monitored' for attempted attacks from outside to in, but no, there is zero monitoring from inside to out, and zero response to abuse reports. In other words, it is a spammers' & abusers' paradise: full core-network bandwidth, full privacy and complete license to do what you want without notice or penalty.

If you glance at the graph on my site abuse report, you will see a background of continuous lo-speed scrape attempts with a hedgehog-hair of very-hi-speed scrape attempts. Each of those latter that I've investigated--max 403 accesses / second--is a corporate / uni / military / government / etc., etc. entity, obviously with core-network bandwidth access. The point is that each can be identified. Your wonderful Telstra Enterprise Firewall hides the identity of each scumbag that commits this abuse. That is a proxy, by action if not name.

Alex Kemp · 2011-08-11 2:32 am

Alex Kemp wrote:

Jambalaya wrote:
You could try raising this matter on http://forums.whirlpool.net.au
I've given your suggestion a go:
http://forums.whirlpool.net.au/forum-re … ?t=1750924

...and got a response!

The security team at Telstra have had an honest try to discover who is responsible. They found 5 different users, 5 different internal hosts, 3 dates: August 4, August 2 + July 11. I thought that their system may baulk at attachments, so sent 2 pages of access-log extracts for each of the 4 IPs. In return they sent anonymised proxy logs for two dates. Here's the discovery, and it's interesting, and has wider implications:

Telstra could NOT find the very many rejected scrape attempts. They COULD find 200 responses for various image files, CSS, etc.. The reason is easy and--in my view--has implications: their proxy logs filter out 403 & 503 responses. Guess what: my site gives these scrapes a 403 or 503 response.

I would expect that filter to be common across most companies: folks tend to keep to defaults. No wonder that all these companies just do not see any problem.

Alex Kemp · 2011-08-11 6:37 am

Alex Kemp wrote:

their proxy logs filter out 403 & 503 responses

I was trying to be kind.

I've received a response from them saying: "Our proxies should record all access attempts. We can verify that other 403 and 503 messages are seen from the same proxies via which connections have been made to your website".

That is a disaster at their end. Vast amounts of abusive traffic are occurring, and none of it appears within their logs, when it should do.

Oh dear.

zaphod · 2011-08-11 9:15 am

But, are you sending warning message 199?

See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html Subsection 14.46 .

Zap

#1 2011-07-22 11:13 am

Is cj.com abusing your website?

#2 2011-07-22 12:21 pm

Re: Is cj.com abusing your website?

#3 2011-07-22 12:54 pm

Re: Is cj.com abusing your website?

#4 2011-07-22 10:08 pm

Re: Is cj.com abusing your website?

#5 2011-07-23 8:53 am

Re: Is cj.com abusing your website?

#6 2011-07-23 4:18 pm

Re: Is cj.com abusing your website?

#7 2011-07-23 4:27 pm

Re: Is cj.com abusing your website?

#8 2011-07-23 11:58 pm

Re: Is cj.com abusing your website?

#9 2011-07-24 1:49 am

Re: Is cj.com abusing your website?

#10 2011-07-24 10:35 am

Re: Is cj.com abusing your website?

#11 2011-07-25 3:45 pm

Re: Is cj.com abusing your website?

#12 2011-07-26 8:16 am

Re: Is cj.com abusing your website?

#13 2011-07-26 9:47 pm

Re: Is cj.com abusing your website?

#14 2011-07-26 11:26 pm

Re: Is cj.com abusing your website?

#15 2011-07-27 12:34 am

Re: Is cj.com abusing your website?

#16 2011-07-27 7:05 am

Re: Is cj.com abusing your website?

#17 2011-07-28 7:45 pm

Re: Is cj.com abusing your website?

#18 2011-08-04 8:32 am

Re: Is cj.com abusing your website?

#19 2011-08-04 8:59 am

Re: Is cj.com abusing your website?

#20 2011-08-05 12:56 am

Re: Is cj.com abusing your website?

#21 2011-08-05 7:39 am

Re: Is cj.com abusing your website?

#22 2011-08-05 7:04 pm

Re: Is cj.com abusing your website?

#23 2011-08-11 2:32 am

Re: Is cj.com abusing your website?

#24 2011-08-11 6:37 am

Re: Is cj.com abusing your website?

#25 2011-08-11 9:15 am

Re: Is cj.com abusing your website?

Board footer