Slashdot Log In
New Spam Frontier: Referer Logs
Posted by
timothy
on Sun Oct 27, 2002 04:45 PM
from the die-spammers-die dept.
from the die-spammers-die dept.
geoffsmith writes "Wired News is reporting that spammers are using referer logs as a cheap new way to
spam small sites. Anyone running a website has probably already seen this phenomenon; I'm thinking of writing a script to remove these entries from my access_log by looking for hits that don't grab my images. (sorry lynx users!)"
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
They will never stop. (Score:4, Insightful)
Re:They will never stop. (Score:5, Interesting)
Lowlife (but capitolist god bless 'em) pigs generate spam to sell their penis enlargement scam and mail clients develop ways to filter and block email. Distraction.
Distributed Denial of Service attacks attempt to shake the very foundations of the NET through bandwidth flooding and sysadmins implement redundancy and load balancing. Jamming - Frequency Hopping.
Remote exploits and virus appear everyday and patches are generated quickly for the more quality OS's and virus updates are required daily for Micro$oft OS's. Infiltration.
Governing bodies exist that the people disagree with such as the RIAA and MPAA. Demonstrations are held in both violent(DDoS) and non-violent(civil disobedience of P2P) manners. Revolution.
Needless to say, civilization has managed to survive for thousands of years despite man's desire to control everything including his fellow men. I think the internet will find a way.
Parent
Re:They will never stop. (Score:5, Informative)
If you want to learn about what civil disobedience really is, check this [eserver.org] or this [actupny.org] out.
If you think that the Internet is the most active battlefield today, you need to visit [un.org] a [disastercenter.com] few [yahoo.com] places [yahoo.com].
Parent
Re:They will never stop. (Score:4, Insightful)
I agree that the Internet can be and is a battlefield. So can Wall Street or the TSE. Or major media sources. Or the telephone. I don't agree that it is the most active - perhaps from a first world perspective, but I try and think a little more globally than that.
And how many people died due to those attacks on the public
Hacking a
(And anyone considering invoking Godwin's law... piss off
I concede that the Internet certainly *is* a battlefield. However, considering that conflict on the Internet barely affects most of the people of the world, I wouldn't rate it so high.
Parent
The spammer speaks... (Score:5, Interesting)
Heh, it's funny that this guy can make this statement and expect to be taken seriously. It's even more pathetic that he actually thinks he's "innnovative".
Re:The spammer speaks... (Score:4, Insightful)
Anyhow, unless the traffic is completely disabling, I don't see this as more than an annoyance that technology will filter out when it becomes sufficiently obnoxious.
Parent
You can do better than that (Score:3, Insightful)
Spam Lite (Score:4, Insightful)
Re:Spam Lite (Score:3, Interesting)
Re:Spam Lite (Score:3, Interesting)
Wish I could remember where I read it, I would give you a link. Best I can find right now is:
http://join.msn.com/?page=features/junkmail&pgm
Re:Spam Lite (Score:3, Interesting)
Lynx users? (Score:5, Funny)
Don't worry. It's highly unlikely that any of the 4 current users will visit your website anyway.
Well.. (Score:3, Insightful)
In other news... (Score:5, Funny)
referer information should be disabled by default (Score:5, Interesting)
Re:referer information should be disabled by defau (Score:4, Insightful)
Then there is the statistics, learn how people navigate around your site. referer can help you see a pattern and improve your layout.
Also it can prevent bandwidth hogs, mostly a issue with ad. graphics and pron sites where people use graphics from others servers on html pages on their own sites but also on free servers where people place graphics and files and link to those directly without using any html and then not showing any of the free servers ad's which provides them with money to run the sites in the first place.
Parent
Re:referer information should be disabled by defau (Score:5, Informative)
Furthermore I can restrict traffic for some areas of my site (like some sites that block links from slashdot) for particular reasons or uses. "You just came from the page of an associate and are able to receive a discount." "This page is restricted to users of xyz.com. Please go there first."
Lastly, it protects my image content. My images are not stellar, and yet other sites continue to use them on their pages. I can use the referrer to limit the damage done by only allowing the images to be referred by pages from my own site.
Referrer information may be annoying to you, but it's an extrememly useful tool. If taken away one restricts opportunities for the site operator to personalize and protect content on their site. Not a huge loss, but it isn't really as great a privacy issue as you seem to believe.
-Adam
Parent
Re:referer information should be disabled by defau (Score:4, Insightful)
This is so damned annoying. If I'm searching for some specific information, I don't give a damn about your idiotic welcome page. I don't care what your website is about or what you have to say on your other pages - all I care about is the specific technical information that google told me you have.
More and more, I'm finding myself using googles cache instead of clicking on the actual links. I know you couldn't care less about my insignificant browsing habits, but the more people start doing annoying crap like this, the more people start using google instead of the web.
"This page is restricted to users of xyz.com. Please go there first."
Do you realize how stupid this is? You're trying to control how I use my browser. Of course I'm not going to go to xyz.com and try to use their idiotic navigation looking for a link to you. You're simply advocating another form of advertisement and I'm not interested. I care about the data you're providing, not how you're getting funded.
I can use the referrer to limit the damage done by only allowing the images to be referred by pages from my own site.
And this is, of course, broken behaviour. Did you know that when you open a new link in Netscape/Mozilla that the browser does not send any referer at all? This means that I can't open your images in new windows and I'm constrained to view your images one at a time. Also, the some browsers change the referer for images when you "save" images (eg, right-click and choose "Save as..." may not send the referer you're expecting).
If taken away one restricts opportunities for the site operator to personalize and protect content on their site.
If you're using this to restrict content to your site ... well, forget it. If you have something I really want, I'll open up a terminal and telnet to port 80. Yes, this is indeed effective restriction. (Quiz to see if you really know what you're doing: how would you set it up so that you know that a user has previously visited another site, with cryptographic confidence?)
As for "personalizing" content, please stop. The only times I've seen that word being used in a web context is to personalize advertising (and also restricting content because I'm not using IE, but don't get me started on that). I've never seen anyone "personalize" a site in a useful way, eg, "You're a C programmer who writes Solaris kernel modules, so you're probably not going to spring for my Herbal viagra scheme and I'm going to cut the marketing BS and give you only useful information."
Why do these "blogs" even keep logs of referer links? This is pure narcisism (and more importantly, a waste of disk space - even though disk is cheap, it's still worth more than someone else's paltry feeling of acceptance). If you're going to say something, just say it. Don't base your life around how many people like what you say. "Ohh, somebody linked to my journal, that means I'm special and I can now feel good about myself." Ahh - get a life.
I swear, "webmasters" piss me off.
Parent
Referer checking for images (Score:4, Interesting)
And this is, of course, broken behaviour.
So do you have an alternative proposal to prevent resource (i.e. bandwidth) theft? That is a very real problem, and no amount of arguing that the current solution is "broken" will get people to change unless you provide them an alternative.
Parent
Re:Referer checking for images (Score:4, Insightful)
Session cookies based a cryptographic hash of browser-identifiable information. Just hashing the IP and some secret string will prevent the bandwidth-stealing problem (not ideal since it breaks with NAT, but that's irrelevant if you're only trying to solve the deep-linking problem).
In php, setcookie('hash', md5($ENV[REMOTE_ADDR] . "TOPSECRET)) on page load, link to a file "image.php" instead of the .jpg and "image.php" does something like this: if (getcookie('hash') != md5($ENV[REMOTE_ADDR] . "TOPSECRET")) { header("Location: /error-documents/403.html"); exit(); }. This isn't complete (probably not even syntactically correct and be careful with what image.php allows one to download), but you get the idea. The actual image files can't be downloaded by apache, but can only be opened and sent to the browser through "image.php". For extra fun, re-generate the secret string from /dev/random every ten minutes (and keep around the last version of the key to avoid breaking on-going sessions).
This stops everyone from stealing bandwidth (including telnet-wielding network programmers like me) and it annoys no one.
Parent
Re:referer information should be disabled by defau (Score:3, Insightful)
Incidentally, I don't know why anyone bothers with logging referrer information. The only use sounds like what the bloggers do. If you're not a blogger, why do you even care who the referrer is? Half the time it's bogus or one of your own pages.
Re:referer information should be disabled by defau (Score:4, Interesting)
It is extremely useful for security purposes.
No, not the security most people are thinking of. Checking to see if the user came from FeedBack.html before executing FormMail.pl is no security, since spammers can forge any referer they want.
I'm talking about security which stops a human user who is logged in to a particular website from being tricked into performing actions they didn't authorise. For instance: I log into my server's adminsitrative area. Then, in another window, I browse someone's blog. And I click on their "search" button. As it turns out, this search button is a trap, which sends me to my own admin area with a command to delete someone's account. I'm logged in, I have a valid network address, I'm active, there's no problem. Except that fortunately my browser sends "Referer: www.blog.org" instead of "Referer: www.admin.com".
That's why referer info is useful: to prevent a user from being hijacked.
Parent
Re:referer information should be disabled by defau (Score:5, Funny)
Yes, referrer information makes an excellent authentication scheme for highly confidential system dealing with transfer of mission critical information. ... Just also check for a magic string in the user agent and voila! trusted computing reinvented. To make it unhackable - just add a few more levels of obfuscation. ;))) The sad part of this, is that I have actually seen authentication schemes like this. Don't know whether I should cry or laugh :)
Parent
Re:referer information should be disabled by defau (Score:3, Insightful)
Just also check for a magic string in the user agent and voila! trusted computing reinvented. To make it unhackable - just add a few more levels of obfuscation. ;))) The sad part of this, is that I have actually seen authentication schemes like this. Don't know whether I should cry or laugh :)
probably cry... what you described could easily be enforced with the DMCA.
If you use wget, watch out when using "--referer" and "--user-agent".... you just might be breaking TEH LAW!!!
Sorry 'bout what? (Score:3, Insightful)
Sorry about what? Why should they care wether you keep them in your log or not?
Boost search engine ranking? (Score:3, Insightful)
From the wired article:
Umm, huh? I don't think the spammers actually link to the sites, they probably just send HTTP requests with faked referrer headers that contain the URLs of the spammer's web site. That won't boost your search engine rankings.
Score another for Opera! (Score:3, Informative)
what is this? (Score:3, Interesting)
Or are they just bots that hit random web sites and send fake referers along?
Either way, I have absolutely no clue why this would be abusive or even annoying? Can someone explain? Do people sit around checking their referers all day long?? (Then again, I don't understand why anyone would run a blog, so maybe I'm just out of touch).
I clean out all my outgoing referers (thanks squid), so maybe I subconciously assume everybody else does too. Never thought of the referers as anything but a silly waste of bandwidth, since they can be forged so easily.
It's ironic... (Score:4, Funny)
"... they are hitting my site so hard. One day there were more than a thousand hits from one single porn site," complained Åsk Wäppling, known to bloggers as "Dabitch."
Ok - this guy's name is Dabitch in an obvious reference to the whole pimp-bitch-rape-mysogynist thing that seems to be all over the Internet (and that I can't stand), and he's complaining that a porn site is effectively doing a denial-of-service. I think it's just desserts.
Re:It's ironic... (Score:5, Funny)
By making over 1000 "hits" in a day, which isn't even one per minute. Sounds like "Dabitch" needs to consider hosting that blog at an ISP instead of over a 28kbps modem.
Parent
referer not trusted information (Score:4, Insightful)
Here's some scripts ... (Score:3, Informative)
Also, showpath.pl and spampath.pl are handy to see where the vister went, and check for obvious robot behaviour:
http://www.turnstep.com/Spambot/Programs/
What do small website admins do with apache logs? (Score:5, Funny)
1. To see how search engine users have found my pages. I think it is amazing that I will get hits from people looking at the 100th page generated by google. I mean, c'mon on, 100 clicks on "next" before you think the I have the info you are looking for?! All I have is crap.
2. To see if slashdotters actually click on my postings' links.
Come to think of it, the only other thing I even look at the logs for are to see what search engines are indexing me (and are they obeying robots.txt) and to harrass people with infected iis machines.
You mean code red is spam? (Score:4, Funny)
Obligatory Simpsons Reference (Score:5, Funny)
"No! That's German for 'the spammers, the.'"
"No one who speaks German could be an evil man."
ewww! (Score:3, Informative)
Old trick (Score:5, Informative)
I asked the owner.... (Score:4, Informative)
From: "[ f ]"
Date: Thu, October 24, 2002 12:19 pm
Subject: Re: logs
Hi!
You can send us your URL and we'll permenantely remove it from our database.
It's not a spider, it reads only the blogs in the database.
Regards,
Francois
And to add to that, one of the comments on my weblog was that there is also an opt-out list [mastodonte.com].
check for valid referrers (Score:5, Informative)
backlink links (Score:4, Interesting)
While the process has gained some popularity [decafbad.com] of late in the blogosphere, the idea has been around [unrealities.com] for quite some time [gmu.edu].
Are tactics like this legitimate forms of marketing? Would you term it Gonzo Marketing [gonzomarkets.com]? Viral Marketing [wilsonweb.com]? Whatever term you use, it will definitely be interesting to see where this one goes.
Script to check it out... (Score:4, Informative)
Backlinking (Score:5, Insightful)
Also, this quote from the article is ludicrous: "bloggers are not thrilled, even though they ruefully admit that the log spamming may falsely boost their ranking on some search engines."
There is no search engine that bases your rank on the number of sites that you LINK to. I believe the bloggers actually mean that they're sorry to see their backlinks (read: link farms) go, since those do in fact raise search rankings. What a travesty- Sites may have to rely on the actual quality of their content, rather than trading links!
Amidst the alarmist cries in the article, "spammers will destroy our practice of posting referral logs," nobody has even mentioned that there is a ridiculously easy technical solution. Before posting a referral link, why not just have your software visit the referring site and detemine if it actually links to your page? This will defeat the referral advertisers.
Guestbook spam (Score:4, Interesting)
For now I'll delete the entries by hand, but if this increases it could get really annoying.
AlpineR
Re:huh (Score:5, Informative)
255.255.255.255 - - [27/Oct/2002:00:00:00 -0000] "GET /perfectly/valid/page/at/yoursite.html" 200 2467 "http://www.wilddonkeysex.com_for_Wild_Donkey_Sex/ " "(SpamBot5000)"
and then people looking at the report would say, "hey, the page at wilddonkeysex links to my perfectly/valid/page and it's getting like 500 hits a day from there, woo! let's click on that url and see what the link to my page looks like!"
-calyxa
Parent
Re:huh (Score:4, Informative)
I think it's more than the web site's owner clicking on the page - a lot of bloggers post a list of "top referrers" on their web site as a way of thanking the referrers, and therefore they generate a lot of traffic to their referrers from their own visitors.
Parent
Re:*sigh* (Score:5, Insightful)
[Wishful thinking mode ON!]
This implies that there are, maybe, all of 10,000 suckers who keep every spammer on the planet in business. If we find them and cut them off, spam response would drop to about 1 per billion and there's just no way they could make any money off of that.
Parent
Re:*sigh* (Score:3, Insightful)
I always use the example of my father, who is your archetypical pre-UNIX geek. He did all the PDP-11 stuff, worked with the VAXes and hacked machine code in ways that I don't yet understand -- an intensely intelligent man. Yet, every few months when I go to visit him, we get to talking about the internet and the first thing he does is talk about what he's bought online. For him, paying spammers is part and parcel of buying online -- he's paid spammers for search engine placings for his personal site, silly trinkets like water pumps and gardening tools and books.
To people who aren't part of the current 'geek' cognoscenti, spam is just another form of valid advertising, like the leaflets they get in the post and the billboards they walk past on their way to work. This isn't a specific group of people -- you can't "find them and cut them off" -- you need to target the problem at its source.
Re:*sigh* (Score:4, Insightful)
The guy who sold it to me was obviouly a late teen, and was making ok money selling shirts at about $5 profit per when I called him.
I think most geeks have no problem with spam itself (in fact targeted spams that interest me often get clicks, I get about two of those a year), they have a problem with the number of scams that are sent using spam.
Parent
Re:I don't know if these are *as* bad. (Score:3, Interesting)
I see a solution in this. It would be the spammer's own DOS attack. If they willing to download
Re:Spamfilter (Score:5, Informative)
(Sorry, Unix-like systems only.)
Parent