Attorney Sues Website Over His Online Rating 207
An anonymous reader writes "The Seattle Post-Intelligencer is reporting that a local attorney is suing legal startup Avvo over a rating that was algorithmically assigned. The story covers the controversy of computers grading humans. 'Browne, who has participated in a number of high-profile cases in the state, including the defense of arsonist Martin Pang, said in an interview that Avvo is being irresponsible with the ratings and called them a fraud. And he questioned why Supreme Court justices and prominent lawyers score so low. Three other attorneys interviewed by the P-I also expressed doubts about the rating system, while News.com reported that the site "seemed to be riddled with bizarre errors."' Such practices are not new: the New York Times earlier this year reported on Google using algorithms to determine applicant suitability. But what happens when you don't like the result? Can a computer program be considered defamatory?"
Just another tool. (Score:5, Insightful)
If I use a hand-held calculator to get a result, and then publish it and that publication defames someone, I can't blame it on the calculator.
In this case, a computer is just another tool used to calculate something - perhaps a tool that many people don't understand as well as they should - but a tool nonetheless.
You use it, you take responsibility for the results. You don't understand how it works? Hire a consultant. The fact that it is a complex tool does not excuse you if you libel someone.
( NB: The above paragraphs presume that there is indeed libel, a fact not yet proven.)
Nice sig (Score:3, Funny)
Parent's Sig: When members of a profession start referring to non-members as 'laymen', it is time to start shooting them.
Shooting whom? The members of the profession or the laymen? If you're going to call for someone's head, you should at least be a little more specific about whose head you want served to you on a silver platter.
Re:Nice sig (Score:5, Funny)
u = "start referring to nonmembers as 'laymen'"
When members of a profession u, it is time to start shooting them.
It becomes clear, now.
Food for thought, or thought for food? (Score:5, Funny)
Those who call lawyers useless aren't being very open minded. Lawyers may be a melamine-free source of protein, but I think my cat would prefer something a bit less bony than the head.
I wonder if the computer program has a way to rate them on flavor?
Re:Food for thought, or thought for food? (Score:5, Informative)
I had to get a lawyer because my landlord was a slimy, conniving liar. And it's turned out to be some of the best money I've ever spent, because before I was spending loads of time doing all the paperwork and covering my own ass. Now the lawyer does it. And he's a nice guy, to boot. Shocking.
Re: (Score:3, Funny)
Re: (Score:3, Funny)
Defense Lawyers = Good
Other Person's Lawyer = Bad
Prosecutor Lawyers = Really Bad
Corporate Lawyers = Really Evil
Re: (Score:3, Interesting)
Our landlord had just purchased the place, and was still making repairs. One thing we stated very clearly was that we NEEDED the garage, because we were 7 people, and I have a home-based business. He said that the previous owner had a lot of garbage in the garage, and it would be cleaned out and we'd be provided access, then he would finish the garage. He also
Re: (Score:3, Funny)
Re:Just another tool. (Score:5, Insightful)
I don't know what to think of this. As a restaurant, you can earn michelin stars based on the grades you get from probably several testers. Did you ever hear of a restaurant that sued michelin for loosing a star? It doesn't seem to make sense.
Rmember, any person, magazine, or website can grade services, but they will only be taken seriously when they have a decent method. Systems like this work when not only the public has faith in the method, but also the people involved (e.g. the chefs), respect the way the testing was done. If restaurants would think that the michelin system is fake, they wouldn't be proud of their stars in the first place.
Since there seems to be quite a group of people that do not agree with the method used by that website, they can of course try to sue them, but I figure that the website will be rendered useless within the trade fast enough that they might as well just ignore the score of the website all together. Sueing might even be counterproductive, I didn't know before that the website existed in the first place. And I think he has very much the right to do so. He never asked to be rated, but at the same time the rating will be of high economic importance to him (getting more high-profile jobs, etc.). Now you could compare this with a restaurant getting the famous michelin stars, it can make or brake the restaurant. Now In cases like that, you need to be able to ask for a second opinion.
Re:Just another tool. (Score:4, Insightful)
Some human made a decision somewhere.
Re:Just another tool. (Score:5, Interesting)
I think not.
Re: (Score:3, Interesting)
The problem here isn't the math, or the low ratings. The problem is idiots like you who don't understand that an industry rating depends on the faith of the people and the respect of the industry.
If the site gets no faith or respect, the site will fail.
Re: (Score:3, Funny)
Re:Just another tool. (Score:5, Insightful)
That's what this situation reminds me of. Credit scores (which is what the above lines reference) are calculated by an algorithm, and they can be wrong. If they are, you can get the company to adjust them, provided that you can provide evidence that their input is faulty. I see no reason that this case should be any different, provided the inputs (reasons for the ranking) are well known.
In the case of Google, however, they are providing a ranking rather than a rating. It is much, much harder to objectively rank webpage relevance based on search terms, and even harder to know whether a given ranking is justified when you don't even know the algorithm used.
Re: (Score:2)
Re:Just another tool. (Score:4, Insightful)
Re: (Score:3, Funny)
I think layman is a lot better than vulgar, don't you, you vulgar person?
Re:Just another tool. (Score:5, Funny)
Have you read some of the comments people put in source code?
Re:Just another tool. (Score:5, Insightful)
I disagree. Programs are merely an extension of the human(s) who designed them, the program does what the human(s) told them to do. Therefore what a program does is the full responsibility of the human who designed it.
I always laugh when a programmer tells me, "there's a bug in my program." My first questions is always, "well, who put that bug there?" Programmers talk of bugs as if they just magically appear, and are not the result of the programmer's error(s).
The comment that programs cannot be defamatory smacks as specious at best. Of course, programs can be defamatory. Programs are written by humans, programs are computerized extensions of humans.
Re:Just another tool. (Score:5, Insightful)
if(applicant.sex == female) rating = 0.0;
That would certainly violate equal opportunity laws, unless the company could prove that the output of said program was not used at all.
One can argue many ways on the X of CS and Law... (Score:2)
if(applicant.sex == female && will_halt(random_bytes())
|| applicant.sex == male && !will_halt(random_bytes()) rating = 0.0;
-- making the one having to prove the violation of the equal opportunity law (presumably NOT the company using the test!) also being able to violate the halting theorem [wikipedia.org]!
Should we j
Re:One can argue many ways on the X of CS and Law. (Score:2)
Re: (Score:2)
Re:One can argue many ways on the X of CS and Law. (Score:3, Insightful)
Huh? (Score:2)
If it is statistically proven that white guys rape black girls significantly more often, or asian girls murder russian ones significantly more often (tried to put as many random inversions as I could, and, I'd guess, still failed to satisfy the PC crowd
But good try, next one, please!
Paul B.
Re: (Score:3, Insightful)
Ranking someone based on your critera is not libel.
Re: (Score:2)
Re:Just another tool. (Score:5, Funny)
Oh yeah?
int main()
{
int x = 2;
return 0;
}
Re:Just another tool. (Score:4, Insightful)
But is it libel? I see very little difference between this and consumer reports - Both take data and attempt to draw a simple rating/conclusion from it. If the lawer wins his case it undermines the whole independent review and ratings system - because anybody giving a negative rating would be open to a lawsuit.
How they weigh their ratings is up to them, just as it's my choice as to how heavily I weigh their ratings (anywhere from not bothering to read them to simply attempting to get the highest rated).
Errors in the data is one thing - for it to be true libel it would probably have to be deliberate. As they get more data into the system, improve accuracy, etc... The ratings should become more accurate.
Re: (Score:2)
No, but the criteria the computer program uses to determine the rankings can be.
Re: (Score:2)
Re:My Opinion? (Score:5, Interesting)
I can think of a few reasons why supreme court judges would be legitimately rated low. Support for blatant ex post facto laws; commerce clause inversion; allowing states to take people's property; allowing wiretapping before a warrant is issued (FISA); Allowing congress to blackmail the states by withholding highway (and other) funds; allowing the right to bear arms to be infringed upon; allowing government support of Christianity; outright ignoring the 10th amendment (see earlier reference to blackmail); cowardly and un-statesmanlike refusal to hear critical cases of government malfeasance (like Robert Newdow's); allowing the state to infringe upon the liberties of the citizens (drug war, censorship, marriage, sexuality, unreasonable copyright and patent terms and mechanisms; allowing the feds to step beyond the enumerated powers without requiring a constitutional convention; restricting freedom of speech (free speech zones, censorship, funeral zones, etc.)... that's all just off the top of my head.
Yeah, I could definitely see low supreme court ratings.
Re: (Score:3, Informative)
Correction: "Robert Newdow" should be "Michael Newdow." Told you it was off the top of my head. :) My apologies for the error.
Re: (Score:2, Offtopic)
I suspect that you mean to say 'justices,' not 'judges.'
commerce clause inversion
Which would be, what? The negative commerce clause?
allowing states to take people's property
Why? States have always had that power. Indeed, the federal Constitution only requires that governments pay fair compensation to the owner when they seize property under the eminent domain power. It not only doesn't prohibit it, but by making it condit
Re:My Opinion? (Score:5, Insightful)
Yep, that'd be close, all right. Perhaps I meant injustices.
That would be the feds marching into California and swooping down on medical marijuana users based on a commerce clause argument that 100% intrastate commerce "could be" or "could have been" interstate commerce, and so the feds claim to have jurisdiction to screw with California law, legislators, and citizens. Which they do not. The ruling and the reasoning is sophist nonsense. The constitution says in sec 8, para 1 through para 3, that The Congress shall have Power To... regulate Commerce... among the several States. That's it. No more than that. It's an enumerated power, and there is no authority implied or specified that allows mucking about with commerce internal to a state. Furthermore, the 10th amendment makes the limit explicit: The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people. So there you have it. They have the authority to regulate intrastate commerce, but not interstate commerce. The court is out of line, and so are the feds.
That observation in no way precludes the fact that there are other ways to build roads, including ways that don't screw with people's properties at all. You can go under them; you can go around them; worst case, you can even go over them, though you certainly ought to pay for that privilege. It is wrong to steal, and it is no less wrong when the government does it.
The idea of "fair compensation" is intellectually bankrupt. If I own a piece of land, and I want to sell it, that is where it can be determined that it has a specific monetary value. The way that is done is that when you offer enough money to satisfy me, I'll let you have it.
If you don't, I won't. But if I own that property and for whatever reason, I do not want to sell it, then you cannot put a value on it that equates to "reasonable compensation." How do you compensate for my ancestors having raised generations there? How do you compensate for the view, or the fishing in my lake? How do you compensate for the fact that my brother died in that house, or that I was married there? Or that I built it by hand? Or that I lost my virginity on the living room couch?
The answer, of course, is that you can't, not that it is fair to use some number a bunch of people I didn't delegate my feelings and associations to invented based on their feelings. I'll tell you how it actually works: force and threat of force. Coercion. There isn't a reasonable step in the entire process.
Sure, I'll point the problem out. It is what we call blackmail, where one party is forced to do something it does not believe is legal, ethical or otherwise proper, by another party that wields a coercive force. I'll point something else out, too: the trust the people put in the government to build and maintain a general infrastructure doesn't include the presumption that said power will be used as a weapon, nor does it include the presumption that the feds won't build roads in some states, while building in others. The collection of taxes is (barely) tolerated with the idea that said collection is done for the common good, not in order to wield a coercive force on the states. The fact that the feds do wield such coercive forces is contrary to article 1, section 8 of the
Re: (Score:2)
Of course, I meant to say: "They have the authority to regulate interstate commerce, but not intrastate commerce." Again, my apologies.
Re: (Score:3, Insightful)
Unless that commerce is interstate commerce despite remaining within the state. Honestly, your argument has lost since Gibbons, which was back in the 1820's.
That observation in no way precludes the fact that there are other ways to build roads, including ways that don't screw with people's properties at all. You can go under them; you can go around them; worst case, you can ev
Re: (Score:3, Insightful)
See, you probably didn't even have to tell us you were a lawyer; your ability to consider an absolutely absurd and contradictory argument perfectly reasonable might have done it. "Unless that is a pillow despite being a knife" "Unless that color is blue, despite being red." "Unless you actually threatened me, despite having never threatened anyone in your life."
Re: (Score:2, Offtopic)
Which is why the government will make an offer based on their appraisal of the value. And if you don't like it, you can negotiate for a better price, and ultimately sue for a better price. Then each side brings in appraisers and a jury decides based upon the evidence presented to them.
This isn't difficult. Courts have to make decisions about how much things cost all the time.
The Ninth amendme
Re: (Score:3, Informative)
Okay, this is the amendment that reads "The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved for the States respectively, or to the people."
So, if the power to spend federal monies was not delegated to the United States, then you'd have a point. Let's see what Article I has to say:
"The Congress shall have power
Re: (Score:2)
Of course it does. The Fifth Amendment says, among other things, "nor shall private property be taken for public use, without just compensation." If the federal government didn't have an eminent domain power, there would be no need to restrict the use of that power; thus, it must have that power.
the property owner "must be justly compensated"
Yes, but the question is, for what? The cost of the property, or that and other associated co
Re: (Score:2)
Here's the relevant part of the fifth, emphasis mine, other subject matter clipped:
So the authority to offer compensation for taking land is there. The problem is that phrase "without just
Re: (Score:3, Informative)
Yes. You've agreed that the government has the power to take land. Remember that word: take. Not 'buy,' or 'purchase,' but take. That is to take it from the previous owner, whether that person likes it or not.
Which brings us to this: No such thing as "just compensation" can be established by third parties if the property is not offered for sale.
Assuming that you're right -- you're not, btw -- then we'd have a little conundrum.
You are saying th
Re: (Score:2, Insightful)
Sure, but you at least need some wiggle room. You need to find, somewhere, some arguable limit on the federal spending power. An implied limit would be fine, a penumbral limit, fine, but so far I'm not seeing any support for it, nor are you showing me any.
The states have the right to set their own speed limit. The tenth amendment shows that since the federal government does not have the power to set a speed limit, the states get it.
Th
Re: (Score:2)
Interesting mixture. I'll abbreviate your post to:
"I don't like the Supreme Court because they do too much bench legislation for causes I don't advocate. I also don't like the Supreme Court because they don't do enough bench legislation for caues I do advocate.
Perhaps you should be asking Congress to solve these problems.
Re: Your opinion - wrong. (Score:2)
How about instead you abbreviate it to the supreme court is ruling in a way that is forbidden, many times over, by the constitution, without supporting amendment, and so they are operating illegally and contrary to their charter, instead? Because if you make that change, you'll enjoy the luxury of actually being accurate. If that's of interest to you.
Re: (Score:2)
There, fixed that abreviation for you.
Re: (Score:2)
Are you kidding? That's what makes Supreme Court justices so great!
Ironically ... (Score:5, Funny)
Re: (Score:3, Funny)
Unless he wins (this is in the US, right?), in which case they'll be obliged to increase his rating.
"Think I'm a bad lawyer? I'll show you!"...."See, I told you I was a good lawyer."
This should set a precedent (Score:2, Funny)
Re: (Score:2, Funny)
It's an input problem (Score:5, Interesting)
An example that most here can relate to is the US News and World Report college rankings. It's a whole other topic in itself, but suffice it to say that there is a lot of discussion about their inputs and how it has influenced the way colleges operate. Most colleges try to get many small donations instead of a few big ones, because the rankings weigh number of donors more heavily than total amount donated. They encourage many, many applications from just about anyone because they get ranked based on the number of applications that they reject.
Once people learn what the inputs are, they just game the system.
Re:It's an input problem (Score:5, Interesting)
Re: (Score:2)
Ugh... (Score:4, Insightful)
Re: (Score:2)
Apparently not. They'd be drooling idiots if they just said "well, it's a computer rating. They're expected to be wrong, I'm fine with it" and did nothing about it.
Re: (Score:3, Interesting)
It happens because human laws have a lot of flexibility that computers don't have. When human logic turns against them, people respond with "don't be a fool, you know what I mean". You can't do that with computers.
It's a simple fact that this guy is not as good as some other lawyers, according to a given set of objective factors. If they change the algorithm to improve his rating, then what about other lawyer
Re: (Score:2)
What? Of course they are flexible. They can be programmed to calculate whatever you want them to, and they can be changed on a whim.
Huh? Of course they can, you just program them to make everybody look good.
"Can a computer program be considered derogatory?" (Score:4, Insightful)
switch (person.getSkinColor()) {
Re:"Can a computer program be considered derogator (Score:5, Interesting)
As an aside, I really doubt that skin colour is going to be expressible as a simple enumeration or integer, so you won't be able to switch on it...
Re: (Score:2)
Anyway, I think the only way an algorithm (or program) would be considered defamatory is if you didn't precisely specify what the output meant. For example, if your algorithm said white lawyers got a valu
GIGO (Score:2)
Program = Algorithm + Data. - Knuth [wikipedia.org].
"if you didn't precisely specify what the output meant"
If you expand that to include input, you will have described what is otherwise known as - GIGO [wikipedia.org].
BTW: A subjective judgement as to wether the "garbage" output by a particular program is/isn't defamatory is outside the realm of computer science.
Re:"Can a computer program be considered derogator (Score:2, Funny)
Fixed:
try {
switch (person.getSkinColor()) {
}
catch( CaucasoidFeaturesException ) {
switch (person.getReligion()) {
switch (person.getCountryOfOrigin()) {
}
finally { person.warilyAccept() }
Defamatory? (Score:5, Insightful)
Re: (Score:3, Interesting)
Everyone in RL judges people based on algorithms in their heads, "He didn't do as well as person Y, and charges more than Z". Typical defamation cases are thrown against people who have messed up algorithms, "He wears white all the time...I hate white" or poor input, "I've only seen him lose". Why shouldn't a
It's called "harassment tax" (Score:2)
Yes, but when you rate lawyers, they will find loopholes in loopholes and mess with you anyhow. Remember that even if they don't win a case, they can make you drive all over, pay attorneys, and hang out in court-houses to counter the charges. I once tried to start up a school bulletin board-like website, but I soon found out that the legal ramifications associated with children can be humongous.
In a perfect world... (Score:3, Funny)
2) Attorney loses, and his rating goes even lower.
3)
4) Profit!
Ok, 3 and 4 aren't really necessary.
Am I a Good Attorney...? (Score:4, Funny)
FICO (Score:3, Interesting)
Reminds me of... (Score:2)
Online Auto Diagnosis Doctor [using AOL voice]: You've got... Leprosy!
Upstairs Downstairs Lawyer on Retainer (Score:2)
Looks like he deserves a low rating to me. Are we sure he's a real lawyer? Or does he just somehow avoid all discussions pertaining to jurisprudence?
Re: (Score:2)
Well, it could be the whole '1' is the best, '10' is the worst type scale, but I figure that any SC justice is probably going to score low because he or she hasn't operated as an attorney for quite a while.
As for the prominent lawyers- I can't be sure. I don't know how they rank stuff.
Thinks I'd figure on:
Years in Practice
Number of cases
Success rate
Appeal rate
Charge Rate
Any malpractice/suits against the lawyer/bar judgements.
Sympathy for the Devil? (Score:2)
Re:Sympathy for the Devil? (Score:4, Insightful)
Re: (Score:2)
Petty (Score:2, Interesting)
Re: (Score:2)
Brown typically represents people accused of high-profile crimes such as murder and rape. Most (all?) of his clients are guilty. Given the trash he represents, it wouldn't surprise me if he has a high loss rate. Everyone deserves a lawyer, and in Washington, when you've chopped up a few people, killed a few in an arson, done a rape or two, if you've got money, you get Brown. You
My online rating (Score:2, Insightful)
bad program but not defamatory (Score:5, Interesting)
It sounds like the rating algorithm isn't very good, but I don't see how this guy can win a suit for defamation. In US law, only false claims of FACT are actionable. If the web site stated that a lawyer had received a reprimand from the bar association when he had not, that would be defamatory. If it said he had cheated a client or bribed a juror and he had not, that would be defamatory. But saying: "This guy is a jackass" or "This guy is a poor lawyer" is not actionable because these are opinions.
I'm not sure what can be made of the use of a poor algorithm. If they disclose the algorithm and say "Here is what we get when we plug in the data we have", so long as the data is accurate and they apply the algorithm correctly, they aren't making any false claims of fact. Ethically, it seems like there should be a penalty if they persist in using an algorithm that demonstrably does not produce output that is reasonably related to what people generally take to be valid measures of lawyer quality and if they deceive people into thinking that it is valid, but I'm not sure how this can be addressed legally. I think you'd have to argue that there is an objective definition of lawyer quality of which the algorithm gives a false view. I don't know if defamation has ever been proven on such a basis.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Couching something as an opinion does not protect you if what you claim is actually a matter of fact, but a statement that really is merely opinion is indeed not actionable. That's black letter law.
Re: (Score:2)
If you don't like it, show its faults (Score:2, Interesting)
If the algorithm has a rule that says "if lastname=Browne then rating=rating-100" that's obviously a bogus algorithm.
Fundamentally flawed business plan. (Score:5, Funny)
This is quite possibly the first time anyone thought they could make money by being sued constantly. Anyone who thought that the dot-com bubble used up all of the reservoirs of stupidity may now rest assured that fresh reserves have been discovered.
Re: (Score:2)
No malice = no defamation (Score:3, Interesting)
This guy is a public figure. In order to successfully claim libel, he must prove malice [eff.org]. An algorithm can't have malice against him in particular. This guy is out of his depth. To paraphrase an old saying in the legal profession, a lawyer who represents himself is an idiot.
Re: (Score:2)
This guy is a public figure. In order to successfully claim libel, he must prove malice [eff.org]. An algorithm can't have malice against him in particular.
Of course, if the algorithm does have malice against him, it didn't get that way by accident.
Shocking (Score:2)
Re: (Score:2, Funny)
It says something about American politics that the first thing that came to mind involved Bill Clinton and an intern.
Attorney's solution. Litigate. (Score:4, Insightful)
Does that mean the company behind the algorithm is biased? Yeah again. In this case (not necessarily Avvo's case) this algorithm is blatantly biased, trying to rate people on their skin color.
Does this mean it's illegal? Not unless the law has changed. If he wants to litigate then he needs to prove the algorithm is biased (and a few anomalies doesn't mean it's biased, it means it has a flaw) Avvo has to be biased in such a way they are making a profit over the difference. Avvo has to be deliberating trying to damage someone's career for it to even be illegal.
If the algorithm is running correctly and there's no X factor (meaningless stastical values, such as the color of the skin) then there's no crime here. They might not have a perfect algorithm but they arn't claiming it.
If the lawyer in question wanted to fix this correctly, he should bring this to the attention of the site, point out numerous cases of people being graded too harshly and then publicize the data to the public if need be. From the sound of it, there's little to no proof except some lawyers feeling they are being treated unfairly. From the sound of it, that's sour grapes, there's no defamation.
Just because an algorithm is poorly designed it doesn't mean it's crap, errors happen even with great algorithm's first iterations. There's still a burden of proof on the lawyer and I'm not hearing any real proof yet.
Credit Ratings (Score:2, Insightful)
Computers Don't Design Algorithms, People Do (Score:2)
Medical Expert Systems (Score:2, Interesting)
IIRC, the hospital chose not to use the system for fear of litigation when the expert systems d
Statistical reality. (Score:2)
You know how dumb the average guy is? Well, by definition, half of them are even dumber than that! -- J.R. "Bob" Dobbs
Mal-2
How to do ratings right (Score:4, Interesting)
We've had to face a similar problem as Avvo with SiteTruth [sitetruth.com], which rates web sites. The answer seems to have two parts - integrity and transparency. This means looking at information that comes from reliable sources other than the thing being rated, and showing the information from which the ranking is derived.
Avvo is trying to do this. Avvo's information comes partly from external sources, like legal directories and records of disciplinary actions. That's less game-able than traditional web search. And Avvo shows that information, so they have transparency.
Google is slowly coming around to this point of view. Originally, Google rankings were opaque, but now they've put in various "Webmaster Console" features to show some of the information that drives their algorithm.
Google faces the problem that some of their metrics for detecting junk web sites are heuristic, and rely on "security through obscurity". They don't want to say exactly how obscure text can be before it's considered "hidden text", or exactly what they consider a "link farm", or they'll be spammed right up to the allowed limit. So they can't have full transparency. They're inherently limited by the approach of primarily looking at the web site itself, which the site operator can change freely, to rate the site.
Google does look at some external non-Web information, but mostly things like how long a domain has been registered.
Avvo has user ratings of lawyers, which probably aren't that useful. User ratings are most valuable when the universe of raters is much larger than the number of things being rated. So it's good for major movies, where there are tens of new movies and millions of fans, marginal for hotels, and weak for businesses few people have heard of. There aren't enough clients per lawyer to get a statistically valid result, and it's too easy to game when the number of raters is small.
Re: (Score:2)
Re: (Score:2)