Washington Finds Computer Simulation Unreliable 277
Toadpipe writes "Washington State Court of Appeals reverses a conviction in which a computer simulation had been the main evidence. Quoting 'At issue was PC-Crash, a computer program distributed by Vancouver, B.C.-based MacInnis Engineering Associates. The program recreates traffic collisions using simulations and reconstructions.
"PC-Crash had not been validated for the purpose for which the evidence was offered, simulation and prediction of multiple-occupant movement within a vehicle during a multiple-collision accident," the Court of Appeals said in ordering a new trial. "There is no general acceptance in the relevant scientific community of the use of the PC-Crash program for the purposes to which it was put."' Here is
the Court's opinion."
What next? (Score:5, Funny)
Re:What next? (Score:2)
JFK Reloaded (Score:2)
Re:What next? (Score:3, Funny)
... and in related news, Electronic Arts has announced that it's pursuing court certification for its Sims games series. "I'm absolutely certain they're certifiable," said a company spokesman, though it was unclear if he was referring to the games or the players. "And we've already incorporated sophisticated PC crash behaviors right into the game," he added. Details to follow at 11.
Eric
William Shatner, the unknown cereal box celebrity [ericgiguere.com]
Darn it (Score:5, Funny)
Re:Darn it (Score:2)
Re:Darn it (Score:2)
Re:Darn it (Score:4, Insightful)
Re:Darn it (Score:2, Informative)
Re:Darn it (Score:2)
I do know that some instructors use it to practice the scanning of instruments, or how to scan them without losing your focus on your surrounding or somesuch.
I once saw a simulator running of a stock PC at a flight school but it wasn't MS'. The controls were made by Jeppesen (I think) so maybe they made the software as well.
PC Crash? (Score:5, Funny)
Re:PC Crash? (Score:2)
Apparently Bill doesn't like it, though: Microsoft's Gates Endures PC Crash During Keynote Speech [osopinion.com].
EricSee your HTTP headers live [ericgiguere.com]
Re:Absolutely, absolutely (Score:3)
In other news (Score:4, Funny)
Re:In other news (Score:5, Informative)
Re:In other news (Score:5, Funny)
Re:In other news (Score:3, Insightful)
Correlation is not causation.
Example: I started losing weight around the same time as I started exercising. Because I started exercising, I lost weight.
The logical fallacy is that two events are linked when there's nothing to link them other than their concurrency. There are other possible explanations, such as dieting, intestinal parasites, or other diseases. Regardless, I don't need to suggest alternate explanations to show the fallacy, simply the fact that there's no illustration of cause throug
Re:In other news (Score:3, Insightful)
When you assume that you cannot determine causality, the best you can do is to create a theory that stands up to testing.
In fact, you cannot say that your dieting is the cause of your weight loss. It may have been coincidence. The best you can say is that it is 100% correlated with a large sample set (and thus high confidence).
Example:
Observation: I started exercising
Observation: I started to lose weight.
These observations are 100% correlated.
Hypothesis: Exercising causes weight loss.
Testing: Exercise,
Re:Human nature will not permit this (Score:3, Insightful)
How is polluting less, or at least differently, going to "terminate their economic development"? Isn't rising to a technical challenge something that our brilliant global economy is supposed to be able to manage, what with Innovation and Freedom and all that?
Re:Global Warming Debate (Score:4, Informative)
Uh, dude? You do know that global warming isn't contested, right?
Really? See http://www.sepp.org/books/hotcold.html [sepp.org] and http://www.spiked-online.com/Printable/00000002D37 1.htm [spiked-online.com] among others.
Not so long ago, we were facing imminent threat of an ice age caused by -- you guessed it -- our polluting ways. The proponents then were as convinced of their inerrancy as you are now.
When your computer model can accurately predict whether it will rain ten years from next Friday, then your inanity will warrant a rethink.
Re:Global Warming Debate (Score:3, Funny)
Re:In other news (Score:2, Insightful)
not unreliable, but... (Score:5, Insightful)
Anyone can say that they're an expert. The court system requires that if you're going to present evidence, you better have some credentials. This program, apparently, did not have the proper credentials.
Re:not unreliable, but... (Score:3, Informative)
and is therefore unreliable. i.e. You can't rely on it in a court of law. If you RTFA
attaching an assessment of Heusser's PC-CRASH simulation from Boyd Allin of MacInnis Engineering Associates, Inc., which is the distributor of PC-CRASH for North America. Similar to McHenry, Allin opined that Heusser's arbitrary 'inputs' made the results of the occupant modeling highly suspect. Allin also stated that the PC-CRASH program could not calculate the speed change of a vehicle when it strikes a p
Duh (Score:4, Insightful)
Every part on a car would need to be tested for strength, width, height, depth, shape, mass, the connections holding it to another part, and that bolt tested...You get the idea. You would also need the conditions that happened the second the crash occured. Road type, amount of friction, temperature, slope, etc. As a juror I would never trust a computer simulation.
Re:Duh (Score:5, Insightful)
Re:Duh (Score:4, Insightful)
Re:Duh (Score:2)
Then there's still the issue of knowing exactly where the ice patch is, or where the wet spot is, simulating the tires to the exact wear level. How much sand or salt was on the road, was it even or not?
In the autos, it requires knowing the angle of the s
Re:Duh (Score:5, Insightful)
In the real world, when being right is more important than having a merely plausible prediction in vibrant colors, people do experiments and laboratory tests.
Simulations by experienced analysts often turn out wrong: data from crash tests is much more trustworthy than the best simulations, let alone simulations performed by cops or prosecutors with some two-bit PC software.
Re:Duh (Score:3, Interesting)
You're still talking about the stages where it's more important to be plausible than correct. That's basically where FEA fits in.
I won't argue that it beats guessing, though. And the full-color plots seem to mesmerize people. Note too, though, that "PC-Crash" is
Re:Duh (Score:5, Interesting)
When Ford, or Daimler Benz goes out to design a car, they know where every bolt, nut, rivet, weld, and cup holder are. That information is fed into a finite element analysis model that breaks the car down into ever finer blocks of deformable material.
They than take that model and bash it against a series of controlled obstructions.
Even then, those simulations are just used to rule out certain design changes. All final designes are bolted to a hydrolic ram, filled with test dummies, and shot into a wall or another vehicle.
And again.
And again.
Yes, the automaker DOES have a model of the car. Yes, that model could be fed into another FEA. But in order to produce any meaningful result you would have to have equally good data about all the occupants in the car. Where everything on the road was, and at which time in the "event."
And did I mention that the simulation is only as good as the least accurate measurement? At best. And most of the data you would have needed is gone as soon as rescue crews arrive and attempt to move the vehicles out of the way of traffic?
Re:Duh (Score:2)
This is because, as we all know, metal weakens as it is heated. This is important because how a centain important part broke (bar reinforcing a sliding door, for instance) can make a big difference in how the other car bouces/hits/goes over the initial car.
Re:Duh (Score:2)
PC-Crash is not one of those pieces of software, however.
Re:Duh (Score:2)
I'm sure companies have the money and processing power for a COMPLETE crash testing of a car/plane but some prosecutor in Pu-Dunk Kansas will not. Good observation smitty45.
Re:Duh (Score:3, Informative)
I did about 4 years working for the Vehicle Crashworthiness Division for the US DOT using the above programs, and they are quite complex and used for mostly research, not accident reconstruction.
Re:Duh (Score:2)
See, you can learn/talk on Slashdot, it's not all trolls!
Re:Duh (Score:5, Informative)
Turned out the simulation that was generated that kinda matched what happened had the data entered at random until it matched "Heusser manipulated data by entering arbitrary 'inputs' such as separation speeds as high as 1,114.8 mph, placing the mailbox pole away from where it was actually located, and having the computer occupant models remain in a default resting position after the collision with the mailbox".
Indeed, the software was described at the end of the trial as "During closing argument to the jury, the prosecutor described PC-CRASH as a
computer program that essentially takes the laws of physics and reduces them to mathematical calculations that can be done over and over again to generate an accurate picture of what happened during a collision based on the tire marks at the scene, based on the physical evidence in the case such as the damage to the car, as well as the conditions that can be observed at the scene.
13 Report of Proceedings at 13. The prosecutor then showed the PC-CRASH
video to the jury, again.
Sipin was convicted as charged."
Whilst the expert opinion from someone who used his brain to see what happened described something completely different. The jury was mislead to how good the software was quite clearly, they were lead to believe that the software was infallible. It is only as good as the person entering the data, and when they choose to ignore data because it is inconvenient
So whilst the guy was stupid for buying a manual car when he had gout and couldn't drive it half the time, he does deserve a retrial.
Re:Duh (Score:2)
Re:Duh (Score:2)
just, PC-Crash doesn't.
Re:Duh (Score:2)
And you're correct that jurors should never trust a computer s
Re:Duh (Score:4, Insightful)
Re:Duh (Score:2)
Re:Duh (Score:4, Insightful)
This is like claiming that we can't calculate the acceleartion on an apple due to gravity, because the actual effect of gravity is dependent on the gravitational force of every atom, etc.
Perfect knowledge is not necessary to acquire a reasonably accuarate simulation or estimation. And the error bars on simulation can easily be small enough that they are irrelevant to the conclusion.
Now, we don't know the particular of this case, but nowhere near the information you seem to think is neccessary is actually relevent to a reasonably and usefully accurate simulation.
LetterRip
Re:Duh (Score:2)
Re:Duh (Score:2)
Re:Duh (Score:2)
hmmm....I think those parameters already exist. Like in the mfr's CAD/CAM system.
Road type, amount of friction, temperature, slope, etc
These are known quantities as well. The only real variable of those is temperature, and that can be found within a very few degrees.
Now...should these simulations be used for specific, down to the millimeter recrea
True (Score:2)
I hope this court decision doesn't have a dampening effect on automobile-accident-simulation research. It would be a shame to lose such a p
Re:Duh (Score:2)
Re:Duh (Score:2)
It reminds me of a news story I saw once. The reporter was standing in front of a car completely wrapped around a power pole in a suburban street.
"Police think that speed may have been a factor"
Re:Duh (Score:3, Funny)
It sticks, but it doesn't stick. It's a miracle!
Re:Duh (Score:2)
Car crash simulations are more than two boxes of different masses hitting each other. That is pretty much how the GTA engine simulates crashes. Sure it looks like a cool mysterious thing but it very simplistic compared to the real world.
PCCrash unreliable? (Score:5, Funny)
Re:PCCrash unreliable? (Score:2)
Re:PCCrash unreliable? (Score:2)
Doesn't matter, every computer crashes, because Every OS Sucks [deadtroll.com]
Non Newsworthy (Score:3, Insightful)
Digital evidence (Score:4, Interesting)
I'd be okay with photo radar and with red light cameras if they were used to bolster the Prosecution/Plaintiff, like if there were a car accident and the red light camera data were used to show that the cited person (by the officer on the scene) had indeed run the light, and that the officer was correct. The current system of using photography with near-automatic conviction deprives people of privacy. If the police want to cite people for speeding or for any other traffic violation then they need to get out there with people who will be required to testify as to what they saw; people who actively claim the count in the charge, not some computer or desk-jockey who analyses data after the fact.
Of course, I also have the opinion that if there's no victim then there's no crime. Take this as you will.
Re:Digital evidence (Score:2)
Re:Digital evidence (Score:4, Insightful)
Do you exceed the posted speed limit?
What defines it as a crime when it hasn't yet been through due process?
It's the job of the constabulary to enforce laws in person to protect the public, and to investigate real crime that has already occurred. Photo Radar for speed enforcement is a stupid idea, and just leads to people finding out where the cameras are that day, speeding everywhere except by the cameras. Traffic used to even require an officer to serve one with a court notice (the traffic citation), same as an officer picking up a wanted criminal to force a court appearance. Many cities don't even use the police departments to run photo radar, they contract it out to companies, who give the city a portion of the money collected. One such company is American Traffic Systems, who has operated in Scottsdale AZ and San Diego CA if memory serves.
By not receiving instant citation, the accused has no opportunity to place any importance on the memories that might help them form a defense. The prosecution/plaintiff is rarely forced to appear in court either, let alone testify to remembering the vehicle as it sped by, or any of that, like a real officer is required to do. A real officer is required to take an oath that he or she isn't committing perjury when they testify. A picture sent to the court isn't, and should be thrown out if the prosecution/plaintiff does not appear to press their side.
Photo Radar is treating people as guilty by default, without requiring individual explanation, or without an arraignment, pre-trial conference, and trial. It's a travesty to justice and a continual erosion of the rights of citizens by the government.
Re:Digital evidence (Score:3, Funny)
Society as a whole loses when people can speed with (almost) impunity. (Due to vastly increased amount of road deaths.)
Re:Digital evidence (Score:2)
Re:Digital evidence (Score:3, Informative)
Teaching to the test (Score:5, Insightful)
The biggest problem I face on the road are tailgaters and the guys who cut me off at interstate speeds and the morons who barrel out of parking lots at 20 mph without checking for traffic and the idiots who think "right turn on red" has right of way over people already on the road. Hell, even the superjock riding his bike far too fast for me to see him approaching as I cross the bike path... and he wrongly believes that he, not I, have right of way. (Pedestrians do, but in this state mounted bikes are "vehicles" and bike paths are "secondary roads.") As if it will matter when he hits my car (or vice versa), other than me suing his estate to repair my car's paint job.
People who run red lights or are speeding between lights on limited access roads? Not A Problem. Maybe once every few years I'll nearly get clobbered by some moron who goes through an intersection at high speed long after the light changed, but that's reckless driving, not merely running a red light. The latter should remain illegal, but a low enforcement priority unless it's an ongoing serious problem at a specific location.
So why do we see more and more red-light X systems? Because they're cheap revenue sources. To actually make driving safer you have to hire more cops and put them in more unmarked cars and get them out on the street where they can nail the guys who really are hazards to other drivers. Not guys going 45 in a 35 zone because that's what the heavy traffic is doing and it would be far more dangerous to obey the law than to break it. Or the guy who's behind a truck and doesn't know the light has turned red until he's already in the intersection.
How long until the laws themselves are written on the basis of what's easily enforceable, not on the basis of what harms others?
And the guy in Denver who put a photo-radar system on the interstate onramp where traffic is always at least 15 mph over the posted speed limit? The cop who lectured my HS class wants to talk to you - he assured tens of thousands of us that no cop would ever, under any circumstances, ticket us for going over the speed limit in order to merge with traffic. (We were supposed to gradually slow down once merged.) Ticket or being flattened by a semi? Hmm, which will it be? Ticket or being flattened by a semi. Gee, that's such a hard decision. Not.
Re:Teaching to the test (Score:2, Insightful)
Being in my 20's and not having any major diseases, driving to
Re:Teaching to the test (Score:3, Insightful)
Red light running is potentially lethal, moreso than tailgating in many situatio
I'll call bullshit (Score:3, Insightful)
That's why those laws are there, and that's why those cameras are there. I'd hardly discount that as "enforcing laws that are easily mechanized, not laws that are most critical to public safety." It _is_ critical to public safety, and if it can be easily mechanized, I for one am all fo
Re:Digital evidence (Score:2)
For example, look at San Francisco's 2004 statistics [highwayrobbery.net] and you'll see that out of 29,335 "raw" violations, only 7,943 citations were issued.
-B
Re:Digital evidence (Score:2)
Come look at the situation here in Arizona and you'll find that they're very much inclined to press anyone involved with the car about who was driving, trying to get them to tip them off, and it's very hard to defend against it when the system itself is slanted so heavily in the plaintiff
Re:Digital evidence (Score:2)
Don't you mean that it encourages people not to speed?
Of course, I also have the opinion that if there's no victim then there's no crime.
I would agree with that in most cases. This instance is an exception. When you violate traffic laws, you make it extremely easy for some random passerby to become a victim. The point of speeding laws is to deter people from speeding before somebody becomes a victim. I do
Re:Digital evidence (Score:3, Insightful)
I'll take it as an excuse to tell a story. :)
A couple of years back a guy ran over and killed a little girl very near my home. This young man was a speed enthusiast, drove a powerful BMW and had "Stockholm Getaway" (a video of crazy speeding through a city) in the glove box. When the accident happened he was speeding through a red light.
Those are the facts. Now, to make this more interesting, let
Re:Digital evidence (Score:2)
Ah, but first, there's intent to cause harm that's sta
Re:Digital evidence (Score:2)
Unrelieable (Score:2)
Verify this (Score:4, Funny)
Regardless of the computer software (Score:2, Insightful)
Re:Regardless of the computer software (Score:2)
Seatbelts will work for low-medium speed collissions but the there's the slight problem of F=MA.
You might get away with whiplash... you might end up dead with a broken neck. You also might end up crushed when the seat rips from its moorings and slams into the dashboard.
Seatbelts are a good thing but they're not a cure-all.
I could be right (Score:2)
Re:I could be right (Score:2)
I actually read the article (Score:5, Insightful)
I would like to see more of this kind of common sense in life today.
The story states both occupants were ejected from the car in the accident. The prosecution is quoted as saying their key element of the case was that part of the passenger door was melted on the dead guy.
So which was it? Did the dead guy stay there and take the burn, or get ejected? Did the car sit for awhile burning, and take off again?
I will make the specific conclusion from the vast amount of data in the article that there was enough doubt to go around in this case.
To often attorneys for both sides put up a George Lucas light show in order to sell their version to a jury. Matters are not helped by the fact that jury selection all to often resembles a Jerry Springer casting call.
I've seen the software in question used in a trial (once). What I saw seemed to be a believable representation of an elastic collision between vehicles. At no time were there any renderings, or mention of what happened INSIDE the vehicles. But then again, you know what they say about prepared demos...
Re:I actually read the article (Score:2, Insightful)
What I found, though, is that a standard trial is a piss-poor way of getting to the truth. It may be "fair", and certainly it is a lot more expedient than a real investigation, b
Re:I actually read the article (Score:3, Interesting)
As well it should be! And I say this as a prosecutor. But the reason for this should be obvious: the truth is very often unknowable. Thus, given that it is impossible for the jury to figure out truth, you narrow the scope a little, and only ask them to resolve certain factual disputes, i.e., who is/isn't lying, whether a story is plausible, what a reasonable person would do, etc. The effect of this is that juries a
Not validated != Unreliable (Score:5, Insightful)
=h=
Unexpected... (Score:4, Funny)
Actual courtroom events (Score:5, Funny)
Re:Actual courtroom events (Score:2)
Results 1 - 10 of about 46,200 for simpsons quote driver. (0.37 seconds)
Did you mean: simpsons quote server
What's so wrong with it? (Score:2, Interesting)
PC-Crash? (Score:2, Funny)
Shishberg Finds Article Title Unreliable (Score:5, Insightful)
At issue was the word "Unreliable", which implies some comment on the accuracy of the software in question. The article, however, consistently states that the software "had not been validated for the purpose for which the evidence was offered", a far more sensible claim.
"Titles of Slashdot posts have not been validated for the purpose for which this one was offered, simulation and prediction of the content of the article itself," a Slashdot representative stated. "There is no general acceptance in the relevant online community of the use of article titles as a substitute for R-ing TFA."
CowboyNeal was not available for comment.
So what? (Score:4, Informative)
The court didn't find that software simulation was categorically disallowed as evidence. It didn't even find that the PC-CRASH application was inadmissible in general. It just found that this particular software in modeling this particular event had not been shown to satisfy expert consensus.
Maybe PC-CRASH will in the future be shown reliable for this type of modeling. Maybe it will be shown to be inaccurate. Maybe the makers will enhance the software to demonstrably cover this type of event. None of these are anything terribly profound, and none have any great moral for the intersection of law and software.
Physics is not incredibly easy (Score:2)
Any simulation represents a subset of reality - - the quality of the simulation depends on how large this subset is.
When a car-crash simulator that accounts not only for the vectors, but also the exact conditions of the road, the exact nature of the cars, and friction (maybe air resistance!), I'll see it acceptable as the crux of an argume
People are too trusting (Score:3, Informative)
I remember ten years ago, gradebook software calculated grades incorrectly. It was used in thousands of schools.
I was a high school student at the time, doing a little part-time work for a company writing a competitor. We actually discovered this after hand testing our calculations by hand. Our calculations were right. But what other tests could we do? We figured we'd try their numbers and see if we got the same results. We didn't. Surely it was our problem, since we weren't on the market yet? No, it was their problem.
I had a 18% grade swing as a result of pointing this out to my chemistry teacher. She apologized over and over again. We also reporterd it anonymously to the company, which fixed it in their next version. But I discovered last year that as of two years ago my high school was still using the same version of the same flawed program, and it was still generating incorrect grades.
A younger friend of mine pointed this out to the teachers. The response was the same as ten years ago: "Of course it's right, the computer did it."
This is absolutely astonishing. It means that final grades produced by thousands of schools are not according to the criteria specified by the teachers and/or school and/or school district. If they are right, it is only the happiest of coincidences.
Re:People are too trusting (Score:2)
Nothing in the school system astonishes me. A while back I was staying with friends who are both teachers. They showed me the handout on evaluating reading problems that a visiting "expert" on reading instruction had given out. It was five or six pages long. It said nothing about the different kinds of reading problems and how you detect them. Basically, all it did was explain how to score the exam. That is, it explained, very slowly and carefully, that you subtract the number of incorrect answers from the
It wasn't the software's fault! (Score:2)
A few facts (Score:5, Informative)
This may be shocking, but I am actually familiar with both this software and the process of giving expert testimony. PC-Crash is one of several *Crash* programs provided by different vendors that share a common lineage. It and its sister programs are used extensively in accident reconstruction and the results are presented to juries every day. The core of these *Crash* programs are a series of well-established (although certainly not perfect) algorithms and physical properties related to vehicle dynamics. The problem here was the extension to occupant dynamics, not the use of simulation programs in general.
You may now return to your regular uniformed ranting.
As the author of "Falling Bodies"... (Score:5, Informative)
You cannot, even in theory, predict how a human with arms and legs banging around will move in a complex crash. It's chaotic, in the formal mathematical sense of the world. That is, an arbitrarily small change in the initial conditions can create a large change in the outcome. In Falling Bodies, if you change the low order digit of a double precision number in the initial conditions for a fall down a staircase, the simulations will start to diverge after about a second, and the fall may end quite differently.
I had this discussion a few years ago with an Army officer who was trying to reduce accidents in parachute landings, and was considering using Falling Bodies. I talked him out of it.
Auto collisions can be simulated well because there's one big mass that dominates the simulation. So you get a deterministic result within some error limits. Multibody systems with joints and links are quite different.
Realistically, you can probably do a sound simulation which predicts how a passenger will bounce around from the beginning of the collision to the first passenger interior collision with the vehicle. Beyond that point, forget it.
Non-determinstic results. (Score:3, Insightful)
Of course, you would still need to validate the results of the monte carlo simulation on (lots of) observed data, both to see if you got all the relevant factors in the determinstic model
Re:don't they realize (Score:2)
Re:don't they realize (Score:2)
PC-Crash, FWIW, is not a *real* vehicle crash simulation package, IMHO. validated and comphrehensive results can be obtained with simulations, and do, every day.
the results of those validated simulations go into the NHTSA rulemaking body, which gets all of the compliance rules together so auto manufacturers aren't allowed to sell cars with tinfoil frames.
Re:another possible idea (Score:2)
I have to admit there is something to that interpretation, though. Every time I think I've got it figured out, I realize I don't. The only sentence that makes sense, the second one, also has nothing to do with the article, so it is no help. Upon reflection, I couldn't make a masterpiece of Rorschach posti
Re:another possible idea (Score:3, Informative)
Well, perhaps you think so. But it would be just as wrong. An electron isn't made of leptons anymore than a swede is made up by europeans.
Re:another possible idea (Score:3, Funny)
err... Ahem. Strike 2.
here ya go. [particleadventure.org]