Court Orders Breathalyzer Code Opened, Reveals Mess 707
Death Metal writes with an excerpt from the website of defense attorney Evan Levow: "After two years of attempting to get the computer based source code for the Alcotest 7110 MKIII-C, defense counsel in State v. Chun were successful in obtaining the code, and had it analyzed by Base One Technologies, Inc. By making itself a party to the litigation after the oral arguments in April, Draeger subjected itself to the Supreme Court's directive that Draeger ultimately provide the source code to the defendants' software analysis house, Base One. ... Draeger reviewed the code, as well, through its software house, SysTest Labs, which agreed with Base One, that the patchwork code that makes up the 7110 is not written well, nor is it written to any defined coding standard. SysTest said, 'The Alcotest NJ3.11 source code appears to have evolved over numerous transitions and versioning, which is responsible for cyclomatic complexity.'" Bruce Schneier comments on the same report and neatly summarizes the take-away lesson: "'You can't look at our code because we don't want you to' simply isn't good enough."
Re:But does it work? (Score:5, Informative)
No. (Score:5, Informative)
Just read Schneier's comments. He cites some of the more important things:
Readings are Not Averaged Correctly: When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed... There is no comment or note detailing a reason for this calculation, which would cause the first reading to have more weight than successive readings.
That alone should be enough -- the readings are not averaged correctly. But it goes on:
The A/D converters measuring the IR readings and the fuel cell readings can produce values between 0 and 4095. However, the software divides the final average(s) by 256, meaning the final result can only have 16 values to represent the five-volt range (or less), or, represent the range of alcohol readings possible. This is a loss of precision in the data; of a possible twelve bits of information, only four bits are used. Further, because of an attribute in the IR calculations, the result value is further divided in half. This means that only 8 values are possible for the IR detection, and this is compared against the 16 values of the fuel cell.
So we know it's buggy and inaccurate, to a moronic degree. If that wasn't enough:
Catastrophic Error Detection Is Disabled: An interrupt that detects that the microprocessor is trying to execute an illegal instruction is disabled, meaning that the Alcotest software could appear to run correctly while executing wild branches or invalid code for a period of time. Other interrupts ignored are the Computer Operating Property (a watchdog timer), and the Software Interrupt.
So, basically, it's designed to always return some value, even if it's wildly inaccurate, and even if the software is executing garbage at the time.
In other words: It appears to be a very low-level equivalent of Visual Basic's "on error resume next".
Whiskey. Tango. Foxtrot.
So to answer your question: No, it does not work. Even if it did somehow work, there's obviously an unacceptably poor level of quality control here.
Re:Lint is crap (Score:2, Informative)
Re:not written to a coding standard? (Score:1, Informative)
If you read the points beyond the first one, they indicate some pretty serious problems: All readings are reported relative to a baseline reading taken when the device first turns on, but there are no sanity checks in place for this reading, nor are there attempts to establish an initial average. The code that's supposed to average the user readings doesn't average, but instead heavily weights the first readings. Error handling is available but disabled; there is no way to determine a malfunctioning device from a functioning one. Known bad values are simply clamped to a valid range.
Granted, the thousands of "errors" mentioned in the article are probably mostly lint warnings... While worrying, they are not actual coding errors.
But it stands that this device could report an alcohol level that is higher or lower than detected in various circumstances. And these cases could have been detected or avoided if the software had been written better.
Looks like the NJ Supreme Court doesn't care (Score:5, Informative)
Re:But does it work? (Score:5, Informative)
>> assigning more weight to the first reading than the subsequent ones.
It seems to apply more weight to later readings:
where a1=1, b1=2, c1=3, d1=4
(A1+B1+C1+D1)/4 = 2.5 (the correct average)
and
(((((A1+B1)/2)+C1)/2)+D1)/2 = 3.125
Re:No. (Score:3, Informative)
Sure, one bit would be enough to make a pass/fail decision. But they throw away info BEFORE making that determination. You can make a determination and round it down to one bit, but you can't round down to one bit and then make an accurate determination, this is an analog sensing device we are talking about. Throw away everything but one bit, and you don't have a yes/no on the legal limit, you have 'above 2.5v, or below 2.5v.' What's the legal limit, translated into volts, hmmm?
Re:But does it work? (Score:5, Informative)
correction, you may refuse a roadside breath test, but not one at the police station.
Re:But does it work? (Score:5, Informative)
"Just because they've never observed it to fail, doesn't mean a thing."
Correct! This is a point that many people fail to understand. Testing can't prove that there aren't bugs. All it proves is that a bug did not occur. Failing a test just proves that a bug exists while passing all test just proves that you failed to find a bug. Passing many tests can boost your confidence that there are no bugs. Verification can prove that your code is correct but for most programs it is unfeasible.
Re:But does it work? (Score:5, Informative)
In all 50 states, refusal to take a breathalyzer at the police station will result in a 1 year(minimum) suspension of your drivers license.
In all 50 states, you can refuse to take a roadside breathalyzer as they're inadmissable in a court of law. If you have had even 1 drink, always ask to go to the station for a real breathalyzer. The PBT's, or portable breath testers, are wildly inaccurate and only give the police probable cause to arrest. It will not work in your favor to take it.
Re:But does it work? (Score:3, Informative)
Several states such as Washington state will let you get a blood test, but the breathalyzer at the station is still legally required and you have to pay for the blood test out of your own pocket.
Re:Bruce Schneier needs a math refresher (Score:1, Informative)
Yeah the math's not right but it's not Schneier's fault. He was quoting the summary.
Re:But does it work? (Score:4, Informative)
It measures a concentration (per unit volume), not how many beers you drank.
It measures a concentration per unit volume of breath. There are many assumptions in extrapolating this to concentration per unit volume of blood.
Re:But does it work? (Score:3, Informative)
Very true. To some extent, it's reasonable to truncate a few bits of precision if the noise floor of the BAC sensor is substantially higher than the dynamic range of a 12-bit ADC. No reason to display a bunch of meaningless flickering digits extending far to the right of the decimal point.
But when you're displaying a decimal value, every place value with full 0-9 range takes about 3.3 bits of precision. If you're going to return numbers like "0.18" from a device with a range of 0.00 to 0.99, you need to keep at least 7 bits intact, or you're making stuff up. If their BAC sensor's uncertainty is insufficient to provide consistent, monotonic 7-bit values, or if they're deliberately throwing away all but four bits, then they're making stuff up.
So, yeah. Conclusion: they're making stuff up.
Re:But does it work? (Score:5, Informative)
If you have a noisy sensor and are trying to keep a low-noise estimate of the input, while that input is changing, you do some sort of filtering on the data. The weighted rolling average described above is nice for a number of reasons, mainly it's simple to implement and simple to analyze. In some cases, other filters are better.
If you have a noisy sensor and want to measure a single unchanging input, you would want a different sort of filter. In this case, the simple arithmetic average works quite well.
As you correctly observe, the two filters of similar complexity. Which one you use should depend on the sort of input you're trying to measure. In this case, they used the former type of filter on the latter type of data, which is a definite no-no. This will result in data that is far noisier than you would otherwise expect from the raw sensor noise and the number of samples taken. When that noise could be the difference between a DUI conviction and and the cop telling you to drive home carefully, I'd say that's worth worrying about.
Re:But does it work? (Score:3, Informative)
A moving average is useful if you don't have a large enough data type to store the sum of all the values.
Re:But does it work? (Score:4, Informative)
In Britain, the breathalysers decide whether or not you get taken to the police station.
When you arrive at the police station, they take a blood sample. They test half of it, and that decides whether or not you get convicted. You get the other half and can arrange for your own tests on it.
Re:But does it work? (Score:3, Informative)
This seems to make sense to me. The breathalizer is supposed to measure the blood alcohol content, and this is done by measuring the alcohol content in air expelled by the *lungs* (with a knowlege of partial pressures).
Actually, it measures presence/absence of ketones in the air. Also, they have you blow "continuously" for several seconds (the mouth doesn't hold that much). Finally, .08 is considerably drunk, but it isn't you're-falling-over-how-could-you-possibly-think-you-could-drive drunk.
But is it broken? HELL YES!!! (Score:5, Informative)
OK, LOTS of strange posts from people who claim to have read the article but only see that it's bad code, not actually broken.
Read it again. It's broken from a legal liability and trustworthiness standpoint. It's broken from a precision standpoint. It's broken from an algorithm standpoint. It is not trusworthy, precise, accurate, or correct.
"It is clear that, as submitted, the Alcotest software would not pass development standards and testing for the U.S. Government or Military. It would fail software standards for the Federal Aviation Administration (FAA) and Federal Drug Administration (FDA), as well as commercial standards used in devices for public safety. This means the Alcotest would not be considered for military applications such as analyzing breath alcohol for fighter pilots. If the FAA imposed mandatory alcohol testing for all commercial pilots, the Alcotest would be rejected based upon the FAA safety and software standards."
Nobody in the government or military would be allowed to trust this, if it weren't already in use.
"Results Limited to Small, Discrete Values"
Sixteen values is all it displays! It throws away almost all of the precision of the 12-bit ADC, and reduces it to 4 bits! This is NOT precise enough!
"Catastrophic Error Detection Is Disabled"
"Diagnostics Adjust/Substitute Data Readings"
"Range Limits Are Substituted for Incorrect Average Measurements"
"The software design detects measurement errors, but ignores these errors unless they occur a consecutive total number of times."
It's not correct. It's not accurate. It's not good enough. The odds are VERY good that some people over the limit have gotten off lucky, and also that some people below the limit now have criminal records.
Re:But does it work? (Score:1, Informative)
He blew over - he was guilty! Why would he even expect to be let off?
Because the initial traffic stop was illegal. The police are not entitled to pull over anyone they like whenever they want - the police need probable cause.
Under the "poison tree" doctrine http://en.wikipedia.org/wiki/Fruit_of_the_poisonous_tree [wikipedia.org] evidence gathered by the police behaving illegally is thrown out of court.
Re:But does it work? (Score:3, Informative)
Re:No. (Score:2, Informative)
Cosmic rays can cause bit flips, but in my experience it is more likely to happen to electrostatic discharge or other electromagnetic interference of terrestrial origin. The odds of cosmic rays hitting your device is partially dependent on altitude.
There was a study done by IBM that indicated that a semiconductor based device could expect one such event every year. Other studies have shown that as the number of transistors in a device goes up, the chances increase. Just because an event occurs does not mean it will be visible. It could happen in unused memory, not affect a calculation significantly, etc.
I dug up an article written by some guys at Cyprus Semiconductor(complete article at http://www.edn.com/article/CA454636.html [edn.com]);
The interesting bit is this: "The potential impact on typical memory applications illustrates the importance of considering soft errors. A cell phone with one 4-Mbit, low-power memory with an SER of 1000 FITs per megabit will likely have a soft error every 28 years. A high-end router with 10 Gbits of SRAM and an SER of 600 FITs per megabit can experience an error every 170 hours. For a router farm that uses 100 Gbits of memory, a potential networking error interrupting its proper operation could occur every 17 hours. Finally, consider a person on an airplane over the Atlantic at 35,000 ft working on a laptop with 256 Mbytes (2 Gbits) of memory. At this altitude, the SER of 600 FITs per megabit becomes 100,000 FITs per megabit, resulting in a potential error every five hours. The FIT rate of soft errors is more than 10 times the typical FIT rate for a hard reliability failure. Soft errors are not the same concern for cell phones as they can be for systems using a large amount of memory."
Re:But does it work? (Score:3, Informative)
So, let's assume that individual readings are the true value plus some zero-mean error (it's supposed to be calibrated, after all). Using the exponential average that was in the actual device's code, your error will never be better than 1/2 of an individual reading (the last one). To beat this with a array to implement a sliding-window average, you'd need only 5 entries to get a more accurate reading than their approach (error is proportional to sqrt(N) for an arithmetic mean). That's 10 bytes. How cheap of a processor do you expect them to be using, such that 10 bytes is unacceptable? With 16 bytes, you get twice the accuracy, as well as the ability to take the average by a bit shift (likely not worth bothering with since you would convert it to base-10 for display anyway).
Re:But does it work? (Score:1, Informative)
Re:But does it work? (Score:3, Informative)
That's the same. You just started n at a different value... :-)