Slashdot Log In
US District Court Says Calculating a Hash Value = Search
Posted by
timothy
on Tue Oct 28, 2008 10:02 AM
from the fair-enough dept.
from the fair-enough dept.
bfwebster writes "Orin Kerr over at The Volokh Conspiracy (a great legal blog, BTW) reports on a US District Court ruling issued just last week which finds that doing hash calculations on a hard drive is a form of search and thus subject to 4th Amendment limitations. In this particular case, the US District Court suppressed evidence of child pornography on a hard drive because proper warrants were not obtained before imaging the hard drive and calculating MD5 hash values for the individual files on the drive, some of which ended up matching known MD5 hash values for known child pornography image and video files. More details at Kerr's posting." Update: 10/28 16:23 GMT by T : Headline updated to reflect that this is a Federal District Court located in Pennsylvania, rather than a court of the Commonwealth itself.
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
It's good to see. (Score:5, Informative)
The courts are finally getting up to speed on technology.
Re:It's good to see. (Score:5, Funny)
Parent
Re:It's good to see. (Score:5, Informative)
It's not that child pornographers shouldn't be prosecuted, but like it or not, they're still entitled to the same due process as normal, "non-pervert" criminals. This "it's for the children" stuff shouldn't fly when we claim to follow the rule of law.
Parent
Re:It's good to see. (Score:5, Insightful)
It's not that child pornographers shouldn't be prosecuted, but like it or not, they're still entitled to the same due process as normal, "non-pervert" criminals. This "it's for the children" stuff shouldn't fly when we claim to follow the rule of law.
And anything we can do to deflate the "think of the children" hysteria will help protect our society. It's not that protecting children is a bad thing, it's that turning people into frothing flesh-rending mobs at the drop of a hat is a bad thing. If I were a nasty sort of black-hatted individual, the quickest way I can think of for destroying an enemy would be planting kiddie porn on his computer and dropping a dime to the authorities. Kiddie porn will be the new "baggie of drugs to plant on a perp." I wouldn't be surprised to see cops dropping usb drives on accidentally shot guys. "No, don't worry, I just planted kiddie porn on the guy. Disciplinary action? We'll probably get a medal for this."
Incidentally, your tagline: "Ein Volk, Ein Reich, Ein Messiah." Is that an inept slam against Obama?
Parent
Re:It's good to see. (Score:5, Insightful)
Speaking of frothing.... This wasn't an "active choice" to free a child molester it was a judge using common sense and realizing that this was a search without a proper warrant and throwing it out just as he would\should if an officer kicked your door down without a proper warrant.
Troll indeed!
Parent
Re:It's good to see. (Score:5, Informative)
also, wouldn't this type of search be pretty useless for identifying kiddy porn images?
md5 hashes are useful for verifying a binary package is in fact what it is supposed to be because it's hard to create a fake or altered program that produces the same md5 hash number as the authentic copy. so it's useful for verifying a "good" file, because presumably a good file won't try to deceive you, and a bad file can't reproduce the same md5 hash.
however, with something like a digital photo, all a user has to do is make a few very minor alterations (like a small watermark) to the image and it would produce a different md5 hash--essentially exploiting the inherent design of the md5 hash algorithm--and be missed by the md5 scan. these small changes could be as simple as flipping a single bit in the file, but with a standard 24-bit RGB bitmap image, each pixel is stored as three 8 bit values representing the red, green, and blue color channels. by flipping the least significant bit in each channel, you can alter up to 1/8th (12.5%) of the file without creating any perceptible changes (to human eyes at least) to the displayed image.
another method would be to employ lossy compression schemes like JPEG image compression. convert all your images to JPEG (or if they are already JPG, just compress it again at minimal compression strength) and the MD5 hashes will be completely altered. yet another method is to resize the image by a small amount--say reduce both width and height by just 1 pixel--using bicubic interpolation to scale the image up or down would preserve the image quality while completely changing the md5 signature of the file.
all of these methods would be simple to automate and allow you to easily hide known child porn images from detection using md5 comparisons.
Parent
Re:It's good to see. (Score:5, Interesting)
Actually, it looks like a pretty good search technique. It's fast, easy to automate, probably a low percentage of false positives, and can be used to link perps together through shared files. As you note, it would be easy for the pervs to block, by dropping a few bits, but I suspect it would be effective for a while.
It's still a search, with all that goes along with that. But it's probably better than having Officer O'Reilly deciding that your picture of your daughter playing at the beach sans diaper is porn.
Parent
Re:It's good to see. (Score:5, Insightful)
Actually, it looks like a pretty good search technique.
No, it's a pretty shitty one. Here's why:
Parent
Re:It's good to see. (Score:5, Insightful)
It's preposterous to assume that any lawyer defending a client accused of possessing child porn would throw up his hands in the face of the authorities only identification being based on a hash. Any non-retarded person would ask the next logical question: did you actually look at the image/video and verify?
This method is a convenience search. The authorities still have to go through all the other steps to identify and verify child porn. If anything this search method is more likely to make authorities lax and catch less people with child porn.
Parent
Re:It's good to see. (Score:5, Interesting)
Yes, easy, but many of the porn collectors aren't going to be bothered with actually doing the edit...
So, go out and make a program that will automatically change a few bits in each file in a directory. Make it a TSR, and watch for all files in a directory. Sell it, profit.
Then the fbi will be after your list of customers (child porn collectors) because it's more complete than theirs.
Shit, the FBI should write this program and sell it from a fake company.
Parent
Re:It's good to see. (Score:5, Funny)
Make it a TSR, and watch for all files in a directory. Sell it, profit.
TSR? What?! Are you still using DOS as your main OS in 2008?
Today we call stuff that run in the background while you do other stuff "Programs", "Services" or "Daemons". Get with the times, man.
Parent
Re:It's good to see. (Score:5, Funny)
TSR? What?! Are you still using DOS as your main OS in 2008?
Today we call stuff that run in the background while you do other stuff "Programs", "Services" or "Daemons". Get with the times, man.
Note the user ID of 1263. I believe you're on his lawn.
Parent
Re:It's good to see. (Score:5, Insightful)
Those who deal in child pornography and prey on children are, to my mind, some of the worst exxamples of humanity out there. I wouldn't bat an eye if they increased the prison sentences for them to life or allowed capital punishment. But it still has to remain within the bounds of our laws, the core of which is the Constitution.
Parent
Re:It's good to see. (Score:5, Insightful)
Those who deal in child pornography and prey on children are, to my mind, some of the worst exxamples of humanity out there. I wouldn't bat an eye if they increased the prison sentences for them to life or allowed capital punishment. But it still has to remain within the bounds of our laws, the core of which is the Constitution.
Granted. Those who take advantage of, say, 5-year-old kids should be flayed and burned where they stand.
It's the grey areas that concern me, though. The difference between a naked 17-yo and a naked 18-yo is 15 years in jail vs. perfectly legal. If you have a picture of a kid a day before his 18th birthday and a day after, what's the huge difference that makes you a heinous pervert vs. just another horney guy?
Parent
Re:It's good to see. (Score:5, Interesting)
You have to set the bar somewhere, and then stick to it. Sure, you can be more lenient on edge cases, but you still need to say "the limit is X", or the whole legal system is a farce made out of "fuzzy rules we're kind of supposed to follow".
In particular, when we get to the 17-yo case, it's as simple as this: did you think, in good faith, that she was of age? If yes, you should be home free. We're talking reasonable doubt here. It's reasonable to think a 17-yo is 18 or 19. If it was publicized as kiddie porn in any way, I don't care if she's 15 or day shy of 18. You had the information available, you're screwed.
Parent
Re:It's good to see. (Score:5, Insightful)
First of all, penal law is immoral, only the victims should have a claim against their aggressor. The victim should present the damage in front of a judge, establish the lack of consent, and the verdict set accordingly.
Be careful with absolutes like that. You just legalized murder, beating somebody to the point of brain damage, racketeering where victims are too afraid for their own safety or that of their loved ones to take people to court, etc.
Perhaps you were only referring to the kiddie porn issue and suggesting that 5 year olds should file charges against their exploiters (often their parents), but even then it's a stance that's kind of hard to understand.
Parent
Re:It's good to see. (Score:5, Insightful)
And the hell of it is, the 22 year old that accidentally slept with a 17 year old; well, he's still going to be forbidden to see his 12 year old sister until she grows up. He's still going to have to find some place to live that's not within ten miles of schools.
These sorts of liminal states are just going to come up more and more, and to be bigger and bigger problems, partly due to the utterly awful sexualization of girls' clothing. At this point, the difference between a 15 year old's clothing and an 18 year old's is likely that the 15 year old's clothing is skimpier and sluttier.
We need to do two things; we need to make some judgments that are currently just binary, i.e. either you're sex offender registry or you're not, into more gradated judgments. And we need to work to reverse the societal trends that are driving people to consider banging people at the edge of consent (and beyond) optimal.
Parent
Re:It's good to see. (Score:5, Insightful)
Well, to my mind, they are still fellow human beings and fellow citizens who deserve every moral and legal right as to the rest of us.
I would shed a tear for each such measure as yet another branch was torn from the tree of liberty. I would mourn the needless waste of human life.
The law, and even constitutions, are ultimately subject to the will of the people. People like you and others in this thread who would rather join a rabid mob than go against one and stand up for what is right. If you're too afraid of unpopularity, or condemnation, or guilt by association, to defend the rights of others, then you don't deserve a single one of those right yourself.
You, and every poster in this thread panders to hysteria by sycophantically declaring your own inflated revulsion at these crimes. Every time you do so, you further strengthen the forces that are eating away at the foundations of law and freedom in the western world. No reasonable person need declare their revulsion for these crimes. Yet everyone insists on doing so, loudly and explicitly at the earliest opportunity.
Because they are afraid.
"Find out just what a people will submit to, and you have found out the exact amount of injustice and wrong which will be imposed upon them" - Frederick Douglass [wikiquote.org]. The west has submitted to the howls, intimidation and demagoguery of the Outrage Brigade. We will suffer whatever injustice or wrong they now choose to impose upon us, and it seems, will do so indefinitely. Please read the rest of the Douglass quote, and think next time before you obediently proclaim your moral standing.
Parent
Re:It's good to see. (Score:5, Insightful)
And now we see exactly what your protestations of outrage are really all about. You would force the rest of us to stand to attention behind you or risk having the vilest of accusations thrown directly against us. You are a pitbull of social reactionaries who will use any weapon, no matter how odious, to chip away at the foundations of our free society and who will without conscience pass within a hair's breath of libel so as to cut most deeply without risk to yourself.
You, and people like you, are destroying the western world, one pointing finger at a time.
Parent
Re:It's good to see. (Score:5, Insightful)
You misunderstand the parent post. He's not saying, "it's only children, who cares," he's saying, "whether or not it's children has nothing to do with whether a suspect's constitutional rights should be violated."
The thing is that you don't have perfect knowledge of whether the suspect is a child pornographer or not. Lacking perfect knowledge, you should seek it out by following the appropriate channels.
If you are sure that someone is involved in any crime (whether or not it involves children), you should be sure enough that you can convince a judge to issue a search warrant. If you don't have enough evidence to convince a judge to set aside this person's rights, then you shouldn't just go ahead and set aside those rights even if you're really, really sure.
That's due process. That's how we protect the rights of innocent citizens from being abused by the power granted to police and other government agents. It completely doesn't matter what the nature of the crime you're investigating is. I'll say that again. It is wholly immaterial what crime you suspect someone has participated in; if you don't have enough evidence to convince a judge to issue a search warrant, you should not take the law into your own hands anyway.
The only time you might convince me otherwise is if there was an imminent threat - such as in the case of kidnappings or (since you're talking about child porn), a live feed of a child being abused, and the only as far as is necessary to secure the immediate safety of that child. This again has nothing to do with it being children though - this is just as true in my mind for securing the immediate safety of adults.
Parent
Re:It's good to see. (Score:5, Insightful)
They didn't know that when they violated his constitutional rights. I'm not talking about whether or not he was guilty... I'm talking about whether or not they had a right to look at all - and without a warrant, they didn't.
The 4th amendment doesn't get suspended just because you incant the word, "children."
Parent
Re:It's good to see. (Score:5, Insightful)
You're speaking in the past-perfect tense. You're speaking only with perfect knowledge of what transpired.
The problem is that is not how decisions are made. Decisions are made with imperfect future knowledge. When they decided to search this guy's computer, the did not know if they would find evidence of child pornography. Whether or not they found anything, once again, is completely irrelevant to whether they should have looked at all, because you cannot know before hand if you will find anything; you can only suspect you will.
I agree, there is a balancing act, and we should balance the rights of victims with the rights of criminals, but also with the rights of non-criminals. Fortunately exactly how we balance that is very clearly defined for us by the legal system. When you suspect someone has committed a crime, and you need to violate their 4th amendment rights to prove it, we have this excellent system already set up to facilitate it. It's called the warrant system, and its whole purpose is to balance the rights of victims with the rights of citizens which we do not yet know to be a criminal or not.
You're completely ignoring an entire class of citizen. There's victims, criminals, but most significantly there are people who are neither. THAT is the purpose of the 4th amendment.
I'm not saying, "4th amendment, therefore you can never search," I'm saying, "4th amendment, therefore you need to follow the procedures we have in place which provide checks and balances to protect innocent citizens from abuse by people in authority."
If this guy wasn't a criminal, he still would have had people searching his stuff. Or maybe you don't believe in privacy for innocent citizens at all. If that's the case, then you and the 4th amendment are incompatible, and you should return to Tudor England and stop taking advantage of the freedoms the blood of patriots have purchased for you.
Parent
Re:It's good to see. (Score:5, Insightful)
>>>the guy had the pictures on his computers. guilty... it's pretty cut and dry.
I agree. In fact I think the police should continue their search for child porn, and start searching all 110 million homes in America, confiscating PCs without search warrant, and comparing hash values on the drives.
We'll start with your house first.
What's that? You don't want the inconvenience of warrantless searches and losing your PC for a month while its scanned? WELL NEITHER DOES ANYONE ELSE YOU DURNED FOOL! That's why the requirement for a judge-issued warrant exists; to stop the government from going house-to-house-to-house harassing citizens!
DUH.
The government is the People's Servant, not the other way round.
Parent
Re:It's good to see. (Score:5, Insightful)
I could hack into your computer and plant pictures. Then you would have the pictures on your computer. Are you then guilty? Is that situation cut and dry?
It seems to me that most people, on this site and elsewhere, don't really believe in evidence, due process, or innocent until proven guilty. They think that suspects are guilty, period. The rest of the stuff is just a formality meant to please the judges. According to this attitude, if the crime is heinous enough and the publicly-available evidence damning enough, the trial becomes redundant and pointless.
This attitude, quite frankly, scares the everliving shit out of me. Everyone deserves a fair trial, and that means properly obtained evidence. You can't simply throw this out because you think a particular crime is really extra special bad.
Parent
Re:It's good to see. (Score:5, Insightful)
And nowhere in the post you quoted was the inference that you applied to it, you're one of the "frothing flesh rending mob" if you believe what you state about the post in question.
No one, not even the leftiest lefty on the left of a leftie is arguing that crimes against children are not abhorrent (maybe my grammar is though - double negatives aside).
The issue here is "do the ends justify the means?" While you may agree that anything should be permitted to catch and convict child molesters and kiddie porn collectors, you have to watch the slippery slope.
If a law enforcement agency can scan your drive and compare MD5 sums without a warrant, you have removed due process from the equation - one of the things that you are entitled to in the US justice system, regardless of your suspected crime, because like it or not, you are innocent until proven guilty.
This whole bollocks of "if you have nothing to hide, you won't mind" is bullshit. If they come to scan your drive with no proof to justify a warrant then they might as well just say that everyone's drives need to be scanned when the law asks, and if they find anything that flags you, you then have the burden of proof on yourself to assert your innocence.
It just doesn't (or shouldn't) work that way.
Do I want child molesters arrested and put away? Absolutely. Do I want them to be arrested through an illegal search of their property? Absolutely not.
It's a hot button issue, much the same as terrorism - we're in danger of severely crippling our society if we stoop to "prove you're not a terrorist/child molester/communist or we'll lock you up!"
Parent
Re:It's good to see. (Score:5, Insightful)
These children will have to one day live with the mess that we have created for them in their name.
Parent
Re:It's good to see. (Score:5, Insightful)
They are the hopes and dreams of the parents who raised them, the future of our society, innocent and worthy of our very best efforts to protect them.
Everyone agrees children need to be protected. But that's not the least bit topical given the context. Just the same, no child in inherently innocent; and that is not a sexual reference. That's a flawed Western-Christian philosophy. I've known far too many children that were far from innocent and far too many parents dismiss their actions simply because they are "innocent children." That child then grows up to be a monster of an adult.
So please stop with the "innocent child" bullshit. Ignorance is not heavenly innocence. A child is well behaved and "good", very poorly behaved and "bad", or fits somewhere in between. Many children have at least some understanding of their actions at very early ages and that doesn't suddenly change at age 18. Even if a child doesn't fully understand the ramifications of their actions (example, pull trigger = death), many do understand it is not something they should be doing - assuming the parents were doing their job in the first place.
Parent
Re:It's good to see. (Score:5, Insightful)
You know, you're right. And I think *you* are a child molester. So much so that I'll report you to the police. Under the new Think of the Children Act, the police I tipped off will be at your door to kick it in, drag you out of your house, and shoot you dead at the side of the road. What, you don't like this idea? Then you support child molesters!
You see how it works? Due process is needed for everyone, no matter how vile.
Parent
Re:It's good to see. (Score:5, Insightful)
>>>>>The man was clearly guilty
A lot of you are missing the point, so let me put it in bold:
Without the requirement for search warrants (obtained from an impartial judge), the police, FBI, or other government officials/politicians can go from house-to-house-to-house taking PCs simply because they feel like it. Do YOU want to be a victim of these random, harassing, and very inconvenient confiscations. I certainly Do Not! The Constitution was written because that's precisely what was happened in the 1760 and 1770s, and the American people were stick and tired of the bullshit.
"[Our government] has erected a multitude of new offices by a self-assumed power, & sent hither swarms of officers to harrass our people" - Declaration of Independence, 1776
So they setup a Supreme Law of the Land that would prevent this from ever happening again.
Parent
Re:It's good to see. (Score:5, Funny)
Without the requirement for search warrants (obtained from an impartial judge), the police, FBI, or other government officials/politicians can go from house-to-house-to-house taking PCs simply because they feel like it. Do YOU want to be a victim of these random, harassing, and very inconvenient confiscations. I certainly Do Not! The Constitution was written because that's precisely what was happened in the 1760 and 1770s, and the American people were stick and tired of the bullshit.
Who cared if the Brits were confiscating PCs, with no electricity to run them anyhow?
Parent
Re:That's a terrible argument (Score:5, Insightful)
Or maybe get a proper warrant and follow procedures properly? Sorry, I am no fan of kiddie abusers but if we bent the rules the way you'd like them for this instance then what comes next? I break down your door as an officer, find nothing, and suffer a fine for having made a mistake? Sorry, the officers must follow rules same as you and I or they will become simple bullies. Oh wait....
Better a few guilty men go free on a technicality than allow officers to become a law unto themselves.
Parent
Re:That's a terrible argument (Score:5, Insightful)
Quite honestly, the judicial tradition of suppressing evidence entirely because it was produced without a proper warrant is absurd.
So you're saying you have no problem with warrentless searches? Shall we continue this thought to it's logical extreme conclusion?
There's a reason the judicial system has the structure it does: so there's a strong trail of evidence, to ensure the rights of everyone involved have not been broken by law enforcement, to ensure nothing has been tampered with.
The law HAS to follow the law, otherwise what authority does it really have to enforce it?
Parent
Re:That's a terrible argument (Score:5, Insightful)
I apologize for interrupting the false dilemma here, but would it be a reasonable option to prosecute both the criminal who was caught and the cop who violated the Constitution to catch him? I know, I know, we've got two guilty people on our hands, and our natural, rational instinct is "let them both go unpunished, then set fire to our own hair"... but perhaps there's a way to disincentivize police excesses without giving criminals a get-out-of-jail-free card.
I suppose there's an argument that anyone who would violate the Fourth Amendment can't be trusted as part of a chain of evidence... but in that case, shouldn't the guilty cop be kicked off the force entirely, not just distrusted regarding a single case?
Those are just thoughts in general, though, not necessarily a recommendation for this particular case. Even if it was admissible, I'm not sure I'd want to prosecute someone with evidence like "Look at what we found on his computer, thanks to the help of some guys who felt cheated by him, took his computer, reported incriminating files to us, and totally pinky swear that neither of them put them there themselves."
Parent
Re:That's a terrible argument (Score:5, Insightful)
How would you feel about this man if it was your child's photograph on this man's notebook.
How would you feel if it was your laptop that was seized without a warrant? "Oh I don't have child porn" you say. Sure...but without that warrant the cops may just plant the evidence. Now what say you?
Or, that friend you let borrow your machine last week, remember him? Yeah, he's not the church going fun loving person you thought. On that USB key with all of his work related stuff was a nice folder of child porn. Its a good thing he copied everything to your machine so you could work together on that big project that boss is asking about.
Or, that teenager in your house, yeah dirty young man. He's out browsing the internet looking for pictures. He accidently clicks on a link with under age "actors". Fortunately, he's a good kid and backs out of the site right away. Didn't look at anything, didn't mean to go there. Hell, you've even trained him well enough to erase cookies and temporary files. Hear that knocking? Yeah, that's the police showing up without a warrent and taking your machine. Oh look, they just found deleted child porn images on your computer. You sick bastard.
Without the warrant you have one more leg to stand on to fight these charges. Its there to protect the innocent.
Parent
Re:That's a terrible argument (Score:5, Insightful)
>>>"Oh I don't have child porn" you say. Sure...but without that warrant the cops may just plant the evidence. Now what say you?
Even if they don't plant evidence, who wants to go through the hassle of losing their PC for one or two months while the cops scan it for hidden porn (or even stashed drugs). It's not about dishonesty by police, but stopping harassment of citizens. Nobody wants one or two months of their lives wasted just because the government agents have nothing better to do than grab private property.
"[the British government] has erected a multitude of new offices by a self-assumed power, & sent hither swarms of officers to harrass our people & eat out their substance;" - Declaration of Independence, 1776
Parent
Re:That's a terrible argument (Score:5, Insightful)
"The law exists to serve the public good"
No, it doesn't. Government exists to uphold rights, and the law exists to provide government one of the tools to do that. Rights belong to individuals, not "the public".
What makes a child pornographer a criminal is the concrete harm he does to an individual -- not some abstract harm to "the public good".
The system is designed around that. The bill of rights gives weight to the rights of the accused for two reasons. First, it is the job of the justice system to protect everyone's rights -- to defeind the rights of the victim while still respecting the rights of the accused. Second, when we don't respect the rights of the accused, we tend to conflate "accused" with "guilty", and then nobody's rights (including the victim) are protected.
If you dont respect the rules of the system even when they make it harder to catch the bad guy, then you're really asking for a rule-less system that enforces your will. But watch out -- yours isn't the will that's going to prevail if the system heads that way.
"With this decision, the courts have just given license to all of those who kidnap or exploit children to make this pornography"
No, they haven't. They have not made child porn legal; they have reminded the authorities that they still have to do their job according to the rules even when it's a job that really needs to be done.
"How would you feel about this man if it was your child's photograph on this man's notebook."
If we left 'justice' in the hands of how those harmed by the crime feel, it would be revenge (which is not the same thing -- and which incidentally doesn't serve the "public good", either).
"the judicial tradition of suppressing evidence entirely because it was produced without a proper warrant is absurd. The man was clearly guilty and the evidence was there. Instead, fine the police for doing the wrong thing"
Here, I agree -- to a point. It doesn't change the fact that in the context of the system as it exists, the court's action is correct, though; today the remedy for illegal search is suppression of evidence.
But yes, I think holding law enforcement personally responsible when they violate the rights of the accused would be more just than penalizing the victim (and any potential future victims) by preventing a conviction when the accused really is guilty -- if such a system can be made to work.
There are two problems with that, though, which I don't know how to resolve:
1) Having performed an illegal search, which results in the conviction of a child pornographer, a police officer goes on trial. What jury will convict him? If the answer is none and that's ok with you, then you're really saying that the accused shouldn't have had rights in the first place.
2) Being personally liable for mistakes can create an incentive to do less work. I'm not saying this justifies a lack of personal accountability in general, but you do have to have a system in which the police are confident "if I do the right thing, I won't be punished". That's harder than it sounds.
Parent
Re:That's a terrible argument (Score:5, Informative)
What evidence? Some md5 hashes that happen to match hashes from a select number of images? Odds are if we hash out every file on your hard drive we will also find matches to that same list.
Actually, odds are the hashes will not match...
Parent
Re:That's a terrible argument (Score:5, Informative)
Odds yes.
But no guarantee.
A better check is hash and file size, since it is more difficult for two files of the same size to have the same hash by chance. Especially using compression due to images or videos of the same dimensions reducing to different sizes.
Hash and file size checks are useful for checking if a file is intact and possibly not altered. They are great for lookups.
But, in the end, you still need the file to validate the correct item is found. Hashmaps store both the key and hash for this very reason. The hash is a quick lookup, but the key is needed to verify the right element has been found.
Unless the hash is the same size as the key.....
Parent
Re:That's a terrible argument (Score:5, Informative)
Yes, that's the birthday paradox. I'm not sure offhand how big the NCMEC database is, which is usually what they're comparing against, but let's try some math.
Let's say your hard drive has N files and the database has M items (so, comparing a list of N to another list of M hashes). Your hard drive doesn't actually contain any of the files used to generate the "bad" hash list. The probability of a hash collision is approximately P = 1 - exp( -N*M / (2 * 2^128) ). Assuming the value in the exponent is small, this is approximately P = N*M/2^129. 2^129 is in the rough vicinity of 10^43. In order for you to have a one in a billion (10^9) chance of a false positive, the product N*M would have to be ~10^34. If the hash list has a billion items (I think it's smaller than that, by quite a lot), you'd need 10^25 files on your disk -- well beyond the capacity of readily-available desktop storage.
MD5 hashes are useful because they're resilient to even birthday collisions. What they're not resilient to, it turns out, is intentionally creating two files with the same MD5 hash. (Even then, it is infeasible to generate two files with the same MD5 hash and the same size.)
Parent
Re:That's a terrible argument (Score:5, Insightful)
To exceed a .1% chance of finding a match with MD5 (a 128-bit hash) you would need to compare
n(p;H) ~ sqrt( 2*H*ln (1/(1-p)) )
n(.001;2^(32-1)) ~ 2^60
pictures. So to have a .1% of finding a collision of a legitimate picture and malicious picture in the FBI database one would have to compare about 830,000,000,000,000,000 pictures (8.3*10^17). You don't understand what it means to say that "MD5 is broken." Please leave the cryptography to the cryptographers.
Parent
Re:It doesn't matter. (Score:5, Insightful)
so you mean youre scared of living in an environment that everyone not on the right has been living in from 2000-2006?
Parent
that's basically what they were doing. (Score:5, Informative)
Re:that's basically what they were doing. (Score:5, Insightful)
The landlord's friend's friend didn't own the laptop. He can no more authorize a search of it than your landlord can authorize a search of the apartment he rents to you.
Parent
Bad way to search for kiddie porn (Score:5, Interesting)
I dont see how the 4th amendment applies here (Score:5, Interesting)
Re:good point (Score:5, Informative)
Parent
Error made by Slashdot in headline (Score:5, Informative)
When I submitted this story, I gave it the headline "US Court:...". Someone changed that to "PA Court Says...". That's wrong. This is a ruling from a US District (Federal) court, not a Pennsylvania state court, and so carries much more weight. ..bruce..
Cops blow it again (Score:5, Informative)
Not only did they search the drive without a warrant, but they also got the defendant to confess to putting the files there by questioning him without reading his rights and telling him that he didn't need an attorney. Genius.
Even dumber: Based on the testimony of the guy who originally found the child porn, they could have gone to a magistrate and gotten a warrant. Then there would have been no issue of a warrantless search.
BTW, for those considering the abandoned-property angle -- the court goes into that. It wasn't a legal eviction and the defendant hadn't abandoned his stuff; he merely hadn't removed it all yet.
Re:search = search (Score:5, Insightful)
To calculate the hash values they had to read the contents of the drive. That is a search of a person's effects without a warrant.
Parent
Re:I love how... (Score:5, Insightful)
Bad police work is bad police work, no matter the criminal.
Here's a clue: be upset with the stupid officers that could've followed procedure and actually nabbed the guy instead of being lazy and screwing up the case instead of the judge for enforcing the law.
These are YOUR freedoms too.
Parent