Forgot your password?
typodupeerror
Bug Crime Government Your Rights Online

US Monitoring Database Reaches Limit, Quits Tracking Felons and Parolees 270

Posted by timothy
from the paid-by-the-row dept.
An anonymous reader writes "Thousands of US sex offenders, prisoners on parole and other convicts were left unmonitored after an electronic tagging system shut down because of data overload. BI Incorporated, which runs the system, reached its data threshold — more than two billion records — on Tuesday. This left authorities across 49 states unaware of offenders' movement for about 12 hours." As the astonished submitter asks, "2 billion records?"
This discussion has been archived. No new comments can be posted.

US Monitoring Database Reaches Limit, Quits Tracking Felons and Parolees

Comments Filter:
  • Now.. (Score:4, Insightful)

    by Anonymous Coward on Saturday October 09, 2010 @04:27AM (#33844000)

    They just need to upgrade it so they can track the other 4 billion properly.

    Damn sick criminals! ALL OF THEM.

    • Re:Now.. (Score:4, Insightful)

      by h4rm0ny (722443) on Saturday October 09, 2010 @04:59AM (#33844092) Journal
      I'm getting this scene in my mind like Austen Powers, where Senator McCarthy is unfrozen and keeps rubbing his hands with glee saying "We'll track one million US citizens." His NSA assistant coughs politely. "Uh, [i]billion[/i], sir".
    • Re: (Score:3, Insightful)

      by arivanov (12034)

      The joy of using centralised versus distributed systems. We live in the age of the network. Load balancing data sets across multiple databases, machines and systems and merging them whenever they are needed is trivial. Designing for such load balancing is trivial as well.

      Anyone designing a system that piles up everything on a single box gets whatever christmas they deserve. By the way, considering the name of the company I am not surprised. It says everything that there is to be said about their design meth

    • Re: (Score:2, Funny)

      by Sulphur (1548251)

      HTTP/1.1 200 OK Date: Sat, 09 Oct 2010 10:31:23 GMT Server: Apache/1.3.41 (Unix) mod_perl/1.31-rc4 Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=iso-8859-1
      OK
      The server encountered an internal error or misconfiguration and was unable to complete your request.

      Please contact the server administrator, admin@fbi.gov and inform them of the time the error occurred, and anything you might have done that may have caused the error.

    • Re: (Score:2, Insightful)

      Are people really this dumb?

      A record does not equate to a single individual. They are tracking movements so in the movement table, I am sure they have a lot of records for each person they are tracking, not to mention the other tables that contain multiple entries for people

      • Re: (Score:2, Funny)

        by Mitchell314 (1576581)
        No, what you're supposed to do is to have each movement have its own column.
    • Re:Now.. (Score:4, Informative)

      by Anonymous Coward on Saturday October 09, 2010 @10:46AM (#33845140)

      Posting anonymously for obvious reasons.

      I was on electronic monitoring for the US BOP (bureau of prisons) through BI incorporated for about 3 years. I had to pay my own monthly bill for monitoring services, which went to BI incorporated in Colorado somewhere.

      How the system works is like this: Your federal probation officer comes to your house and installs a box that looks kind of like a cable TV box. It connects to your telephone line (you must have a land line phone to be on electronic monitoring) on one end and also plugs into power. The box is pretty heavy because it has some rechargeable batteries in it so it can operate for some time if the power goes out.

      You get an ankle bracelet installed that is pretty permanent - rubber band with a steel core around your ankle, and a pager-like device attached to it. Now, the device is pretty simple. Whenever you go out of range (about 100-200 ft.) of the box, it dials one of BI's modems and reports that you left. Whenever you come back in range of the box, it dials out and reports that you arrived home. If you disconnect it from power, or the power goes out, it also dials in and reports the power outage (you are never supposed to unplug it, but sometimes power outages happen). When the power comes back on, it dials in and reports the power is back online. Even if you never leave your house at all that day, it still dials in once a day to report it's status.

      The purpose of this EM (electronic monitoring) system is to allow people to be on home confinement and still leave the house to go to work, get groceries, etc, but not be out at all hours of the night committing crimes.

      I can easily see how 2 billion records are in the database. There are not 2 billion criminals. These are just 2 billion date/timestamp entries saying prisoner #X left their house, prisoner #X returned, etc.

      I found the entire 36 month or so experience pretty surreal. The most difficult thing was wearing baggy pants to hide the ankle bracelet at work. For obvious reasons I didn't want to advertise to the world that I was a federal prisoner. It also says a lot about a society and judicial system where there are so many prisoners that they need to outsource the imprisonment of non-violent offenders to a corporation. But who am I to complain? I'm just a felon who committed a victim-less drug crime.

    • Re: (Score:3, Funny)

      by Phat_Tony (661117)
      But this just tracks US sex offenders.

      It will be no time before the fear mongers on the evening news are bandying about the new statistic,
      600% of the population of the US are sex offenders.
  • by Fry-kun (619632) on Saturday October 09, 2010 @04:29AM (#33844006)

    MS Access can't possibly handle 2 billion records, no matter how much hardware you throw at it.

    • by sakdoctor (1087155) on Saturday October 09, 2010 @04:54AM (#33844086) Homepage

      65536 Excel rows should be enough for anyone

    • by Eskarel (565631)

      Nor should it, if you've got 2 billion records use something designed for that kind of load.

      • Re: (Score:3, Interesting)

        by maxume (22995)

        Yes, that was the joke. See, GP poster is implying that even though the system should have been using something designed for the load, since it is a government contract, they used Access.

    • by Sycraft-fu (314770) on Saturday October 09, 2010 @07:40AM (#33844388)

      It seems to be the crap database of choice these days, especially for consulting companies. Friend of mine got a job not long ago as a consultant for a consultant. Yes really, he consults for a consulting firm. Not like he is someone they hire out, he is a consultant they hire to work on jobs they've been hired to work on. The thing that got him the job was his Quickbase experience. This company loves them some Quickbase for some reason. However they are always bashing in to limits it has. Had they used MSSQL or Oracle they'd be fine, but they didn't. So a major thing he does is work around those limits in various creative ways. Retarded, but that's what they want and they'll pay for it.

    • Re: (Score:3, Interesting)

      by assertation (1255714)

      If you haven't, rent a copy of the documentary "Hacking Democracy".

      Diebold chose to use MS Access as the backend for voting machines

  • by Adambomb (118938) * on Saturday October 09, 2010 @04:31AM (#33844010) Journal

    BI increased its data storage capacity to avoid a repeat of the problem.

    ONCE AND FOR ALL.

    • by siddesu (698447)

      Yes, I heard that after a long and challenging, but well-planned and spotlessly executed migration, the system now works on a future-proof fully 32-bit operating system, capable of accessing more than 640k of RAM.

  • by Anonymous Coward

    Seems like it took them a few hours to change the key column from unsigned +/- 2^31 to signed 0-2^32-1

    • Thus solving the problem once and for all. [youtube.com]
    • by hedwards (940851)
      You got that backwards. They went from signed to unsigned, giving them roughly double the space. But, what they're going to have to do is remove old records from the primary database, and put them into a secondary one after they're no longer tracking them. Because at the very least, it slows things down and increases the likelihood that a server crash will lead to a prolonged outage of service.
  • 2 billion... (Score:5, Interesting)

    by onion2k (203094) on Saturday October 09, 2010 @04:36AM (#33844026) Homepage

    Assuming that's a normal "US" billion, and assuming it's a journal of historical data going back a few years, I don't think it's unreasonable to think there could be information in there on a couple of hundred thousand people each of whom has been track for an average of at least 6 months. So, approximately and with some guesses, that's around 55 [wolframalpha.com] records per prisoner per day. 1 update every 30 minutes? That sounds about right, maybe a little on the low side if anything.

    What is surprising is that they were running some sort of database process that maxxed out at 2 billion records, and that it just stopped once it hit that limit rather than failing over to a backup process. But then, this is a government IT contract, so maybe it's not too surprising.

    • Re:2 billion... (Score:5, Interesting)

      by Statecraftsman (718862) * on Saturday October 09, 2010 @05:27AM (#33844130) Homepage
      If you track 16000 people and store their location once per second, you'll only need 1.55 days to reach 2^31 records. Once per minute only gives you 90 days. Once every 10 minutes, less than 3 years... I wonder if anyone is on the user end of this system that can comment.
    • Re:2 billion... (Score:5, Informative)

      by bertok (226922) on Saturday October 09, 2010 @06:26AM (#33844238)

      it just stopped once it hit that limit rather than failing over to a backup process.

      "just over 2 billion" is almost certainly 2^31 (2 147 483 648), or the maximum number representable by a signed 32-bit integer. People usually think of "over 4 billion" (2^32) as the integer limit, but that's for unsigned integers only, which are rarely used, especially in databases. I'm willing to bet that they used an "int" as a primary key in one of their tables, and simply overflowed the maximum possible value.

      This kind of bug has impacted lots of systems in the past. If it happens, there's no "fail over" that could possibly save the system. The replica would have the same data, and hence the same issue, and would have failed as well. The usual fix is to extend the key type to 64-bits or longer (e.g.: GUIDs), but for a 2 billion row table, that's going to take hours at best, probably days.

      Most database systems do not provide a warning when the keys start to approach large values, so it's easy to miss.

      • Re: (Score:3, Insightful)

        by Toze (1668155)

        unsigned integers only, which are rarely used, especially in databases.

        What? Intro DB courses all mention using unsigned columns for numeric/incremented indices. I use unsigned ints by habit for numbered indices. I'll grant you I've seen plenty of really terrible DB designs in the wild that happen to use signed ints, but "especially in databases" unsigned ints are more frequently used, at least by the competent pros I've met.

        Also, if the system's already down, there's no load on it. The admin can mount the schemas (maybe rollback a bit), apply the changes, and go get some coff

    • We consistently see this in monitoring systems designed by other companies. In our own monitoring systems we make extensive use of sparse appends (i.e. data only gets added to when there is a significant change, and we maintain a timestamp for the last update for each entity being monitored so we know that monitoring is actually taking place.) Of course this puts a lot more up front effort into actual system design.

      There seems to have been a period, roughly when hard drive capacity was rising more rapidly t

      • Re: (Score:3, Interesting)

        by MichaelSmith (789609)

        And maybe they don't need every GPS position in the database. It could be there just to cover for legal requirements in which case they could append it to a binary file and open a new file every day. Compress the old files with bz2 and archive all data more than a year old.

    • Probably a signed 32bit primary key....
    • Re:2 billion... (Score:4, Informative)

      by Anonymous Coward on Saturday October 09, 2010 @10:55AM (#33845202)

      Posting anonymously for obvious reasons.

      I was on electronic monitoring for the US BOP (bureau of prisons) through BI incorporated for about 3 years. I had to pay my own monthly bill for monitoring services, which went to BI incorporated in Colorado somewhere.

      How the system works is like this: Your federal probation officer comes to your house and installs a box that looks kind of like a cable TV box. It connects to your telephone line (you must have a land line phone to be on electronic monitoring) on one end and also plugs into power. The box is pretty heavy because it has some rechargeable batteries in it so it can operate for some time if the power goes out.

      You get an ankle bracelet installed that is pretty permanent - rubber band with a steel core around your ankle, and a pager-like device attached to it. Now, the device is pretty simple. Whenever you go out of range (about 100-200 ft.) of the box, it dials one of BI's modems and reports that you left. Whenever you come back in range of the box, it dials out and reports that you arrived home. If you disconnect it from power, or the power goes out, it also dials in and reports the power outage (you are never supposed to unplug it, but sometimes power outages happen). When the power comes back on, it dials in and reports the power is back online. Even if you never leave your house at all that day, it still dials in once a day to report it's status.

      The purpose of this EM (electronic monitoring) system is to allow people to be on home confinement and still leave the house to go to work, get groceries, etc, but not be out at all hours of the night committing crimes.

      I can easily see how 2 billion records are in the database. There are not 2 billion criminals. These are just 2 billion date/timestamp entries saying prisoner #X left their house, prisoner #X returned, etc.

      I found the entire 36 month or so experience pretty surreal. The most difficult thing was wearing baggy pants to hide the ankle bracelet at work. For obvious reasons I didn't want to advertise to the world that I was a federal prisoner. It also says a lot about a society and judicial system where there are so many prisoners that they need to outsource the imprisonment of non-violent offenders to a corporation. But who am I to complain? I'm just a felon who committed a victim-less drug crime.

      Another thing to mention is that what you see on TV or in the movies is pretty false. These are not GPS enabled tracking devices that can pinpoint your location on a map so they can hunt you down anywhere in the country. These are dumb radio devices that only have a 100-200 ft. range and the box uses dial-up modem technology from the 90s. I wouldn't be surprised if they ran the entire monitoring center on a few old PC servers.

  • about 16000 (Score:5, Interesting)

    by roman_mir (125474) on Saturday October 09, 2010 @04:49AM (#33844072) Homepage Journal

    Prisons and other corrections agencies were blocked from getting notifications on about 16,000 people, BI Incorporated spokesman Jock Waldo said on Wednesday.

    - interesting number. Anyway, it's not about the number of people in the database, it's about some number of records associated with each person presenting their location, so probably GPS coordinates taken at some time intervals.

    Also note that they are still logging the data, they just can't read it, so it's an application for displaying the coordinates that is failing. Quite possible that the actual problem is in filtering the data, maybe they are just trying to view data for an entire time period per person rather than looking at latest records, something like: 'last month only'. But this is, in the words of infamous W, 'speculaaation'.

    • by Kjella (173770)

      So on a wild guess there's a function "int recordId()" that'll return -1 on problems (network failure, database connection failure, whatever). Sounds like a completely reasonable design except it's time to move to int64, 2^63 records ought to be enough for everyone. Though with the US prison population, who knows...

    • by julesh (229690)

      Anyway, it's not about the number of people in the database, it's about some number of records associated with each person presenting their location, so probably GPS coordinates taken at some time intervals.

      Interestingly, if you assume they've averaged half the number of people currently being tracked (not an unreasonable assumption if the popularity of such tracking has been steadily increasing), and hourly logging, then you get 2^31 / 8000 / 24 / 365 = about 30 years, which is perhaps not coincidentally t

  • by mykos (1627575) on Saturday October 09, 2010 @05:02AM (#33844096)
    "According to the U.S. Bureau of Justice Statistics (BJS): "In 2008, over 7.3 million people were on probation, in jail or prison, or on parole at year-end — 3.2% of all U.S. adult residents or 1 in every 31 adults."

    This doesn't make me feel safe.
    • by GMThomas (1115405) on Saturday October 09, 2010 @05:35AM (#33844146) Homepage
      Right? You shouldn't feel safe. Not because of the "criminals" but because of the reason why there are so many "criminals." Have a joint on you? You're a criminal. Do you know how many people are in jail because of simple drug-related offenses? Be afraid. http://www.whitehousedrugpolicy.gov/publications/factsht/crime/index.html [whitehousedrugpolicy.gov] Look at that. 25% of federal inmates are in there for drug possession. I bet you a good amount of these people wouldn't rob you at gunpoint. Good luck, America!
      • Re: (Score:3, Informative)

        by jfengel (409917)

        Not that I'm disagreeing with your point, but I think you're misreading that page. That 25% figure is for people who were high at the time of the offense. (I assume you're looking at table 2).

      • by mbone (558574)

        The entire reason the Federal penitentiary system was created and the first Federal penitentiary (in Atlanta) was built was to handle drug criminals. Before that, except for the military, there basically weren't any Federal prisoners.

      • by tverbeek (457094) on Saturday October 09, 2010 @10:23AM (#33845014) Homepage

        25% of federal inmates are in there for drug possession. I bet you a good amount of these people wouldn't rob you at gunpoint.

        Not before their incarceration, no. But after surviving lock-up in a Darwinian environment in which "fittest" equates to "most dangerous", then re-entering a society in which convicts are denied the right to a good job, there's a pretty good chance they will. We have a criminal justice system that develops criminals.

    • Re: (Score:2, Funny)

      by ieatcookies (1490517)
      No no it's fine, they're all being monitored. Oh wait....
  • by WidgetGuy (1233314) on Saturday October 09, 2010 @05:27AM (#33844134)
    The actual data was only about 500K. The rest was XML markup.
  • by Co0Ps (1539395) on Saturday October 09, 2010 @05:47AM (#33844166)
    2 billion? That's awkwardly close to 2147483647... This is why your ID field should be BIGINT and not INT.... They where probably logging coordinates etc.
    • Re: (Score:2, Insightful)

      by HyperQuantum (1032422)

      2 billion? That's awkwardly close to 2147483647... This is why your ID field should be BIGINT and not INT....

      And I see no reason why someone would use a signed integer for an ID field. You're wasting half of the type's range (assuming negative ID's are not used).

    • by Surt (22457)

      Seems like a complete waste to use BIGINT if your db is never going to approach a billion rows. Obviously they should have used BIGINT in THIS case, but in general? Or did you only mean this specific case?

  • I'm not sure any data has been lost. Say they have a table with the following columns:

    id (auto increment)
    felonid
    gps
    timestamp ...

    If the 2 billion number is simply id that has run over, there's still enough data in the database to recreate the felons whereabouts using the gps and timestamp columns. Might be a problem in the system pulling data (based on id), but probably no data has been lost.

    • Re: (Score:2, Informative)

      by Grygus (1143095)
      You're right. From the press release: '“Importantly, the monitoring system continued to operate and gather information, but transmissions were delayed until the system was restored. Offender activity logged while the server was being worked on was effectively processed at 7:25 p.m. MT when the system was restored. Alerts that may have occurred during this period were transmitted to our customers at that time."'
  • by Chris Snook (872473) on Saturday October 09, 2010 @06:46AM (#33844284)

    Anyone remember when Slashdot hit 16,777,215 comments, and overflowed MEDIUMINT? The ALTER TABLE statement that fixed it took hours to run. I shudder to think how long it'll take to fix this, even with the problem diagnosed.

    • by Surt (22457)

      What DB was slashdot running on .. oracle and sqlserver will both do this sort of upgrade in very little time.

  • Hmmmm (Score:4, Interesting)

    by luis_a_espinal (1810296) on Saturday October 09, 2010 @07:04AM (#33844314) Homepage
    Sharding? Partitioning? But most importantly, using 64bit int types (or bigger) rather than 32-bit ints for primary indexes? I mean, what the hell they were using to store that data anyways? A Visicalc spreadsheet running on a TRS-80?
  • by assertation (1255714) on Saturday October 09, 2010 @08:20AM (#33844498)

    Maybe the answer isn't better software, but fewer criminals to fill up the database with.

    I keep seeing articles here and there how the U.S. has more people imprisoned than China. A large chunk of the prison population are inmates convicted of drug crimes and a large portion of that set of people were convicted on marijuana laws.

    I don't smoke, but as a tax payer I would rather see the government make marijuana into a tax revenue generator instead of a huge expense to paid for with taxes.
     

    • by WinstonWolfIT (1550079) on Saturday October 09, 2010 @08:42AM (#33844562)
      The Chinese use a simpler, more lethal solution to prison overcrowding.
    • Re: (Score:3, Informative)

      by tverbeek (457094)

      In addition to having the largest number of prisoners by headcount, the US has a comfortable lead in the largest percentage of its population (715/100K) in prison. Russia and Belarus (core of the former Soviet Union) are the closest competitors (554-585/100K), followed by an assortment of various small "third world" countries, other former Soviet-bloc states, South Africa, and Singapore. Not great company. The first western-European country – the societies that the US is supposedly closest to in c

  • We don't know!

    (check the name of the BI Inc spokesman)

  • As far as I can tell the population of the USA (http://www.wolframalpha.com/input/?i=population+of+USA) is around 312 million.
    Which is not even close to 2 billion....

    So how can the US be monitoring over 6 times it population?

    • by Surt (22457)

      Maybe they keep more than one record per offender in their movement monitoring database. Shocking, I know. Imagine the phone call: hello, there was an attack at XYZ Elementary School last week, we think it might have been wisnoskij, can you tell us where he was thursday at 2pm? No, you can only tell us where he is now? Your system only keeps one record per offender?

      • OK i guess record probably just means piece of data, meaning every single person would have many.

  • One record per second for the year, tracking your location, 31 million records per day, per offender. A decade of records, a dozen offenders, 3 billion records. Oops ... they're not even tracking them that accurately. Maybe they're only tracking minute by minute, but for a couple of thousand offenders. I sure hope it's not hour by hour, it'd be too easy to get to a school, do something horrible, and get back before the system would notice.

"Marriage is like a cage; one sees the birds outside desperate to get in, and those inside desperate to get out." -- Montaigne

Working...