Berners-Lee Calls For Government Data Transparency

eldavojohn writes "Two months ago, Tim Berners-Lee unveiled a UK Government data project with the goal to make government data more useful for everyone. Today he is calling on the rest of the world's governments to become more transparent with their nonsensitive data. After only a few months, his project boasts around forty applications for using government data (screenshot example here). The BBC article notes the interesting uses of public data in India and Brazil that are disappointingly lacking in other countries — even the United States. Hopefully the US's data.gov will evolve to hosting apps instead of just data."

Yeah yeah, he's a smart dude

[BadAnalogyGuy]

The thing that gets me is that TBL designed the internet protocols we use every day. Yet they are so full of plaintext and the technology to process it is all based around slicing and dicing this data up to turn it back into usable binary data that it's amazing we've come this far on such a rickety technology.

Re:

[eldavojohn]

The thing that gets me is that TBL designed the internet protocols we use every day. Yet they are so full of plaintext and the technology to process it is all based around slicing and dicing this data up to turn it back into usable binary data that it's amazing we've come this far on such a rickety technology.

"Rickety technology?!" Well, TBL should be so lucky to have you waste time posting about him on Slashdot. We're all waiting for your revolutionary code to be donated free of licenses. Are you working with Stanford on their clean slate project announced three years ago [slashdot.org]? How's that going?

For all the crap people give the current internet, there's a whole lot of talk and not a lot of work being done. Keep in mind that if you don't license it for free, it's going to perish like the U of MN's gopher pro [wikipedia.org]

The web is built upon hack after hack.

[Anonymous Coward]

Mr. Linux Nutsack, I respect your opinion on this matter, but BadAnalogyGuy is actually correct, whether he was trying to be funny or not.

The World Wide Web is built upon a base of rickety technology. Basically every web-related technology is a hack. JavaScript is one of the most significant hacks, in order to add interactivity. Cookies are a hack, in order to get around a lack of state storage. The various HTTP headers relating to caching are one of the most miserable of hacks. The ability of browsers to a

Re:

[Lunix Nutcase]

Mr. Linux Nutsack, I respect your opinion on this matter, but BadAnalogyGuy is actually correct, whether he was trying to be funny or not.

I wasn't saying he was wrong. The point, though, is that BadAnalogyGuy posts things in order to bait idiots such as eldavojohn who bite.

Re:

[Canazza]

what's even scarier is that it's eldavonjohn, so "you must be new here" doesn't work :S

U of MN's gopher protocol

[falconwolf]

I still have gopher sites in my bookmarks, it's likely their dead links though.

Falcon

Re:

[falconwolf]

Dang it, it's likely they're dead links.

Falcon

Re:

[Rogerborg]

Its like we're all enjoying transporting drunken cheerleaders across State lines in our fancy pants flying cars, but they're still using the same old steam engine that TBL designed under the hood.

(Psst... you're slipping)

Re:

[longacre]

Is that a new Bangbus site?

Re:

[Anonymous Coward]

BinaryXML (EXI) licenses aren't cheap. Until binary XML is standardised and affordable, we're stuck with legacy sub-optimal bloated plaintext technology.

Re:

[thijsh]

compress(XML) should do the trick... parsing time is negligible, the only advantage would be an index, but there are other tricks for that like embedding it in the XML or creating a separate smaller index file.
I like the trick DriveImage XML uses: one binary blob with all files after each other and a plaintext XML file with filenames and byte positions, this is a great example of the strength of XML and binary data combined.

Re:

[K. S. Kyosuke]

Ehm [sourceforge.net], what non-cheap licences?

Re:

[K. S. Kyosuke]

Browsers? I haven't seen anyone mentioning browsers.

Re:

[Anonymous Coward]

You don't just consider the format in isolation, you need to consider the application.

If we are to create, transmit and parse optimal, efficiently compressed content on the interwebz, the browsers need to be able to parse it and tools, scripting languages, etc. need to be able to read/write the format.

EXI appears to be a good solution, but there is no low cost C/C++ implementation that I am aware of.

Agile Delta (EXI inventor) have a C/C++ solution, but it's $loadsamoney.

An unrestricted (ie. not GPL or dual

Re:

[MrMr]

00101000100011110101000101010?

Re:

[drgould]

Yet they are so full of plaintext and the technology to process it is all based around slicing and dicing this data up to turn it back into usable binary data that it's amazing we've come this far on such a rickety technology.

The advantage of transferring data as plaintext is that you can slice and dice it to your heart's content. Instead of transferring it in some binary format that may be proprietary or non-extensible or out-dated in a few years.

In other words, plaintext is a feature not a bug.

Re:

[drgould]

> In other words, plaintext is a feature not a bug.

They are not directly human readable (the only advantage of plaintext), but you can convert to/from a human readable form whenever this is required.

I would submit that the tools to slice and dice plaintext are easier to write and debug than binary tools. And since I assume in this context "plaintext" really means "XML", there are tested, debugged, standardized libraries for importing, exporting and manipulating XML for all languages and all platforms th

Re:

[hydroponx]

One word, gzip [wikipedia.org].

gzip is great as long [microsoft.com] as [microsoft.com] you're [microsoft.com] not [microsoft.com] supporting [microsoft.com] ie6 [microsoft.com]

Re:

[hydroponx]

Unfortunately, there's still a lot of businesses running ie6 internally, so if you are (or work for) a software vendor that happens to sell/distribute/support web apps it's still a big consideration.

Re:

[drgould]

If I want to read an array of floats (say a bunch of 3D points), it's easier to read them in as a bunch of 4 byte binary chunks than to parse a text string.

Big or little endian? X86, ARM or IBM or other processor? IEEE 128, 64, 32 or 16 bit format? Gotta know exactly what format those binary data points are in or you're screwed. Plaintext... not so much.

We don't store images in plaintext because binary is too hard to work with.

But you can easily convert binary to plaintext using base64 [wikipedia.org] or similar conversi

Re:Binary XML (library support)

[advance-software]

> I assume in this context "plaintext" really means "XML" In this context plaintext does refer to XML including, but not limited to optimal XHTML compression, decompression & parsing. >And you can convert to/from binary format whenever this is required. Which (C/C++) libraries support binary XML serialization in EXI format ?

Re:

[tbuskey]

The thing that gets me is that TBL designed the internet protocols we use every day. Yet they are so full of plaintext and the technology to process it is all based around slicing and dicing this data up to turn it back into usable binary data that it's amazing we've come this far on such a rickety technology.

The thing that gets me is Microsoft designed the file formats we use in Office every day. Yet they are so full of binary that's not portable and subject to endianess issues. The tech to slice & dice the data to make it available on portable media such as the web or non intel cpus has to be all reverse engineered. Even Microsoft has issues with backward compatibility. It's amazing this rickety technology has lasted so long.

TBL didn't use a binary format for a reason. I have LaTeX documents created o

Asking the U.K. Government For Transparency

[Anonymous Coward]

is analogous to asking for banking reform in the United States of Amerika.

Yours In Perm,
K. Trout

This is how the transparency will be implemented

[K. S. Kyosuke]

You will see everything in front of the government, you will see everything behind the government, but you won't see what the government is today and what it is doing, because the government will be, well, transparent.

Anti-social

[TypoNAM]

Of particular note is the ASBOrometer which is a mobile application (iPhone and Android) that measures levels of anti-social behaviour at your current location (within England and Wales) and gives you access to key local ASB statistics. This app was number one in the top free UK iTunes app store last week.

So, this application keeps tracks of all nerds like me? Pretty harsh for not going outside...

Re:

[tehcyder]

I think most nerds are asocial rather than anti-social.

A private public service project in Portugal

[Sr. Zezinho]

An interesting project coming from a private foundation, instead of the government, is Pordata, a database of statistical data about Portugal:

http://www.pordata.pt/ [pordata.pt]

No, the Government should not host "apps".

[Animats]

"data.gov" should not host "apps". Just release the raw data, and let others analyze it.

If the Government provides "apps", they will be limited in annoying ways and won't be integrated with data from other sources.

Re:

[migla]

How do you know this as a fact?

Maybe the apps would be open source and allow extensive configuration. Besides, maybe the raw data would be there as well.

After all, it is a government by, for and of the people, isn't it?

Re:

[egcagrac0]

"Apps" may be bad, but a usable API to the raw data might be good.

Like "look, we get that you might want to look at only a subset of our 6 terabyte database. We'll let you run a limited number of queries per minute and return just that relevant subset of data for you." could be really handy.

It's not hosting apps - it's "linking" to them

[mattpalmer1086]

But data.gov is not hosting apps, just the data. It's doing this webby thing TBL invented called "linking" to them.

Almost magically, they are actually hosted and written by by entirely different people and organisations, and yet you can access them from data.gov's own pages.

Maybe you should click on one of those "links" at the top of this page and RTFA...

TED Talk

[jduhls]

Here's his TED Talk [google.com] on the subject.

Hey Lady!

[AP31R0N]

Pick a name!

Vancouver data is open..still waiting for StatsCan

[el chief]

City of Vancouver data is open now, at: http://data.vancouver.ca/ [vancouver.ca] Still can't find a goddam list of all the buildings in Vancouver. How hard is that? Statistics Canada are still dinosaurs. They charge for access to data we paid them to collect.