Saw this on Huffington Post via Tim O’Reilly, about how datafeeds will allow light to penetrate into dark places:

It seems to me that we also need more regulation, but we certainly need to make sure whatever is imposed is also as technologically sophisticated as the financial transactions it’s meant to monitor!

One Internet innovation, automated data feeds, might simultaneously meet the smart regulation requirement, improve banks’ internal operating efficiency, and even win back public trust as well. Obama has specifically called for data feeds from government agencies as part of his transparency strategy to rebuild public confidence in government, especially as a result of the bailout crisis.

Sound marvellous at frst reading - but then, you think a bit - OK, I can hear you lining up with The Big Objection - but hang on……

Technological limits used to mean that any kind of data had to be laboriously culled and aggregated before being submitted to regulators. Now, innovations such as XML tags and data syndication in formats such as RSS (as a Huffington subscriber, you get the Post via an RSS feed) or KML (used for Google map mashups) make it simple for companies to automatically combine the data and release it, whether internally or to regulators — and even to the public.

The District of Columbia, long plagued by corruption, began a transparency initiative under former Mayor Anthony Williams. It shifted into high gear under Mayor Adrian Fenty, and CTO Vivek Kundra. They now publish, on a real-time basis, more than 260 different data streams of statistics as varied as violent crime, building starts, and even requests to fill potholes. All of those statistics are available for anyone to analyze and interpret, and current uses range from tracking development around the new Nationals Park to showing crime reports on a Google Map.

And even better….

If the same system was applied to banking, critical statistics could flow automatically to federal regulators, while also being available to the banks’ own staff — many of whom have never had real-time data access in the past. Combined with innovative web-based tools to turn obscure data into easy-to-understand visualizations, for the first time the workforce, as well as regulators, would have the kind of real-time information that’s essential in today’s global economy, whether to regulate businesses or to run them.

OK, its time for the Big Objection - Garbage In, Garbage Out.

Undoubtedly using the most modern tech to get data from systems is a Good Idea, but that doesn’t guarantee its accuracy in and of itself. Who shall ensure that what goes into the feeds is accurate? The minute this data becomes valuable, it will be gamed. As anyone who has implemented a new system will verify, accuracy comes from cleaning up databases, putting in workflows that cannot be subverted, and ensuring management oversight (or other forms of effective oversight). Except in these cases its harder, as management (the ones wot got us into all this) oversight needs to be replaced by independent 3rd party oversight.