Interview with W. David Stephenson on Dueling Platforms and a Dash of Transparency

Hi, Dave -- want to start off by thanking my high school typing (that's what it was in the day...) teacher, Mrs. Taughner, for giving me nimble fingers.

I came to this work through my work on what I call "networked homeland security," which argues that the advent of networked communication devices and Web 2.0 apps, especially mobile social network ones, have fundamentally changed the relationship of government and the people, and have empowered us whether government wants that or not.

When I first found out about what I guess we'll call "public data" -- ie, public release of data streams in easily-used formats such as XML, RSS and KML, that seemed to be a logical extension of the same change toward empowering the public and unleashing the "wisdom of crowds"

In the US, there are only a few government agencies (TO MY KNOWLEDGE -- WOULD LOVE TO KNOW OF OTHERS!!) working actively in this area. The RI Secretary of State, for example, releases a wide range of data streams, but they don't provide a lot of information on how to use them, so it's largely for the geek population.

In the US, the best example is the District of Columbia: its Citywide Data Warehouse releases -- at last count -- 216 different REAL-TIME (that's important, because real-time data allows you to interpret the data in real-time (duh) and that allows any analyses to really be valuable management tools, rather than just historical.

However, bar none, the most creative effort of this sort that I'm aware of is in the UK, where the government is actually asking the public for our ideas !!! on how to use public data better to serve the public interest. The "Show Us a Better Way" competition is actually posting the entries to the web as they are received (compare that to most contests, where only the judges ever see any but the winning entries), so that the public in essence already "wins" by learning a wide range of practical and impractical ideas, and they're offering a sizeable pot of money to the best ideas to begin their implementation. I hope that this gains a huge amount of attention, and that it forces the new US president to follow suit!

It seems to me that -- as the tremendous number of creative Google mashups that have been created in just a few years demonstrates -- there are limitless ways in which data can be used, by entrepreneurs, by watchdog groups such as the Sunlight Foundation, and by people with a strong interest in a given subject, to analyze possible trends, convergences, causality, etc. As a result, almost any kind of data stream that doesn't affect security or personal privacy is probably fair game for release in easy-to-use streams such as XML, KML and RSS. I'll be urging the attendees to err on the expansive side: you should have to justify keeping a data stream private, rather than the reverse!

Bear in mind that David Robinson, the lead author, is a self-described libertarian, and thus isn't a huge fan of government in general, so it's understandable that he'd take a position (OK, this is an over-simplification) that the best thing government can do is to publish data streams and then get out of the way of activist groups and entrepreneurs who can interpret it.

I think that under-estimates the way that smart government employees can use data internally to do a better job. I forgot to mention that Vivek Kundra, the DC CTO (who will be on the 9/9 panel!) is doing a great job of using data streams internally to undercut agency and program "silos," encourage collaboration, empower individuals, etc, with demonstrable benefits in terms of more efficient government operations, better services, etc!

Number one, understandably, would be fear of the unknown. This might seem risky if you've never done it before, and they also worry that statistics might be taken out of context (as if government officials hadn't ever done that themselves....) That's why I favor an approach of releasing the data internally first, as DC has done. That way you'll see some of the potential benefits, get familiar, understand possible pitfalls, etc. THEN release them publicly.

However, for those who still don't want to do it, I answer that ALL YOUR DATA ARE BELONG TO US!!! -- smart folks can already "scrape" much of it from other sources. If that's the case, I think the battle is already lost, and agencies would instead be smart to start enjoying some of the benefits of releasing it.

BTW: I don't want to underestimate the critical importance of a rigorous review process so that data relating to national security or personal privacy is protected -- much better than it is at present!

This little exercise resulted in some munching on humble pie on my part: the Obama campaign specifically endorses public streaming of data in its position paper on transparency and technology. I thought it would be cool to not wait for next January to test this commitment, so I set up a "topic hub" on IBM's "Many Eyes" data visualization site, called "Obama platform," linked in a wide range of data that others had already posted to other "topic hubs," relating to issues as varied as foreign aid, mortgage foreclosures, etc., and then invited people who were participating in the Obama "Listening to America" project to involve the public in writing the Democratic Platform to contribute their own data, use the Many Eyes visualization tools, and -- most important -- to discuss them using the site's threaded discussions.

I found out what happens if you create a data hub that falls in the forest and no one hears it -- to my knowledge it hasn't been used, but hope springs eternal!

1) they are showing a tremendous amount of humility in running the event: check the site for the exact verbiage -- but, in essence, they're saying you (us) have much better ideas than we do (that may be a little too Uriah Heep, obsequious, and selling bureaucrats short, but you get the picture)
2) the way they're publicizing it and offering cash prizes should raise visibility (and, subsequently, the pressure to deliver on new projects!)
3) most important, I forgot to mention that they're coupling the contest with massive releases of new data streams, which will continue to be available long after the contest is over.

Years ago I was in PR, and it strikes me that this is a bit of razzle-dazzle with real substance, and I hope it will kick off a series of virtuous competition in which other governments around the world will also release data and offer prizes.

Let 1,000 mashups bloom!

I do think the private sector has a role, but it ought to be on an even footing. When you have widespread release of public data, big companies can run with it, but so can individuals such as Rami Tabello in Toronto, who pursues illegal billboards, or the Council on National Priorities (hope I've got the name right on them )-- an activist group in Northampton, MA that does clever ways of illustrating guns and butter trade offs on national spending.

It's great if entrepreneurs can create profitable new services from this data (Google should be all over this...), but let's make sure EVERYONE has the same access.

I really like the IBM Many Eyes site (Swivel is also good, but I started using Many Eyes first and never really found a reason to leave...) because it offers a wide range of visualization styles (so you can experiment until you find one that seems best able to convey your point. It also requires that you post the URL where you got the data, so others can check to make sure you didn't doctor it.

Perhaps the most important thing, however, is the number of Web 2.0 tools that it uses: tagging, "topic hubs," and threaded discussions. They have a great mission statement that say that, as cool as any data visualization is, it really comes alive when it is shared by a community. I think that's true, and that the general public includes so many different perspectives, life experiences, expertise, etc., that the chance of totally novel insights resulting from the "wisdom of crowds" emerging from this dialogue is perhaps THE most important benefit of public data & data visualizations.

I don't know of ANY governmental site (my bet will be that the UK will be the first to pull it off...) that equals Many Eyes in this regard, so until some agency does pick up the challenge, Many Eyes is a great place for people who want to try the approach to do so. I ain't no stinkin' statistician (although I sure have a lot more respect for them now and am trying to really get up to speed!), but it took me less than an hour to find statistics, scrape them, upload them to Many Eyes, and do my first visualization. Try it yourself, folks!

Thanks, David. I hope that a lot of people in the DC area join us. Vivek is really pioneering transparent, data-driven government, and it promises to be a lively seminar!