Dan O’Neil on WBEZ re: the Limits of Open Data

WBEZ LogoOn Monday, Smart Chicago Executive Director Dan O’Neil went on WBEZ’s Tech Shift to talk about what the Homan Square story says about open data in Chicago.

Dan wrote a blog post on both the Smart Chicago blog and his own personal blog with his thoughts on the issue.

Dan spoke about how he’s been a big fan of the open data policy, but that we’ve run right against the limits of open data. Some things are just not publishable, and data that does get published has limited utility. The crime incident data, for instance, has always been limited and the city’s always been upfront about it. (More here.)

Dan also spoke about how the Open Data movement has had the general idea that if we release data, steps 2-10 (the civic innovation) will occur all on it’s own and how this may not be true.

Dan stated that we can’t have a data-first solution for civic tech. We have to start with people and with what they know.

Boohood also asked about the recent discovery of missing crime data on the data portal that was uncovered by the Crime in Wrigleyville and Boystown blog.  Dan responded by stating that the people who ran the blog did a great service – they used the portal to find a specific case – spoke out about it – and resulted in more data being added.

Here’s the whole interview:

Add your own data sets to Plenar.io

Today, Plenar.io released a new feature that allows you to add your own data sets to Plenar.io.

Brett Goldstein Presenting Plenar.io at Code for America Summit 2014

Plenar.io was conceived as a centralized hub for open datasets from around the country. Funded by the National Science Foundation and the MacArthur Foundation, and led by a team of prominent open data scientists, researchers, and developers, it is a collaborative, open-source solution to the problems inherent to the rapid growth in government data portals.

Today, the team added a new feature that allows people to submit their own datasets to be used by Plenar.io. Currently, Plenar.io is able to accept any URL to a comma separated value (CSV) or link to a dataset on a Socrata data portal (like data.cityofchicago.org) that has fields with the following attributes:

  • Unique id: a field that is guaranteed to contain a unique number for every row in the dataset, even if rows are updated
  • Observation date: a date or datetime field for each observation
  • Latitude/Longitude or Location: either two fields with latitude and longitude , or a single field with both of them formatted (latitude, longitude)

If you have a dataset that has these feilds you can enter them on the Plenar.io website and it’ll be reviewed by the team. 

Tonight on Smart Chicago Live: The City of Big Data at OpenGov Hack Night

At tonight’s OpenGov Hack Night, Bo Rodda will talk about his work with the interactive 3D city model at the City of Big Data Exhibition.

Rodda is currently working on a platform to allow anyone to submit their own visualization for public display. Learn more tonight at hack night (every Tuesday at 6:00pm at 1871) or by tuning into our live stream at smartchicagocollaborative.org/live.

Chicago: City of Big Data

City of Big Data exhibit at the Chicago Architecture Foundation. Photo by John Tolva

We’ll post the live stream below the fold at start time!

Continue reading

Dan O’Neil and Lola Chen on Big Data at PechaKucha Night with the Chicago Architecture Foundation

Tonight I co-presented at the Chicago: City of Big Data Pecha Kucha with my colleague and friend Lola Chen.

Here’s the presentation, along with complete text below.

1.
I’m Dan O’Neil, and I run the Smart Chicago Collaborative, a civic organization devoted to improving lives in Chicago through technology. I’m here with Lola Chen, a community advocate here in Chicago. We are going to talk about the role of humans in big data in an urban environment.2.
I think it has a great role to play in helping understand how to run a city. The understanding of facts is critical to a just society. And what makes sense for other segments of our culture and economy can make sense for government.

3.
And much of my career has been devoted to data. I’ve made data-driven web products for the last decade. Smart Chicago Collaborative is a national leader in the creation of civic apps. We were the impetus behind bringing Open 311 to Chicago. I guess the point is, I know of what I speak.

4.
In my work at Smart Chicago, I’ve come to deeply appreciate the value of humans. They make all data. Data is a subset of humanity, not the other way around. I’ve seen first-hand what happens when the fetish of data can make everything go wrong.

5.
So I am dubious of any discipline that seeks to help people that doesn’t seem to really include people in meaningful ways. Remember how stoked Burgess Meredith was in the Twilight Zone when all the people were gone and he was left with his books?

6.
Pretty much every time I see something in the world of “big data” or “predictive analytics”, there is never any mention of humans. As if the machines are autochthonous, indigenous, comes from nowhere and knows everything. Empty of humans.

7.
But of course humans have made everything. And they are the most versatile and capable objects on earth. Burgess Meredith got pretty bummed when he immediately broke his glasses and couldn’t read any of his glorious books. His myriad word repositories were of no use. If only there was one other human left to read to him.

8.
I’ve come to know Lola through the OpenGovChicago meetup and she’s helped me greatly in my work at Smart Chicago. She is an amazing Chicago resident. She values data and technology, and is one of the best humans I know.

9.
Lola Chen is the master of the email. As I was preparing for this event, and I was pondering the value of humans in big data, she wrote me one of her missives. In it, she wrote, “Any alert person can ride the streets of Chicago and see the pattern of pothole problems. The ride might take 4 hours or so. The making notes might take 1 hour.” This is what I mean. This is the value of humans. So I yield the remainder of my Pecha Kucha to the great Lola Chen.

10.
Hi there, my name is Lola Chen, a self-confessed extreme data hog. I moved to Lincoln Park in 1969, right around when the federal government declared the area the first urban renewal blight zone. I first bought properties in Bucktown in 1984. I moved to East Garfield Park in 1998. All throughout, I collect data.

11.
Everywhere I go I ask the question “when might data be flawed”. I interviewed all sorts of residents and visitors from all over the world.  All have seen Chicago Potholes. Data can be incomplete, biased, omitted, inaccurate, misclassified, or falsified. I have seen all of these.

12.
Here’s a practical example of the lack of data sharing. I had parked far from the curb due to a deep pothole. They gave me a ticket for being more than 13 inches from the curb. One piece of data that should relate to another. I GOT OUT OF THE TICKET.

13.
Data can be falsified faster than you think. I monitored grass cutting in vacant lots owned by the City. The workers knew they had GPS installed on the tractors, and they went up and down the lot, showing through through data that the lot was cut. But they lifted the blade so that no grass was cut. The lot was marked as done. It was not.

14.
Here we have a KINZIE INDUSTRIAL CORRIDOR series of long potholes going down the street that are DEEP. Chicago is currently promoting a return to manufacturing with hopes of creating new jobs. THAT IS WHY THESE POTHOLES ARE RELEVANT. There are patterns, visible, if you look.

15.
This lovely Lincoln Park ALLEY Pothole has a mural as a backdrop. Stanley’s is a neighborhood fruit market institution opened by Greek immigrants in the 1960s who have succeeded in expanding almost every decade. DON’T REALLY THINK AN ALGORITHM COULD PREDICT POTHOLE/MURAL

16.
This Humbolt Park CATCH BASIN Pothole seems to be accessorized with roadwork paraphernalia.  Today the paraphernalia runs for almost 1 block it has been there so long. The paraphernalia becomes permanent. COULD AN ALGORITHM SPEED UP ROAD REPAIR?

17.
This sewer pothole was misclassified as fixed. It was not, however, fixed. This is shoddy workmanship that leads to multiple visits to the same issue, leading to more work for contractors and more dollars out of our pockets. The data saw “fixed”, but it was nothing of the sort.

18.
Here you can see the impetus of my note to Dan. Any alert person can see the issue with the seam in the asphalt. I’ve seen it all over, and reported it to a number of commissioners. The Inspector General is now looking into this.

19.
The City collects and stores enormous amounts of data, but the data is flawed, and there’s not enough. We need drones, satellites, patrol cars, garbage trucks, all collecting data and making 311 requests. There are no mechanisms to address these flaws. We need more people— smart City workers who know the data— cleaning this up. Let’s do it.

20.
So thank you to all the people who helped me put this together. If anybody in the world knows how to fox potholes, please send us some ideas!

Video Interview with Joe Olson of Foodborne Chicago

Foodborne Chicago is a website that connects people who complain about food poisoning  on Twitter to the people who can help them out—  the Chicago Department of Public Health. This Smart Chicago Collaborative Project recently had it’s one year anniversary and has responded to over 300 food illness tweets and has initiated over 60 different restaurant inspections in the City of Chicago.

Foodborne Chicago Schematic

An early schematic of Foodborne Chicago

To learn more about how the site works, we sat down with Joe Olson to talk about how they’re using machine learning and APIs to help the health department combat foodborne illness.

Continue reading