The Smart Chicago Collaborative provided server space on our Amazon Web Service account so that a local developer to develop an algorithm to classify tweets in the service of public health. Here’s how Joe Olson of Tracklytics describes the work:

Tracklytics partnered with Smart Chicago and DePaul University to enter a the Now Trending: #Health in My Community contest sponsored by the Department of Health and Human Services. The purpose of the contest was to spur innovation in analyzing social media outlets to detect global health trends. Specifically, the contest requirements were to use Twitter to identify trending topics relating to disease, such as West Nile virus and Legionnaire’s disease.

Tracklytics designed and implemented the solution, DePaul helped with testing and algorithm validation, Smart Chicago funded the computer time.

We did not win the contest, but several innovations were developed. We now have the ability to collect over 1,000,000 disease-related tweets per day. A subset of these tweets are run through a machine learning algorithm, and can be filtered down to a much smaller set of tweets where someone is reporting having an instance of a disease, as well as their location. Current accuracy of this is near 85%. We’ve reduced the cost of classifying tweets down to .0000057 cents/tweet, storing them for analysis down to 0.000000235 cents/tweet/month, and archiving them down to 0.000000004 cents/tweet/month.

You can view the site at http://hhs.tracklytics.com

We are currently working on modifying this technology to track food poisoning cases in Chicago. Once legitimate food poisoning cases reported by individuals are identified by Twitter using the machine learning algorithm, we can send to the person reporting the case a tweet containing a URL directing them to a form to collect more specific information (do you suspect the incident occurred at a restaurant, was from tainted food, etc). This information can them be entered into the City’s 311 system via the Open311 interface, and directed to the City’s Health Department for further analysis.

Lots more to come from this work— stay tuned!