CUTGroup #28 – Chicago Open Data Portal

CUTGroup Test in Progress on mobile deviceFor our twenty-eighth Civic User Testing Group (CUTGroup) session, we tested the newly redesigned homepage for the City of Chicago’s Open Data Portal. The Open Data Portal allows users to find various datasets regarding the City of Chicago. The City of Chicago Department of Innovation and Technology (DoIT) is working with Socrata to redesign the Open Data Portal, focused currently on the homepage, to be more user-friendly for users finding datasets and represent data and technology initiatives and applications created with open data.

The main goal of this test was to understand how testers who have some familiarity with the data portal (even minimum) respond to the changes made to the homepage. We wanted to capture how residents with different levels of digital and data skills search and what is the best structure for the homepage to make searching easier. Lastly, we wanted to see how responsive testers are to the other content that relates to the programs and tech initiatives of the City of Chicago DoIT.


On September 22, 2016, we sent an email to 1,172 CUTGroup testers who live in Chicago, and notification via text message to our SMS-preferred testers. We wanted to know if they would be available for an in-person test on September 28. When segmenting our testers, we were interested in testing on a variety of devices. We wanted to include testers who have used the Open Data Portal in the past and those who have never used the Open Data Portal before. We also wanted to include testers who have all levels of data experience to see how user-friendly the search functionality is.

Screening Questions

During our initial call-out for testers, we heard from 60 CUTGroup members. We asked how familiar CUTGroup members are with the City of Chicago’s open data portal and learned:

5 – Very familiar                  8% (5)
4 – Familiar                          22% (13)
3 – Neutral                            20% (12)
2 – Not very familiar     30% (18)
1 – Not at all familiar          20% (12)

23 out of 60 respondents (38%) had used the Chicago Open Data Portal before. 6 of those respondents specifically used the open data portal to search for crime data.

Test Format

For this in-person test, each tester was paired up with a proctor who was either a City of Chicago DoIT employee involved in the project or a proctor from the CUTGroup proctor program. Proctors requested testers to complete tasks on the Open Data Portal beta, observed the results, and took notes on the testers’ experiences and feedback.We also wanted testers to test either on laptops that we provided or their own mobile device. We tested with 6 testers on laptop devices, 6 testers on Android mobile devices, and 5 testers on iOS mobile devices.


Contributions to this test were made by our CUTGroup proctors. Erik Hernandez, Peter McDaniel, Christopher Gumienny, Steven Page, and April Lawson helped facilitate this test.  CUTGroup proctor, Christopher Gumienny, also helped write a lot of the analysis report for this test. The CUTGroup Proctor Program trains once highly active CUTGroup testers to learn more about usability (UX) testing and CUTGroup test processes.

On September 28, and on a very rainy day, 17 CUTGroup testers tested Chicago Open Data Portal beta at the Woodson Regional Library located in the Washington Heights neighborhood.

This presentation was shared with the City of Chicago DoIT team that highlighted top results from the test. 

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents, but many expected access to city services.

At the end of this CUTGroup test, we asked testers if they felt that they are the target audience for the new, redesigned Open Data Portal and 13 testers (76%) said “yes.” Our testers included individuals who had no, little, or some familiarity with the current data portal, and 14 testers mentioned that this site appeared to be designed for the general public and residents.

3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness. 1 tester mentioned it was for people interested in data analysis. Another tester said it was built for business owners, while another said it was built for developers.

One concern is that testers, even after reviewing the website, still expected more access to city services and resources that could be found on the official City of Chicago website. There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website.

Identify access points to city services or action steps for residents.

There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal. A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage.

A second, more complex level is at the category level. For instance, if a user clicks on the “Events” category there could be a link to the City of Chicago events’ page before users even begin to explore datasets. This level of access would require reviewing all of the categories and understanding if users associate city services with those category types.

The most complex level is at the dataset level. Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information, this would require additional user feedback. The suggestion is to connect relevant datasets to the appropriate city services. Therefore, if a user is reviewing data about potholes, as one example, there could be a link to make a pothole service request.

Utilize the header image to clarify purpose about the site.

Data Portal Beta Homepage Header

“What’s this sales thing? It makes it look like it’s advertising stuff,” GPBlight (#4) said in response to reviewing the header images. When the homepage first loaded, a bug was immediately noticeable because the header images would stack and cause the page to jump upon load. While that is a detectable fix, we identified a larger opportunity to share information about what the website does. Very few testers reacted to the initial “data sale” image and those who did react had negative responses. The first header image, before rotating to either the crime or Divvy data, should define the purpose to the site while being as clear and direct as possible. The current “data sale” language and images confuse some users.

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task.

We asked testers to search for six different datasets; one was open-ended, while the rest proctors requested that they find specific datasets: building permits, a map of crime data, recently fixed potholes, active business licenses, and Chicago’s community boundaries map. We were interested to see how testers would search and find the datasets and what issues they came across during the process. While 12 testers (70%) indicated that finding these datasets was “easy” or “very easy” we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant, but not requested dataset.

Categories were often chosen over search bar, but technical bugs and design influenced those choices.

Only 1 tester out of 17 (6%) used the search bar consistently for all of the dataset search tasks, whereas 7 testers (41%) used categories to complete all dataset search tasks. The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find.

On mobile devices, the magnifying glass of the search bar was not working properly. Therefore, if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results.

When doing the initial review of the page, only 1 tester mentioned the search bar, which indicates that it is not a prominent part of the page. When asked about what actions they knew they could do from reviewing their page, only 2 testers mentioned the search bar. When asked to take their first action, 3 testers (18%) searched whereas 10 (59%) clicked on a data catalog category.

Data Portal Homepage search/categories

We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes. Nonetheless, we realized that the search bar was not very prominent for testers. Suggested improvements included larger font, higher contrast, and a more prominent location on the page.

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible.

While testers knew where to find building permit and crime data, finding recently patched potholes, active business licenses, and community boundaries was more challenging. In one example, 10 out of 11 testers who found the active business licenses dataset had explored other categories first.

Testers expected to find the potholes data in these categories: Public Safety, Environment, and Sanitation. Testers expected to find active business licenses in Administration and Finance. Lastly, testers looking for community boundaries looked in the Community category.

Some testers did use the help text on the categories to make decisions about which category to choose, but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category. Testers did not use the filters on the search results page, and upon reviewing a few datasets without identifying a relevant one, would often rely on search or finding another category. Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience. Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly.

We also witnessed that testers did not always distinguish between the type of data– whether it was a dataset, data lens, filtered view, or map they were choosing. This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list.

Data Portal Search Results Page

A primary search method should be defined for the user. Currently, users can choose a category or search, but both have their challenges. Outlined above are results and improvements for the categories since that was a prominent way testers searched. For each task, 11 or more testers used the categories in completing their search. If the search box was more prominent on the page and functioned better on mobile, we could identify if search was a better way for testers to complete these tasks. We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience. The bug on mobile also caused a stopping point that forced testers to use categories. If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources.

Testers liked the “Digital Chicago” & “How to Videos” resources on the homepage, but the “Chicago apps” were seen as being the most relevant.

Testers reviewed the other sections of the homepage including the “Digital Chicago” section that showcases recent articles by the City of Chicago’s Department of Innovation and Technology, “How to Videos” that showcase how to use the data portal and other tools, and “Chicago apps” tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker).

48% of testers thought the “Digital Chicago” section was “relevant” or “very relevant,” but testers felt that “Digital Chicago” did not mean much to them (calling this section “news” could be an improvement), they had questions about the articles and what they meant, and this section lacked descriptive content for residents.

66% of testers thought the “How to Videos” section was “valuable” or “very valuable” and a lot of testers liked receiving information via video, although there was a consensus that they should be shortened and users should know that they will be directed to YouTube.

76.5% of testers were likely to use the apps under the “Chicago apps” section because they saw them as useful to have for residents. 15 of our 17 (88%) testers were not familiar with these apps prior to visiting the website. The “Chicago apps” section met expectations from testers who at the beginning searched for or were interested in city services. There was a positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future.

Data Portal Homepage Chicago apps section

For the resident user, we would suggest placing the “Chicago apps” section higher on the page because it relates to their needs. To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website, adding a descriptive sentence about what “Chicago apps” is and why it’s on this page would be valuable. Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives.

Screenshot of misaligned layout on mobile devices

Next steps

Based on this CUTGroup test, work is already in progress to make changes that respond directly to our CUTGroup testers’ feedback. DoIT and Socrata are considering changing the Open Data portal’s layout to include:

  • Featuring the “Chicago apps” section higher on the homepage
  • Changing the design of the search box to be more prominent
  • Distinguishing between sections to make each section more apparent and separate from the next
  • Improving layout to be work better on mobile devices

DoIT is also thinking about creating shorter, more digestible tutorial videos for the “How to Videos” section. Finally, the banner will be reviewed and redesigned to be more user-friendly on all devices.

We look forward to future iterations of the Open Data Portal, and seeing how resident feedback was included in that process.

Final Report

Here is a final report of the results:


Here is the raw test data:

CUTGroup #28 Open Data Portal

Recap of CUTGroup Collective Call with CUTGroup Detroit

On November 7, we conducted our first CUTGroup Collective Call with about 20 individuals who are either interested in or working towards a CUTGroup model for their own city or wanted to hear about recent developments in the CUTGroup program. In this first call, we highlighted CUTGroup Detroit who, in the last seven months, built a CUTGroup tester base and conducted their first test on a city tool that displays commercial property information. The CUTGroup Detroit team– Noah Urban, Meghin Mather, Ayana Rubio from Data Driven Detroit (D3) and Ivoire Morrell, Civic Tech Fellow with Microsoft – provided great insight about tester recruitment, test process and results from their first test.

Here is their presentation:

Here are the full notes from the call, which includes a link to the audio:

Some takeaways

As someone who leads CUTGroup efforts in Chicago, and has been assisting cities like Detroit and Miami with their own CUTGroups, I still learn a lot from hearing CUTGroup Detroit’s perspective on their work.

Gift cards create program legitimacy

“He wasn’t sure if it was a legitimate enterprise until he received his $5 gift card in the mail… that $5 was confirmation that it was a legitimate organization.” -Ayana, D3, shares the benefit of sending a $5 gift card to testers upon sign-up.

This quote really stood out to me when listening to the CUTGroup Detroit team answer a question about the benefits of sending a $5 gift card. I am often asked why we send $5 gift cards for people to sign-up for CUTGroup and I respond with multiple answers ranging from people like money to confirming address eligibility to the value we see in getting device information during sign-up. To hear how the gift card helps create program legitimacy in Detroit gives me additional insight of the importance of using gift cards.

Tell your story during recruitment

At Smart Chicago, we are very good at documenting what we do and how we do it, but I was impressed with how CUTGroup Detroit told a story about their recruitment process. Their recruitment strategy was to have a street team and a social media campaign and to write about their experiences each week. The street team, led by Ivoire, walked through communities to hand out flyers and encouraged the people they met to sign up and follow CUTGroup Detroit on social media. They also connected with community organizations to do additional outreach with those networks.

CUTGroup Detroit invited a photographer to document, and they wrote (fun) blog posts about their experiences through this on the ground, recruitment effort over four weeks. Here’s just one snippet with lessons learned after the CUTGroup Detroit street team walked through Midtown neighborhood in the scorching heat of summer to recruit:

What were the lessons learned this week? Well, one the big things is that we need to memorize our license plate numbers so we can park without worrying about tickets because the city is handing them out like fliers. We got a ticket for being a few minutes over the time and we couldn’t pay over the app because no one knew the license plate number of the car we were using. So memorize your license plates people. Secondly, we are not as hardcore as we thought we were. Walking around midtown is no joke. As I write this blog my heels are still recovering from all that walking. We also decided that we need to be proactive in reaching out to organizations well ahead of time so they are more receptive to us when we show up and talk to them about CUTGroup Detroit. Our street team was also pivotal in making week two of recruiting successful. We thank them for all the effort they put in.

It was great to follow CUTGroup Detroit on their journey and feel like I was a part of it even from Chicago.

Cross-collaboration model across institutions can help build a CUTGroup

Detroit was unique when they approached me about building a CUTGroup because it involved so many different people and partners, whereas previously CUTGroups were built and maintained by one group or organization. Shelley Stern Grach, Director of Civic Engagement for Microsoft Chicago, described CUTGroup Detroit as a “perfect storm” of partners coming together. The collaboration happened very naturally and through the training we did together, individuals defined their roles.

“It’s actually has been really helpful to have the City [of Detroit] involved from the beginning. The City has provided insight about the strategic goals of the civic tech ecosystem and the CUTGroup Detroit team has been able to shape the program to complement the bigger strategy.” -Noah, D3, describing learnings from working with city partners/institutions.

Next steps

We look forward to continuing the CUTGroup Collective calls on other topics and to highlight experiences faced by cities building CUTGroup in their own city. If you have any questions about CUTGroup or CUTGroup Collective, please contact Sonja Marziano at

Digital Inclusion Innovation in Chicago: Wi-Fi at Windsor Park

Windsor Park Lutheran Evangelical Church in Chicago’s South Shore is the home of a collaborative, community-based connectivity initiative. Along with Leave No Veteran Behind, Cambium Networks, and a neighboring local restaurant, the Church set up a Wi-Fi network —  something to power job searches, homework, and communication even when libraries and computer centers are closed.


I had the opportunity to visit the project in November and invited other members of the Connect Chicago Meetup to join me. I’ve had my eye on this initiative, not only because it was hosted in a faith community, but also because of its smart combination of partners: an anchor institution, a training nonprofit, an equipment donor (Cambium Networks), and a broadband provider (American Wide Broadband). During our visit we met with the champions that pieced together their strengths and expertise to make this work happen: Ray Savich of Cambium Networks, Kitty Kurth a consultant assisting with the project, Alvyn Walker of Leave no Veteran Behind, and Will Williamson of Windsor Park Lutheran Church.

Rural Solutions in City Neighborhoods

One obstacle that the group met when advocating for the project was the attitude that free Wi-Fi was plentiful in all parts of an urban area. “Why don’t people just go to Starbucks?” was a common question they encountered. Of course, you can’t see a Starbucks in every other block of South Shore as you do in other Chicago neighborhoods or in the Loop. Plentiful public or semi-public Wi-Fi is simply not there and, despite being in one of the largest cities in the U.S., the South Shore’s public connectivity lagged behind.

The solution implemented by partners was a fixed wireless network, powered by an antenna placed on the steeple of Windsor Park Evangelical Lutheran Church. Fixed wireless connections, Ray Savich of Cambium pointed out during our tour, are more commonly seen as a connectivity solution in rural areas. In this case, it also suited the needs of the South Shore community.

The network has been used by the church itself, supporting summer programming and community food pantry operations. Community programs supporting veterans and seniors have also utilized the connected space. Alvyn Walker of Leave No Veteran Behind described the reentry process for Veterans, sometimes additionally complicated when online resources are hard to access. Job training and searching, accessing medical services, signing up for public benefits, and navigating bureaucracy is that much harder without the Internet and digital literacy, he explained.



The Compelling Partnership Equation

This project’s model is one to watch and potentially replicate in neighborhoods with similar challenges and community assets. There were complementary partners with different resources, but a common mission:


As with many neighborhoods and cities, churches and other communities of faith play an important center of gravity for civic life. Voting, charity work, training, and language programs (just to name a few) are housed in these spaces and fueled by the volunteers and community networks that the buildings host. It is almost natural that such a place could be the great home for a hyper local connectivity solution as well.

Here are more pictures from Windsor Park:
Windsor Park Lutheran Church November 2016

If you’re interested in this model for community connectivity or would like to help host and plan an event about public Wi-Fi in Chicago, email

October Connect Chicago Meetup Recap: Refurbished Devices

The digital divide in Chicago is not just about Internet access and access to digital learning opportunities. It’s also about hardware. Not every Chicagoan or Chicago household has the tools it needs to succeed in the digital age, even if the question of Internet access is in place.

Here is one piece of a longer infographic we published recently:


Some of Chicago’s households don’t have computing devices or, if they do, they rely exclusively on handheld devices. Imagine relying on a phone to apply to college, apply for jobs, or create content. A recent piece from Shorenstein also points out that mobile devices hold residents back when it comes to civic information and news consumption.

Device lending programs get computers into the hands of students and households who might not have had them before. Computer refurbishing programs also insert lower cost devices into our ecosystem and can help narrow technology access divides. Programs like PC Rebuilders & Refurbishers (PCRR) and FreeGeek Chicago take donated, used devices, refurbish them, and sell them for discounted rates. This can be an easier way for individuals and even institutions to get the hardware they need.

At our last Connect Chicago Meetup we explored these programs and also learned about how to buy refurbished devices — what certifications to look for, prices and quality to expect, and the overall benefits. Sarah Cade from PCRR led the discussion and created resources for digital inclusion practitioners and trainers across Chicago.

Here is Sarah’s presentation:

Sarah also shared PCRR’s “What Device is Right for You?” guide to help people just starting to invest in personal devices and technology.

All of the resources that Sarah has created will be incorporated onto the new Connect Chicago website, a page aggregating information on digital inclusion resources, stories, and programs in Chicago.

At the Meetup we learned that, in addition to churning refurbished devices for people and nonprofits, PCRR also tries to be inclusive in its employment and training. At the Meetup, we learned that about half of PCRR employees are ex-offenders.  


Below are pictures from the Connect Chicago Meetup:

Connect Chicago Meetup Oct 28, 2016: Computer Refurbishing in Chicago

More Connect Chicago Meetup resources:

Here is a video of the entire Meetup:

Thank you to all who joined us for this important conversation! The digital inclusion field in Chicago is a vibrant group of advocates and practitioners. We learn so much and leverage more resources when we work together. To join the Connect Chicago Meetup community, go to our Meetup page.

‘Tis the Season for Strategy

In December 2011, I arrived as the second employee of the Smart Chicago Collaborative bringing over 20 years of human service experience with me through the door. I was challenged and excited about being part of a new organization whose vast and innovative mission was to improve the lives of Chicago residents using technology. It has been 6 months since I transitioned into my role, as Interim Executive Director, and we’ve been busy. I’m very proud to lead and work with this super smart and competent team of all women, comprised of Sonja Marziano, Denise Linn, Leslie Durr, and Phaedra Studt. Time certainly flies and the work continues.

Smart Chicago Collaborative was created over 5 years ago by the City of Chicago, the Chicago Community Trust, and the John D. and Catherine T. MacArthur Foundation in response to the report The City that NetWorks. With this institutional positioning, Smart Chicago Collaborative has successfully served as a launch pad for innovation with great potential to influence the civic technology ecosystem in Chicago to increase access, skills, and data use. Our recent organizational evaluation identified two key recommendations: first, that, we refine our strategic role in relation to our fast-paced, ever-evolving technology ecosystem; and second, consistent with our role as a launch pad, that we identify and transition several of our promising innovations and projects into sustainable community-based opportunities to further fortify our ecosystem. The Smart Chicago Collaborative, utilizing our existing leadership and talent supported by our Advisory and Operations committees, will continue to digest the findings of our evaluation and align with Chicago’s existing economic development and smart city plans. Check back on our Smart Chicago Evaluation page as we share our progress and emerging ideas during this process.

Structural Success in Civic Tech

This blog post is also published on Data-Smart City Solutions and is by Glynis Startz — Smart Chicago’s 2016 Harvard Ash Center Summer Fellow. Glynis is assisted with Smart Chicago’s Array of Things Civic Engagement work, among other smart cities-focused projects. Glynis is a Master in Public Policy Candidate at the Harvard Kennedy School.

As a Harvard Ash Fellow this summer, I spent 10 weeks working at the Smart Chicago Collaborative, a civic organization dedicated to improving lives in Chicago through technology. I was brought on to work on Smart Chicago’s resident engagement for Array of Things, a series of environmental sensors being installed around the city. In addition to assisting with the day to day aspects of that project, I got to step back and think more closely about the Internet of Things and civic engagement. You can find the resulting blog posts here. In addition to this more tangible work, I also was able to learn about the breadth of projects Smart Chicago works on, and see how they, as an organization, function.


Glynis, our 2016 Summer Ash Fellow, greets Pilsen residents at our Array of Things Public Meeting in Lozano Library

Smart Chicago has a unique structure, functioning between the governmental and non-profit spheres. It was founded jointly by the City of Chicago, the John D. and Catherine T. MacArthur Foundation, and the Chicago Community Trust, and sits within the latter. This structure puts Smart Chicago in a privileged space. They have both stability and a lean, agile structure difficult to maintain in a government office. From this position Smart Chicago has prioritized reaching out to communities while maintaining ties to the government, philanthropy, and tech spaces — being a bridge between these communities in addition to their specific project work. Sitting at Smart Chicago this summer, I was exposed to many different aspects of the civic tech sphere as well as the wider community, but it also made me aware how rare that position was. Most cities are without a Smart Chicago, and many may never be able to create a similar model.

Smart Chicago is very open about its successes, lessons, and individual project information. It’s also open to spreading their unique model, but it’s hard to see how to export that to cities or areas who cannot provide that stable foundation. How do you take success in one arena and hack it together to work in in areas with less (or different types of) local support? It’s something I thought about more generally this summer, watching the civic tech ecosystem in Chicago. How can we export all these successes in major urban centers to small cities or counties that operate under even greater financial and expertise constraints?

To some extent this is already being done. Areas and organizations who want to prioritize civic tech aren’t waiting for a perfect solution or someone to give them resources, they’re figuring it out. But there are many more areas that don’t have the internal drive or knowhow to push through new projects but which could benefit just as much. Smart Chicago has been working to help adapt some of their successful projects to other cities. A great example of this is their Civic User Testing Group (CUTGroup) which brings residents in to provide user feedback for civic technology projects. This summer Smart Chicago helped Detroit adopt the model.

Not everything can be exported in this way, however. Some innovations are simply too expensive or complicated to run, or may not work the same way in a new location. Smart Chicago’s government-philanthropy model may be one of these goods, not feasible for every city at present, but an innovation with outcomes to try for. I think what we all need to strive for is the ability to disentangle the essential from the fluff in any venture. When you strip it way down, what are the wires that hold a project or organization’s value? Can these be achieved by a different method or set of founding partners? For Smart Chicago, the essentials seem to be the smarts and dedication of the people who work there, and the little extra freedom to test things out and experiment. Smart Chicago strikes me as an exemplar of what most of us internally understand works, but still struggle to create, particularly within government: a place where people are supported enough to be comfortable going out on a limb and stable enough to spend time spreading their successes.


Thank you for join us this summer, Glynis!