CUTGroup #28 – Chicago Open Data Portal

CUTGroup Test in Progress on mobile deviceFor our twenty-eighth Civic User Testing Group (CUTGroup) session, we tested the newly redesigned homepage for the City of Chicago’s Open Data Portal. The Open Data Portal allows users to find various datasets regarding the City of Chicago. The City of Chicago Department of Innovation and Technology (DoIT) is working with Socrata to redesign the Open Data Portal, focused currently on the homepage, to be more user-friendly for users finding datasets and represent data and technology initiatives and applications created with open data.

The main goal of this test was to understand how testers who have some familiarity with the data portal (even minimum) respond to the changes made to the homepage. We wanted to capture how residents with different levels of digital and data skills search and what is the best structure for the homepage to make searching easier. Lastly, we wanted to see how responsive testers are to the other content that relates to the programs and tech initiatives of the City of Chicago DoIT.

Segmenting

On September 22, 2016, we sent an email to 1,172 CUTGroup testers who live in Chicago, and notification via text message to our SMS-preferred testers. We wanted to know if they would be available for an in-person test on September 28. When segmenting our testers, we were interested in testing on a variety of devices. We wanted to include testers who have used the Open Data Portal in the past and those who have never used the Open Data Portal before. We also wanted to include testers who have all levels of data experience to see how user-friendly the search functionality is.

Screening Questions

During our initial call-out for testers, we heard from 60 CUTGroup members. We asked how familiar CUTGroup members are with the City of Chicago’s open data portal and learned:

5 – Very familiar                  8% (5)
4 – Familiar                          22% (13)
3 – Neutral                            20% (12)
2 – Not very familiar     30% (18)
1 – Not at all familiar          20% (12)

23 out of 60 respondents (38%) had used the Chicago Open Data Portal before. 6 of those respondents specifically used the open data portal to search for crime data.

Test Format

For this in-person test, each tester was paired up with a proctor who was either a City of Chicago DoIT employee involved in the project or a proctor from the CUTGroup proctor program. Proctors requested testers to complete tasks on the Open Data Portal beta, observed the results, and took notes on the testers’ experiences and feedback.We also wanted testers to test either on laptops that we provided or their own mobile device. We tested with 6 testers on laptop devices, 6 testers on Android mobile devices, and 5 testers on iOS mobile devices.

Results

Contributions to this test were made by our CUTGroup proctors. Erik Hernandez, Peter McDaniel, Christopher Gumienny, Steven Page, and April Lawson helped facilitate this test.  CUTGroup proctor, Christopher Gumienny, also helped write a lot of the analysis report for this test. The CUTGroup Proctor Program trains once highly active CUTGroup testers to learn more about usability (UX) testing and CUTGroup test processes.

On September 28, and on a very rainy day, 17 CUTGroup testers tested Chicago Open Data Portal beta at the Woodson Regional Library located in the Washington Heights neighborhood.

This presentation was shared with the City of Chicago DoIT team that highlighted top results from the test. 

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents, but many expected access to city services.

At the end of this CUTGroup test, we asked testers if they felt that they are the target audience for the new, redesigned Open Data Portal and 13 testers (76%) said “yes.” Our testers included individuals who had no, little, or some familiarity with the current data portal, and 14 testers mentioned that this site appeared to be designed for the general public and residents.

3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness. 1 tester mentioned it was for people interested in data analysis. Another tester said it was built for business owners, while another said it was built for developers.

One concern is that testers, even after reviewing the website, still expected more access to city services and resources that could be found on the official City of Chicago website. There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website.

Identify access points to city services or action steps for residents.

There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal. A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage.

A second, more complex level is at the category level. For instance, if a user clicks on the “Events” category there could be a link to the City of Chicago events’ page before users even begin to explore datasets. This level of access would require reviewing all of the categories and understanding if users associate city services with those category types.

The most complex level is at the dataset level. Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information, this would require additional user feedback. The suggestion is to connect relevant datasets to the appropriate city services. Therefore, if a user is reviewing data about potholes, as one example, there could be a link to make a pothole service request.

Utilize the header image to clarify purpose about the site.

Data Portal Beta Homepage Header

“What’s this sales thing? It makes it look like it’s advertising stuff,” GPBlight (#4) said in response to reviewing the header images. When the homepage first loaded, a bug was immediately noticeable because the header images would stack and cause the page to jump upon load. While that is a detectable fix, we identified a larger opportunity to share information about what the website does. Very few testers reacted to the initial “data sale” image and those who did react had negative responses. The first header image, before rotating to either the crime or Divvy data, should define the purpose to the site while being as clear and direct as possible. The current “data sale” language and images confuse some users.

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task.

We asked testers to search for six different datasets; one was open-ended, while the rest proctors requested that they find specific datasets: building permits, a map of crime data, recently fixed potholes, active business licenses, and Chicago’s community boundaries map. We were interested to see how testers would search and find the datasets and what issues they came across during the process. While 12 testers (70%) indicated that finding these datasets was “easy” or “very easy” we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant, but not requested dataset.

Categories were often chosen over search bar, but technical bugs and design influenced those choices.

Only 1 tester out of 17 (6%) used the search bar consistently for all of the dataset search tasks, whereas 7 testers (41%) used categories to complete all dataset search tasks. The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find.

On mobile devices, the magnifying glass of the search bar was not working properly. Therefore, if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results.

When doing the initial review of the page, only 1 tester mentioned the search bar, which indicates that it is not a prominent part of the page. When asked about what actions they knew they could do from reviewing their page, only 2 testers mentioned the search bar. When asked to take their first action, 3 testers (18%) searched whereas 10 (59%) clicked on a data catalog category.

Data Portal Homepage search/categories

We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes. Nonetheless, we realized that the search bar was not very prominent for testers. Suggested improvements included larger font, higher contrast, and a more prominent location on the page.

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible.

While testers knew where to find building permit and crime data, finding recently patched potholes, active business licenses, and community boundaries was more challenging. In one example, 10 out of 11 testers who found the active business licenses dataset had explored other categories first.

Testers expected to find the potholes data in these categories: Public Safety, Environment, and Sanitation. Testers expected to find active business licenses in Administration and Finance. Lastly, testers looking for community boundaries looked in the Community category.

Some testers did use the help text on the categories to make decisions about which category to choose, but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category. Testers did not use the filters on the search results page, and upon reviewing a few datasets without identifying a relevant one, would often rely on search or finding another category. Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience. Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly.

We also witnessed that testers did not always distinguish between the type of data– whether it was a dataset, data lens, filtered view, or map they were choosing. This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list.

Data Portal Search Results Page

A primary search method should be defined for the user. Currently, users can choose a category or search, but both have their challenges. Outlined above are results and improvements for the categories since that was a prominent way testers searched. For each task, 11 or more testers used the categories in completing their search. If the search box was more prominent on the page and functioned better on mobile, we could identify if search was a better way for testers to complete these tasks. We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience. The bug on mobile also caused a stopping point that forced testers to use categories. If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources.

Testers liked the “Digital Chicago” & “How to Videos” resources on the homepage, but the “Chicago apps” were seen as being the most relevant.

Testers reviewed the other sections of the homepage including the “Digital Chicago” section that showcases recent articles by the City of Chicago’s Department of Innovation and Technology, “How to Videos” that showcase how to use the data portal and other tools, and “Chicago apps” tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker).

48% of testers thought the “Digital Chicago” section was “relevant” or “very relevant,” but testers felt that “Digital Chicago” did not mean much to them (calling this section “news” could be an improvement), they had questions about the articles and what they meant, and this section lacked descriptive content for residents.

66% of testers thought the “How to Videos” section was “valuable” or “very valuable” and a lot of testers liked receiving information via video, although there was a consensus that they should be shortened and users should know that they will be directed to YouTube.

76.5% of testers were likely to use the apps under the “Chicago apps” section because they saw them as useful to have for residents. 15 of our 17 (88%) testers were not familiar with these apps prior to visiting the website. The “Chicago apps” section met expectations from testers who at the beginning searched for or were interested in city services. There was a positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future.

Data Portal Homepage Chicago apps section

For the resident user, we would suggest placing the “Chicago apps” section higher on the page because it relates to their needs. To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website, adding a descriptive sentence about what “Chicago apps” is and why it’s on this page would be valuable. Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives.

Screenshot of misaligned layout on mobile devices

Next steps

Based on this CUTGroup test, work is already in progress to make changes that respond directly to our CUTGroup testers’ feedback. DoIT and Socrata are considering changing the Open Data portal’s layout to include:

  • Featuring the “Chicago apps” section higher on the homepage
  • Changing the design of the search box to be more prominent
  • Distinguishing between sections to make each section more apparent and separate from the next
  • Improving layout to be work better on mobile devices

DoIT is also thinking about creating shorter, more digestible tutorial videos for the “How to Videos” section. Finally, the banner will be reviewed and redesigned to be more user-friendly on all devices.

We look forward to future iterations of the Open Data Portal, and seeing how resident feedback was included in that process.

Final Report

Here is a final report of the results:

 

Here is the raw test data:

CUTGroup #28 Open Data Portal

Recap of CUTGroup Collective Call with CUTGroup Detroit

On November 7, we conducted our first CUTGroup Collective Call with about 20 individuals who are either interested in or working towards a CUTGroup model for their own city or wanted to hear about recent developments in the CUTGroup program. In this first call, we highlighted CUTGroup Detroit who, in the last seven months, built a CUTGroup tester base and conducted their first test on a city tool that displays commercial property information. The CUTGroup Detroit team– Noah Urban, Meghin Mather, Ayana Rubio from Data Driven Detroit (D3) and Ivoire Morrell, Civic Tech Fellow with Microsoft – provided great insight about tester recruitment, test process and results from their first test.

Here is their presentation:

Here are the full notes from the call, which includes a link to the audio:

Some takeaways

As someone who leads CUTGroup efforts in Chicago, and has been assisting cities like Detroit and Miami with their own CUTGroups, I still learn a lot from hearing CUTGroup Detroit’s perspective on their work.

Gift cards create program legitimacy

“He wasn’t sure if it was a legitimate enterprise until he received his $5 gift card in the mail… that $5 was confirmation that it was a legitimate organization.” -Ayana, D3, shares the benefit of sending a $5 gift card to testers upon sign-up.

This quote really stood out to me when listening to the CUTGroup Detroit team answer a question about the benefits of sending a $5 gift card. I am often asked why we send $5 gift cards for people to sign-up for CUTGroup and I respond with multiple answers ranging from people like money to confirming address eligibility to the value we see in getting device information during sign-up. To hear how the gift card helps create program legitimacy in Detroit gives me additional insight of the importance of using gift cards.

Tell your story during recruitment

At Smart Chicago, we are very good at documenting what we do and how we do it, but I was impressed with how CUTGroup Detroit told a story about their recruitment process. Their recruitment strategy was to have a street team and a social media campaign and to write about their experiences each week. The street team, led by Ivoire, walked through communities to hand out flyers and encouraged the people they met to sign up and follow CUTGroup Detroit on social media. They also connected with community organizations to do additional outreach with those networks.

CUTGroup Detroit invited a photographer to document, and they wrote (fun) blog posts about their experiences through this on the ground, recruitment effort over four weeks. Here’s just one snippet with lessons learned after the CUTGroup Detroit street team walked through Midtown neighborhood in the scorching heat of summer to recruit:

What were the lessons learned this week? Well, one the big things is that we need to memorize our license plate numbers so we can park without worrying about tickets because the city is handing them out like fliers. We got a ticket for being a few minutes over the time and we couldn’t pay over the app because no one knew the license plate number of the car we were using. So memorize your license plates people. Secondly, we are not as hardcore as we thought we were. Walking around midtown is no joke. As I write this blog my heels are still recovering from all that walking. We also decided that we need to be proactive in reaching out to organizations well ahead of time so they are more receptive to us when we show up and talk to them about CUTGroup Detroit. Our street team was also pivotal in making week two of recruiting successful. We thank them for all the effort they put in.

It was great to follow CUTGroup Detroit on their journey and feel like I was a part of it even from Chicago.

Cross-collaboration model across institutions can help build a CUTGroup

Detroit was unique when they approached me about building a CUTGroup because it involved so many different people and partners, whereas previously CUTGroups were built and maintained by one group or organization. Shelley Stern Grach, Director of Civic Engagement for Microsoft Chicago, described CUTGroup Detroit as a “perfect storm” of partners coming together. The collaboration happened very naturally and through the training we did together, individuals defined their roles.

“It’s actually has been really helpful to have the City [of Detroit] involved from the beginning. The City has provided insight about the strategic goals of the civic tech ecosystem and the CUTGroup Detroit team has been able to shape the program to complement the bigger strategy.” -Noah, D3, describing learnings from working with city partners/institutions.

Next steps

We look forward to continuing the CUTGroup Collective calls on other topics and to highlight experiences faced by cities building CUTGroup in their own city. If you have any questions about CUTGroup or CUTGroup Collective, please contact Sonja Marziano at smarziano@cct.org.

Digital Inclusion Innovation in Chicago: Wi-Fi at Windsor Park

Windsor Park Lutheran Evangelical Church in Chicago’s South Shore is the home of a collaborative, community-based connectivity initiative. Along with Leave No Veteran Behind, Cambium Networks, and a neighboring local restaurant, the Church set up a Wi-Fi network —  something to power job searches, homework, and communication even when libraries and computer centers are closed.

30719151055_4e0f37157b_o

I had the opportunity to visit the project in November and invited other members of the Connect Chicago Meetup to join me. I’ve had my eye on this initiative, not only because it was hosted in a faith community, but also because of its smart combination of partners: an anchor institution, a training nonprofit, an equipment donor (Cambium Networks), and a broadband provider (American Wide Broadband). During our visit we met with the champions that pieced together their strengths and expertise to make this work happen: Ray Savich of Cambium Networks, Kitty Kurth a consultant assisting with the project, Alvyn Walker of Leave no Veteran Behind, and Will Williamson of Windsor Park Lutheran Church.

Rural Solutions in City Neighborhoods

One obstacle that the group met when advocating for the project was the attitude that free Wi-Fi was plentiful in all parts of an urban area. “Why don’t people just go to Starbucks?” was a common question they encountered. Of course, you can’t see a Starbucks in every other block of South Shore as you do in other Chicago neighborhoods or in the Loop. Plentiful public or semi-public Wi-Fi is simply not there and, despite being in one of the largest cities in the U.S., the South Shore’s public connectivity lagged behind.

The solution implemented by partners was a fixed wireless network, powered by an antenna placed on the steeple of Windsor Park Evangelical Lutheran Church. Fixed wireless connections, Ray Savich of Cambium pointed out during our tour, are more commonly seen as a connectivity solution in rural areas. In this case, it also suited the needs of the South Shore community.

The network has been used by the church itself, supporting summer programming and community food pantry operations. Community programs supporting veterans and seniors have also utilized the connected space. Alvyn Walker of Leave No Veteran Behind described the reentry process for Veterans, sometimes additionally complicated when online resources are hard to access. Job training and searching, accessing medical services, signing up for public benefits, and navigating bureaucracy is that much harder without the Internet and digital literacy, he explained.

30682260376_293c69bb65_o

30630634021_6a9d83f3ae_o

The Compelling Partnership Equation

This project’s model is one to watch and potentially replicate in neighborhoods with similar challenges and community assets. There were complementary partners with different resources, but a common mission:

30630631371_d9cf60e4a5_o

As with many neighborhoods and cities, churches and other communities of faith play an important center of gravity for civic life. Voting, charity work, training, and language programs (just to name a few) are housed in these spaces and fueled by the volunteers and community networks that the buildings host. It is almost natural that such a place could be the great home for a hyper local connectivity solution as well.

Here are more pictures from Windsor Park:
Windsor Park Lutheran Church November 2016

If you’re interested in this model for community connectivity or would like to help host and plan an event about public Wi-Fi in Chicago, email dlinn@cct.org.