Kyla Williams Co-Presents Today at Philanthropy Ohio’s Annual Conference

Today, Leon Wilson, CIO of the Cleveland Foundation, and I will co-present at the Philanthropy Ohio’s annual conference with a theme this year of “Philanthropy Forward” and a concentrated discussion on Digital Civic Engagement & Community-Centered Design. Philanthropy Forward ’17 is set to inform practices, strategies and goals and connect peers in the field of philanthropy. The conference will also focus on the future of philanthropy with insight into the current state of the sector – fueled by recent research – addressing transitions, change and the leadership pipeline. With several networking and roundtable discussions, attendees will discover how to shift failures to successes, effectively fund advocacy and civic engagement and hear from  exceptional leaders across the state and country.

Leon and I also presented in April 2017 at the Council on Foundations Annual Conference “Leading Together” as part of a panel discussion with: Aaron Deacon, Managing Director, Kansas City Digital Drive; Elizabeth Reynoso, Assistant Director of Public Sector Innovation, Living Cities; and Lilly Weinberg, Program Director/Community Foundations, John S. & James L. Knight Foundation on “Supporting Civic Engagement through Technology and Community-Centered Design”. After finishing that presentation we decided more collaborative sharing between cities was necessary and lead to this opportunity at Philanthropy Ohio.

Community building in the digital era requires providing a space for the public sector and local communities to interact. Building solutions with peoplenot just for them – by using community-centered design can have profound social impact. This has been central to Smart Chicago’s work and has lead to the building of processes, products, services, and other lightweight tech solutions that have been helpful.

Our presentation today has the learning objectives:

  • To introduce different models developed in communities to address civic engagement digitally
  • To encourage the consideration of embedding support for digital civic engagement into existing grantmaking & advancement efforts

You can follow the happenings of the conference on Twitter @PhilanthropyOH and @SmartChgoKyla or by using the hashtag #PhilFWD17.


Good News!!! The Smart Chicago team is moving and now will be co-located with the City Digital Team at UI Labs. As such, our individual emails will be changing to:

Kyla Williams 

Sonja Marziano

Denise Linn     

Leslie Durr       

Our new mailing address is 1415 N. Cherry Avenue Chicago, IL 60642 and general phone number is 312.281.6900.

Please check our website at or follow us on twitter @smartchicago for more updates.

We appreciate your patience during this time of transition.

Civic User Testing Group (CUTGroup): Remarks at Code for America 2015 Summit

Today, I will talk about the Civic User Testing Group (CUTGroup) during the 2015 Code for America SummitIn 2013, Dan O’Neil presented at the CfA Summit about the CUTGroup as a model for changing the relationship between government and residents. Since that summit two years ago, we have doubled the number of CUTGroup testers from 511 to over 1,000 testers, we have tested sixteen websites and apps, we have expanded to all of Cook County, and we continue to add processes to engage with people in the CUTGroup. The work is never done.

I now run the CUTGroup project for Smart Chicago. Here are my thoughts about how not only to run a CUTGroup, (we lay this out in detail in the CUTGroup book and blog posts) but how to sustain a CUTGroup by leading with community engagement.


The Civic User Testing Group (CUTGroup) is a new model for UX testing, digital skills, and community engagement. What makes the CUTGroup the CUTgroup is the merging of these three components into one experience. To build a CUTGroup, you need to devote time to all these components equally.

The CUTGroup is a central program for Smart Chicago because it cuts across our three areas of focus: access, skills and data. Access: we conduct the majority of our tests in public computer centers and libraries in the community. Skills: for the tester who is introduced to new technology, and for the developer who learns ways to design tests and engage residents. Data: we help improve existing technology and encourage the creation of better technology.


The CUTGroup is the community of people in Chicago and all of Cook County who come together in libraries and public computer centers to test and have conversations around technology. In Chicago, we have testers from every part of the city, all 50 wards and all 77 community areas. We are now reaching the rest of Cook County.


These residents are paid to test websites and apps to help create better technology. We pay every CUTGroup member $5 for signing up, and then $20 when they participate in a test. As of today, we have done 19 tests that cover a wide range of topics such as schools, transportation, social services, neighborhood information, and more.


The motto below is key to this work. It removes the idea that residents do not understand technology as well as technologists. Instead, it permits testers to participate, give their feedback and show that their ideas are a valuable part of the process.


We talk about the methods and processes in the CUTGroup book to help other cities run their own CUTGroup.


Asking how to start or run a CUTGroup is important, but I think the question below is more important.


The answer is easier than we think.


So, when is it not a CUTGroup?


It’s not a CUTGroup when you are only testing civic apps. In other words, it’s not a CUTGroup if you put the tech first. The “civic” in Civic User Testing Group has to describe the group before it describes the user testing. At Smart Chicago, we are beginning to move away from the language of “testing civic apps” because there are more important criteria in determining what to test. We determine whether or not to test technology based on these criteria:

  • Interest and desire to do CUTGroup testing and talk with residents (commitment to be part of the process!)
  • The technology reaches a large and diverse group of residents and could have or has an impact on their lives
  • Willingness to listen and then respond to the feedback and make changes

Not everyone who seeks CUTGroup testing might see that the tech they created is a “civic app,” and that is ok.


A CUTGroup needs to lead with engaging with people around technology and that becomes easier when you know that people want to help make tech better. They want their voices heard and they know so much about how they use technology and how they want technology to work for them.





The first step to engaging through UX testing is inviting and recruiting people to join the CUTGroup and recruit from inside your networks as well as outside of it. It is crucial to open the CUTGroup to everyone and, then once they are in the group, to build one-on-one relationships with new testers. That way, no matter how they were recruited – by a friend, from a flyer in a library, from an organization – they are on the same level with all of the testers in the group. It does not matter how they get to you, but how they are included in the CUTGroup experience.


I cannot emphasize the statements below more. Recruit anywhere and everywhere. If we only recruited from one place or one group, or stayed with the same method of recruitment, we would miss so many people who want to join and participate. The tech we test cannot define who is in our CUTGroup.



Here are a couple of very specific ways that we try to be inclusive in CUTGroup.

First, we use physical gift cards because it’s the closest thing to cash that we can get. We want to give testers something that they can use anywhere they normally would shop. Even though physical gift cards cost more than digital gift cards we use them because they are easier to use and can be used anywhere.  We also cannot assume that people shop online in their normal day-to-day.


Recently, we incorporated text messaging into the CUTGroup process to reach people who do not have regular access to the internet. Out of our 1,000+ CUTGroup members today, 29% of our testers said their primary form of connecting to the Internet is either via public wifi or their phone with data plan. Testers will be able to sign up for CUTGroup and receive text notifications when new tests come up and being to respond to participate.




The next step centers on how we communicate with our testers and how we keep in contact with them. First, we never share information with our CUTGroup testers about anything other than CUTGroup. This maintains a relationship that only centers on the program that they signed up to be a part of.

We keep regular and open communication about upcoming tests, and explain the reasons behind why or why not we picked them for a specific test. If they are not selected for a test, we still want to share what the test was about so that they feel included in what is happening and can still interact with the website on their own.


When testers sign up for CUTGroup, we  do not gather information about demographics, and prefer to ask questions that relate to their devices and the ways they connect to the Internet.  This keeps our sign-up form simple and easy to fill out. Also, we are more interested in learning about the tester’s non-technical and technical experiences, and we value learning about these experiences more than gathering demographic information. We do not have a database of testers’ age, race, income levels, or education. These things are just not as important.


We do design the screening questions to gather better information about testers for that specific test. If necessary, this is when we capture information about demographics.


This allows testers to voluntarily give us more information. If they do not feel comfortable with answering the questions for a specific test, they do not have to, and they can still be part of the CUTGroup. Letting people choose how they interact with us is important and we want to feel comfortable in these interactions.



When it comes to the CUTGroup test, it is hard to listen and leave expectations aside. When you bring in developers and project managers, they have their own assumptions of how the test will go (and so will you).  However, the unexpected solutions that testers come up with can sometimes be the most valuable part of UX testing. It is important to step back, listen, and not tell testers why they are right or wrong  or what they should have done. This is hard because we want to help if they are getting stuck. What is better is to really focus on understanding how they are doing the task you asked of them.


We generally test in branches of the Chicago Public Library because it allows us to visit new neighborhoods and reach new people. We are very lucky to have such a great resource in the Chicago Public Library since there is a library in every neighborhood. It would be really easy for us to test in our own offices, but meeting people in their community allows new people to participate in testing. We love going to them.


It is important to talk about the person’s experiences not just about how they use technology. We want to ask how testers get information, how they do things in their normal day-to-day, and how they see technology fitting into those experiences.


Once testing begins, we keep asking questions to learn more about what testers are doing and their expectation. We ask them why they clicked on that button/link or why they are using that feature. Their responses help us gather insight not only about the tech being tested but also tech in general. Their UX experience is influenced by other tech they know and learning about those expectations helps understand how to build tech better.


We also always ask testers for very specific ways to improve the website or app. “Tell us anything and everything you can” is a phrase I use often because I never want testers to feel their improvements are too big or too small. I want to hear all of their ideas. We consistently ask this question across tests, and it gives testers to contribute to the creation.


Another big part of our work is teaching developers how to incorporate user testing in the development of their technology. By requiring that the developer be part of the process, we put them in front of testers to see and hear for themselves how real people would use their tech.   



The last piece of engaging with the CUTGroup is making the changes that the testers suggested.


Once a test is completed, I spend a lot of time working on the analysis. I take a look at each question and provide quantitative and qualitative data. We want to see how easy it was for testers to finish a task as much as we want to understand their expectations of the technology and its functions. We also want to learn about whether or not they think this website is for them, if they like it, and if they would use it again.


The last step is the hardest, and not always completely in our control, but my goal is always to find better ways to help developers make changes on their website or app. This is a process of collaboration and there needs to be a commitment made at the beginning of the process that the developers, project managers, or organization staff will make some changes based on the conversations they had with CUTGroup testers.

After that happens, I want to show testers what they helped to create and show them that we listened.


It all comes back to making change happen because if we are inviting, asking, listening, but not responding — the test is hollow. We are not respecting our testers and their suggestions and it comes full-circle to our motto. If it doesn’t work for them… it doesn’t work.


To follow my presentation at the summit, check out some of these hashtags #CUTGroup and #CfASummit. You can also follow me at @ssmarziano.

Here is my entire presentation:


Digital Inclusion Meets Civic Tech: Remarks at Code for America 2015 Summit

Today I will take part in the Digital Inclusion Meets Civic Tech panel at the 2015 Code for America Summit. It’s great to talk about such a timely issue with Deb Socia of Next Century Cities, Demond Drummer of CoderSpace, Chike Aguh of EveryoneOn, and Susan Mernit of Hack the Hood.

Since important conversations like this never seem long enough, I wanted to share my thoughts here.


I’m the new Program Analyst at Smart Chicago managing the Connect Chicago initiative and other projects like the Chicago School of Data. I care about open data, Internet access, faster networks, and improving digital skills. A question I’m particularly interested in is this:


I’ve noticed that a lot of common answers involve versions or combinations of the following:


Do we think these answers are enough? When the White House released its analysis of the digital divide in the U.S., they defined being on the right side of the digital divide as having Internet access in your home. While increasing at-home subscriptions is certainly a desirable trend, is it enough to declare victory in a city?

I would say no — not in 2015. Since Internet access has become more essential and its place in our hierarchy of needs has shifted, we should expect that percentage to increase naturally, even without policy interventions. We should acknowledge that, in 2015, Internet access in your home does not necessarily give you equal opportunity in the digital economy. Things like speed, type of online activity and skill are just as key to unlocking the potential of a connection.

Also an increase in citywide broadband adoption doesn’t speak to geographic and demographic gaps in Internet access; rather, the disadvantaged or historically underconnected people and neighborhoods that see that increase are the marginal successes we care about.

We need to go further:


Getting more people online in a city, getting a faster network, or having a robust nonprofit sector does not necessarily mean that city is digital inclusive. We should care about gaps in skill and use in addition to gaps in adoption. We should acknowledge that connections themselves are not the end game— rather, educational attainment, technology sector growth for all, workforce development and increasing civic engagement are the true outcomes. As program managers and policymakers, we should plan our evaluations around these truths.

Also, faster networks alone are not enough to make a city digital equitable. It’s what you do with the network that matters. Last year as a graduate student at the Harvard Kennedy School, I wrote, “A Data-Driven Digital Inclusion Strategy for Gigabit Cities.” You can see a summary one-pager here or read a blog post about it here. One thing I observed was that some high poverty urban census tracts had very high connectivity. Why? Because they tended to be dense, walkable, and house several community anchor institutions – schools, churches computer labs or community centers. While digital inclusion programing is often built on top of these trusted neighborhood institutions, cities should care about digital deserts – areas with low connectivity and low access to digital assistance.

Another question of interest:


Here are the “almost” answers – a vision of the civic technology movement that is admirable, but arguably incomplete:


The answers below go one step further. Much of this sentiment is captured in the great work of Laurenellen McCann in Experimental Modes of Civic Engagement in Civic Tech and Sonja Marziano in the Civic User Testing Group (CUTgroup).


The benefits of civic technology do not travel in one direction. Civic hackers have as much to gain from including diverse, non-expert and non-technical residents in their work as the residents themselves do. Residents seek ways to learn about solving social problems with technology, data or mobile applications. Civic hackers seek to create tools that solve relevant problems and truly work for everyone.


Despite the way we talk about them, civic tech and digital inclusion are not separate movements with separate missions. Both seek to make residents’ lives better through technology. The differences lie in the method and associations. When a resident thinks of a civic hacker, they might think of a person who seems smarter than them coding away at a hackathon. When a resident thinks of a digital trainer, they might think of the volunteer in the library public computing center. Wouldn’t it make both jobs easier (and the city better) if the civic hacker could also talk to the library trainees and the digital trainer attended the hackathon? Let’s make that happen.


Digital inclusion professionals have a lot to offer the civic tech community. These trainers and program professionals are experts in community outreach, skilled in training, and are the boots on the ground in their neighborhoods.

Unfortunately, digital inclusion professionals are not always paid to or encouraged to think about civic tech. I feel lucky in this respect because I work for Smart Chicago – an organization built for Chicago specifically to care about both civic tech and digital equity. The only thing I have to do to form a digital inclusion-civic tech partnership is Slack Sonja Marziano or wheel my desk chair three feet behind me to her work station!

These are her people:


These are my people:


This is just one place where civic tech meets digital inclusion in Chicago.


We’re thinking of ways Connect Chicago (a network of aligned programs, public computing centers and trainers across the city) and the CUTgroup can work together to make Chicago the most connected, skilled, digitally dynamic city in America.

One idea? Let’s get feedback from CUTgroup’s 1000+ testers on the digital skill offerings in the city. How easy is it for them to learn what they want? What resources are lacking? What resources exist that they don’t know about? We want to collect all the unknown unknowns. Chicago can better understand the “user experience” of its residents, not in relation to a new application or website, but in relation to the digital access and skills ecosystem in their city and community. As it turns out, a significant chunk of our CUTgroup testers rely on mobile and public Wi-Fi:

Are there other creative ways digital inclusion projects and the civic tech community can partner and strengthen one another? We’re confident. Let’s have that conversation. If you have ideas, we want to hear about them.

Talking to each other is fun, but doing stuff is better. Here are actionable items we can take home after the Summit:


Civic hackers, pledge to involve five people you don’t currently know in your next project. Embrace civic user testing groups as opportunities to learn, teach and inspire; measure success not only the number of apps created and tested, but in the number of people engaged. Digital trainers and digital inclusion program managers, go to a hackathon or civic tech convening, present a specific wish list, and take your trainees and co-workers with you.


To follow the panel on Twitter, see our hashtags #CfADigInc and #CfASummit and follow me at @DKLinn.

Here’s the presentation as a download:

Smart Chicago + Code for America Summit 2015

Radar screen

The Code for America Summit, “a roll-up-your-sleeves conference that brings together innovators from hundreds of governments across the U.S. along with civic-minded technologists, designers, community organizers, and entrepreneurs” starts on September 30.

Smart Chicago has a unique relationship with Code for America and performs a singular role in the community of civic-minded people and organizations here in Chicago and across the country.

Here’s a look at some of the presenters and speakers at this year’s conference and they work we’ve done with them over the years. Lots of the support we provide is quiet and under the radar, so we thought we’d make some noise and put some blips on the green screen.

Continue reading