Once the program's over, leaders ask: What just happened?
Project managers call it monitoring and control. We gather data to measure progress toward our goals. Nonprofit funders may require it. When children are research subjects, sponsors may set ground rules. In any case, youth program leaders admit the challenges of collecting and presenting the data.
At the Hive Chicago Buzz hackathon, members of a data working group dubbed the Think Tank pledged to create a self-evaluation tool, plus professional development to help members conduct their own analysis or work with outside consultants. I'm a journalist fellow in a member group, The News Literacy Project.
We started by listing own data challenges: tracking the reach of our programs, surveying young participants and presenting the results. Many of us have similar goals – to teach critical thinking, raise engagement or encourage careers – so common benchmarks could help us compare outcomes. Not all of us have resources at hand to create or analyze a database.
Brainstorming sessions were led by Jeremy Dunn of the Chicago Public Library and Eve Gaus of the Field Museum. Stephanie Levi of Chicago STEM Pathways Cooperative presented a starting point for self-evaluation, a checklist based on Harvard research on after-school programs. Tene Gray of Digital Youth Network advocated digital badges as way to demonstrate and track what students have accomplished.
During a lunch break, I chatted with with Christie Thomas of the University of Chicago about Lawrence Lessig's lectures on "institutional corruption." (For the news media, that means forces undermining their public-interest mission.) After that ethics appetizer, Thomas led the group in exploring the issues of working with children.
Research rules are well established, dating to the government's 1979 Belmont Report. But parents can find them intimidating when they're outlined on a consent form. Are students giving informed consent when blogging or posting their cellphone selfies, and are their parents on the same page? Does putting children in a control group deny them benefits? Would a program for high school students be more effective in middle schools?
The group wrapped up by documenting their thoughts on an evaluation toolbox for Hive members. Future training could cover survey construction and emerging standards for connected learning. Work will continue in monthly meetups at the Harold Washington Library Center.This article also appears at Civic Artworks.