Be Adaptive: Using Information to Adapt Programming (Stage 3)

Stage Three of a Four-Part Blog Series

There are various ways media can influence audiences, leading them from awareness to action. In the previous parts of this series, we introduced the four stages of media campaigns and associated evaluative thoughts:

• Determine what you are trying to accomplish with media content
• Develop and test your content
• Distribution of content
• Reflecting on the results of your work

Now that you have determined what you would like to accomplish and developed and tested your content, it’s time to disseminate the content. How do you know if your content is making the impact you expected? How do you know if you are on the right track to achieving your campaign goals? What if something you can’t control happens?

In this post, we focus on Stage 3 (of 4) of a media campaign: Distribution of content. During this stage, we are assessing change in the community and consequently adapting the programming of the campaign.

How evaluation helped a station adapt to changes in the community

In 2013, a public media television station in St. Louis was working to support an initiative to reduce the high school drop-out rate. The station created content intended to highlight the positives of youth, with the campaign goal to improve adult attitudes toward youth at risk of not graduating, as well as drive viewers to become champions of youth. The station designed a content plan and implemented it.

Reflecting the notion that no plan survives in a changing environment, the region experienced many significant events in 2014. A police shooting of an African-American youth, Michael Brown, received a lot of public attention. Additionally, local public schools lost their accreditation, resulting in youth being given the option of attending other regional public schools. These events, subsequent protests, and media coverage served as factors that negatively affected the public’s attitudes toward youth. Where there was hope for youth, there was now a feeling of derision toward them.

The public media television station evaluated its programming effects on viewership throughout the campaign. The station saw that public views toward youth were souring and subsequently adapted content to address the issues and concerns that were affecting its viewers. A nationally covered town hall was implemented and content shifted, with voice given to all sides in a safe and open environment. These changes not only ensured that the campaign achieved its goals of reinforcing positive views of youth; the evaluation also clearly demonstrated to the station and its funders that without its influence, viewers’ attitudes toward youth would have significantly declined.

Four key steps to evaluation in the distribution stage

There are four key steps in the distribution stage—all of which must be implemented on an iterative basis. Since change occurs quickly in a community, your team is advised to address these four steps on an annual, semi-annual, or even quarterly basis—to accurately assess and adapt your program to meet your campaign goals.

1. Adopting your data collection tools and approach

The previous stages of evaluation work provided you with:

• Clearly articulated changes you wish to create in the community (Stage 1)
• A sense of what you think should happen and when (Stage 1)
• Tools you created to assess the impact of content on audiences (Stage 2)

The challenge in this phase is adapting the tools to reach a representative group of your target audience and determine when to ask them to share their thoughts. An easy and efficient tool for reaching most groups is an online survey. Here are a few surveys that can be adapted to your needs:

• Specific show
• Coverage of a topic
• Station specific with multiple topics

Many media organizations can leverage the viewership and sponsorship lists of their distributors/broadcasters to find individuals to survey. Partner organizations also maintain contact lists—often used for marketing purposes. These are usually acceptable proxies for the larger viewership/listenership communities. The important thing to remember is that you want these individuals to represent the community you are trying to influence or affect. So, it is important that you select your participants carefully.

If the information collected from surveys doesn’t provide enough detail, consider other evaluative methods including interviews, focus groups, or listening sessions. If you aren’t comfortable conducting these types of sessions, TCC Group as well as other evaluators in your community can help.

2. Plan data collection and conduct your research

To do this, you need to determine when you would want to engage these representatives. This is based on your expected rate of change and how often you can reasonably ask them to participate in your data collection. Based on these factors, you can determine when it is best to poll your audience.

One cost-saving method that helps minimize participation fatigue is to randomly select individuals to participate in a survey from your larger pool of representatives. By selecting say a quarter of the participants for a bi-annual survey, you should be engaging a good representation of the community and the most anyone will be asked to participate is twice in one year. Your capacity to do this does depend on the number of people you have access to and how often you really need information from them. However, this is a good basic rule to follow.

Another cost saving method is to combine evaluative questions across multiple projects into one survey. The various questions may collect information supporting several campaigns and/or to simply learn more about your audience.

Where possible, it is recommended that you share your findings with the community. If people understand that they are helping, they may be more likely to engage in the future.

3. Compare your results with the campaign’s expectations

Data analysis comes next. Depending on the complexity of the social issue and the questions you ask the community representatives, the analysis may be simple or complex. Survey data that can be easily quantified tends to support easier data analyses (e.g., values going up or down). Interview and focus group data tends to be more complicated, as you search for themes and changes in those themes.

Regardless of the method of data collection you use and how you analyze it, the critical step is to use the data to critically assess your campaign. Here we ask:

• Are we on the right track?
• What is working?
• What isn’t working?
• What else might be influencing our audience(s)?
• What do we need to change in the campaign?

4. Revise your campaign as needed

By comparing your results with your expectations, you will gain a better sense of your community’s current stance on the issue at hand. You will also know what you have done to affect change. Finally, you will be able to tell whether what you are doing has had the influence you expected. Now is the opportunity to adjust your campaign work if necessary. This can be accomplished by:

• Increasing distribution of relevant content
• Modifying current content targeted to increase its effectiveness
• Developing and distributing new content

Final thoughts

This isn’t easy work. Following Stages 1 through 3 can appear to be straightforward and simple. However, in practice it can be very challenging. If you are thinking about evaluating a media campaign or even just the effects of one show and this feels a bit daunting, you might consider engaging evaluators who have experience studying media and community effects. They can provide you with coaching to help you through the process or take on the process for you.

Coming up next, we will discuss Stage 4 of measuring media outcomes—reflection—explaining how to both learn from the campaign experience as well as share important findings with funders and communities.

Want to learn more about media evaluation services & tools? Email the author Charles Gasper: cgasper@tccgrp.com.