Arushi Saxena, Leonie Bolte, Rachel Gibian, Nick Anway
Our seminar discussion featured Jane Lytvynenko, a senior reporter at Buzzfeed News whose work focuses on disinformation and online investigations. Jane led the Assembly team through a post-election analysis of her reporting on disinformation, reflecting on how she and other journalists focused on disinformation during the election period and what they learned in the process. Her seminar covered: (1) how she and other journalists are thinking about disinformation; (2) the growing global problem of online manipulation; (3) the professionalization of disinformation; and (4) a deep dive on the U.S. election, conspiracies, extremists, and “voter fraud.”
We focused on headlines in the online information system
Headlines are an important component of our online information ecosystem. In a 2016 study, researchers at Columbia University and the Microsoft Research-Inria Joint Centre estimated that 59% of links mentioned on Twitter are not clicked at all, confirming that users share articles without reading them first. The Headline Game Activation was thus designed to build on Jane’s seminar by putting the cohort in the role of journalists writing headlines, in an effort to drill down on their role in disinformation. The game assigned each participant to a role as a journalist from a list of media sources, ranging in political ideology, and asked them to compose a headline from the perspective of that media source. Each participant was given a common fact pattern but the facts were rearranged for each, priming participants to emphasize different parts of the story. During the session, we played the Headline Game, in which participants tried to distinguish their “fake” headlines from real headlines, and to identify which source the headlines came from. The Activation discussion was designed to reflect on and learn from this process, focusing on a central question: What is the responsibility of a headline?
The Headline Game pursues three goals
- Understand how different news outlets might report on the same facts based on their subjective perspective / lens
- Develop understanding on journalistic responsibility and role of headlines on social media
- Develop view on best practices for writing headlines intended for online audiences
The activation recipe consists of a pre- and an in-session part
- Assemblers receive a pre-work role assignment and materials, a fact pattern to use as a headline writing prompt, and a link to the headline submission form
- Activation team compiles headline submissions in Google Slides
- Intro (5 min)
- Play the headline game (20 min)
- Go through a Guessing Game exercise live, as a full cohort
- Exercise is for us to walk through a deck together. Each slide contains two headlines; one is true, one is a fabricated headline from someone in our cohort. Participants are asked to guess (1) which headline is true and (2) which news outlet is represented on the slide
- Individuals have 30 seconds to guess and submit their responses into Zoom chat
- Correct answer is revealed; brief discussion
- Each news outlet will have a dedicated slide, so we expected to present 10-15 slides during the exercise
- We allocated 20 minutes for this portion (so 1-2 minutes for the slide)
- To keep time and maintain a game-like spirit, we will consider using a loud buzzer so that participants know 30 seconds are up, 1 minute left, etc.
- Discussion (15 mins)
- What makes for a “good headline”?
- Are any of the headlines we wrote misinformation? Why or why not?
- What is the responsibility of a “good headline”?
- What is your concrete recommendation for writing a good headline?
- Is your recommendation universal (independent of the outlet)?
- How do you balance your recommendation with editorial, click-economy, institutional incentives?
- Wrap up (5 mins)
The session generated four outputs
- Pre-session Activation submission
- Fact pattern source (NYT)
- Google form for headline submissions
- In-session slides (including headlines created by the cohort)
Participants leaned on heuristics to determine if a headline was false or real
The headline game itself generated two interesting insights. First, participants leaned on heuristics to determine if a headline was false or real, choosing fake headlines based on questions like whether they felt like an appropriate voice for the publication, whether they were worded strangely or not, whether they were narrative or wire headlines, whether they were edgy or not. Second, these heuristics did not lead to accurate predictions of whether a headline was fake or not. As one participant put it, “they’re almost impossible to distinguish.”
Here are two examples. Can you determine the authoring publication, which is real, and which is fake?
Answer: Option A was a real headline from The Federalist.
Answer: Option B was a real headline from The Epoch Times.
The post-game discussion also turned up several interesting insights from the cohort. Several headline authors mentioned that stepping into the role of their assigned media source influenced them to focus on a subjective portrayal of the source’s voice more than an objective description of the facts. This led to interesting insights regarding the subjective incentives that influence headlines, including differential click economy incentives of subscription-funded versus advertising-funded media sites. The influence of structural factors was also discussed, like the separation of headline and article editors, and the use of recommender algorithms on news sites. Finally, the cohort discussed the inherent tension in regulating the responsibility of a headline: folks thought that news organizations have an obligation to frame their headlines in a manner that mitigates disinfo, but the use of regulatory requirements—by newsrooms, algorithms, or laws—can have a chilling effect on speech that raises concerns.
Future Assemblers should focus on the time limit to improve this exercise
The biggest challenge that we faced in facilitating the Headline Game was the time limit. We assigned headline writing pre-work in order to free up more time, but we were still only able to discuss seven headlines as a group within the time limit. This did not seem to impede discussion, but future Assemblers should focus on the time limit if replicating the Headline Game; future iterations may identify new ways to adapt.