Tabloid Media Campaigns and Public Opinion: Quasi-Experimental Evidence on Euroscepticism in England

Abstract

Whether powerful media outlets have effects on public opinion has been at the heart of theoretical and empirical discussions about the media’s role in political life. Yet, the effects of media campaigns are difficult to study because citizens self-select into media consumption. Using a quasi-experiment—the 30-year boycott of the most important Eurosceptic tabloid newspaper, The Sun, in Merseyside caused by the Hillsborough soccer disaster—we identify the effects of The Sun boycott on attitudes toward leaving the EU. Difference-in-differences designs using public opinion data spanning three decades, supplemented by referendum results, show that the boycott caused EU attitudes to become more positive in treated areas. This effect is driven by cohorts socialized under the boycott and by working-class voters who stopped reading The Sun. Our findings have implications for our understanding of public opinion, media influence, and ways to counter such influence in contemporary democracies.

Introduction

Are powerful media outlets able to shape public opinion? This question is central to political science and also extensively debated in other disciplines (Bartels 1993; Horkheimer, Adorno, and Noeri 1972; Lazarsfeld, Berelson, and Gaudet 1948; McQuail 1985; Mutz and Martin 2001; Zaller 1996). The media’s role in influencing public opinion is contentious, from both a normative and an empirical point of view. While theorists have either attributed a crucial information and enlightenment role to the media (Holmes 1991) or warned about the media’s ability to spread propaganda (Horkheimer, Adorno, and Noeri 1972; Lippman 1921), empirical political scientists have questioned the media’s ability to shape public opinion. Most prominently Klapper (1960) concluded that, if anything, the media have “minimal effects,” and this view has been shared by other important scholars (Lazarsfeld, Berelson, and Gaudet 1948). At best, some have argued, media exposure should lead to the reinforcement of existing attitudes (Sherrod 1971; Shrum 2002).

However, the relationship between media exposure and public opinion is difficult to disentangle. Citizens choose which media outlets they consume, and the media not only set the agenda but also respond to public opinion shifts (Baum 2002; Kinder 1998). The “minimal effects” view has been forcefully challenged (Zaller 1996). Lately, field experiments (Gerber, Karlan, and Bergan 2009; King, Schneer, and White 2017) and carefully designed observational studies (Gentzkow, Shapiro, and Sinkinson 2011; Grossman, Margalit, and Mitts 2020; Ladd and Lenz 2009) show that exposure to newspapers can affect some electoral behaviors (Gentzkow, Shapiro, and Sinkinson 2011; Gerber, Karlan, and Bergan 2009; Ladd and Lenz 2009) and political discussion (King, Schneer, and White 2017). However, there are fewer studies providing robust evidence that the media can change opinions. Gerber, Karlan, and Bergan (2009), which was the only field experiment that attempted to test the effects of randomly assigned newspaper subscriptions on stated opinions, report null effects. However, the time frame of their study is limited and it is difficult to speak of “the media” and “media effects” as though we were studying a homogeneous actor. Some TV stations and newspapers, for instance the BBC or the Washington Post, attempt to report news in a neutral or factual manner, while other outlets explicitly try to “create opinion” (Arceneaux and Johnson 2013): consistent with this idea, quasi-random exposure to campaigning outlets such as Fox News led to an increase in Republican vote shares (DellaVigna and Kaplan 2007; Martin and Yurukoglu 2017) and populist cable news in Italy affected perceptions of crime rates (Mastrorocco and Minale 2018).

In this study we show that a campaigning tabloid newspaper was able to shape public opinion in the long run, with potentially important policy consequences. For causal identification, our study relies on a specific historical event, the Hillsborough disaster, a human crush at Hillsborough soccer stadium in Sheffield, England, in 1989 in which 97 Liverpool Football Club (F.C.) supporters lost their lives. ¹ The biased and slanderous reporting by the UK’s most popular tabloid newspaper, The Sun, on the disaster and its victims led to an unexpected and sudden boycott of The Sun in the region of Merseyside (UK) covering Liverpool. Importantly, for the purposes of this study, The Sun also happens to be the most widely read Eurosceptic newspaper in the UK and the second most widely read newspaper in Europe (Campaign Magazine 2002). Since the boycott was not triggered by the Eurosceptic slant of The Sun, but by its extremely biased reporting on the UK’s most deadly sporting disaster, the boycott was not a function of EU attitudes in Merseyside.

We believe that our study makes several important contributions to the literature on media effects, public opinion, and the rise of Euroscepticism and has broader implications for how we understand the role of the media in contemporary democracies. The fact that this large-scale boycott of the nation’s most widely read newspaper has endured until the present day is a sufficiently large and important event to allow us to estimate the effects of a powerful medium on Euroscepticism, addressing the concern that the effects of important causes are usually extremely difficult to identify. Following individual attitudes in Merseyside and other Northern English counties for more than 30 years, we provide evidence of the long-term influence of sustained media campaigns on public opinion. Using a difference-in-differences design based on yearly British Social Attitudes data from 1983 to 2004, we show that The Sun boycott caused a sharp drop in Sun readership in Merseyside. We also find that respondents’ attitudes toward the EU got significantly more positive in Merseyside after the onset of the boycott compared with attitudes of respondents in other Northern English areas and in other areas of England as a whole. We substantiate these findings by providing empirical evidence consistent with the explanation that this decline in Euroscepticism in Merseyside was driven by the boycott: The decline in Euroscepticism was most pronounced among cohorts that came of age during the boycott and among working-class voters who were most likely to read The Sun before the boycott. Moreover, we show that the long-lasting “Sun” boycott negatively affected the “Leave” vote in Merseyside during the 2016 EU referendum. We thus provide evidence that long-term attitudes toward the EU were influenced by media reports and that the unfolding disintegration crisis of the EU is to some extent a function of media campaigns that started more than 30 years ago. While this might not be surprising to EU scholars, estimating the causal long-term effect of a Eurosceptic media campaign in a quasi-experimental setting is a novel contribution. After all, the British Brexit vote has been one of the most consequential public policy decisions taken by referendum in the past decades.

Media Campaigns and Euroscepticism

When theorists worry about the persuasive power of the media, they usually refer to sustained one-sided campaigns by a medium or multiple media outlets that are meant to explicitly influence public opinion in a specific direction (Horkheimer, Adorno, and Noeri 1972; Lippmann 1921). Media outlets can act as “political actors” (Page 1985) and “issue entrepreneurs” (de Vries and Hobolt 2020; Hobolt and de Vries 2016), raising the salience of a specific political issue while at the same time providing a strong frame that citizens rely on to interpret the issue (Leeper and Slothuus 2020). As Page (1985, 20) writes, we should understand certain media outlets as political actors who “try and change the beliefs and policy preferences of mass and/or elite audiences, which would presumably affect subsequent policy decisions.” This assumes that these media outlets or their owners act purposefully to influence public opinion and that citizens can be receptive to such influences.

The latest experimental (Arceneaux and Johnson 2013; Arceneaux and Kolodny 2009; Broockman and Kalla 2016) and quasi-experimental (Jäger 2020) literature on issue-based campaigns and framing effects suggests that persuasion effects can materialize under favorable scope conditions: when audiences are not self-selecting into receiving political messages, but receiving them as a by-product (Arceneaux and Johnson 2013), when campaigns target emerging rather than highly salient or polarized issues (Arceneaux and Kolodny 2009), and when messaging is one-sided or loop-sided (Jäger 2020). As Zaller (1992) has shown, messages should be more effective at influencing opinions if the audience is not highly politically interested or engaged. This leads to the apparent paradox that those most likely to accept persuasive messages are least likely to receive them (Zaller 1992). Building on these insights, Arceneaux and Johnson (2013) have argued that audiences that consume a medium for entertainment purposes should be more likely to change their opinion if they receive political news via that medium than audiences that consume the medium because they intend to consume political news.

Arceneaux and Kolodny (2009) show that voters changed their minds when exposed to messages on birth control but not on the highly salient issue of abortion. On highly salient issues, citizens will already hold crystallized attitudes that are difficult to move. This is consistent with findings by Gerber, Karlan, and Bergan (2009) who report null effects of subscribing to either the Washington Post or the Washington Times on opinions toward the Iraq War in 2005. Clearly, Iraq was a highly salient issue in the United States in 2005. Moreover, the framing literature suggests that one-sided messaging is particularly effective (Chong and Druckman 2007). We should thus expect media effects to materialize in political environments characterized by one-sided messaging on emerging issues. We therefore deduct that if exposure to strong frames happens in the absence of effective counter frames, the media should be able to shape strong attitudes in relation to an issue. As Chong and Druckman (2013, 13) write, once “strong attitudes are accessible, counter-frames are rejected and may even serve to reinforce the original attitude.” Strong attitudes are then difficult to change going forward and will be less likely to decay.

One of the key remaining challenges in the study of media effects is thus to identify the consequences of sustained exposure to media content, so-called media campaigns, on specific issues of policy importance. A prime example of such a media campaign is the one-sided negative coverage that the European Union (EU) was subject to over three decades in the British tabloid press (Hobolt and Tilley 2014; Jackson, Thorsen, and Wring 2016). While the EU issue in the 2010s had become highly salient and during the EU referendum campaign reporting was balanced, English voters had been exposed to Eurosceptic content for decades (Jackson, Thorsen, and Wring 2016; Levy, Aslan, and Bironzo 2016). One remarkable aspect of this exposure is that a lot of it happened inadvertently while consuming tabloid media for their celebrity or sports coverage (Rooney 2000).

Eurosceptic media outlets and the reporters associated with them had consistently portrayed European integration as a threat to sovereignty and the EU as an inflexible and irrational bureaucracy imposing layers of red tape on member states (Jackson, Thorsen, and Wring 2016). While many researchers (de Vreese 2007; Hooghe and Marks 2007) and columnists (Bennhold 2017; Martinson 2016) have wondered whether the tabloid press have contributed to the rise of Euroscepticism—the (qualified) rejection of European integration (de Vries 2018; Vasilopoulou 2013), across Britain and other European countries—testing this claim is difficult. Researchers have relied mostly on cross-sectional surveys (Carey and Burton 2004) or panel studies to control for time-invariant confounders (de Vreese and Boomgaarden 2006). Survey-experimental studies suggest that there can be immediate effects of Eurosceptic media frames on public attitudes, for instance toward EU enlargement (de Vreese, Boomgaarden, and Semetko 2011). Thus, while there is evidence that media slant is correlated with public attitudes and experiments show that Eurosceptic frames and primes can have immediate effects in the direction of the information, it remains unclear whether sustained Eurosceptic media campaigns are able to consequentially influence public opinion in the long run.

The Case: Hillsborough and the Merseyside “Sun” Boycott

To study such sustained media effects in a real-world setting, we need to observe substantial and relatively sudden change in media landscapes. Yet, media landscapes are normally relatively stable and change slowly; on rare occasions media outlets enter or exit existing media markets, offering opportunities to study the effects of newspapers or TV channels on political outcomes (DellaVigna and Kaplan 2007; Gentzkow, Shapiro, and Sinkinson 2011). However, the decision to enter or exit a media market can be endogenous to political or economic considerations such as consumer demand or the lack thereof (Gentzkow, Shapiro, and Sinkinson 2011). Normally, media boycotts also follow a political logic. We overcome this endogeneity concern by relying on the case of the long-lasting “Merseyside Sun boycott,” which was caused by an exogeneous event, the Hillsborough sporting disaster involving Liverpool F.C., and the ensuing slanderous media coverage in The Sun tabloid, which led to a geographically restricted boycott of the paper.

On April 15, 1989, Liverpool F.C. played Nottingham Forest in the semifinals of the British Football Association (FA) Cup in the Hillsborough stadium. Due to overcrowding and an ensuing mass panic, 97 Liverpool supporters lost their lives, hundreds were injured, and thousands traumatized (Scration 2004; Wright 1993; Wright, Gaskell, and O’Muircheartaigh 1998). ² As pictured in Figure 1, The Sun’s sensationalist coverage of the disaster was particularly one-sided and falsely claimed that “the truth” about the disaster was that Liverpool supporters were responsible for the chaotic escalation, and ultimately, for their own death.

Figure 1.

The Sun’s Hillsborough and EU coverage

Sources: British Newspaper Archive (British Library) and Woodhouse, Cole, and Pettit (2016).

[Figure omitted. See PDF]

Based partly on false information by a South Yorkshire police inspector, The Sun claimed that Liverpool supporters had stolen from the dead as the disaster unfolded. According to The Sun’s source, one of the victims had “numerous wallets” on him, and was likely “one of the Liverpool pickpockets.” Despite what would turn into a commercial disaster for the paper, The Sun remained unapologetic. This stubbornness led to a boycott of The Sun in the Merseyside area. Soccer is very important to Liverpudlians’ identities and to the city of Liverpool (Alrababah et al. 2021). The Sun boycott was initiated by football supporters and their family and friends stopping purchasing the tabloid in protest of The Sun’s coverage, and soon the boycott was coordinated by the Hillsborough justice campaign (BBC 2012). The boycott has not only been supported by fans of Liverpool F.C., the most popular soccer club in Merseyside, but even supporters of Premier League rival Everton F.C. showed their solidarity with Liverpool supporters and the Hillsborough 97 and vouched never to buy The Sun again. Until today, this boycott is ongoing. In 2017 after speaking to several victims of the Hillsborough disaster, the club owners and the manager Jürgen Klopp decided to ban all “Sun” journalists from entering their stadium at Anfield Road and their training grounds (Conn 2017).

The Sun’s version of the Hillsborough disaster was comprehensively contradicted by multiple reports released by the official Hillsborough Independent Panel, which pointed to the catastrophic mishandling of the situation by the police (House of Commons 2012). The Guardian estimates that in the wake of the Hillsborough disaster, The Sun’s circulation fell from 55,000 copies to just around 12,000 copies in Merseyside, although these figures cannot be independently verified (Brook 2005).

Thus, to achieve valid and reliable information about the extent of the boycott in Merseyside, we conducted a telephone survey. More precisely, in January and February 2021 we surveyed the entire population of newsagents and cornershops located in Merseyside and two directly adjacent counties, Lancashire and Cheshire, listed on Yelp where a functioning telephone number could be located either on Yelp or Google Maps. ³ Newsagents were interviewed about newspaper sales, with no reference to Hillsborough or The Sun boycott. The results clearly confirm that The Sun boycott in Merseyside is widespread. Overall, 62% of newsagents’ in Merseyside reported not selling The Sun. This figure increases to 86% in the city of Liverpool. The survey also shows that the boycott is clearly geographically limited: 90% of newsagents in Lancashire and 88% of newsagents in Cheshire report selling The Sun. Asked about the number of copies of The Sun the newsagent sold on an average weekday, 34% of newsagents who did not sell The Sun mentioned the boycott unprompted. One newsagent went on to explain:

You’re in Liverpool mate, not sure if you’re a football fan, but no Scouser would dare read it. You might get the odd one or two in the city, maybe the odd tourist, but that’s about it. Ever since Hillsborough, nobody in this city touches it.

“It’s Liverpool, innit?” was a common theme when interviewing newsagents’ about how many copies of The Sun they sold. The top panel in Figure 2 displays the estimated number of newspaper sales on an average weekday per newsagent in Merseyside compared with adjacent counties. With an estimated six copies sold per newsagent per day, sales in Merseyside are only around one third of those in Cheshire and one fourth of those in Lancashire.

Figure 2.

Estimated Extent of the Boycott in Merseyside and Adjacent Counties

Note: Top: Average number of papers sold per newsagent per county (source: original newsagents telephone survey, N = 165 shops); whiskers show 95% confidence intervals. Bottom: Location of boycotting shops (source: https://totaleclipseofthesun.org/shops).

[Figure omitted. See PDF]

The bottom panel of Figure 2 then reports shops boycotting The Sun in Merseyside according to a webpage associated with the “Total Eclipse of the S*n”-campaign. Both independent data sources provide complementary information on the extent of the boycott and show that the boycott is strong across Merseyside—strongest in its core—and does not significantly extend to areas outside of Merseyside.

Even after The Sun’s unreserved apology in 2012 under the headline “Hillsborough: The real truth,” which admitted that their reporting on Hillsborough was the “gravest error” in the tabloid’s history and acknowledged that “the people of Liverpool may never forgive us for the injustice we did to them” (Moriarty, Veevers, and Dunn 2012), the boycott is ongoing to this day (Conn 2017). In fact, a strong social norm of boycotting The Sun developed in Merseyside. The norm extends not only to boycotting the newspaper itself but also to interviews by public figures such as Merseyside MPs and celebrities who are publicly sanctioned when breaking the norm (The Liverpool Echo 2011; Thorp 2018). Due to the UK’s reliance on newsagents for newspaper sales, before the advent of the Internet in the 2000s, norm violations were easy to police and the initial boycott easier to enforce. Three newsagents interviewed mentioned unprompted that they were not selling the paper because they were worried about customers’ reactions: “We stopped selling The Sun because we don’t want any trouble—people were coming in and complaining.” One newsagent reported that last time they tried selling The Sun, people were “throwing things.” There is also evidence that the boycott led to habit formation that lasted into the Internet age. To get a better understanding of how often people in Liverpool access The Sun online, we report Googletrends data (more information: Appendix A.1), which is available at the city level in England from 2004 until today.

Googletrends displays the relative amount of searches for a respective term, meaning that we can not know how many people in Liverpool searched for The Sun on Google but only the relative number in relation to the region where most people searched (Wolverhampton = 100%). In Liverpool we find by far the lowest search amount in any British city, with a 30% search share. Even in urban, cosmopolitan areas such as London, Oxford, or Cambridge the search amount is still around 50%. Overall, this suggests we have little reason to question that people from Merseyside bypass the boycott of The Sun or seek access to The Sun via the Internet, even today.

The occurrence of the Hillsborough disaster, thus, gives us the rare opportunity to identify the causal effect of a sustained media boycott on attitudes toward the EU because the circulation of the most important Eurosceptic tabloid was significantly reduced due to a sporting disaster, which was unrelated to the tabloid’s EU coverage. Readers did not choose to stop reading The Sun due to its EU coverage but due to a clear cause, which is exogenous to The Sun’s anti-EU campaign. Thus, our research design addresses the methodological issues of selection and reverse causation discussed above.

The Sun’s EU coverage

The Australian-born media mogul Rupert Murdoch bought the British newspaper The Sun in 1969 and turned it into a “red top” tabloid paper (Rooney 2000), directly competing for readers with the Daily Mirror, a title that dominated the tabloid market in the UK in the 1960s and 1970s. The British newspaper market can broadly be divided into two sections: “Quality papers,” often referred to as “broadsheets,” such as The Guardian and The Times, and “tabloids,” which are defined by their sensationalist coverage and eye-catching title pages. By the end of the 1970s, The Sun had overtaken The Mirror as the most widely read tabloid in the UK (Johansson 2007). In 1992, The Sun had a daily circulation of around 3.6 million and was read by around 10 million individuals (Rooney 2000). At the end of the 1980s, The Sun and The Mirror combined sold more copies than all other English newspapers taken together (Johansson 2007).

However, in contrast to its main competitor, The Mirror, The Sun took a strong negative editorial stance on the European Union during the 1990s and 2000s (see Figure A.4 and Figures A.5 in the Appendix for the number of articles and proportion of editorials that contain EU-specific search terms). As Nigel Farage commented in a recent BBC interview, “I think Murdoch helped to create an environment in which Brexit could happen with the Sun’s pretty constant Eurosceptic criticism” (Roberts 2020).

In Table 1 we display a comprehensive content analysis of a random sample of editorials published in The Sun and The Mirror that contained search terms associated with the “European Union.” ⁴ Between 1996 and 2016, the period for which editorials are available in the Factiva database, 92% of The Sun’s editorials displayed a negative tone toward European integration and 80% were classified as “Eurosceptic,” based on the definition by Vasilopoulou (2013) provided to coders (see Appendix A.3.1). This overwhelmingly negative slant of The Sun compares with the relatively neutral to positive slant that is visible in The Daily Mirror throughout the same period. Only 10% of The Mirror’s editorials were classified as “negative” in tone toward European integration by coders and only 6% could be classified as “Eurosceptic.” ⁵

Table 1. Editorial Slant of the Sun and the Mirror, 1996–2016

	The Sun	The Mirror	% agreement	Kappa	Pearson’s r
% EU	56.9	49.4	83.6	0.67
N	610	352	481	481
% negative	91.6	10.3	92.3	0.83
0–100 negativity scale	62.0	41.3			0.75
% Eurosceptic	80.1	6.3	76.9	0.55
N	347	174	221	221	200

While The Sun’s Online Archive only starts in 1996, in Appendix Figure A.8 we present qualitative evidence that The Sun’s Eurosceptic coverage was pronounced around the time of the Hillsborough disaster in the early 1990s when the paper opposed then Commission President Jacques Delors in his feud with British Prime Minister Margaret Thatcher about further European integration, British participation in monetary union, and British contributions to the EC budget. Now famous articles and editorials from 1989 and 1990, the period directly following the Hillsborough disaster, are shown in Figure 1 and A.8 in the Appendix.

Moreover, The Sun became famous for inventing and spreading so-called Euromyths, some of the most famous being that Brussels was trying to ban “bendy bananas” and regulate the shape of cucumbers (see Figure A.9 in the Appendix for a famous Sun story). All stories portray EU bureaucrats as out-of-touch and trying to impose unnecessary red tape on Britain. The persistence of Euromyths among the British public was a problem important enough that the European Commission dedicated an entire website to debunking these myths during the 2016 EU referendum campaign (Hobolt and Tilley 2014). While there are other Eurosceptic papers in the UK, notably the Daily Mail and the Daily Telegraph, The Sun has by far the highest circulation numbers. As Figure A.2 in the Appendix shows, in 1992 The Sun had around 3.6 million readers compared with the Mail’s 1.7 million readers and the Telegraph’s 1 million readers. Moreover, the class appeal of Eurosceptic papers in the UK is markedly different. The Sun’s readership is significantly more working class than the readership of other papers (Ladd and Lenz 2009; Rooney 2000; see also Figure A.15 in the Appendix). During most of the period we study, The Sun supported the Conservative party under Margaret Thatcher (PM from 1979–1990) and John Major (PM from 1990–1997). While the tabloid switched support to New Labour under Tony Blair and Gordon Brown from 1997 until the 2010 General Election, it remained steadfast in its Eurosceptic slant and anti-EU coverage throughout UK Labour’s last period in office (Ladd and Lenz 2009; Roberts 2020).

The Sun was a major media player in the EU referendum campaign of 2016 when it forcefully advocated for a “Leave” vote (see Figure A.7). A comprehensive analysis of EU referendum media coverage shows that The Sun was classified as one of only two papers in the UK (the other being the Daily Mail) that was both Eurosceptic in its positioning and had a large reach and volume, meaning it was read by a large proportion of the electorate (Levy, Aslan, and Bironzo 2016, 17). Importantly, The Sun published more than twice as many articles on the referendum than the second most important tabloid, the Mirror, which took a pro-EU stance (Levy, Aslan, and Bironzo 2016, 12). Emphasizing the papers’ perceived contribution to Brexit, the editor of The Sun, Tony Gallagher, texted the Guardian Newsroom on the morning after the referendum with the words “So much for the waning power of the print media” (Martinson 2016).

Research Design

The unexpected occurrence of the Hillsborough disaster allows us to estimate the causal effect of a widespread, but geographically restricted, boycott of the most important Eurosceptic tabloid newspaper in the UK, caused by a plausibly exogenous event, on attitudes toward leaving the EU. In our framework, all parliamentary constituencies that happen to be located within Merseyside county would be assigned to treatment, the tabloid boycott. Our main analyses are therefore “intent-to-treat” analyses. We conceptualize the boycott as a large number of shops refusing to sell the paper and a majority of citizens refusing to read The Sun. Both dynamics are strongly and positively correlated. In our data we do not observe evidence consistent with a displacement mechanism: shops in Merseyside that continue to sell The Sun do not sell more copies than shops in places where the boycott is not operational. In auxiliary analyses we therefore measure the extent of today’s boycott via two distinct but complementary proxies: the number of stores that participate in the boycott according to a list provided on the “Total Eclispe of the S*n” campaign website that we scraped from the Internet and geolocated within parliamentary constituencies ⁶ and based on the proportion of shops boycotting The Sun as recorded in our newsagents survey.

Given the strong anti-EU stance of The Sun, we expect that due to the boycott induced by the Hillsborough disaster, Euroscepticism should decrease in Merseyside compared with a counterfactual where Hillsborough would not have happened. To construct this credible counterfactual, we rely on a difference-in-differences design (DiD) (Angrist and Pischke 2009; Dunning 2012), assuming that Euroscepticism in Merseyside would have continued on a similar trend to the rest of Northern England were it not for The Sun boycott directly induced by the newspaper’s libellous reporting on the Hillsborough disaster.

Estimation

To estimate the DiD, we allocate respondents to the treatment group if they reside in parliamentary constituencies within Merseyside and to the control group if they reside in the remaining Northern English parliamentary constituencies. The reason that we define other Northern English constituencies as the control in our main specification is that they were subject to similar macroeconomic and socioeconomic trends over period that we study (Wilks-Heeg 2003). Regions such as London and the South of England had seen faster economic growth in the 1980s and 1990s than the North of England. Note, however, that our findings are robust to including all of England as a control group (Appendix A.7).

We apply a two-way fixed effects specification, which allows us to adjust for any time-constant differences across constituencies: (1) $$ \mathrm{leaving}\;{\mathrm{EU}}_{i,c,t}={\alpha}_c+{\gamma}_t+{\delta}_{\mathrm{DID}}{T}_{c,t}+{\varepsilon}_{i,c,t}, $$ where Leaving EU_i,c,t is respondent i’s support for leaving the EU in constituency _c at year _t ; α_c is a constituency fixed effect that rules out omitted variable bias from unobserved constituency characteristics that are invariant over our study period; γ_t are time fixed effects (year, quarter) that control for common factors changing across time; T_i,c,t is our binary treatment indicator, that equals 1 for Merseyside after the Hillsborough disaster; and $ {\varepsilon}_{i,c,t} $ is the error term. The term δ_DID is the estimand of interest, which identifies the effect of Hillsborough on Euroscepticism in Merseyside. Because the sampling frame of the survey is stratified by constituency, we cluster our standard errors at the constituency level. ⁷

As a robustness test, we use constituency-specific linear time trends instead of the simpler constituency and time fixed effects specification, outlined above. The model we estimate is as follows: (2) $$ {\displaystyle \begin{array}{l}\mathrm{leaving}\;{\mathrm{EU}}_{i,c,t}={\alpha}_c+{\gamma}_t+{\alpha}_c\times {\gamma}_t+{\delta}_{\mathrm{DID}}{T}_{c,t}+{\varepsilon}_{i,c,t}\end{array}}. $$ This allows treatment and control constituencies to follow different trends (Angrist and Pischke 2009, 238). Furthermore, we conduct analyses using alternative sample compositions (Appendices A.6 and A.7), investigate potential spillover effects to adjacent constituencies (Appendix A.8), conduct placebo tests (Appendix A.9), use matching techniques (Appendix A.10), and provide additional information and data that helps us appreciate the exclusion restriction to further substantiate the robustness of our empirical findings.

Our attitudinal analyses are based on the long-running and high-quality British Social Attitudes (BSA) Survey—an annual cross-sectional survey of public opinion toward social and political issues in Britain (National Centre for Social Reserach 2004). ⁸ We rely on the BSA Survey because it is the only survey in the UK that covers a sufficiently long period before and after the Hillsborough disaster and provides information about the location of the interview at the constituency level. We cover the years from 1983 to 2004, the last year in which parliamentary constituency identifiers are included in the BSA, allowing us to match respondents to the area in which they live. From 2005 onward, the BSA no longer includes any location identifiers smaller than the subregion. ⁹ Of the 35,204 BSA respondents, 908 live in one of the 15 parliamentary constituencies within Merseyside and were directly exposed to the “Sun” boycott. ¹⁰ We measure Euroscepticism by relying on the question asking respondents whether “Britain should continue its EEC/EU membership.” Our dependent variable Leaving EU is then coded 1 if respondents answered that Britain should withdraw from/leave the EEC/EU and 0 otherwise. ¹¹ As we use the BSA Euroscepticism measurement instrument over a 30-year period, concerns over measurement error linger. In Appendix A.4 we validate the measurement instrument by reporting a strong correlation between national trends in Euroscepticism as recorded in the BSA and the most frequently used measure based on Eurobarometer data.

Because we are dealing with repeated cross-sectional data, we control for respondents’ gender, age, education, ethnicity, self-reported social class, and party identification. As the BSA reports the interview dates for each respondent, we can directly identify which respondents were interviewed before and after the 19th of April 1989—the day The Sun published its first of several slanderous front pages on the Hillsborough Disaster.

Results

Figure 3 shows the results of the manipulation check, the effect of Hillsborough on self-reported print media consumption among respondents sampled in parliamentary constituencies located within Merseyside compared with that of respondents sampled from the remaining parliamentary constituencies located in the North of England. The Sun does not release circulation data at the county level or below, so we are unable to estimate the effects of the Hillsborough disaster on actual “Sun” readership in Merseyside. While self-reports can be a function of social desirability bias, in this case this would confirm the existence of a strong social norm against reading The Sun in Merseyside.

Figure 3.

Changes in Self-Reported Newspaper Readership in Merseyside and Control Counties Post- versus Pre-Hillsborough

Note: Control (top row) includes all Northern English counties except Merseyside; changes in predicted probabilities derived from multinomial logistic regression surrounded by 95% confidence intervals.

[Figure omitted. See PDF]

Figure 3 compares changes in self-reported newspaper readership in Merseyside post- versus pre-Hillsborough (row B) with change in readership in other Northern English counties (row A). The changes in predicted probabilities derived from the multinomial logistic model that is plotted in Figure 3 can be directly interpreted in percentage points, meaning that post-Hillsborough, “Sun” readership in Merseyside declined by 11 percentage points compared with no change (a precisely estimated 0 percentage points) in other Northern English counties. To be precise, self-reported Sun readership among respondents living in parliamentary constituencies within Merseyside decreased from 18 to 7%.

Note also that Figure 3 shows a substitution effect of “Sun” readership to pro-EU papers, coded as Daily Mirror, Independent, Guardian, the Financial Times, and The Times ¹² instead of substitution to anti-EU newspapers (Daily Mail, Daily Telegraph, Daily Star, and Daily Express). Crucially, The Daily Mirror becomes by far the most popular national paper in Merseyside post-Hillsborough. This is reflected in both the self-reported readership data and the newsagents survey. Based on the BSA, among newspaper readers, 26% report reading The Mirror. Readership of all other pro-EU papers combined is only 6%. Moreover, based on the newsagents survey we estimate that the Mirror sells an average of 34 copies per newsagent daily in Merseyside, whereas the number is half that figure in the adjacent counties of Cheshire and Lancashire. Substitution from The Sun to The Mirror is plausible because many readers consume The Sun based on its cultural appeal and sports coverage, which is most closely reflected by The Mirror, not The Daily Mail or The Daily Telegraph (Johansson 2007; Rooney 2000). As Rooney (2000, 107) writes, The Sun and The Mirror “both thought the same readership, they both eschewed the serious in favor of the nonserious. Central to their editorial agendas was the pursuit of the sensational.” While The Sun’s and The Mirror’s readership bases are mostly working class, the Mail’s and the Telegraph’s bases are (lower) middle class. While the DiD analyses that follow will not allow us to empirically distinguish between the effects of nonreadership of Eurosceptic material in The Sun and potential substitution with pro-EU material in The Mirror during the 1990s and early 2000s, which is the period covered by the public opinion data, in contrast to The Sun, The Mirror published very few editorials pertaining to European integration (Figure A.4 in the Appendix). In fact, The Mirror published only one editorial that included keywords relating to European integration in the years between 1996 and 2000 compared with The Sun, which published 62 editorials over the same period. Also, 92% of the “Sun” editorials showed a negative slant toward the EU. Therefore, we think that it is highly unlikely that the drop in Euroscepticism that we observe as a function of the boycott was in large part due to the Mirror’s pro-EU coverage as opposed to the absence of The Sun’s Eurosceptic content.

Figure 4 plots the percentage of respondents who support leaving the EEC/EU sampled from parliamentary constituencies within Merseyside and the control group consisting of respondents living in all remaining constituencies in the North of England, along with a dashed line projecting counterfactual trends in Merseyside. We estimate this counterfactual by relying on the trends in the control group to extrapolate the trends for Merseyside. Because sample sizes are small for some years, we always pool two successive years in this figure. Before the Hillsborough disaster, respondents’ opinions sampled within Merseyside constituencies followed parallel trends on Euroscepticism to opinions of respondents sampled in other Northern parliamentary constituencies. These parallel pretrends give credibility to the assumption that in the absence of the Hillsborough disaster, respondents in Merseyside would have followed opinion patterns in Euroscepticism similar to those in the rest of Northern England. There are many similarities between Merseyside and the remaining North of England (e.g., voting patterns). ¹³ Although we estimate that Merseyside was more Eurosceptic before Hillsborough, during the boycott we estimate that attitudes toward EU membership in the early 1990s became significantly more positive in Merseyside, compared with the counterfactual, and stayed more positive throughout the entire posttreatment period.

Figure 4.

Trends in Euroscepticism in Merseyside and Control Counties before and after Hillsborough

[Figure omitted. See PDF]

Table 2 reports the main findings of the formal DiD models. All models use the DiD strategy outlined above. The first model reports a simple bivariate comparison, and the following two models add fixed effects for constituencies, year (2) and months (3), respectively. Model (4) then adds all control variables listed above. Models (5) and (6) then introduce interactive fixed effects. Because some covariates are not recorded in the years before 1985 (e.g., education) the main findings are based on the years 1985–2004. Our findings are entirely robust to the inclusion of the entire time span (Appendix A.6).

Table 2. Difference-in-Differences Estimates of Hillsborough-Induced Sun Boycott on Euroscepticism

	(1)	(2)	(3)	(4)	(5)	(6)
	Support leaving the EU (0,1)
δ DiD	–0.070	–0.172	–0.170	–0.166	–0.120	–0.114
	(0.016)	(0.047)	(0.049)	(0.044)	(0.032)	(0.033)
Constant	0.212	0.219	0.219	0.231	0.215	0.220
	(0.007)	(0.003)	(0.003)	(0.028)	(0.002)	(0.029)
Constituency FE		✓	✓	✓	✓	✓
Year FE		✓	✓	✓	✓	✓
Quarter FE			✓	✓	✓	✓
Constituency FE × year					✓	✓
Constituency FE × quarter					✓	✓
Controls				✓		✓
Obs	10,384	10,384	10,384	10,384	10,384	10,384
Nconstituencies	172	172	172	172	172	172
Adj. R 2	0.00	0.05	0.05	0.08	0.05	0.08
Adj. R 2 (within)	0.00	0.00	0.00	0.03	0.00	0.03
RMSE	0.41	0.40	0.40	0.39	0.39	0.39

Note: Period: 1985–2004. Clustered standard errors by constituency. Controls: age, gender, education, religion, social class, party-ID. Constituency and time fixed effect estimates omitted from table.

Throughout all models in Table 2, we estimate a theoretically meaningful, large effect of Hillsborough on respondents’ attitudes toward leaving the EEC/EU in Merseyside. Depending on the models we estimate, the effects range from a 7- to 17-percentage-point decrease in Euroscepticism. Our preferred model, model 6, reports an 11-percentage-point decrease in Euroscepticism. Thus, we find a statistically significant and substantially meaningful decline in Euroscepticism due to The Sun boycott following Hillsborough. Note that effect sizes and significance levels are comparable across models 2–6 once we add more fixed effects, covariates, and constituency-specific time trends. It is important to note that most of the decline in Euroscepticism occurs between 1989 and 1994, exactly at the time when The Sun made its first big stand on European integration, vehemently opposing, ridiculing, and deriding then Commission President Jacques Delors and Britain’s adaption of the ECU, the predecessor of the EURO. Articles and editorials from the crucial 1989 and 1990 period are displayed in Appendix A.3.2.

The analyses presented here do not yet incorporate the extent of the boycott. To do so, we rely on the web-scraped information on boycotting shops along with our telephone survey of newsagents to estimate the extent of the boycott across English constituencies and counties. In Appendix A.13, we use these measures to reestimate our DiD model, first using the extent of the boycott as measured by the logged number of boycotting shops per constituency scraped from the “Eclispe of the S*un” website and, second, based on an instrumental variable approach. We use the number of boycotting shops per county as the endogenous variable and location within Merseyside post-Hillsborough as the exogenous instrument, and second we use the proportion of boycotting shops per county as estimated based on the newsagents survey as the endogenous variable. In both instances we distinguish between the city of Liverpool and other places in Merseyside to account for the essential geographical variation in the strength of the boycott. ¹⁴ Our findings are robust to both 2SLS specifications.

We also report further tests in the Appendix. First, we report the results using respondents sampled in all remaining English parliamentary constituencies as the control group. Reestimating the DiD models does not change our findings (Appendix A.7). Second, we investigate whether there are spillover effects into counties adjacent to Merseyside. Spillover effects could imply that we underestimate the effect of reading The Sun on Euroscepticism because in the models reported in Table 2 these constituencies are part of the control group. However, we do not find a pattern of effects that would be consistent with large spillover into adjacent areas (Appendix A.8). This is consistent with the results from our newsagents survey, which show that the boycott is geographically limited to Merseyside. Third, the negative effect of the boycott on Eurosceptic attitudes might, at least in part, be a function of an increase in “don’t know” responses in Merseyside and could mask a smaller movement away from “Leave.” Yet, we cannot reject the null hypothesis that there was no effect of the Sun boycott on “don’t know” responses to the Euroscepticism question. Appendix A.5 shows point estimates ranging from +1.6 to -4.6 percentage points, but none of these estimates is significantly different from zero and their size cannot account for the large media effects reported in Table 2. While we cannot identify individual-level mechanisms with repeated cross-sectional data, the observed patterns would be consistent with substantial movement from “Leave” to “Remain.” Fourth, the decrease of Euroscepticism might not be unique to Merseyside, but subject to a more general pattern across English constituencies. Therefore, we randomly reassigned the treatment into other parliamentary constituencies in England using a (permutation test) and find that the drop of Euroscepticism in parliamentary constituencies located within Merseyside is statistically distinct from the simulated sample of estimates (Appendix A.9). Fifth, we used matching on observables to address issues of comparability between treatment and control group across space and time. Again our findings remain robust (Appendix A.10). Sixth, we reestimate the DiD clustering at the county level and including county-level fixed effects. The results remain consistent (Appendix A.11).

Excludability

For our results to be valid, we need to assume that Hillsborough in April 1989 did not coincide with any other significant event or phenomenon of similar importance that differentially applied to Merseyside compared with other Northern English counties and could (to a large extent) explain the large decline in Euroscepticism in Merseyside that we observe compared with control areas. This assumption is often summed up as the “no compound treatment assumption.” There are three alternative developments, an increase in Labour Party support under Tony Blair during the 1990s and 2000s, an increase in EU structural funds to Merseyside, and deindustrialization due to globalization, that could potentially provide threats to inference. In what follows, we provide evidence that we think helps us appreciate the validity of the excludability assumption.

First, during the mid 1990s and early 2000s, the UK saw a decline in Conservative party support and a shift to the Labour Party, first lead by Neil Kinnock, then by the late John Smith, and from 1994 onward, by Tony Blair. Thanks to its industrial heritage and radical political tradition (including the Militant tendency), by the late 1980s Merseyside was a bastion of the UK Labour Party. Importantly, by 1983, the beginning of our time-series data, the Labour Party had decisively abandoned its Euroscepticism under its socialist leader Michael Foot (1979—1983), and both the pretreatment 1987 and the posttreatment 1992 Labour Party manifestos authored under Neil Kinnock’s leadership consistently committed Labour to staying in the EEC. Recall that for the DiD to be valid, trends need to be parallel, not levels. Consistent with the excludability assumption, a study of Merseyside politics by Wilks-Heeg (2018) finds that an increase in Labour Party support occurred in the 1980s, and then again after 2010. Beyond this qualitative illustration, we try to address any remaining concerns empirically in our matching approach in Appendix A.10, where we match respondents in treatment and control on party identification. We show that our results are robust to including party identification (which is plausibly endogenous) as one of the variables we match on.

Second, during the 1990s Merseyside was named a priority region for the receipt of EU structural funds (Objective 1 spending) by the UK government led by Conservative Prime Minister John Major. Thus, an increase in EU structural funds above what other Northern counties received could potentially provide an alternative explanation for our findings. However, Merseyside was only dedicated a priority region for EU structural funds from 1994 onward, for the 1994–1999 funding round (Di Cataldo 2016). Before that, for the crucial 1988–1993 funding round that directly coincides with Hillsborough, Merseyside benefited from structural funds equally with other Northern counties. Figure 4 clearly shows that the largest drop in Euroscepticism in Merseyside occurs in the early 1990s, not in the mid 1990s, which would have pointed to some influence of EU structural funds. This assumption is also consistent with a recent paper using a RDD to identify the effects of EU structural funds on Euroscepticism, which reports null effects (Schüssler 2019).

Third, previous scholarship contends that globalization shocks contribute to explaining the recent success of challenger parties (Autor, Dorn, and Hanson 2013) and the rise in Eurosceptic attitudes (Colantone and Stanig 2018). If Merseyside was affected differently by globalization at the exact time as The Sun boycott occurred, this might explain the decline of Euroscepticism. In this context it is important to note that Liverpool was worst hit by deindustrialization in the 1970s and the early 1980s (Wilks-Heeg 2003) and that the economic revival of the region did not start in the early 1990s. Liverpool’s population continued to decline (albeit at a lower pace) until 2001, and economic growth only returned in the 2000s (Wilks-Heeg 2003; Jones and Wilks-Heeg 2004). Moreover, in Appendix A.12 we merge the BSA data with information about the “offshorability” of respondents’ jobs—a valid approximation for individuals’ economic affectedness by globalization (see Mahutga, Curran, and Roberts 2018)— and control for “offshorability,” and our findings remain unaffected.

Mechanisms

To further understand which subpopulations drive our global finding, we rely on a set of difference-in-difference-in-differences (DiDiD) designs. Doing so allows us to better understand which mechanisms could potentially explain the main findings reported above.

First, we investigate how the boycott of The Sun affected generations that were socialized into politics while the boycott was ongoing. Research on political socialization posits that political identities tend to develop in younger age and frequently remain fairly stable across the life cycle (Dinas 2013). These “formative years” are described as a laboratory in which ideas are exchanged, challenged, dropped, and renewed. Applied to our case at hand, this might mean that people going through their formative years during the boycott (a) might be more likely to support the boycott and, thus, (b) be more supportive of the EU due to the absence of The Sun’s Eurosceptic slant. In contrast, both effects should be weaker for people who were past their formative years when the boycott started.

To test this idea we coded all respondents as “experiencing their formative years during the Sun boycott” if they were born after 1972. The reason for choosing this year as our cutoff is that these respondents were 16 or younger at the time the Merseyside region decided to boycott The Sun. We then use this dummy and interact it with the Merseyside indicator, such that our treatment group consists of respondents born after 1972 in Merseyside. Table 3 reports our findings for “Sun” readership and support for leaving the EU. Indeed, for the models we estimate, we find a stronger effect during the formative years. Respondents experiencing their formative years during the boycott are significantly less likely to read The Sun and to be Eurosceptic. This finding is even more telling if we keep in mind that according to our data, respondents born after 1972 are overall more likely to read The Sun and to be Eurosceptic outside of the Merseyside region. This further underpins our previous findings and adds the perspective that the boycott indeed might shape political attitudes in Merseyside, particularly for younger cohorts born after the boycott.

Table 3. DiDiD Results: Effect of The Sun Boycott Conditional on Respondents’ Birth Year

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	Sun readership (0,1)				Support leaving the EU (0,1)
Merseyside × born after 1972	-0.094	-0.080	-0.082	-0.085	-0.066	-0.065	-0.061	-0.060
	(0.021)	(0.024)	(0.025)	(0.026)	(0.033)	(0.030)	(0.030)	(0.029)
Constant	0.124	0.126	0.125	0.180	0.215	0.207	0.207	0.310
	(0.005)	(0.005)	(0.005)	(0.022)	(0.007)	(0.006)	(0.005)	(0.027)
Birth year FE		✓	✓	✓		✓	✓	✓
Study year FE			✓	✓			✓	✓
Controls				✓				✓
Obs	10,225	10,223	10,223	10,223	10,350	10,348	10,348	10,348
NConstituencies	172	172	172	172	172	172	172	172
Adj. R 2	0.00	0.00	0.00	0.04	0.00	0.02	0.05	0.07
Adj. R 2 (within)	0.00	0.00	0.00	0.04	0.00	-0.00	-0.00	0.02
RMSE	0.33	0.33	0.33	0.32	0.40	0.40	0.40	0.39

Note: Period: 1985–2004. Standard errors clustered by constituency in parentheses. Controls: gender, education, religion, social class, party-ID. Fixed effect estimates omitted from table.

Second, social class is an important predictor of whether a respondent reads The Sun. Therefore, the effects of the successful “Sun” boycott should be more pronounced among working-class respondents, with middle-class respondents (who were unlikely to read The Sun in the first place) acting as an additional control group within Merseyside.

We report our procedure in detail in Appendix A.15. We find that before the Hillsborough disaster, unskilled and semiskilled workers were much more likely to read The Sun than were skilled workers, persons in intermediate occupations, or professionals. Therefore, we should expect that the effect of The Sun boycott should be most pronounced among unskilled and semiskilled workers. Figure 5 reports the marginal effects of the three-way interaction between the DiD estimand and social class. We use three class indicators in our analysis: low (unemployed, unskilled, semiskilled), medium (skilled), and high (intermediate and professional). ¹⁵ The results of the DiDiD model vary as expected across social classes—we observe a very large and significant decrease of Euroscepticism for unskilled and semiskilled working-class respondents, whereas we estimate smaller effects for skilled workers and middle-class respondents that were far less likely to report reading The Sun before Hillsborough. Thus, the decrease of Euroscepticism in Merseyside after the Hillsborough disaster reported in the first part of our analyses appears to be driven by working-class respondents.

Figure 5.

DiDiD Results: Effect of The Sun Boycott Conditional on Social Class

Note: The figure displays the CATEs stemming from a DiDiD specification interacting the standard DiD estimand with BSA respondents’ self-reported social class (unskilled working class served as the baseline): never had job, unskilled, skilled working class: partly skilled, skilled, middle: intermediate, professionals). The point estimates are plotted (scatter) surrounded by 95% confidence intervals (whiskers).

[Figure omitted. See PDF]

2016 Referendum

As outlined in our case description, The Sun played an important role during the 2016 Brexit referendum campaign. Thus, in this section we look at the long-term effects of Hillsborough on the 2016 EU referendum. Figure 6 plots remain vote shares in the “Brexit” referendum across England.

As shown in Figure 6, the city of Liverpool voted significantly more “Remain” than did the rest of England. In Table 4 we use the same DiD identification strategy as in the previous analyses. We rely on a counting-area-level dataset of referendum vote shares in the 1975 EEC accession referendum and the 2016 Brexit referendum, which also includes time-invariant and time-variant socioeconomic and political control variables (Becker, Fetzer, and Novy 2017). As in all previous analyses, we use other Northern English counting areas as the control group. We code 2016 as the post-Hillsborough period and 1975 as the pre-Hillsborough period and again interact this variable with the Merseyside treatment area indicator. We estimate the following fixed effects model: (3) $$ {\displaystyle \begin{array}{l}\mathrm{Leave}\;{2016}_{i,t}={\alpha}_i+{\gamma}_t+{\delta}_{DID}{T}_{i,t}+\zeta^{\prime }{X}_i+{\varepsilon}_{i,t}\end{array}}, $$

Figure 6.

Remain Vote Share in the 2016 EU Referendum across England

[Figure omitted. See PDF]

Table 4. Difference-in-Differences: Effect of Hillsborough on 2016 Leave Vote Share

	(1)	(2)	(3)	(4)	(5)
	Leave vote share
δ DiD	-0.082	-0.083	-0.082	-0.088	-0.088
	(0.031)	(0.032)	(0.031)	(0.025)	(0.026)
Constant	0.453	0.453	0.453	0.601	0.446
	(0.005)	(0.005)	(0.002)	(0.052)	(0.051)
Merseyside FE	✓	✓		✓	✓
Region FE	✓			✓	✓
Year FE	✓		✓	✓	✓
Counting area FE			✓
Region × year FE		✓
Controls				✓	✓
Observations	102	102	102	102	102

Note: Standard errors in parentheses; model (5) uses bootstrapped standard errors. Omitted controls: shares of EU migrants, A10 migrants, non-EU migrants, financial sector employment, manufacturing sector employment, residents > 60, tertiary education, and median wage.

where Leave 2016_i,t is the share of the “Leave” (2016) or “No” (1975) vote in the EU referendums in counting area i in year t, α_i is a region or counting area fixed effect (depending on specification) that rules out omitted-variable bias from unobserved region/counting area characteristics that are invariant over our study period, γ_t is a fixed effect that accounts for common factors that change between 1975 and 2016, X is a matrix of time-variant counting area covariates collected in 2001 and 2011 by Becker, Fetzer, and Novy (2017), T_i,t is our binary treatment indicator (Merseyside in 2016 after the Hillsborough disaster), and $ {\varepsilon}_{i,t} $ is the error term. The term δ_DID is the estimand of interest that identifies the effect of Hillsborough on the Eurosceptic vote share in Merseyside. We cluster our standard errors at the counting-area level. All details about the data and statistical analysis can be found in the Materials and Methods section in Appendix A.16.

Table 4 shows that counting areas located in Merseyside are consistently estimated to be around 8–9 percentage points less “Leave” voting in the 2016 EU referendum than other Northern English counting areas, compared with the 1975 referendum. This effect is robust to the inclusion of time-variant control variables related to EU/Non-EU migration as well as economic and sociodemographic variables reflecting changes in age and educational composition that are strongly correlated with Brexit vote shares (Table 4, columns 4–5, and Table A.13 in the Appendix). These results indicate that the consistently more pro-European attitudes in Merseyside as a result of The Sun boycott also translated into higher “Remain” vote shares in the Brexit referendum.

Discussion and Conclusion

Evidence from an exogenously induced boycott of the most important Eurosceptic newspaper—The Sun—in Merseyside as a consequence of the paper’s reporting on the Hillsborough sporting disaster—caused a decrease of Euroscepticism in Merseyside. Consistent with our hypothesis that the decline in Euroscepticism in Merseyside post-1989 was driven by The Sun boycott, treatment effects are stronger among generations that came politically of age during the boycott and among unskilled and semiskilled workers, the social group that was most likely to read The Sun before Hillsborough. The effects we identify cannot be explained by differential trends in party preferences between Merseyside and other Northern English counties, nor are they likely to have resulted from differential access to EU structural funds or differential effects of globalization on offshorability.

Therefore, our study makes an important contribution to our understanding of the long-term effects of sustained media campaigns on public opinion. The study of over-time processes and temporal dynamics in media persuasion research is “in need of considerably more theoretical and empirical attention” (Leeper and Slothuus 2020), and instances that allow for such study are rare. Besides our contribution to the over-time study of media effects, we also address another related challenge of research on the media and public opinion—namely, that most studies can only look at the effects of a specific article or news story rather than the effects of an entire media campaign over an extended period. As former Conservative Chancellor Michael Heseltine said in a recent BBC interview speaking about the Sun’s anti-EU campaign, “That drip drip drip of European poison decade after decade had a most unfortunate effect on British public opinion” (Roberts 2020). No matter where one stands on the normative question, it is not inconceivable that as a discipline we perceive media effects to be small because we usually study small, one-off, interventions.

To what extent are the results generalizable to other contexts beyond the Hillsborough case, given that our estimates are local to Merseyside and the North of England? While generalizability must ultimately remain an empirical question, in this paper we clearly outline the scope conditions that we think need to hold for media campaigns to result in large and lasting opinion change. In highly saturated and polarized campaign environments (Kalla and Broockman 2018), on highly salient issues, and where strong counterframes are available (Chong and Druckman 2007), we would expect the media to have minimal or, at best, small effects. We would expect substantively large effects where political contexts allow for sustained and one-sided media coverage on emerging issues. Moreover, we think that tabloid media outlets can be particularly influential if they decide to take a stance on politics because readers are likely to select into consumption for entertainment purposes (Arceneaux and Johnson 2013). Thus, tabloids overcome the problem that those individuals who should be most likely to receive persuasive content are least likely to change their mind because they are highly attentive to politics in the first place (Zaller 1992).

One counterfactual worth considering when thinking about the generalizability of our results is whether The Sun boycott would have produced equally large effects had it happened for instance in Hampshire, in the South of England, as opposed to Merseyside, in the North of England. One key question is whether former “Sun” readers would have substituted their media diets with equally Eurosceptic papers or whether they would have followed Merseyside’s lead on substituting a Eurosceptic with a pro-EU tabloid. Because we think that substitution occurred based on cultural, not political preferences (Rooney 2000), we would expect that we should have seen similar effects in places that have a similar class composition of newspaper readership.

While instances that allow for the causal study of a sustained one-sided media campaign are rare (Leeper and Slothuus 2020), that does not mean that sustained one-sided media campaigns are rare per se. Besides the Murdoch-owned Eurosceptic tabloid press in the United Kingdom, important cases that immediately come to mind are the emergence of another Murdoch-owned medium, Fox News, in the United States (DellaVigna and Kaplan 2007), Berlusconi-owned TV channels in Italy (Mastrorocco and Minale 2018), an Adelson-owned newspaper in Israel (Grossman, Margalit, and Mitts 2020), and Blocher-owned newspapers in Switzerland (Spirig 2020). These outlets all push clear political agendas over a sustained period—they raise the salience of specific issues and provide a strong frame with which to interpret them. Whether these “issue entrepreneurs” (de Vries and Hobolt 2020) are ultimately successful also depends on whether other actors engage in time and counter the frames provided. As George Osborne, the former chancellor and one of the key government figures in the “Remain” campaign wrote in 2018, “We were too late in the day trying to explain some of the benefits of EU membership” (BBC 2018).

Beyond adding to our understanding of media effects on public opinion in general, what does this study tell us about the rise of Euroscepticism and the 2016 Brexit vote in particular? Although our main results focus on the effects of the boycott on public opinion toward leaving the EU, official counting area data from the 1975 and 2016 EU referendums also suggest that The Sun boycott might have decreased the “Leave” vote share in Merseyside in the 2016 EU referendum. Because this auxiliary analysis covers a longer period, we account for potential time-variant confounders such as changes in education levels and EU migration. Combining the analysis of the public opinion data spanning 30 years and the data on the referendum votes in 1975 and 2016 in a DiD framework, our study indicates that sustained media campaigns on emerging issues can have large, lasting, and ultimately consequential effects on public opinion and public policy.

In addition to the well-researched economic and social factors that have contributed to Eurosceptic opinion and to the Brexit vote in 2016 (Becker, Fetzer, and Novy 2017; de Vries 2018), we show that, at least in part, public opinion and policy-relevant public decisions were endogenous to tabloid media reporting. While this result may not be counterintuitive or come as a big surprise to those who follow British politics, it is one thing to suspect that a causal relationship exists and another to provide empirical evidence consistent with this assumption. Moreover, our article shows that the decline of Euroscepticism in Merseyside following The Sun boycott largely reflects a decrease in Euroscepticism among unskilled and semiskilled working-class voters, who made up a large share of Sun readers before the disaster. This finding is consistent with Ladd and Lenz’s (2009) study of the effects of the Sun’s endorsement switch from Conservatives to Labour at the 1997 General Election, where persuasive effects were also more pronounced among working-class “Sun” readers. By documenting the role of the tabloid media in affecting attitude change toward European integration among working-class voters, we highlight an important nonstructural factor that contributed to the increase in Euroscepticism. This does not mean that EU immigration, regional inequalities, or austerity did not matter or that The Sun caused Brexit, but this study provides evidence that the tabloid press played an important role in shaping support for “Leave” among working-class voters.

Supplementary Materials

To view supplementary material for this article, please visit http://doi.org/10.1017/S000305542100085X.

Data Availability Statement

Research documentation and code that support the findings of this study are openly available at the American Political Science Review Dataverse: https://doi.org/10.7910/DVN/NYPOQD. Limitations on data availability are discussed in the text.

Acknowledgments

Both authors contributed equally to this paper; the order of the authors’ names reflects the principle of rotation. We thank our research assistants in Liverpool, Steven Daniels and Tom Laing, who implemented the telephone survey and coded the editorials, and our RAs at the LSE and the University of Zurich, Ceren Cinar and Natalia Podany. We are also particularly grateful for the constant help and support from Stuart Wilks-Heeg at the University of Liverpool. For comments and suggestions we thank Lucy Barnes, Alex Coppock, Torun Dewan, Elias Dinas, Gemma Dipoppa, Andy Eggers, Katjana Gattermann, Guy Grossman, Dominik Hangartner, Simon Hix, Sara Hobolt, Josh Kalla, Lucas Leemann, Arndt Leininger, Moritz Marbach, Judith Spirig, Florian Stoeckel, Christian Traxler, Elisa Volpi, Markus Wagner, and Catherine de Vries. We also benefited from feedback during the 2018 Conference of the ECPR Standing Group on the European Union; the 2019 APSA, 2018 MPSA, and 2018 EPSA annual conferences; invited talks at the Hertie School; the LSE’s Methodology Department; the University of Konstanz; the University of Geneva; the POLECONUK seminar; and the publication seminar at the Department of Political Science at the University of Zurich. Thomas Willi helped us to revise the visualization of Figure 2 (map); thanks, Thomas. Finally, we thank three anonymous reviewers and the editors for their helpful feedback, which improved the paper tremendously.

Funding Statement

Daniel thankfully acknowledges funding from the Swiss National Science Foundation (SNF Ambizione Grant, NO. 179938). We are thankful for financial support during the revisions from the Government Department at the LSE and the Department of Political Science at the University of Zurich. The original newsagent telephone survey was conducted with funds from the London School of Economics and Political Science and the University of Zurich.

Conflict of Interest

The authors declare no ethical issues or conflicts of interest in this research.

Ethical Standards

The authors declare the human subjects research in this article was reviewed and approved by the Research Ethics Committee of the London School of Economics and Political Science and certificate numbers are provided in the appendix. The authors affirm that this article adheres to the APSA’s Principles and Guidance on Human Subjects Research.

¹ Initially, 96 people died in the disaster. This number was recently updated to 97 (Conn and Vinter, 2021).

² Half an hour before the scheduled start of the match, a large crowd of predominantly Liverpool F.C. supporters gathered in front of the stadium. Police officers decided to open the exit gates to let supporters in. This uncontrolled in-streaming of ever more people led to the overcrowding of the side pens (Jemphrey and Berrington 2000).

³ The response rate to the telephone survey was 25% for an overall sample of N = 165. More details can be found in Appendix A.1.

⁴ The search terms used were “EU,” “European Union,” “European Community,” “European Commission,” “Brussels,” “EURO,” and “Europe,” excluding terms associated with sports competitions, fashion, music, and travel.

⁵ A detailed discussion of the methodology, the intercoder reliability of the measures and instructions given to coders are available in Appendix A.3.1. Due to national lockdown restrictions due of COVID-19, we were unable to extend the content analysis to the years from 1983–1995. The Sun’s online archive is only available from 1996 onward and the British Library, which has archived all editions of The Sun on microfilm in London, was closed during two national lockdowns in England.

⁶ According to the website, there are shops that participate in the boycott and are not located in Merseyside. We also geolocate these shops in their corresponding parliamentary constituencies. We use this measure as an alternative specification of the treatment in our DiD framework (see Appendix Table A.11) and as the endogenous variable in a 2SLS regression reported in Appendix Table A.12.

⁷ Clustering can follow two types of logic: sampling. or assignment (Abadie et al. 2017). The BSA followed a two-stage sampling process, where a subset of constituencies was randomly sampled from the population of all constituencies in the first stage and respondents were sampled randomly from the sampled constituencies in the second stage. Thus, there are constituencies in the population that we do not see in the sample and we should therefore cluster at the constituency level. Clustering can also be driven by design considerations: one might argue that the treatment is assigned at the county level and that we should therefore cluster our standard errors at that level. We do so in Appendix A.11. Our conclusions remain unaffected by this decision. Because we have fewer than 50 counties in our data, standard errors are biased downward when clustering at the county level. We address this issue by applying a wild bootstrap (Esarey and Menger 2019).

⁸ The yearly BSA survey data are available from the UK Data Service: https://beta.ukdataservice.ac.uk/datacatalogue/series/series?id=200006. NatCen who conduct the BSA do not allow researchers to deposit replication datasets. On the APSR Dataverse (https://doi.org/10.7910/DVN/NYPOQD) we provide the complete code required to build the dataset for analysis, based on the original datasets stored with the UKDS, and to reproduce all tables and figures in the text, appendices, and online supplementary information. We also provide the full replication data and code to reproduce all other analyses in the paper.

⁹ We also considered using Eurobarometer data (EB), the British Household Panel Study (BHPS), and the British Election Study (BES). While providing excellent information on Euroscepticism, the EB does not provide any geolocations prior to 1990 and the BHPS only started in 1991. The BES 1987–1992 panel study would allow us to investigate two years, 1987 and 1992, but there are only 46 respondents interviewed in Merseyside post-Hillsborough. All of these shortcomings make it either impossible to use these data sources (EB, BHPS) or less useful than the BSA (BES).

¹⁰ Several covariates have missing values for respondents. We mean-impute all missing values in the analyses reported below; our findings are robust to excluding all missing observations (see Online Supplementary Information section S.2).

¹¹ From 1983—1992, 0 comprises “continue” and “don’t know.” From 1993 onward, the BSA introduced six answer categories to the same question: “uk leave ec” (coded as 1), “stay+reduce ec power,” “leave as is,” “stay+incr.ec power,” “single ec govt,” and “don’t know” (all coded as 0). Because this change in the measurement instrument does not coincide with the treatment and happens in both treatment and control areas, it should not violate the exclusion restriction. All results are robust to excluding the 1993–2004 period and to an alternative specification of the dependent variable where “stay+reduce ec power” is also coded as 1.

¹² There is some ambiguity about The Times’s position. The Times endorsed “Remain” in the 2016 Brexit referendum. Reclassifying The Times as a Eurosceptic paper does not change the results.

¹³ In Appendix A.7 we specify all remaining English constituencies as the control group. Although the pretrends diverge slightly between 1983 and 1984 in this specification, our findings remain entirely robust.

¹⁴ Since we are unable to go back in time, we only hold information about the extent of the boycott as of today. For our 2SLS specification, we need to make the assumption that the boycott began in 1989 and that the geographic variation of the boycott (between Liverpool and other places in Merseyside, and between Merseyside and other counties) was stable over time. While this corresponds to an approximation, we think that the relative strength of the boycott across counties is unlikely to have differed systematically.

¹⁵ To ease interpretation of these findings, we rely here on a simpler DiD strategy. We interact a Merseyside dummy with a post-Hillsborough indicator and the social-class identifier instead of relying on a two-way fixed effects estimation on the constituency level as outlined above. This allows us to plot the pre–post estimates separately for Merseyside and the rest of the country. The interpretation of the findings remains unaffected by this simpler strategy.