Jonathan J. Pierce, Seattle University
Holly L. Peterson, Oregon State University
Michael D. Jones, Oregon State University
Samantha Garrard, Seattle University
Theresa Vu, Seattle University
To better understand how the Advocacy Coalition Framework (ACF) is applied, this article catalogues and analyzes 161 applications of the ACF from 2007 to 2014. Building on a previous review of 80 applications of the ACF (1987–2006) conducted by Weible, Sabatier, and McQueen in 2009, this review examines both the breadth and depth of the framework. In terms of breadth, there are over 130 unique first authors from 25 countries, in almost 100 journals applying the framework, including a majority outside of the United States. In terms of depth, a plurality of applications analyzes environment and energy, subsystems at the national level, and utilizes qualitative methods of data collection and analyses. This review also explores how the three theoretical foci of the framework—advocacy coalitions, policy change, and policy-oriented learning—are applied. Our findings suggest that the ACF balances common approaches for applying the framework with the specificity of particular contexts.
policy process, public policy, literature review, policy change, learning
Paul Sabatier in “The Need for Better Theories” (1999), argued that theories of the policy process should seek to meet the following criteria: (1) have concepts that are internally valid, include causal mechanisms, falsifiable hypotheses, and be broad in scope; (2) be subject to empirical testing that may lead to conceptual and theoretical development; (3) seek to explain much of the policy process and have normative elements; and (4) address both actors and institutions. Toward this end, Sabatier’s edited 1999 volume—Theories of the Policy Process—presented multiple frameworks and theories of the policy process that aspire to achieve these lofty expectations. One such framework within this now classic text is the Advocacy Coalition Framework (ACF) developed by Paul Sabatier and Hank Jenkins-Smith.
For at least the past two decades, the ACF’s emblematic concepts have been a staple of both policy process scholarship and policy focused graduate programs around the world. These concepts include policy subsystems, advocacy coalitions, belief systems, and policy-oriented learning. Along the way, frequent assessments of the framework have described and updated these concepts, discussed the proliferation of applications, and charted a path for future research (Jenkins-Smith, Nohrstedt, Weible, & Sabatier, 2014; Sabatier & Jenkins-Smith, 1999; Sabatier & Weible, 2007; Weible, Sabatier, & Flowers, 2008; Weible, Sabatier, & McQueen, 2009, Weible, Siddiki, & Pierce, 2011). These assessments cite archetypal examples and present an aspirational framework providing guidance to scholars on how to understand and apply the ACF in a manner that is likely to achieve Sabatier’s criteria. However, do applications of the ACF meet the aspirations and guidelines of the framework? The only previous study to address this question using a comprehensive systematic approach was Weible et al. (2009), which analyzed 80 applications of the ACF from 1986 to 2006. It found that many applications fall short of Sabatier’s lofty goals. Since 2007, the ACF has developed new concepts and has been cited over 1,000 times. Therefore, one purpose of this article is to address the question of whether applications of the ACF are meeting the aspirational standards of the framework.
This article describes how the ACF has been applied from 2007 to 2014 and analyzes how these applications compare to the initial aspirations of the framework. In doing so, however, analyses are also preeminently concerned with a recent ambition articulated by Jenkins-Smith, Nohrstedt, et al. (2014) that the ACF should seek balance between generalizable applications of the framework and problem-focused applications relevant to particular contexts (p. 207)—an ambition that merges two goals frequently at odds with one another. Thus, a goal of this research is also to assess how ACF scholarship, in the aggregate, balances these goals of meeting general and aspirational guidelines while still addressing diverse phenomenon in specific contexts.
The ACF is designed to guide policy research by providing a common language and focusing on relevant analytical components and relationships within a policy subsystem (Weible, Sabatier, et al., 2011). The framework is especially helpful in explaining public policy during contentious processes that may involve substantial conflicts over goals and technical and scientific information (Pierce & Weible, 2016). It has been revised multiple times with the most prominent revisions occurring in edited volumes of the Theories of the Policy Process edited by Paul Sabatier and, more recently, by Christopher Weible (e.g., Jenkins-Smith, Nohrstedt, et al., 2014; Sabatier & Jenkins-Smith, 1999; Sabatier & Weible, 2007).
Figure 1. Flow Diagram of the Advocacy Coalition Framework.
Source: Jenkins-Smith, Nohrstedt, et al. (2014).
Figure 1 provides a flow diagram of the ACF. In this figure, coalitions compete within a policy subsystem to translate their beliefs into policies. They use strategies to influence government authorities, which eventually influence policy. Coalition beliefs and actions are impacted by long- and short-term opportunities, constraints, and resources that are affected by both relatively stable parameters and external sub-system events. The impacts of government decisions feed back into the subsystem, and also may affect factors external to the subsystem (see Jenkins-Smith, Nohrstedt, et al., 2014, for a further description).
For the ACF, the primary unit of analysis is the policy subsystem. A policy sub-system is comprised of all relevant actors trying to influence policy and politics involved in a bounded geographic area and/or authority or potential authority in relation to a specific policy issue (Jenkins-Smith, Nohrstedt, et al., 2014). Policy sub-systems may be nested either vertically through levels of government, or horizontally across different jurisdictions or topical issues (Sabatier & Jenkins-Smith, 1993). A strategy for studying policy actors in a policy subsystem is to organize them into advocacy coalitions based on their policy core beliefs (Weible & Nohrstedt, 2012). Policy actors are boundedly and instrumentally rational individuals that use belief systems or sets of abstract patterns and causal relationships to make sense of the world (Jenkins-Smith, Nohrstedt, et al., 2014). In addition, based on prospect theory, the ACF assumes that people remember losses more than gains (Quattrone & Tversky, 1988). This means that individuals are susceptible to a “devil shift,” whereby they overestimate both the power and the malice of opponents.
Policies are projections of the beliefs and subsequent behavior of coalitions and their members (Lasswell & Kaplan, 1950). Last, the ACF suggests a time period of at least a decade to understand policy process and change (Weible & Nohrstedt, 2012). For the ACF, public policy in many ways is the translation of the winning coalition’s beliefs (Jenkins-Smith, Nohrstedt, et al., 2014).
Coalition Hypothesis 1—Allies and Opponents: On major controversies within a policy subsystem when policy core beliefs are in dispute, the lineup of allies and opponents tends to be rather stable over periods of a decade or so.
Coalition Hypothesis 2—Policy Core Beliefs: Actors within an advocacy coalition will show substantial consensus on issues pertaining to the policy core, although less so on secondary aspects.
Coalition Hypothesis 3—Secondary Beliefs: An actor (or coalition) will give up secondary aspects of her (its) belief system before acknowledging weaknesses in the policy core.
Coalition Hypothesis 4—Official Policy Actors: Within a coalition, administrative agencies will usually advocate more moderate positions than their interest group allies.
Coalition Hypothesis 5—Unofficial Policy Actors: Actors within purposive groups are more constrained in their expression of beliefs and policy positions than actors from material groups.
Policy-Oriented Learning. Policy-oriented learning occurs when policy actors consider alternative forms of beliefs associated with obtaining a goal (Sabatier & Jenkins-Smith, 1993). Alternatives in beliefs might refer to new problem definitions, policy solutions, or strategies for influencing government decisions (Jenkins-Smith, Nohrstedt, et al., 2014). Research regarding policy-oriented learning might focus on identification of learning among or between coalitions, policy brokers (policy actors operating between coalitions), or belief change. ACF hypotheses related to policy-oriented learning include (Jenkins-Smith, Nohrstedt, et al., 2014, pp. 199–200):
Learning Hypothesis 1—Learning Across Coalitions: Policy-oriented learning across belief systems is most likely when there is an intermediate level of informed conflict between the two coalitions. This requires that (1) each have the technical resources to engage in debate, and (2) the conflict be between secondary aspects of one belief system and core elements of the other or, alternatively, between important secondary aspects of the two belief systems.
Learning Hypothesis 2—Learning Professional Forums: Policy-oriented learning across belief systems is most likely when there exists a forum that is (1) prestigious enough to force professionals from different coalitions to participate, and (2) dominated by professional norms.
Learning Hypothesis 3—Quantitative Learning: Problems for which accepted quantitative data and theory exist are more conducive to policy-oriented learning across belief systems than those in which data and theory are generally qualitative, quite subjective, or altogether lacking.
Learning Hypothesis 4—Normative Learning: Problems involving natural systems are more conducive to policy-oriented learning across belief systems than those involving purely social or political systems because, in the former, many of the critical variables are not themselves active strategists and because controlled experimentation is more feasible.
Learning Hypothesis 5—Technical Information: Even when the accumulation of technical information does not change the views of the opposing coalition, it can have important impacts on policy—at least in the short run—by altering the views of policy brokers.
Policy Change. Public policies are the translations of beliefs of past winners of policy processes (Pierce & Weible, 2016). Therefore, policies can be analyzed in terms of belief systems. The ACF associates major policy change with changes in policy core beliefs and minor policy change with changes in secondary beliefs (Sabatier & Jenkins-Smith, 1999). The framework identifies four major pathways to policy change within the policy subsystem: external shocks (Sabatier & Weible, 2007), internal subsystem events (Sabatier & Weible, 2007), policy-oriented learning (Jenkins-Smith, Nohrstedt, et al., 2014), and negotiated agreements (Sabatier & Weible, 2007). Research focusing on policy change often targets the type of policy change and pathway. A fifth alternative form of policy change may occur if there is a change in the governing coalition or an imposition of a policy change by a superior authority. Hypotheses for policy change include:
Policy Change Hypothesis 1—Bottom-up Policy Change: Significant perturbations external to the subsystem, a significant perturbation internal to the subsystem, policy-oriented learning, negotiated agreement, or some combination thereof are necessary, but not sufficient, sources of change in the policy core attributes of a governmental program (Weible & Nohrstedt, 2012, p. 133).
Policy Change Hypothesis 2—Top-down Policy Change: The policy core attributes of a government program in a specific jurisdiction will not be significantly revised as long as the subsystem advocacy coalition that instated the program remains in power within that jurisdiction—except when the change is imposed by a hierarchically superior jurisdiction (Jenkins-Smith, Nohrstedt, et al., 2014, pp. 203–4).
The first step in our review process was to produce a list of peer-reviewed jour- nal articles. We utilized the Web of Science database to create a list of peer-reviewed journal articles in English that cite at least one of the following six ACF origin and revision publications: Paul Sabatier, Journal of Public Policy (1986); Paul Sabatier, Policy Sciences (1988); Paul Sabatier and Hank Jenkins-Smith (eds), Policy Change and Learning: An Advocacy Coalition Approach (1993); Paul Sabatier, Journal of European Public Policy (1998); Paul Sabatier and Hank Jenkins-Smith, “An Advocacy Coalition Framework: An Assessment” in Theories of the Policy Process (1999); Paul Sabatier and Christopher Weible, “The Advocacy Coalition Framework: Innovations and Clarifications” in Theories of the Policy Process, Second Edition (2007). These six journal articles, book, and book chapters were utilized because they establish the theoretical basis and development of the ACF. They describe the assumptions, scope, and hypotheses of the framework. A previous study of ACF applications by Weible et al. (2009) examined peer-reviewed journal articles, books, and book chapters from 1987 to 2006 (n580) and was the starting point for this research; however, this study methodologically departs from Weible et al. (2009), who utilized previous lists of applications found in Sabatier and Jenkins-Smith (1999) and Sabatier and Weible (2007), as well as searches for key words on Web of Science and Google Scholar.
This review’s search criterion deviates from the aforementioned study for a few reasons. First, this search does not use the lists of past applications by Sabatier and Jenkins-Smith (1999) and Sabatier and Weible (2007) as these are prior to this study’s range of interest, 2007–2014, and the list of applications in Jenkins-Smith, Nohrstedt, et al. (2014) was published after this data set was collected. Second, searches of Web of Science were used instead of Google Scholar to avoid data collection errors associated with Google Scholar’s duplicate citation listings, its limit on results returned from a single search, and the search engine’s restrictions regarding accessing numerous search results in a single session. Third, using Web of Science’s Cited Reference Search rather than keywords provided the advantage of a systematic search algorithm that produced replicable results. Lastly, the search criteria were limited to peer-reviewed journal articles for three reasons: to establish a bound of quality agreed upon within the field, to increase comparability across cases, and practical reasons related to conducting content analysis on hundreds of documents such as ease of document sharing and limits on document length. The search criteria included only English language peer-reviewed journal articles between 2007 and 2014. This initial search resulted in a total of 1,067 articles. Not every application of the ACF between 2007 and 2014 is captured here; rather, this design attempted to include as many applications as possible given these search parameters.
Content analysis was conducted on the articles in two rounds. First, five coders recorded the bibliographic information of each article. This included 10 identification codes such as title, author, journal name, etc. Four codes were utilized to differentiate between applications and those articles that only cited one of the six theoretical foundational documents. These codes include: (1) the frequency that the keywords “coalition,” “learn,” or “advocacy” occur in the title, abstract, and the (2) frequency that the six theoretical foundational documents are cited. Only articles with two or more of the keywords or two or more citations in the text were included. This process led to the removal of about half of the articles, leaving 512 potential applications.
Using the frequency of keywords and citations alone can lead to Type I errors if all 512 articles are used. Thus, the following additional criterion was applied to determine if an article is an application: (3) data and/or a case study, (4) be about a topic, and (5) analyzes one or more theoretical component of the ACF (coalitions, policy change, and/or policy-oriented learning). For example, Albright (2011) and Ansell, Reckhow, and Kelly (2009) were included as applications because they fit the following criteria: (1) multiple use of keywords “advocacy” and “coalition”; (2) multiple citations of theoretical foundation documents; (3) case study and/or data analyzed; (4) analyze specific topics; and (5) Ansell et al. (2009) analyze advocacy coalitions, and Albright (2011) analyzes policy change. Conversely, Alcantara, Lerone, and Spicer (2012) and Jenkins-Smith, Silva, Gupta, and Ripberger (2014) were not included as applications. Both journal articles meet the first two criteria of keywords and citations, but did not meet either of the additional criteria. Alcantara et al. (2012) examine variation in implementation using learning as a variable of interest, but do not examine policy-oriented learning from an ACF perspective. Jenkins-Smith, Silva, et al. (2014) propose that cultural theory should be integrated into the ACF to explain hierarchical belief systems, but do not include a case study or analyze data, and was not about a topic. Using this subjective process, we ended up with 161 articles identified as applications. To mitigate subjectivity, inter-coder reliability assessments for this coding were acceptable with more than 50 percent of a random sample of articles being reviewed by an inter-coder.1
All the 161 applications are referenced in this article through in-text citations and listed in the references. In the second round of coding, seven coders conducted content analysis. The entire codebook is available in the Supporting Information. This codebook analyzes the articles for scope, purpose, methods, coalitions, learning, policy change, and additional notes. Overall, the codebook includes 15 codes in the first round and 53 codes in the second round for a total of 68 codes.
To ensure reliable results, the codebook uses specific wording to minimize interpretation and binary coding for presence. To determine inter-coder reliability, a random number generator was utilized and 87/161 articles (54 percent) were coded by two coders. This sample is sufficient to determine inter-coder reliability at a 95 percent confidence level with 6 percent confidence interval. All codes showed greater than or equal to 80 percent agreement, which is considered reliable (Lacy & Riffe, 1996), and a Cohen’s Kappa produced a score of 0.40 or greater for 41 nominal codes (considered a moderate level of agreement—see Landis & Koch, 1977). Therefore, based on both percentage agreement and Cohen’s Kappa, these codes achieve acceptable levels of inter-coder reliability.
The following section presents content analysis of data collected from 161 peer- reviewed articles applying the ACF. It reflects how the ACF struggles to balance specificity of unique contexts with generalizable concepts and methods. This section is divided into two parts: description of applications and theoretical components.
Our first series of analyses provide a general description of the breadth of the ACF applications by examining the popularity, flexibility, and general utility of the framework. Descriptors include author information, journal information, policy domains, geographic area studied, governance level, and methods used.
Author and Journal Information. The wide range of author institutions, journals, as well as the volatile increase in the number of publications since 2007 indicates that the ACF is a popular and robust theory of the policy process. Of the 161 articles analyzed, 138 have different first authors with only 14 having the same first author on multiple articles. The most prolific ACF first authors are Christopher Weible (5) (e.g., Weible, 2008), Daniel Nohrstedt (4) (e.g., Nohrstedt, 2010), and Karin Ingold (4) (e.g., Ingold & Gschwend, 2014). In total, there are 326 authors of applications of the ACF. Applications of the ACF tend to have multiple authors. Seventy-nine (49 percent) applications only had one author, while 82 (51 percent) had multiple authors with the median and mean number of authors on an application being two.
First authors represented 122 different universities or institutes, 25 different countries, and 4 continents. An example of an institute is the Forest Research Institute in Zvolen, part of the Ministry of Agriculture in Slovak Republic (e.g., Sarvašová, Šálka, & Dobšinská, 2013). Twenty-two universities or institutes produced multiple applications with the most frequent being University of Berne, Switzerland (7) (e.g., Ingold & Fischer, 2014); University of Colorado Denver, USA (7) (e. g., Weible, Pattison, & Sabatier, 2010); University of Copenhagen, Denmark (4) (e.g., Nedergaard, 2007); and Uppsala University, Sweden (4) (e.g., Nohrstedt, 2013).
Based on first author university affiliation, the most frequent countries of author- ship are the United States (52) (e.g., Schilling & Keyes, 2008), United Kingdom (18) (e.g., Smith, 2013), Switzerland (12) (e.g., Fischer, 2014), Canada (11) (e.g., Fitzpatrick, Fonseca, & McAllister, 2011), and Sweden (10) (e.g., Hysing & Olsson, 2008). However, based on continent of authorship, the most frequent is Europe (77) (e.g., Eriksson, Karlsson, & Reuter, 2010). This is followed by North America (65) (e.g., Vergari, 2007), Asia (13) (e.g., Jang, Kim, & Han, 2010), Oceania (7) (e.g., Beem, 2012), and Africa (3) (e.g., Marfo & McKeown, 2013). The ACF is predominantly an international policy process theory with the majority of authors coming from outside of North America. Figure 2 depicts the number of applications distributed by the author’s country of origin based on university or institution.
Figure 2. First Author University or Institute Affiliation by Country (n 5 161).
The high percentage of unique first authors, and variation among universities/ institutes and countries demonstrates that the ACF has diverse users. The ACF has diffused across the globe, indicating its popularity, durability, and tractability.
The ACF is applied in 98 unique journals. The journals that publish applications of the ACF at least five times include: Policy Studies Journal (16) (e.g., Ansell et al., 2009), Journal of Public Administration Research and Theory (6) (e.g., Brecher, Brazill, Weitzman, & Silver, 2010), Review of Policy Research (6) (e.g., Elgin & Weible, 2013), Journal of Comparative Policy Analysis (5) (e.g., Gupta, 2014), Forest Policy and Economics (5) (e.g., Sotirov & Memmler, 2012), and Educational Policy (5) (e.g., Cibulka & Myers, 2008). Public policy journals most frequently publish ACF applications. This demon- strates the popularity of the framework among policy process scholars. There are 73 journals (45 percent) that published only a single application of the ACF. These journals vary greatly in their scope and purpose. They include political science, American Journal of Political Science (e.g., Leifeld & Schneider, 2012); region or country specific, Philippine Political Science Journal (e.g., Lansang, 2011); domain specific, Energy Policy (e.g., Jegen & Audet, 2011); and interdisciplinary, Innovation: The European Journal of Social Science Research (e.g., Roßegger & Ramin, 2013). Taken in tandem, the range of journals publishing the ACF along with the finding of over 100 unique first authors exhibit the breadth of research agendas and the overall malleability of the framework.
Figure 3. ACF Applications by Year, 2007–14 (n 5 161).
Figure 4. ACF Applications by Policy Domain (n 5 161).
The publication of ACF applications has been volatile year to year. Since 2007, published ACF applications range from a low of 10 articles in 2009 to a high of 27 articles in 2014. In addition, from 2010 to 2014, the mean number of articles published annually is 24 with only 1 year below that total. This demonstrates that in recent years the ACF has achieved a sustained level of over 20 applications annually. In contrast, Weible et al. (2009) found that 10 was the maximum number of applications between 1987 and 2006 while also counting books and book chapters. Therefore, the ACF is growing in its popularity among scholars. Figure 3 only includes peer-reviewed journal articles and indicates clear growth in the number of ACF applications since 2007.
Policy Domains. Nine unique policy domains including an “Other” category were identified (see Figure 4). Applications were coded for either having a single domain or if they had multiple domains they were coded as comparative. There are only four applications that compare across different policy domains (e.g., Nedergaard, 2009). Any domain that has less than five applications, including the comparative applications, is included in the “Other” category. The “Other” includes such domains as: criminal justice (e.g., Bromfield, 2012), sports (e.g., Parrish, 2008), tourism (e.g., Airey & Chong, 2010), and emergency management (e.g., Albright, 2011). The most frequent policy domain representing a clear plurality of applications is environment and energy with 70 (43 percent) applications (e.g., Blatter, 2009; Bukowski, 2007; Hansen, 2013). Other common policy domains include: public health (15) (e.g., Breton, Richard, Gagnon, Jacques, & Bergeron, 2008; Poulsen, 2014), education (14) (e.g., Beverwijk, Goedegebuure, & Huisman, 2008; DeBray, Scott, Lubienski, & Jabbar, 2014; Shakespeare, 2008), social welfare (12) (e.g., Klindt, 2011), science and technology (12) (e.g., Amougou & Larson, 2008; Kettell & Cairney, 2010), defense and foreign policy (8) (e.g., Pierce, 2011), economics and finance (7) (e.g., Buller & Lindstrom, 2013; Dressel, 2012), and urban planning and transportation (5) (e.g., Olsson, 2009). It is evident that the general focus of scholars applying the ACF is on environment and energy issues, but the framework is flexible enough to be applicable to a wide range of public issues. Weible et al. (2009) found that a majority of applications were about the environment and energy; this demonstrates a continuation of this trend, but it no longer represents the majority of applications.
Geographic Area Studied. The ACF is applied to policy subsystems around the globe. There are applications discussing 54 unique countries.2 The total number of countries identified is 201 including duplications and 20 applications with multiple countries, such as Huntjens et al. (2011) comparing 11 countries. Jenkins-Smith, Nohrstedt, et al. (2014) state that there are no comparative applications across political systems systematically comparing policy subsystems, coalition behavior, and policy processes. However, we find that there are multiple studies comparing aspects of the ACF across political systems. For example, Montpetit (2011) compares the role of scientists as policy actors in Canada, the United States, United Kingdom, and France. Montpetit (2009) compares policy actor learning in the European Union (EU) with Canada and the United States. Huntjens et al. (2011) compare policy-oriented learning among policy actors in eight different river basins in 11 different countries including Uganda, Rwanda, Uzbekistan, the Netherlands, and the Ukraine. These are some examples of the 20 applications that analyze policy subsystems in multiple countries.
The most frequent single country is the United States (53), (e.g., Fisher, Leifeld, & Iwaki, 2013), but this only represents about a third of the applications. This is followed by the United Kingdom (15) (e.g., Dudley, 2007), Switzerland (14) (e.g., Mavrot, 2012), and Sweden (10) (e.g., Sandström, 2010). Some examples of applications in more-remote and less-common countries include: Burkina Faso (Cherlet & Venot, 2013), Bulgaria (Brusis, 2010), China (Francesch-Huidobro & Mai, 2012; Li, 2012), Estonia (Adams, Cotella, & Nunes, 2014), France (Bandelow & Kundolf, 2011), Greece (Stamelos & Kavasakalis, 2013), Iceland (Nedergaard, 2009), India (Rastogi, Hickey, Badola, & Hussain, 2013), Indonesia (Fidelman et al., 2014), Ireland (Adshead, 2011), Israel (Lahat, 2011; Mandelkern & Shalev, 2010), Kenya (Kingiri, 2011), Liberia (Runkle, LaFollette, & Alamu, 2013), Papua New Guinea (Babon et al., 2014), Portugal (Huntjens et al., 2011), South Africa (Hirschi & Widmer, 2010), and South Korea (Kim, 2011). Figure 5 illustrates a total of 201 applications by country.3
Figure 5. ACF Applications by Country Including Multiple Country Applications (n 5 201).
Figure 6. ACF Application by Level of Government (n 5 175).
The continent that is the most frequently studied using the ACF is Europe with 111 applications including 16 EU-only applications (e.g., Sloboda, Szabó-Gilinger, Vigers, & Šimičić, 2010). This is followed by North America with 64 applications (e. g., Sistrom, 2010), Asia with 25 applications (Han, Swedlow, & Unger, 2014), Africa with 13 applications (Kingiri, 2014), and Oceania with 5 applications (Battams & Baum, 2010). The only continent other than Antarctica that does not have an application is South America. Also, the applications in North America are only in the United States and Canada. Weible et al. (2009) did find multiple applications from South America but found comparatively fewer applications in Asia and Africa. This may be due to the language restriction of only including English-language peer-reviewed journal articles. Overall, the frequency of the ACF applied to Europe demonstrates that it is not a U.S.-centric policy framework, and the presence of applications in countries such as China (e.g., Li, 2012), Kenya (Kingiri, 2011), and Liberia (Runkle et al., 2013) show its utility in less-democratic and developing countries that were once criticisms of the ACF (Andersson, 1998; Kübler, 2001; Parsons, 1995).
Governance Level. To investigate the governance level of the subsystem(s), the following categories are used (see Figure 6): local, state, regional, national, and transnational (EU and other international governing institutions). This coding does not capture policy actors that may be from different levels of government but rather focuses only on the level of government identified by the policy subsystem. There are 10 applications that analyze more than one level of government. For example, Leach, Weible, Vince, Siddiki, and Calanni (2014) analyze marine aquaculture partnerships at three levels of government: local, regional, and state. Therefore, because these 10 applications include subsystems at multiple levels of governance, there is a total of 175 levels of governance from the 161 applications. The results show that 86 subsystems are at the national level (e.g., Leifeld, 2013). In addition, there are several other applications, 22 that either focus on the EU (e.g., Mailand, 2010) or other international institutions such as the Nordic Council of Ministers (Nedergaard, 2009) that are identified as “transnational.” Local governments account for 28 applications (e.g., Lubell, 2007), state governments 27 (e.g., Heikkila et al., 2014; Heinmiller, 2013), and regional governments 12 (e.g., Van den Bulck & Donders, 2014). Weible et al. (2009) did not collect data on the level of government analyzed in the policy subsystem so a comparison is not available. Our data, however, show that since 2007 the ACF is primarily used to explain national policy processes.
Methods Used. Coders identified articles as applying one of the following methodological categories: only quantitative, only qualitative, or mixed methods (both quantitative and qualitative). For purposes of coding, quantitative is any numerical data collection and analysis, qualitative is any linguistic-based data collection and analysis, and mixed methods include both quantitative and qualitative elements such as combining qualitative interviews with quantitative surveys (e.g., Henry, 2011). The most frequent type of methodology and a clear majority are qualitative methods with 107 (66 percent) applications (e.g., Meijerink, 2008). Mixed methods account for 39 (24 percent) applications (e.g., Nohrstedt, 2010) and there are 15 (9 percent) (e.g., Henry, Lubell, & McCoy, 2011) quantitative-only applications.4 Mixed methods and qualitative applications account for 146 applications. Mixed methods and quantitative applications combined account for 54 applications. Therefore, the vast majority of applications include a qualitative component of data collection or analysis, while in comparison about one third of the applications include any numerical data collection and analysis, indicating a clear preference for qualitative data and/or analysis.
The most frequent method of data collection is interviews in 101 (63 percent) applications (e.g., Knox-Hayes, 2012; Marichal, 2009). The number of interviews conducted varies greatly from only 5 by Klindt (2011) to 250 interviews by Fischer (2014). In comparison, Weible et al. (2009) found that 30 percent of applications utilized some type of interview data. The number of applications using interviews has doubled. A majority, 94 (58 percent) applications use some form of self-reported document analysis. These also represent a wide range in terms of the number of observations. Interviews and document analysis are frequently utilized in tandem. In total, 65 (40 percent) applications combine interviews and document analysis (e.g., Hirsch, Baxter, & Brown, 2010; Nedergaard, 2008). The preferred method of data collection for applying the ACF are interviews and document analysis. This is a recent phenomenon as Weible et al. (2009) found only 10 percent of applications utilized both interviews and document analysis.
In many cases, the number of documents analyzed are not provided (e.g., Princen, 2007; Roßegger & Ramin, 2013; Stensdal, 2014). For example, Afonso (2014) states, “The analysis relies primarily on newspaper reports, government reports, responses to government consultations, party manifestos and secondary literature” (p. 573). In contrast, some applications identify and analyze hundreds or even over a thousand documents, such as Montefrio and Sonnenfeld (2011) and Fisher et al. (2013). There were 14 (9 percent) applications that do not indicate any form of data collection (e.g., Smith, 2009). In total, 52 (32 percent) applications do not specify how many observations are made. For example, Adams et al. (2014) state the following about data collection “The documentary analysis was supplemented by a series of face-to face and telephone interviews with selected key stakeholders” (p. 714). The organization categories for stakeholders are specified, but not how many stakeholders are interviewed, the interview questions, how many documents, and how they are analyzed. By not specifying and making transparent the methods of data collection, replicability becomes impossible. This is a similar issue identified by Weible et al. (2009), who found 41 percent of applications did not specify the methods of data collection. These applications demonstrate a marked improvement of about 10 percent, but still about one third of applications do not specify how data collection occurs beyond stating that document analysis or interviews were conducted. This explains the tension in balancing common approaches for applying the framework, with addressing diverse issues in unique contexts.
Other common forms of data collection that are identified include: surveys in 34 (21 percent) applications (e.g., Leach et al., 2014), participant observation in 13 (8 percent) applications (e.g., Fleury, Grenier, Vallée, Hurtubise, & Lévesque, 2014), and focus groups in 5 (3 percent) applications (e.g., Wilson, Barakat, Vohra, Ritvo, & Boon, 2008). In comparison with Weible et al. (2009), there is little change in the number of applications that use surveys (17 percent); however, there are minor increases in participant observation, which was identified by Weible et al. (2009) in only 3 percent of applications and focus groups were not identified at all. This represents a diversification in terms of data collection among current ACF applications.
Overall, the typical application of the ACF is conducted by European or North American scholars; studies environment and energy issues in Europe or North America at the national level of governance; and uses qualitative methods, including interviews and document analysis. However, the framework is also being applied by scholars in Asia, Africa, and Oceania, studying a wide range of public issues in over one hundred countries including comparative studies at all levels of governance, and using both qualitative and quantitative methods. On balance, the ACF is growing in popularity among all scholars and subsequently becoming a pluralistic framework being used on multiple continents by hundreds of authors to study diverse subsystems.
The ACF has three general theoretical foci. These are advocacy coalitions, policy change, and policy-oriented learning (Jenkins-Smith, Nohrstedt, et al., 2014). Data were collected from the applications examining each of these foci as well as identifying the presence of theoretical components such as belief systems of advocacy coalitions and the different pathways to policy change.
Advocacy Coalitions. One of the basic concepts of the ACF is the advocacy coalition, the policy actors that coordinate their actions in a nontrivial way to achieve policy objectives (Jenkins-Smith, Nohrstedt, et al., 2014). Applications of the ACF range widely on how they identify advocacy coalitions and what they focus on. Data were collected from applications about advocacy coalitions including: (1) how many advocacy coalitions are identified (0, 1, 2, or more), (2) what beliefs are identified (deep core, policy core, or secondary), (3) if coordination is identified, and (4) whether the coalitions exhibit stability and/or defection over time.
A vast majority of applications identify at least one advocacy coalition. In total, 143 (89 percent) applications identify at least one advocacy coalition, which means 18 (11 percent) did not identify any advocacy coalitions (e.g., Beard, 2013; Montpetit, 2011; Ripberger, Gupta, Silva, & Jenkins-Smith, 2014). The applications that did not identify any advocacy coalitions tend to focus on specific components of the ACF such as deep core beliefs of policy actors (e.g., Ripberger et al., 2014). In addition, 22 (14 percent) applications only identify a single advocacy coalition (e.g., Kwon, 2007; Michalowitz, 2007; Parsell, Fitzpatrick, & Busch-Geertsema, 2014; Patel, 2013). In comparison Weible et al. (2009) found that 9 percent of applications did not identify an advocacy coalition, and only 1 percent identified only a single advocacy coalition. This demonstrates that about 10 percent of applications continue not to identify an advocacy coalition and an increasing trend of studying single-coalition subsystems.
The vast majority of applications, 121 (75 percent), identify two or more advocacy coalitions (e.g., Dela Santa, 2013; Dougherty, Natow, Bork, Jones, & Vega, 2013). These applications may identify competing advocacy coalitions as dominant and minority (e.g., Montefrio, 2014) or may not indicate such qualifying differences (e.g., Miller, 2011). Many applications identify more than two advocacy coalitions such as Ingold (2011) that identifies three advocacy coalitions. In comparison, Weible et al. (2009) found that 90 percent of applications before 2007 had two or more advocacy coalitions. This demonstrates a trend in applications studying the ACF that are no longer dependent on identifying two or more advocacy coalitions.
The ACF posits that policy actors have belief systems that are hierarchical. These beliefs are the glue that brings together policy actors to engage in the policy process (Jenkins-Smith, Nohrstedt, et al., 2014). These beliefs are identified as either deep core, policy core, or instrumental/secondary beliefs. For identifying advocacy coalitions and understanding policy change the focus tends to be on policy core beliefs. In general, 145 (90 percent) applications identify coalition beliefs whether specifying the category or not. However, 16 (10 percent) applications did not identify any beliefs (e.g., Frahsa, Rütten, Roeger, Abu-Omar, & Schow, 2014; Howarth, 2013; Ness, 2010). There are 50 (31 percent) applications that identify beliefs in general, but did not specify the category (e.g., DeBray-Pelot, Lubienski, & Scott, 2007; Van Overveld, Hermans, & Verliefde, 2010). This demonstrates that a large minority of applications (a combined 41 percent) are not categorizing belief systems as prescribed by Jenkins-Smith, Nohrstedt, et al. (2014). Weible et al. (2009) did not report results about belief systems.
Applications were also analyzed to determine if they explicitly discussed whether members of the advocacy coalitions coordinate.5 In the past, failure to examine coordination was identified as a major limitation of the ACF (Schlager, 1995). Indeed, Weible et al.’s (2009, p. 132) review of past ACF applications noted a lack of coalitional coordination in ACF applications, identifying only a handful of applications discussing coordination (Abrar, Lovenduski, & Margetts, 2000; Farquharson, 2003; Sato, 1999) and only two using data to investigate the concept (Weible, 2005; Weible & Sabatier, 2005). Following these indications that coalitional coordination was systematically overlooked in ACF applications, Jenkins-Smith, Nohrstedt, et al. (2014) developed explanations of coalitional response to collective action threats. In a departure from previous works, this analysis finds 58 (36 percent) applications explicitly identify coordination (e.g., Baumann & White, 2014; Caveen, Gray, Stead, & Polunin, 2013; Crow, 2008), evidence of a notable increase since Weible et al. (2009). Now a full third of applications identify coordination, suggesting the concept is finally receiving attention.
A final theoretical focus is whether ACF applications find that policy actors defect from advocacy coalitions or if membership is relatively stable over time. Stability is much more frequent in comparison to defection. Coalition stability is identified in 27 (17 percent) applications (e.g., Heikkila et al., 2014; Ingold & Varone, 2012), while five (3 percent) applications explicitly discuss defection (e.g., Svihula & Estes, 2007). Weible et al. (2009) identified stability or defection among 16 percent of applications. Therefore, the study of stability and defection among advocacy coalitions remains relatively stable.
Policy Change. Content analysis of policy change includes whether it is identified, if it is major and/or minor change, and if a pathway(s) to policy change are identified: (1) external events or perturbations, (2) internal events within the policy subsystem such as policy failure, (3) policy-oriented learning that influences policy change, and (4) negotiated agreement as well as the presence of a policy broker (Jenkins-Smith, Nohrstedt, et al., 2014). Overall, 67 (42 percent) ACF applications analyze either poli- cy change or stasis (e.g., Nohrstedt, 2008, 2010, 2011; Stich & Miller, 2008).
A minority of applications, 19 (12 percent), qualify policy change as either major or minor. Sabatier and Jenkins-Smith (1999) describe this difference as a major strength of the ACF. They argue that major policy changes are changes among the policy core beliefs including the objectives of a government policy or program, while minor policy changes are in relation to the secondary or instrumental aspects of the policy. Only one application analyzes both major and minor policy changes (Fischer, 2014). There are 12 applications that analyze major policy changes (e.g., Winkel & Sotirov, 2011). In contrast, eight applications analyze minor policy change (e.g., Penning-Rowsell, Priest, & Johnson, 2014). While applications of the ACF should categorize policy change as either major or minor (Jenkins-Smith, Nohrstedt, et al., 2014), this is not the case in practice.
Overall, the most frequent source of policy change is policy-oriented learning, which was identified in 46 (29 percent) of ACF applications (e.g., Heikkila et al., 2014). Learning may facilitate minor policy changes over time such as the utilization of policy analysis as an enlightenment function (Sabatier, 1988), or it may lead to major policy changes in conjunction with internal or external events (Jenkins-Smith, Nohrstedt, et al., 2014). For example, Heikkila et al.’s (2014) research studying regulation of hydraulic fracturing finds that policy-oriented learning occurred among an advocacy coalition and may have been a predecessor to policy change. Weible et al. (2009) identify 20 (25 percent) applications of policy-oriented learning hypotheses, but these are not explicitly connected to policy change. This may explain the slight increase in the number of applications applying learning to policy change.
A second pathway to policy change is attributing the change to a source external to the policy subsystem. These may include changes in socioeconomic conditions, regime change, outputs from other policy subsystems, and extreme events such as crises and disasters (Jenkins-Smith, Nohrstedt, et al., 2014). External events like these increase the likelihood of major policy change, but are also dependent on the actions of advocacy coalitions (Sabatier & Weible, 2007). In total, 45 (28 percent) applications identify a source external to the policy subsystem as influencing policy change (e.g., Dougherty, Nienhusser, & Vega, 2010; Nohrstedt, 2008). Weible et al. (2009) found 18 (22 percent) applications tested the hypothesis that external perturbations are a necessary, but not sufficient, condition of policy change. This indicates a slight increase in the application of external events as a source of policy change.
In juxtaposition to external sources of policy change, there are also events internal to the policy subsystem such as crises or policy failures within the territorial boundaries and/or topical area of the policy subsystem that may lead to policy change (Sabatier & Weible, 2007). Ten (6 percent) applications identify this as a source of policy change (e.g., Diaz-Kope, Lombard, & Miller-Stevens, 2013). Weible et al. (2009) did not report the number of applications describing internal events as influencing policy change because it was not introduced until Sabatier and Weible (2007).
The final pathway to policy change is negotiation between coalitions, which was analyzed in 22 (14 percent) applications associated with policy change (e.g., Ingold, 2011; Ley & Weber, 2014). Applications identifying negotiation also tend to identify a policy broker (e.g., Ingold, 2011). According to Sabatier (1993), policy brokers are often elected officials or civil servants who may favor one coalition over another, but whose dominant concern is keeping the level of political conflict within acceptable limits and reaching a reasonable solution (p. 27). Weible et al. (2009) do not identify negotiation because it was not introduced until Sabatier and Weible (2007). Since its introduction in Sabatier and Weible (2007), negotiation has become a frequent pathway to analyze policy change. Therefore, new concepts such as internal events and negation are being applied.
Policy-Oriented Learning. Policy-oriented learning has always been a central focus of the ACF and is defined as “enduring alterations of thought or behavioral intentions that result from experience and which are concerned with the attainment or revision of the precepts of the belief system of individuals or of collectives (such as advocacy coalitions)” (Sabatier & Jenkins-Smith, 1993, p. 42). Policy-oriented learning then could mean learning by an individual, an advocacy coalition, or between advocacy coalitions. It may or may not be connected explicitly to a policy change. Overall, policy-oriented learning is identified in 48 (29 percent) applications (e.g., Cairney, 2007; Heikkila et al., 2014; Nohrstedt, 2011; Weible, 2008). In comparison, Weible et al. (2009) identified 20 (25 percent) applications that explicitly applied a hypothesis related to policy-oriented learning. The number of applications applying policy-oriented learning has been relatively stable.
The process and influence of policy-oriented learning varies greatly among the applications. Learning is identified at different levels within and between coalitions, and can influence belief and policy change. In the articles reviewed, learning is identified at the individual level (e.g., Kingiri, 2011; Weible, 2008), the coalition level (e.g., Cairney, 2007), across coalitions (e.g., Nohrstedt, 2013; Weber, Driessen, Schueler, & Runhaar, 2013), and within coalitions, for instance in strategy alterations (e.g., Han et al., 2014; Kingiri, 2014; Nohrstedt, 2011). Learning across coalitions was also identified as leading to coordination between the coalitions (e.g., Johnson, Payne, McNeese, & Allen, 2012) and to changes in coalition alignments and the emergence of new alignments among policy actors (e.g., Kuebler, 2007). In these applications, learning within a coalition was also found to lead to a coalition giving up secondary beliefs to maintain policy core beliefs (e.g., Ellison & Newmark, 2010). Additionally, learning is identified as a function of new scientific and technical information (e.g., Stensdal, 2014) and the exchange of ideas in professional forums and government committees (e.g., Beem, 2012; Nedergaard, 2009). Finally, learning also leads to policy change (e.g., Bandelow & Kundolf, 2011) sometimes identified as minor as it relates to instrumental policy beliefs (e.g., Schröer, 2014; Van Gossum, Ledene, Arts, De Vreese, & Verheyen, 2008). Based on this analysis, learning within the ACF occurs at various levels of analyses and has various causes and influences related to individuals, coalitions, and policy.
Integration and Comparison with Other Frameworks, Theories, and Concepts. The ACF is applied on its own and either in comparison or integrated with other theories and frameworks. There are 83 (52 percent) applications explicitly comparing or integrating the ACF with frameworks or theories of the policy process, or other theoretical concepts. This demonstrates that a slight majority of applications are not solo ACF applications, but rather seek to provide further understanding about the policy process by either integrating or comparing the ACF with other frameworks, theories, and/or theoretical concepts. In comparison, Weible et al. (2009) found that 36 (45 percent) applications used another theory or framework in addition to the ACF. During its history, about half of ACF applications are not solo ACF applications, but instead either compare or combine the ACF with other theories.
Many policy process frameworks and theories are utilized in these applications including: punctuated equilibrium theory (e.g., Beard, 2013; Dziengel, 2010), multiple streams (e.g., Olsson, 2009; Van Gossum et al., 2008), network analysis (e.g., Ansell et al., 2009), regime theory (e.g., Blatter, 2009), narrative policy framework (e.g., Shanahan, McBeth, Hathaway, & Arnell, 2008), social construction and policy design (e. g., Weible, Siddiki, et al., 2011), diffusion of innovation (e.g., Amougou & Larson, 2008), and institutional analysis and development (e.g., Cheng, Danks, & Allred, 2011; Lansang, 2011). The ACF is often compared or integrated with various theories and theoretical concepts that stretch across policy, public administration, and political science including: resource dependence (e.g., Leifeld & Schneider, 2012), discourse coalitions (e.g., Leifeld, 2013; Szarka, 2010), agenda setting (e.g., Smith, 2009), policy paradigms (e.g., Quaglia, 2010), cultural theory (e.g., Nohrstedt, 2013), policy entrepreneur (e.g., Mann & Gennaio, 2010), epistemic communities (e.g., Francesch-Huidobro & Mai, 2012), socio-ecological systems (e.g., Weible et al., 2010), and stakeholder analysis (Weible, 2007). This demonstrates the flexibility of the framework to be inclusive or comparable with various theories and theoretical concepts.
Applications of the ACF identify advocacy coalitions with a growing number only identifying a single coalition. The beliefs of these coalitions are identified, but a large minority does not use the hierarchical categories of the ACF. Also, a majority of applications still do not discuss cooperation and/or coordination among policy actors. Theories of policy change and policy-oriented learning are each analyzed in about a third of applications. There is a balance between the three theories of the ACF with no single theory dominating the rest. Advocacy coalitions are identified among almost all applications because they are necessary before exploring theories of policy change and policy-oriented learning. About half of the applications of the ACF explicitly compare or utilize other frameworks, theories, or theoretical concepts along with the framework. This demonstrates innovation on the part of those adopt- ing the framework, and the flexibility of the framework insomuch that it is responsive to such innovations.
The above analyses concentrated on describing the breadth and depth of the applications. This section assesses how applications of the ACF compare to the aspirational criteria Sabatier (1999) first laid out in the Theories of the Policy Process. However, for contemporary comparison and to better reflect the current state of the ACF, this assessment utilizes the four criteria as prescribed by Cairney and Heikkila (2014) that were adapted from Sabatier (2007), which are based on Sabatier (1999). Table 1 below restates the evaluation by Cairney and Heikkila (2014) of the current state of the ACF, and also summarizes this assessment of the ACF.
A way to assess the overall health of a framework and research agenda is to notice how prolifically it is published. This review’s initial population of English language peer-reviewed journal articles citing the core theoretical texts during 2007–14 produced 1,067 unique citations. After removing articles that only cited the framework and did not apply it directly, 161 or 15 percent are applications, whereas other reviews of the policy process theories such as Pierce et al. (2014) found 111 applications of the social construction and policy design from 1993 to 2013,6 and Jones et al. (2016) found 311 applications of multiple streams approach from 2000 to 2013. In comparison this review of ACF applications is over a much shorter period of time and still produced a relatively robust number of applications. Finally, the number of applications has relatively doubled from 2007 to 2014 in comparison to the 80 applications during 1987–2006 as found by Weible et al. (2009). It is evident that the framework is analytically applied and results are published in peer-reviewed literature.
Data were collected and analyzed based on the policy subsystem and included policy domain, geographic domain, and level of government. Nine unique policy domains are identified with at least five applications, and the policy domain of environment and energy is identified among a plurality of applications. This supports a possible continued bias toward environment and energy as argued by Cairney and Heikkila (2014); however, the majority of applications are no longer applied to this domain as found by Weible et al. (2009).
Applications are identified in 54 unique countries not including the applications that focus generally on the entire EU. For a single country the most frequent application is in the United States, but based on continent of application, Europe has the majority of applications. Also, applications are identified in Asia, Africa, and Oceania, but none in South America. While there may have been an initial bias toward the United States as found by Weible et al. (2009) and argued by Cairney and Heikkila (2014), this is no longer the case.
Cairney and Heikkila (2014) and Weible et al. (2009) do not mention level of government, but this study takes it into account. It finds that a slight majority of applications focus on the national level of governance. While the ACF has been applied at other levels of government there appears to be a bias by scholars to apply it at the national level of government. This may be troubling for ACF scholars as Sabatier (1993) argues:
To examine policy change only at the national level will, in most instances, be seriously misleading. In the United States and many other countries, policy innovations normally occur first at a subnational level and then may get expanded into nationwide programs, (p. 17)
The ACF was intended to better capture bottom-up policy processes operating at the subnational levels of government, but recent applications have focused more so at the national level. This demonstrates a clear difference between how the ACF is applied and its original intention.
Cairney and Heikkila (2014) argue that the methods applied in ACF applications are mixed. Weible et al. (2009) found that a majority of applications were either unspecified or dependent solely on interviews for data, making them qualitative. This research supports that the majority of recent applications are qualitative. Also, a majority utilize interviews and/or documents for data collection, and approximately one third do not specify how many observations are made. There is a mixture of methods, but the balance is clearly toward qualitative work. This might be expected as a strength of qualitative research is explaining how and why a phenomenon occurs (Goertz & Mahoney, 2012), so it is of particular interest to those applying theories of policy change and policy-oriented learning. The framework would benefit from increased quantitative investigation, which may help in terms of validity and replicability. Research exploring specific operationalizations of ACF phenomena, particularly over time, would be especially helpful. Integration with other frameworks such as policy diffusion (e.g., Berry & Berry, 2014) or PET (e.g., Baumgartner & Jones, 1993) may help to facilitate quantitative designs and protect against theory tenacity and confirmation bias (Weible, 2014). Whether qualitative or quantitative methods are used, all scholars should be transparent about how data are collected and analyzed, which currently is not being practiced by about one third of applications.
Cairney and Heikkila (2014) state that:
The ACF’s core studies treat key concepts and their interaction consistently and coherently—but with considerable scope for independent scholars to use the ACF very loosely, without testing any of its hypotheses... ACF also has shared approaches and protocols that are commonly made available to scholars, but the consistency in application of these protocols is less clear. (p. 374)
One of the strengths of the ACF is that applications utilize the core theories related to advocacy coalitions, policy change, and/or policy-oriented learning. For example, the vast majority of applications identify at least one coalition, and a majority of applications identify deep core, policy core, or secondary beliefs. However, although this review finds evidence of an increase in explicitly identifying forms of coordination, the concept remains underutilized by a majority of ACF applications. The other major theories of the ACF—policy change and policy-oriented learning—are also well represented among the applications. Overall, the vast majority of policy change applications identify and apply at least one of the four pathways of policy change described by Jenkins-Smith, Nohrstedt, et al. (2014). Policy-oriented learning is identified in a minority of applications and is much more ambiguous than pathways to policy change. This is not surprising as Jenkins-Smith, Nohrstedt et al. (2014) argue “if there is an understudied area in the ACF, it is the topic of policy-oriented learning” (p. 205). Policy-oriented learning remains an area that needs conceptual and methodological development.
Jenkins-Smith, Nohrstedt, et al. (2014) call for balancing general methods and concepts with specific contexts. They argue that there are three issues to advance the ACF: the application of the ACF in different governing systems, guidelines for data collection, and best methods for analyzing theories of the ACF (p. 207). While this research does not explicitly compare governing systems, it did identify applications in the United States, Switzerland, Sweden, China, South Korea, Kenya, Mozambique, Israel, and Burkina Faso that represent different governing systems. While there are a variety of forms of data collection, the majority of applications are not applying universal or theory centric guidelines. Finally, Sabatier and Jenkins-Smith (1993) provide a methodological appendix for content analysis of documents that can be utilized to collect data about policy actor belief systems and could be adopted for interviews. Such guidelines could be considered best methods at least for the study of belief systems of policy actors. Such guidelines for more quantitatively orientated designs may also prove useful for those applying the framework. Overall, best practices and guidelines (whether universal or theory specific) are necessary.
According to Cairney and Heikkila (2014), “The framework has maintained its basic assumptions, but hypotheses and concepts have been modified over time” (p. 374). The core assumptions of the framework have not been undone and the core theories about coalitions, policy change, and policy-oriented learning continue to be applied. Even the major components of these theories such as hierarchical belief systems and the various pathways to policy change are well established and consistently applied. Some of the recent developments of the framework identified by Sabatier and Weible (2007) have been highlighted by this research and are now frequently applied. The ACF continues to evolve and be revised based on empirical findings (i. e., Jenkins-Smith, Nohrstedt, et al., 2014; Jenkins-Smith, Silva, et al., 2014; Sabatier, 1998; Sabatier & Jenkins-Smith, 1999; Sabatier & Weible, 2007), and subsequently scholars applying the ACF have responded incorporating these revisions and suggestions into applications.
The ACF has a productive and expanding research agenda. It has been applied at least 161 times since 2007 in nearly 100 different peer-reviewed English language journals. Since 2010, the ACF has been steadily producing about 25 applications annually, which is significant growth compared to the 10 or fewer applications identified annually for 2000–06 by Weible et al. (2009) (including non-peer-reviewed sources). The ACF is a popular framework among policy process as well as other scholars.
The ACF has a high level of portability and appeal to scholars throughout the world. There were 138 unique first authors writing from 25 different countries applying the framework to 54 unique countries with multiple applications in every continent except for South America and Antarctica. The framework has been applied frequently across five different levels of government and to 16 different policy domains not including four applications that applied the framework comparatively over multiple domains. Given the diversity among applications, there are also clear tendencies. The majority of applications have multiple authors that are either European or from the United States. Applications tend to focus on a European country or the United States at the national level and tend to examine environmental and energy issues. There is a distinct lack of applications of the ACF in Spanish or Portuguese speaking countries of Latin and South America.
Methodologically, these are primarily qualitative single case studies that utilize interviews and/or document analysis. Most of these qualitative studies meet empirical standards (Goertz & Mahoney, 2012), but a third do not identify how or what observations are made. This practice limits the accumulation of knowledge through reliable and replicable methods. However, the ACF has pluralistic purposes of generating both generalizable knowledge of the policy process, as well as knowledge about specific phenomenon (Jenkins-Smith, Nohrstedt, et al., 2014). This is a strength of the ACF as a framework that it can include common approaches while still capturing and representing unique contexts.
ACF applications do share a common research agenda. The applications do work within the ACF’s theories of advocacy coalitions, policy change, and policy-oriented learning. Applications utilize a policy subsystem approach even if it tends to be at the national level of government. Components of advocacy coalitions such as hierarchical beliefs are common and a majority of studies examining policy change identify at least one pathway. However, there are still too many applications that only refer to beliefs in general without using the hierarchical categories. Another common limitation is the absence of discussing coordination among policy actors. Policy-oriented learning also remains a problematic area of research for ACF scholars.
There are two main limitations to this research. First, it does not capture every application of the ACF from 2007 to 2014. Only applications that were in English and in peer-reviewed journals are included. Therefore, non-English applications would have been missed such as possible applications in Sweden, Germany, or in Spanish-speaking countries. Applications that are books, book chapters, and other mediums are also not included. Second, content analysis is conducted on these 161 applications utilizing multiple coders. The level of interpretation by these coders is mitigated by having codes focusing on presence rather than frequency or strength. Also, multiple forms of inter-coder reliability are tested and found acceptable. While systemic issues may have been mitigated there will be systematic errors when utilizing almost 5,000 pages of material as the source of data.
Jenkins-Smith, Nohrstedt, et al. (2014) state that the “future trajectory of the ACF depends on the innovative and creative efforts of numerous analysts from around the world” (p. 204). Based on the variation of levels of government, policy domains, journals, and countries of application, ACF scholars are an innovative and creative group. However, the accumulation of general knowledge will also depend on how it is applied. Paul Sabatier (1999) called upon all policy process scholars to hold their research to rigorous theoretical and empirical standards. In 2009, Weible et al. found that a large number of ACF applications relied upon unspecified methodologies, and this taking stock of the ACF exercise has yielded quite similar results. Weible et al. (2009) argued that moving forward “the goal should be intersubjectively reliable methods that serve as the cornerstone for making research transparent and provide a basis for collective learning” (135). Given this analysis of ACF applications, the need for increased transparency, reliability, and falsifiability unfortunately remains. ACF scholarship should diligently move toward these goals. However, Jenkins-Smith, Nohrstedt, et al. (2014) discuss the need to balance transparent replicable methods with being able to apply the framework to diverse and unique contexts. Jenkins-Smith et al. raise a difficult question for the ACF and any scientific research agenda, what is the proper balance between specificity and generalizability? This data and analyses show that the scales of the ACF tilt toward specificity and likely do so at the expense of generalizability. Many of the applications reviewed here could devote more attention to transparency of method without sacrificing or altering content or findings.
Today, there is a strong core of methodologically rigorous and prolific ACF scholars across the world using transparent replicable methods testing theories and hypotheses as described by Sabatier and Jenkins-Smith (1993, 1999). Conversely, however, the majority of applications are by scholars who only apply the ACF once to explore a question about a specific case study. No doubt, the presence of both of these types of applications—theory testing and framework adopting—reflects the strength of the ACF. The ACF is a popular, durable, and flexible framework that grows with each adoption of the framework exploring specific contexts and cases, as well as a rigorous core seeking to refine theories through hypothesis testing.
Jonathan J. Pierce is an assistant professor at Seattle University’s Institute of Public Service.
Holly L. Peterson is a doctoral candidate at Oregon State University’s School of Public Policy.
Michael D. Jones is an assistant professor at Oregon State’s School of Public Policy. His research interests include the policy process, narrative and public policy, and science communication.
Samantha P. Garrard is an M.B.A. student at Seattle University. She has an under- graduate degree in Public Affairs from Seattle University’s Institute of Public Service.
Theresa Vu is a graduate of the Masters of Public Administration program at Seattle University’s Institute of Public Service.
The authors would like to thank some of the additional coders on this project, Laura Crandall and Jennifer Pazar. Also, Chris Weible provided important feedback and input on the development of the codebook.
1 A total of 256 articles were randomly selected for inter-coder reliability during this coding.This sample is sufficient to determine inter-coder reliability given the population, a 95 percent confidence level and 5 percent confidence interval. Five coders had an inter-coder reliability rate of greater than 80 percent on applications. Inter-coder reliability at or above the 80 percent threshold is considered reliable for a random sample of this size (Lacy & Riffe, 1996; Lombard, Snyder-Duch, & Campanella Bracken, 2002; Riffe, Lacy, & Fico, 2005).
2 All of these reported descriptive statistics do not include the 16 applications that only examined the European Union (EU) as one geographical and political entity (e.g., Feindt, 2010; Quaglia, 2012). In addition, Montpetit (2011) compares policy subsystems of the EU with the United States and Canada making a total of 17 applications examining the EU as one political entity.
3 Figure 5 includes applications with multiple countries (e.g., Mann & Gennaio, 2010) and excluding the 16 European Union only applications.
4 Weible et al. (2009) do not identify the category of methodology utilized in applications, but rather focus on the specific method of data collection such as interviews or content analysis.
5 Coding for this concept included both coordination as well as collaboration.
6 Pierce et al.(2014) also included books and book chapters as well as peer reviewed journal articles.