A 2014 meta-analysis of the data from five studies by Keith Humphreys, Janet Blodgett and Todd Wagner concluded that "increasing AA attendance leads to short and long term decreases in alcohol consumption that cannot be attributed to self-selection."[4] Austin Frakt, writing for The New York Times, discusses how the study's methodology minimizes outside factors, such as how motivated the people who succeed at becoming abstinent are. Frakt sums up the results of the research as follows: "Going to an additional two A.A. meetings per week produced at least three more days of alcohol abstinence per month". The article concludes that "A.A. helps alcoholics, apart from the fact that it may attract a more motivated group of individuals. With that established, the next step is to encourage even more to take advantage of its benefits."[11]

In 2009, Lee Ann Kaskutas performed a meta-analysis of other studies looking at how effective Alcoholics Anonymous is. The article notes that "rates of abstinence are about twice as high among those who attend AA" but concluded that whether Alcoholics Anonymous has a specific effect is unclear (a specific effect, in this context, is whether it's the actual Alcoholics Anonymous program which helps keep people sober, instead of other factors, including the fact that people more motivated to stay sober will go to more meetings, or that the group support helps alcoholics regardless of the actual program, etc.), stating that there were "2 trials finding a positive effect for AA, 1 trial finding a negative effect for AA, and 1 trial finding a null effect."

The study reported that while the evidence base for twelve step groups from experimental studies was weak, "other categories of evidence... are overwhelmingly convincing". Specifically, the correlation between exposure to AA and outcome, the dose-response relationship, and the consistency of the association were found to be very strong. In other words, the frequency by which individuals attend meetings appears to have a statistically significant correlation with maintaining abstinence. Kaskutas noted two studies which both found that 70% of those who attended twelve-step groups at least weekly were abstaining from alcohol consumption at follow ups two and sixteen years later. Those who attended less than once per week showed about the same success rate as those who didn’t attend meetings. Kaskutas also found AA to function consistently with known behavioral change theories and substantial empirical support for specific mechanisms through which AA facilitates change.[24]

The 2006 Cochrane systematic review reviewed studies published between 1966 and 2005 that investigated the efficacy of AA and twelve step facilitation (TSF) found no significant difference between the results of AA and twelve-step participation compared to other treatments, stating that the "available experimental studies did not demonstrate the effectiveness of AA or other 12-step approaches in reducing alcohol use and achieving abstinence compared with other treatments". This conclusion was based on a meta-analysis of the results of eight trials involving a total of 3,417 individuals;[16] the authors note that further efficacy studies are needed, and mention the presence of flaws in one included study regarding the definition of success of interventions.[16]

The review only looked at "experimental studies"; in the case of this Cochrane Review that meant that the only studies they considered were randomized controlled trials. Observational studies, including longitudinal studies, were not utilized. Kaskutas, in a 2008 letter criticizing the Cochrane review, said, among other things, that non-experimental research should have been considered:

I also urge providers to consider the mounting body of non-experimental research on AA's effectiveness, in judging what to tell patients. Experimental evidence is but one of the six criteria for establishing causation; for the other five criteria, the evidence for AA is compelling.[25]

The authors of the Cochrane review responded to this statement:

randomized controlled trials are only unnecessary when the strength of the association between two variables is sufficient to identify the treatment effect signal from the noise from bias and confounding. Nevertheless, [...] there are aspects of problematic alcohol use which can be difficult to study in experimental conditions.[25]

The review used the following studies: Brown 2002, Cloud 2004, Davis 2002, Kahler 2004, MATCH 1998, McCrady 1996, Walsh 1991, and Zenmore 2004

There have been multiple experimental studies showing Alcoholics Anonymous and twelve step facilitation effectiveness which are more recent than this Cochrane review, such as Litt et al. 2009 and Walitzer 2009.[4][17][18][12] John Kelly, who is working on the updated version of the Cochrane review concerning Alcoholics Anonymous, says that there are "positive results in favor of Twelve-Step Facilitation treatments that have emerged from the numerous NIH-sponsored randomized clinical trials completed since the original review published in 2006."[26]

A 2006 study by Rudolf H. Moos and Bernice S. Moos saw a 67% success rate for the 24.9% of alcoholics who ended up, on their own, undergoing a lot of AA treatment: Of subjects highly involved with AA in their first year of alcohol treatment, 67% were sober 16 years later. The study looked at the outcome of a group of alcoholics seeking treatment over a 16-year period. The subjects decided on their own whether to use AA, and how much AA treatment they got. The study stated that "individuals who participated in AA for 27 weeks or more had better 16-year outcomes", showing that "only 34% of individuals who did not participate in AA in the first year were abstinent at 16 years, compared to 67% of individuals who participated in AA for 27 weeks or more."[3]

The study's results may be skewed by self-selection bias.[21]

Fiorentine 1999 was a 24-month longitudinal study measuring the effectiveness of AA and Narcotics Anonymous.

At the 24 month follow up, 74.8% of people who went to one or more meetings a week self-reported being completely sober, but only 40% of people who did not go to 12-step meetings claimed to be sober. Urinalysis showed 96.6% of people regularly going to meetings as sober, in contrast to the 88.9% of people who didn't go to meetings once a week or more whose urine sample was alcohol-free.[20]

A review of older experimental trials on AA, carried out by Kownacki and Shadish in the late 1990s, showed that randomized controlled trials showed a negative effect for AA, suggesting that coercion to AA yields worse outcomes than the outcomes of people who received other types of addiction treatment and those who received no treatment. However, other trials showed that AA had a positive effect.

This paper used the following coerced experimental trials where the AA treatment was, according to Kownacki and Shadish 1999, a conventional AA meeting: Brandsma et al. (1980), Ditman et al. (1967), and Walsh et al. (1991).[27]

Project MATCH was an 8-year, multi site, $27-million investigation that studied which types of alcoholics respond best to which forms of treatment. MATCH studied whether treatment should be uniform or assigned to patients based on specific needs and characteristics. The study concluded that 12-step facilitation was as effective as the other psychotherapies studied.[28]

In 1991, Walsh and other researchers carried out a randomized controlled trial among 227 subjects. They were randomly assigned to mandated inpatient treatment, mandatory AA meetings, or allowing the subject a choice of options.[29]

This study was one of the eight studies used to determine the conclusions for Cochrane 2006.[16]

While the subjects assigned to inpatient treatment did better than the subjects only assigned to AA meetings, the inpatient treatment consisted of three AA meetings a week while in treatment, followed by a year of three AA meetings a week after the inpatient treatment ended; the study shows that inpatient treatment consisting of AA is more effective than just AA meetings.[24][12]

The 1980 book Outpatient Treatment of Alcoholism describes a mid-1970s study of 260 individuals, 184 referred by the courts and 76 self-referred or referred by other agencies. Participants were assigned randomly within five treatments, including an Alcoholics Anonymous-like meeting.[30] The study found that AA was more effective than no treatment, and about as effective as the other alcoholism treatments. Since the "Alcoholics Anonymous" treatment patients underwent in the Brandsma study did not use community Alcoholics Anonymous meetings, and since there was no effort to stop the people in the control group from attending real Alcoholics Anonymous meetings,[31] later analysis says that there are "concerns with the Brandsma trial which call its experimental results into question".[24]

A 1967 study saw a judge in San Diego randomly assign (ordered) offenders to either clinical treatment, AA treatment, or to a no-treatment (control) group (but not denied AA meetings). After one year, 68% in the clinic group were rearrested, 69% in the AA group were rearrested, and 56% were rearrested in the group receiving no treatment. No statistically significant differences between the three groups were discovered in recidivism rate, in the number of subsequent rearrests or in time elapsed prior to rearrest.[32] Like the Brandsma study, the study did not stop people not assigned to AA treatment from going to AA meetings.[31]

In 1992, the United States Census Bureau and the National Institute on Alcohol Abuse and Alcoholism (NIAAA) conducted the National Longitudinal Alcohol Epidemiologic Survey (NLAES). Direct face-to-face interviews were conducted with one randomly selected respondent, 18 years of age or older, in each of 42,862 households within the contiguous United States and the District of Columbia. NLAES respondents were asked whether they ever attended an Alcoholics Anonymous meeting for any reason related to their drinking. 1106 respondents (2.6% of the general population) stated they had attended an Alcoholics Anonymous meeting for their drinking prior to the past year. 348 respondents stated they attended an Alcoholics Anonymous meeting prior to the past year and continued to attend Alcoholics Anonymous during the past year, a 31% continuance rate.[33]

In 2001–2002, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) conducted the National Epidemiological Survey on Alcoholism and Related Conditions (NESARC). Similarly structured to the NLAES, the survey conducted in-person interviews with 43,093 individuals. This time, respondents were asked if they had ever attended a 12-step meeting for an alcohol problem in their lifetime (the question was not AA-specific). 1441 (3.4%) of respondents answered the question affirmatively. Answers were further broken down into three categories: disengaged, those who started attending at some point in the past but had ceased attending at some point in the past year (988); continued engagement, those who started attending at some point in the past and continued to attend during the past year (348); and newcomers, those who started attending during the past year (105).[34] In their discussion of the findings, Kaskautas et al. (2008) state that to study disengagement, only the disengaged and continued engagement should be utilized (pg. 270).[34]

To measure the retention of members, the 1968 AA General Service Conference voted to begin surveying those participating in AA.[35] A survey was conducted beginning that year and subsequent surveys have been conducted about every three years (triennially), the latest of which was from 2014.[35][36][37][38][39][40] The basic results of the surveys are made available in pamphlet form to AA members. Additional comments and analysis intended for academic and professional audiences were written to supplement the survey results from 1970 to 1990.[35] Non-alcoholic board of trustees member Dr. John [Jack] Norris wrote the second and third analyses. The second was presented at the North American Congress on Alcohol and Drug Problems,[41] the third was presented at the International Congress on Alcoholism and Drug Dependence.[42]

The 1990 commentary evaluated data of triennial surveys from 1977 through 1989 and found that after the first year, the rate of attrition slows. Only those in the first year were recorded by month. The survey states that the data "strongly suggests that about half those who come to A.A. are gone within three months."[43] Comments published by AA about this survey claim that 26% of people who attend AA meetings continue attending for more than one year; this is very close to the results from several independent assessments.[44][45] In the previous surveys, this group (those remaining active for ninety days) would be the only ones considered to have "tried AA".[35] After the first year, the rate of attrition slows. The nature of the survey questions asked did not allow a direct comparison between the twelfth month of the first year and the first month of the second year. Only those in the first year were recorded by month.[43] The necessity of an introductory period was not considered in the 1990 analysis, and the concept was not present in its analysis.[35]

Dodes, in The Sober Truth, says that most people who have experienced AA have not achieved long-term sobriety, stating that research indicates that only five to eight percent of the people who go to one or more AA meetings achieve sobriety for longer than one year.[14] Gabrielle Glaser used Dodes' figures to state that AA has a low success rate in a 2015 article for The Atlantic, which says that better alternatives than Alcoholics Anonymous for alcohol treatment are available.[15]

The 5–8% figure put forward by Dodes is controversial;[13] Thomas Beresford, MD. says that the book uses "three separate, questionable, calculations that arrive at the 5–8% figure."[46][47] The New York Times calls The Sober Truth a "polemical and deeply flawed book".[48] John Kelly and Gene Beresin state that the book's conclusion that "[12-step] approaches are almost completely ineffective and even harmful in treating substance use disorders" is wrong (Dodes responded by pointing out that "I have never said that AA is harmful in general"), noting that "studies published in prestigious peer-reviewed scientific journals have found that 12-step treatments that facilitate engagement with AA post-discharge [...] produce about one third higher continuous abstinence rates."[12][49] Jeffrey D. Roth and Edward J. Khantzian, in their review of The Sober Truth, called Dodes' reasoning against AA success a "pseudostatistical polemic."[50]