SPECIAL TOPIC

Designing a Natural Experiment to Evaluate a National Health Care–Community Partnership to Prevent Type 2 Diabetes

Ronald T. Ackermann, MD, MPH; Ann M. Holmes, PhD; Chandan Saha, PhD

Suggested citation for this article: Ackermann RT, Holmes AM, Saha C. Designing a Natural Experiment to Evaluate a National Health Care–Community Partnership to Prevent Type 2 Diabetes. Prev Chronic Dis 2013;10:120149. DOI: http://dx.doi.org/10.5888/pcd10.120149.

PEER REVIEWED

Abstract

To address the growing incidence of type 2 diabetes in the United States, UnitedHealth Group, the YMCA of the USA, and the Centers for Disease Control and Prevention have partnered to bring a group-based adaptation of the Diabetes Prevention Program lifestyle intervention to a national scale. Researchers at Northwestern and Indiana universities are collaborating with these partners to design a robust evaluation of the reach, effectiveness, and costs of this natural experiment. We will employ a quasi-experimental, cluster-randomized study design and combine administrative, clinical, and programmatic data from existing sources to derive reliable, timely, and policy-relevant estimates of the program’s impact and potential for sustainability. In this context, evaluation results will provide information about the unique role of a health care–community partnership to prevent type 2 diabetes.

Top of Page

Introduction

An estimated 79 million Americans have prediabetes and are at high risk for developing type 2 diabetes in the next 5 to 10 years (1,2). Intensive population-based efforts are needed to reduce the development of type 2 diabetes, over a short time, among people who have prediabetes (3). To help address this issue, UnitedHealth Group (UHG), the YMCA of the USA (the Y), and the Centers for Disease Control and Prevention (CDC) have partnered to create a low-cost, group-based adaption of the Diabetes Prevention Program’s (DPP’s) lifestyle intervention for implementation on a national scale.

The DPP clinical trial demonstrated the efficacy of a behavior-based lifestyle intervention to prevent or delay more than half of new cases of type 2 diabetes among adults at high risk (4). Because the DPP promotes healthful diet and moderate increases in physical activity to achieve modest weight loss, it also has benefits beyond diabetes prevention, such as improving other cardiovascular risk factors, reducing health care expenditures, and enhancing well-being (5–9). The DPP’s high programmatic costs and the frequency of ongoing face-to-face visits have made it challenging to implement routinely in the real world (10).

Community delivery of adapted versions of the DPP have demonstrated promise for achieving weight losses consistent with the DPP trial for about one-eighth the cost of the original intervention design (11–13). In 2010, UHG partnered with the Y and CDC to develop 1) standards for recognition of community organizations that offer a program consistent with the DPP; 2) new infrastructures for the training of a nonclinical diabetes prevention workforce to deliver such a population-based program; 3) processes targeting employers, health professionals, and high-risk health plan enrollees to identify people with prediabetes in the general population; and 4) initiatives to encourage such high-risk people to enroll in a community-based DPP intervention. UHG and the Y also collaborated to develop a payment structure that encourages maximal attendance and achievement of at least a 5% weight loss goal for each participant. By combining new analytic and outreach procedures with performance-based payments for the DPP, UHG has constructed a novel preventive-health benefit design that aims to expand the reach and cost-effectiveness of the diabetes prevention programming that the Y offers nationally.

The success of this initiative depends on the efficient identification of high-risk adults in the population and the willingness of those adults to enroll and maintain participation in the program (13). However, the optimal mix of strategies to maximize program participation is unknown, and the potential for financial sustainability of the program depends on whether the health improvements achieved by greater participation in the DPP are associated with reductions in future health care expenditures. Learning whether the costs and benefits of the program are distributed equitably among all high-risk people in the population, regardless of age, race, culture, or economic context, is also important.

UHG, the Y, and researchers at Northwestern and Indiana universities have partnered to design an evaluation of this natural experiment that will be both pragmatic and rigorous. Our aims are to evaluate whether 1) UHG efforts to identify and engage high-risk adults can efficiently promote use of the Y program; 2) participation in this model for DPP delivery results in meaningful weight loss; 3) use of the program reduces the need for medications to treat diabetes, high blood pressure, or high cholesterol; and 4) DPP participants have lower overall health care use and costs.

Top of Page

Evaluation Design

Our evaluation will focus on the combined elements of CDC workforce development, UHG engagement activities, and a Y model for DPP delivery that involve performance-based payments from UHG to maximize DPP participation and weight-loss effectiveness. This natural experiment will include more than 10,000 DPP participants in approximately 500 community-based DPP program sites in 44 cities throughout the United States.

Data sources and outcome metrics

Data from existing administrative, clinical, and Y sources will be evaluated. A dedicated electronic tracking and billing database, developed by UHG to help the Y administer the DPP, will allow us to analyze attendance and weight loss for program participants. Medical, pharmacy, and laboratory claims, available for all UHG enrollees, will allow us to compare changes in total and sector-specific (eg, inpatient) health care expenditures among different groups of enrollees regardless of DPP participation. Pharmacy claims will enable us to assess changes in treatment intensity for conditions linked to obesity and high metabolic risk (high blood pressure, high cholesterol, diabetes) (14). For subgroups of UHG enrollees with available test results associated with laboratory claims (approximately 20% of claims submitted nationally), we will evaluate changes in total cholesterol, low-density lipoprotein cholesterol, and hemoglobin A1c levels. Finally, data provided by the Y will enable us to explore whether regional variation in the structure and process of program implementation can help explain geographic differences in key program outcomes.

Sampling and analysis plan

One strategy used by UHG to identify prediabetes and to enroll high-risk adults in a Y-based DPP intervention involves large-scale blood glucose or hemoglobin A1c testing as part of a workplace wellness or risk assessment initiative. Because these initiatives are deployed at an employer level, we have designed a cluster-randomized encouragement trial (CRET) as an innovative component of our research design (15,16). The CRET design will allow for comparisons with a randomized control group without interfering with the natural implementation of the program.

In select regions, UHG will provide the research team with a list of large employers whom it would aim to engage prospectively for on-site testing. We will randomly assign these employers to sequential waves of outreach and release them back to UHG to initiate encouragement procedures at a rate that matches the capacity of the UHG outreach team. As some employers are released into the active encouragement arm, others will remain in abeyance to serve as controls until being released at a future date (Figure). By using an intent-to-treat framework, clients who are actively encouraged (and far more likely to participate in the DPP) can be compared with clients who are not encouraged until a later time (17).

Figure. Cluster randomized encouragement design to be used in the Diabetes Prevention Program (DPP) by UnitedHealth Group (UHG). [A text description of this figure is also available.]

Although people cannot be randomized to attending the DPP, the CRET design will also enable us to analyze the treatment response among those who do elect to participate (18). Typically, comparisons of such self-selected subgroups can introduce selection bias. However, through the use of instrumental variables analysis (18–21), the CRET design will enable us to use the status of random assignment (ie, encouragement now vs encouragement later) as an instrumental variable to construct a robust estimate of the treatment effect of the program while minimizing the threat of selection bias.

The evaluation will focus on comparable samples of high-risk UHG clients who have prior claims-based evidence of diagnosed prediabetes (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] code 790.21 or 790.22) or other indications of high metabolic risk (eg, diagnosed metabolic syndrome [ICD-9-CM code 277.7] or multiple metabolic traits such as overweight and obesity; high blood pressure; abnormal blood cholesterol). In this sample, the study will compare differences between randomized groups in the changes in laboratory tests for blood cholesterol and hemoglobin A1c, medication treatment intensities, and patterns and overall expenditures for health care. Total per-person-per-month (PPPM) health care expenditures will be compared by using 24 months of baseline and 12 month of follow-up data. We expect the cluster-randomization to yield a sample of approximately 60 employers and a minimum of 12,500 high-risk employees in each arm. Under reasonable assumptions that fewer than 30% of the observations may be missing (ie, as clients withdraw from the health plan) and that the intracluster correlation coefficient (ie, within employer) will be no more than 3%, we should have more than 80% power to detect mean differences in total health care costs as small as $300 during the 12 months of follow-up.

One possible challenge of using a CRET design is that it may prove more powerful for evaluating the effects of UHG’s encouragement efforts than for evaluating direct effects of exposure to the DPP. If, for example, the number of employers randomized to the “active encouragement” arm exceeds the capacity of UHG to successfully engage them to identify high-risk employees, then the overall “dose” of DPP exposure in the active arm will be lower than anticipated, and statistical power of the study could be reduced.

Top of Page

Conclusion

By using strong quasi-experimental methods mapped to the naturally occurring rollout plans of UHG and the Y, we aim to implement a robust study of this adaptation of the DPP program and to guide future policies about its role in the ongoing national fight against type 2 diabetes. In addition to informing UHG and the Y, our evaluation will provide information about how CDC and other public health and policy stakeholders can leverage natural experiments to build the knowledge base necessary for identifying effective policies to battle this and other population health challenges.

Top of Page

Author Information

Corresponding Author: Ronald T. Ackermann, MD, MPH, Northwestern University Feinberg School of Medicine, 750 N Lake Shore Dr, Chicago, IL 60611. Telephone: 312-503-6417. E-mail: r.ackermann@northwestern.edu.

Author Affiliations: Ann M. Holmes, Chandan Saha, Indiana University School of Medicine, Indianapolis, Indiana.

Top of Page

References

1. Diabetes statistics. American Diabetes Association; 2011. http://www.diabetes.org/diabetes-statistics.jsp. Accessed May 22, 2011.
2. Narayan KM, Boyle JP, Thompson TJ, Sorensen SW, Williamson DF. Lifetime risk for diabetes mellitus in the United States. JAMA 2003;290(14):1884–90. CrossRef PubMed
3. Narayan KM, Williamson DF. Prevention of type 2 diabetes: risk status, clinic, and community. J Gen Intern Med 2010;25(2):154–7. CrossRef PubMed
4. Knowler WC, Barrett-Connor E, Fowler SE, Hamman RF, Lachin JM, Walker EA, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 2002;346(6):393–403. CrossRef PubMed
5. Ackermann RT, Edelstein S, Narayan KM, Zhang P, Engelgau MM, Herman WH, et al. Changes in health state utilities with changes in body mass in the Diabetes Prevention Program. Obesity (Silver Spring) 2009;17(12):2176–81. CrossRef PubMed
6. Herman WH, Hoerger TJ, Brandle M, Hicks K, Sorensen S, Zhang P, et al. The cost-effectiveness of lifestyle modification or metformin in preventing type 2 diabetes in adults with impaired glucose tolerance. Ann Intern Med 2005;142(5):323–32. PubMed
7. Ratner R, Goldberg R, Haffner S, Marcovina S, Orchard T, Fowler S, et al. Impact of intensive lifestyle and metformin therapy on cardiovascular disease risk factors in the Diabetes Prevention Program. Diabetes Care 2005;28(4):888–94. CrossRef PubMed
8. Ackermann RT, Marrero DG, Hicks KA, Hoerger TJ, Sorensen S, Zhang P, et al. An evaluation of cost sharing to finance a diet and physical activity intervention to prevent diabetes. Diabetes Care 2006;29(6):1237–41. CrossRef PubMed
9. Diabetes Prevention Program Research Group. The 10-year cost-effectiveness of lifestyle intervention or metformin for diabetes prevention: an intent-to-treat analysis of the DPP/DPPOS. Diabetes Care 2012;35(4):723–30. CrossRef PubMed
10. Garfield SA, Malozowski S, Chin MH, Venkat Narayan KM, Glasgow RE, Green LW, et al. Considerations for diabetes translational research in real-world settings. Diabetes Care 2003;26(9):2670–4. CrossRef PubMed
11. Ackermann RT, Finch EA, Brizendine E, Zhou H, Marrero DG. Translating the Diabetes Prevention Program into the community. The DEPLOY Pilot Study. Am J Prev Med 2008;35(4):357–63. CrossRef PubMed
12. Ackermann RT, Marrero DG. Adapting the Diabetes Prevention Program lifestyle intervention for delivery in the community: the YMCA model. Diabetes Educ 2007;33(1):69, 74–5, 77–8. CrossRef PubMed
13. Ali MK, Echouffo-Tcheugui J, Williamson DF. How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program? Health Aff (Millwood) 2012;31(1):67–75. CrossRef PubMed
14. Schmittdiel JA, Uratsu CS, Karter AJ, Heisler M, Subramanian U, Mangione CM, et al. Why don’t diabetes patients achieve recommended risk factor targets? Poor adherence versus lack of treatment intensification. J Gen Intern Med 2008;23(5):588–94. CrossRef PubMed
15. Bradlow ET. Encouragement designs: an approach to self-selected samples in an experimental design. Mark Lett 1998;9(4):383–91. CrossRef
16. Hirano K, Imbens GW, Rubin DB, Zhou XH. Assessing the effect of an influenza vaccine in an encouragement design. Biostatistics 2000;1(1):69–88. CrossRef PubMed
17. Brown CH, Wyman PA, Guo J, Peña J. Dynamic wait-listed designs for randomized trials: new designs for prevention of youth suicide. Clin Trials 2006;3(3):259–71. CrossRef PubMed
18. Zhou XH, Li SM. ITT analysis of randomized encouragement design studies with missing data. Stat Med 2006;25(16):2737–61. CrossRef PubMed
19. Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. J Am Stat Assoc 1996;91(434):444–55. CrossRef
20. Keogh-Brown MR, Bachmann MO, Shepstone L, Hewitt C, Howe A, Ramsay CR, et al. Contamination in trials of educational interventions. Health Technol Assess 2007;11(43):iii, ix–107.
21. Sussman JB, Hayward RA. An IV for the RCT: using instrumental variables to adjust for treatment contamination in randomised controlled trials. BMJ 2010;340:c2073. CrossRef PubMed

Top of Page

---

The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors' affiliated institutions.