Chapter 18: Surveillance Indicators

Manual for the Surveillance of Vaccine-Preventable Diseases

Printer friendly version [7 pages]

Author: Sandra W. Roush, MT, MPH

Role of Surveillance in Disease Elimination Programs

In routine disease control programs, traditional, passive disease surveillance systems are usually adequate to meet program demands despite their limitations. In contrast, in disease elimination or eradication programs, routine surveillance activities are inadequate once the goal is near. In advanced disease elimination and eradication programs, every case counts. Without adequate surveillance, elimination of vaccine-preventable diseases cannot be achieved and sustained. This chapter describes the surveillance needs for diseases in various stages of prevention and control and discusses surveillance indicators that have been developed to evaluate the appropriateness, completeness, accuracy, and timeliness of surveillance systems.

Traditionally, communicable disease surveillance programs have relied on passive reporting, in which reports are received from physicians and other providers. For diseases and conditions for which laboratory confirmation is routinely obtained, laboratory-based reporting has virtually replaced traditional provider-based reporting in many jurisdictions, because case ascertainment is far more complete.[1, 2] However, even when supplemented by laboratory-based reports, reporting in traditional passive surveillance systems remains incomplete. Despite this limitation, these data remain useful because they are used primarily for monitoring trends in disease occurrence rather than for initiating public health action in response to each individual case.

In disease elimination programs, the role of surveillance is different. To achieve a goal of zero cases of a disease, aggressive efforts must be made to identify factors that allow cases to continue to occur despite the low incidence of disease. The occurrence of these cases may indicate the need for new prevention strategies, but in order to track the impact of any such strategies, surveillance data are essential. In addition, timely notification is necessary so that public health action can be taken to limit spread of disease.

This was illustrated during the global smallpox eradication program. The continued occurrence of cases of smallpox despite high vaccination coverage led to the development of a new strategy for smallpox eradication; i.e., a wide circle of contacts around each case-patient was identified and vaccinated, creating a wall of immunity around the remaining patients. This led ultimately to the global eradication of smallpox.[3] It could not have been achieved without recognition of the need for an additional strategy and without the ability to rapidly identify and respond to individual cases. Andrews and Langmuir wrote in 1963, “To achieve and maintain the eradication status of a specific disease within an area, it is necessary 1) to obstruct transmission until endemicity ceases, and 2) to prevent or nullify the reestablishment of the disease from carriers, relapsing cases, or imported sources of infection. Accordingly, an adequate surveillance organization must be developed to identify and cope with these threats to the achievement of disease eradication.”[4]

Top of Page

Development of Surveillance Indicators

Because of the essential role of surveillance in disease elimination, methods to monitor its quality were developed in 1988 by the Pan American Health Organization (PAHO) as part of the polio eradication effort in the Western Hemisphere. Surveillance indicators included measures of surveillance infrastructure (e.g., the number of reporting units reporting on a weekly basis), timeliness of notification (e.g., the interval between case onset and notification), adequacy of case investigation (e.g., the proportion of cases with appropriately timed laboratory specimens obtained), and timeliness of laboratory testing.[5] Although not generally done outside of evaluation projects in routine disease control programs, monitoring these attributes would undoubtedly provide useful information for any surveillance system. These attributes overlap with those recommended by CDC for evaluation of surveillance systems[6] (see Appendix 21, "Updated Guidelines for Evaluating Public Health Surveillance Systems").

Indicator of reporting completeness

The unique requirements of surveillance in disease elimination programs led PAHO to also develop an indicator that allowed monitoring of the completeness of reporting. In disease elimination programs, it is critical to have some measure of the adequacy of case ascertainment as well as a measure of how well cases were investigated once they are reported as suspected cases. It is not sufficient to adequately investigate the reported cases if most of the cases are never reported. More importantly, as disease incidence declines, it becomes increasingly difficult to interpret the absence of reported cases. How can you tell if zero means zero? Does it mean there were no cases, or does it mean no one looked?

PAHO developed one successful strategy to address this problem during the polio eradication effort in Latin America. Surveillance was performed not for paralytic poliomyelitis but for a syndrome that includes both paralytic polio and other conditions, including Guillain-Barré Syndrome (GBS), among children younger than 15 years of age—that is, the surveillance system was organized to identify cases that were clinically consistent with polio (suspected cases), and then to track them as laboratory investigation was performed to either accept or rule out a diagnosis of polio due to wild poliovirus. If adequate laboratory testing was not obtained to definitively determine or rule out the diagnosis of polio, the case was classified as compatible and considered a failure of case investigation and surveillance. Because in the absence of polio, GBS and other conditions causing acute flaccid paralysis (AFP) in children occur at a fairly constant rate over time, the adequacy of ascertainment of suspected cases of polio could be monitored by tracking the incidence of AFP among children younger than 15 years of age. In countries or regions reporting rates of AFP of 1 per 100,000 children younger than 15 years of age and without confirmed or compatible cases of polio, one could be reasonably confident that the absence of reported cases of polio in fact meant the absence of polio. In contrast, if AFP rates were less than 1 per 100,000 among children in this age group, the absence of cases might reflect inadequate surveillance rather than the absence of polio. Monitoring the rate of AFP reporting in Latin America was a critical component of PAHO’s effort to monitor the adequacy of polio surveillance. By tracking this closely at the regional and national level, investigators could identify and assist areas with inadequate surveillance and document resulting improvements.

Unfortunately, few other examples of vaccine-preventable diseases exist for which indicators analogous to the AFP rate are known. No external standard for determining the completeness of measles surveillance exists that would be equivalent to the rate of AFP in the surveillance of polio.[7]

While monitoring all cases of AFP is highly sensitive, it is not specific. Another part of the PAHO approach is essential—that is, classifying incompletely evaluated cases as “compatible.” In a disease elimination program the aim is to capture all the true cases by having a case definition that is very sensitive; nonetheless, it is also important to exclude non-cases by adequate case investigation and laboratory testing. The PAHO strategy captured both these elements, enhancing sensitivity and specificity of the surveillance system.

Top of Page

Surveillance Indicators in the United States

The purpose of vaccine-preventable disease surveillance indicators in the United States is to ensure adequate performance of the essential components of surveillance and case investigation, and to identify components of each that need improvement. Surveillance indicators for selected vaccine-preventable diseases were proposed by CDC and approved by the Council of State and Territorial Epidemiologists (CSTE) in 1994. Since then, the indicators have continued to evolve to maximize their usefulness. CDC currently monitors the following indicators on a regular basis.

Indicators for measles surveillance

• The proportion of confirmed cases reported to the National Notifiable Disease Surveillance System (NNDSS) with complete information (clinical case definition, hospitalization, laboratory testing, vaccination history, date reported to health department, transmission setting, outbreak related, epidemiologic linkage, date of birth, and onset date)
• The interval between date of symptom onset and date of public health notification
• The proportion of confirmed cases that are laboratory confirmed
• The proportion of cases that have an imported source
• The proportion of cases for which at least one clinical specimen for virus isolation was submitted to CDC
• The number of discarded measles-like illness (MLI) reports (discontinued January 2006)

Indicators for mumps surveillance

• The proportion of confirmed cases reported to NNDSS with complete information (clinical case definition, hospitalization, laboratory testing, vaccination history, date reported to health department, transmission setting, outbreak related, epidemiologic linkage, date of birth, and onset date)
• The interval between date of symptom onset and date of public health notification
• The proportion of confirmed cases that are laboratory confirmed
• The proportion of cases that have an imported source

Indicators for rubella surveillance

• The proportion of confirmed cases reported to NNDSS with complete information (clinical case definition, hospitalization, laboratory testing, vaccination history, date reported to health department, transmission setting, outbreak related, epidemiologic linkage, date of birth, and onset date)
• The interval between date of symptom onset and date of public health notification
• The proportion of confirmed cases that are laboratory confirmed
• The proportion of cases that have an imported source
• The proportion of confirmed cases among women of child-bearing age with known pregnancy status

Indicators for Haemophilus influenzae type b invasive disease surveillance

• The proportion of cases reported to NNDSS with complete information (clinical case definition—species, specimen type; vaccination history; and serotype testing)
• The proportion of cases among children younger than 5 years of age with complete vaccination history
• The proportion of cases among children younger than 5 years of age in which an isolate was serotyped

Indicators for pertussis surveillance

• The proportion of cases reported to NNDSS with complete information (clinical case definition, complications, antibiotic treatment, laboratory testing, vaccination history, and epidemiologic data [e.g., outbreak/epidemiologic linkage])
• The interval between date of symptom onset and date of public health notification
• The proportion of cases meeting clinical case definition that are laboratory tested
• The proportion of case-patients with complete vaccination history

Additional Approaches and Future Directions

Although these indicators have proved useful for identifying major problems with case investigation and reporting, given the small number of cases of most vaccine-preventable diseases now reported in the United States, a critical issue remaining is the sensitivity of the surveillance system, i.e., does the absence of cases from a particular jurisdiction indicate that there were in fact no cases?

One approach to improving the completeness of reporting is to implement active surveillance, that is, to make contact and solicit reports from all providers and institutions responsible for reporting on a regular basis. Active surveillance has been shown to increase reporting of measles, rubella, salmonellosis, and hepatitis in demonstration projects but is generally too expensive to perform routinely.[8, 9]

Active surveillance is supported by the following assumptions:

• Cases are occurring in the community.
• Case-patients seek medical attention or otherwise come to the attention of institutions subject to reporting requirements.
• The condition is recognized by the provider or institution.
• Cases are not reported because filling out reporting forms or calling the health department is too much trouble.
• If the administrative reporting burden for providers is reduced, cases will be reported.

For rare diseases (i.e., most vaccine-preventable diseases in the United States) these conditions are rarely met. Indeed, previous demonstrations of the usefulness of active surveillance have focused on diseases that were relatively common or at least endemic in the population under surveillance. In many communities and states, no cases of measles or rubella have occurred in years, and in the absence of a large, ongoing outbreak, participating in active surveillance for these conditions is unlikely to be of much interest to providers.

As part of the polio eradication effort in the Western Hemisphere, PAHO instituted a system of weekly negative reporting that allowed them to monitor the surveillance infrastructure (i.e., the number of clinics and other facilities that participated in the surveillance system). Each reporting unit was to include in the weekly notifiable diseases report not only cases of disease identified, but for AFP only, a negative report if no cases were identified that week (i.e., “no cases of acute flaccid paralysis”). It was implicitly assumed that any such cases would be recognized because the patient would seek medical care. This was an attempt to gain the benefits of active surveillance within a passive surveillance system without the investment of resources needed to conduct active surveillance. However, an evaluation in one country suggested that at the local level, negative reporting was not accompanied by efforts at case finding, and substantial training was needed to make negative reporting meaningful at the local level.[10]

What approach can provide firm evidence that the absence of reported cases means the absence of disease in the population? Several methods may be useful: application of external standards, identification of imported cases, monitoring the level of reporting for suspected cases that are ruled out as cases by epidemiologic and laboratory investigation, monitoring diagnostic effort, and monitoring circulation of the organism.

External standards

As discussed above, monitoring the rate of AFP among children younger than 15 years of age was found to be a powerful tool in ensuring the adequacy of surveillance during the polio eradication program in the Western Hemisphere. Unfortunately, a similar external standard does not exist for measles or for most other vaccine-preventable diseases. However, an external standard may exist for invasive disease due to Haemophilus influenzae type b. Data from an active laboratory-based surveillance system suggest that among children younger than 5 years of age, non-type b invasive disease occurs at a rate of about 1.6 per 100,000.[11] If this rate is relatively stable over time in different geographic areas, it can serve as an external standard for monitoring the quality of reporting of type b invasive disease. In 1991, H. influenzae invasive disease became nationally notifiable; cases caused by type b and non-type b strains are included in the NNDSS. Because invasive disease due to non-type b H. influenzae strains are not prevented by vaccination in any age group and because type b cases continue to occur among adults, the absence of reported cases of invasive H. influenzae disease of any type in any age group in a jurisdiction strongly suggests that surveillance is inadequate.

Identification of imported cases

One indirect measure of the quality of case ascertainment at the national level is the demonstration that a surveillance system is sufficiently sensitive to detect imported cases. At the state level, if no importations are identified and reported, this may reflect either the absence of disease or the absence of effort to identify cases. Cases in persons who are not permanent residents of the United States are probably less likely to be reported and adequately investigated than cases in permanent residents for a number of reasons: visitors may not have access to medical care, may be only briefly in an area (making it difficult to complete an adequate case investigation), or may avoid contact with authorities if they are in the United States without appropriate documentation. Single cases of measles—usually with no or very little spread—are often reported, investigated, and confirmed in the United States.[12] In jurisdictions in which no US-acquired cases are reported, the demonstration of imported cases provides good evidence for a well-functioning surveillance system. This concept is listed as a measles surveillance indicator (the proportion of cases that have an imported source).

Endemic transmission of measles has been eliminated in the United States; evidence for this determination rests on the performance of the surveillance system.[13-15] Although measles is now rare throughout the Western Hemisphere, it is endemic in many countries of Western Europe and Asia. Endemic transmission of rubella has also been eliminated from the United States, although international importations continue to be identified. Importation of measles or rubella by travelers from foreign countries occurs frequently and is expected, especially from countries with endemic disease and substantial numbers of international travelers. Failure to detect such cases would suggest that, at the national level, surveillance is not sensitive enough to detect individual, US-acquired cases.

Monitoring cases that are ruled out

Another approach to tracking the quality of case ascertainment is to track the number of cases of suspected disease that are reported, investigated, and ruled out as cases. This approach was employed by PAHO in the polio eradication program in the Western Hemisphere. Even though polio had become an extremely rare disease, suspected cases continued to be reported throughout the region and were aggressively evaluated, including obtaining appropriately timed laboratory specimens. In this way, thousands of cases were demonstrated not to be polio, providing a measurement of system performance. Likewise, cases of acute flaccid paralysis that were not adequately investigated were classified as compatible and indicated a failure of surveillance and case investigation.

In 1997, surveillance for discarded measles-like illness (MLI) was established and has been used to track the quality of measles surveillance and case investigation at the state level.[16] When such information was available, the simultaneous demonstration that

1. many cases were reported and
2. nearly all were ruled out as measles by appropriate investigation provided some assurance that efforts were being made to identify cases of measles and that once a case was reported, investigation was adequate.

The assurance of the strength of the surveillance system provided support for the determination that indigenous transmission of measles had been eliminated in the Unites States.

With elimination of indigenous measles transmission in the United States, discarded MLI as a surveillance indicator was no longer useful and was discontinued in the United States as of January 1, 2006. Collection of MLI data was difficult in some areas, and it required collecting a good deal of information on cases that ultimately were ruled out, which, outside of special evaluation projects, might be considered an inappropriate use of limited resources. Also, in the United States, there is great variation in the delegation of responsibility for case investigation; in many states, it is delegated to city and county health departments. When cases were diagnosed at the local level and measles was almost always ruled out, requiring that every suspected case of measles be reported to the state was challenging. Therefore, although state-level staff may have recognized the usefulness of collecting this information as a performance measure, the necessary information may not have been available at their level. At present, without an external standard, uncertainty remains regarding how many cases of suspected measles should be reported and investigated in a population in the absence of the introduction and circulation of measles virus.

Monitoring diagnostic effort

Given the difficulties in collecting data on reported cases that are ruled out as cases, another approach to surveillance assessment could be to measure diagnostic effort. Diagnostic effort indicates the level of suspicion of a vaccine-preventable disease; if disease is suspected, appropriate laboratory testing should be done to confirm (or rule out) that suspicion. For example, this may be used for evaluation of pertussis surveillance; tracking the number of pertussis specimens submitted over time, even if none are positive, provides good evidence that the diagnosis is being considered even if no cases are found. A similar approach could be used for other vaccine-preventable diseases by tracking submission of laboratory requests for diagnostic testing (e.g., IgM antibody tests for measles, mumps, or rubella). If no testing is being done, no one is looking.

Consolidation of laboratory functions and development of standards and systems for electronic reporting of laboratory data make this approach feasible without developing new data collection systems. If testing occurs, the diagnosis is being considered, so the absence of reported cases is more likely to reflect the absence of disease. Without an external standard, how much testing is “enough” is still open to question, but this approach does capture those suspected cases that are evaluated in the private sector but are not reported as “suspected cases.”

Monitoring circulation of the organism

One adjunct to case surveillance is surveillance for the agent (the virus or bacterium that causes the disease). Molecular typing methods exist for measles, rubella, diphtheria, pertussis, and polio and have been used to supplement the information collected in case surveillance for all these diseases. Monitoring the organism can provide information about its origin, evidence of repeated introduction from multiple sources, and evidence of endemic transmission. For example, the demonstration of endemic transmission of multiple strains of toxigenic Corynebacterium diphtheriae in a Northern Plains Indian community provided evidence of an ongoing public health problem in the absence of reported cases.[17] Molecular epidemiology has also been critical in demonstrating the interruption of endemic transmission of measles in the United States and the increasing importance of importation of measles cases.[18] Similar methods applied to isolates of rubella virus from infants with congenital rubella syndrome and persons with rubella in the United States[19] have been instrumental in documenting the elimination of endemic transmission of rubella in this country.[20] Ultimately, as diseases progress toward eradication, monitoring circulation of the organism becomes an essential component of surveillance activities.

 Top of Page

---

References

1. Harkess JR, Gildon BA, Archer PW, Istre GR. Is passive surveillance always insensitive? An evaluation of shigellosis surveillance in Oklahoma. Am J Epidemiol 1988;128:878-81.
2. Standaert SM, Lefkowitz LB, Jr., Horan JM, Hutcheson RH, Schaffner W. The reporting of communicable diseases: a controlled study of Neisseria meningitidis and Haemophilus influenzae infections. Clin Infect Dis 1995;20:30-6.
3. Foege WH, Millar JD, Lane JM. Selective epidemiologic control in smallpox eradication. Am J Epidemiol 1971;94:311-5.
4. Andrews JM, Langmuir AD. The philosophy of disease eradication. Am J Public Health 1963;53:1-6.
5. Pan American Health Organization. Polio eradication field guide. Washington, DC: PAHO, 1987.
6. CDC. Updated guidelines for evaluating public health surveillance systems: Recommendations from the Guidelines Working Group. MMWR 2001;50(No. RR-13):1-35.
7. CDC. Measles eradication: recommendations from a meeting cosponsored by the World Health Organization, the Pan American Health Organization, and CDC. MMWR. 1997;46(No. RR-11):1-20.
8. Vogt RL, LaRue D, Klaucke DN, Jillson DA. Comparison of an active and passive surveillance system of primary care providers for hepatitis, measles, rubella, and salmonellosis in Vermont. Am J Public Health 1983;73:795-7.
9. Thacker SB, Redmond S, Rothenberg RB, Spitz SB, Choi K, White MC. A controlled trial of disease surveillance strategies. Am J Prev Med 1986;2:345-50.
10. Wharton M. Weekly negative reporting of acute flaccid paralysis: Venezuela, 1991. EPI Newsletter 1992;14:4-5.
11. CDC. Progress toward eliminating Haemophilus influenzae type b disease among infants and children—United States, 1987-1997. MMWR 1998;47:993-8.
12. Vitek CR, Redd SC, Redd SB, Hadler SC. Trends in importation of measles to the United States, 1986-1994. JAMA 1997;277:1952-6.
13. Harpaz R, Papania MJ, McCauley MM, Redd SB. Has surveillance been adequate to detect endemic measles in the United States? J Infect Dis 2004;189 (Suppl 1):S191-5.
14. Harpaz R, Papania MJ, Fujii KE, Redd SB, Wharton ME, Redd SC, et al. Lessons learned from establishing and evaluating indicators of the quality of measles surveillance in the United States, 1996-1998. J Infect Dis 2004;189 (Suppl 1):S196-203.
15. Harpaz R. Completeness of measles case reporting: review of estimates for the United States. J Infect Dis 2004;189 (Suppl 1):S185-90.
16. Harpaz R, Papania MJ. Can a minimum rate of investigation of measles-like illnesses serve as a standard for evaluating measles surveillance? J Infect Dis 2004;189 (Suppl 1):S204-9.
17. CDC. Toxigenic Corynebacterium diphtheriae—Northern Plains Indian Community, August-October 1996. MMWR 1997;46:506-10.
18. Rota JS, Heath JL, Rota PA, King GE, Celma ML, Carabana J. Molecular epidemiology of measles virus: identification of pathways of transmission and implications for measles elimination. J Infect Dis 1996;173:32-7.
19. Frey TK, Abernathy ES. Identification of strain-specific nucleotide sequences in the RA 27/3 rubella virus vaccine. J Infect Dis 1993;168:854-64.
20. CDC Elimination of rubella and congenital rubella syndrome—United States, 1969-2004. MMWR 2005;54(11):279-82.

 Top of Page