Partner Services Program Management & Evaluation Tools Sexually Transmitted Diseases

5 Documentation, Data Collection and Evaluation

How information is documented influences what data is collected, which will determine exactly what pieces of a program can be monitored, evaluated, and improved upon. Conversely, the activities a program evaluates will determine what data needs to be collected and how it will be documented. Developing documentation, data collection, and evaluation plans in concert helps to ensure that the right information is collected in order to monitor and evaluate an IPS program’s progress, quality, and effectiveness, and identify possible improvements to the program. Program activities that are not documented or evaluated present programmatic blind spots that are often undetected and unaddressed.

Currently, there is not a standardized way to document and evaluate IPS efforts. Programs use different data management systems and the information they collect varies widely. Below is an example of an IPS documentation form. See Appendix F for additional samples of documentation logs and Appendix G for examples of documentation procedures.

Sample Log Sheet

INTERNET PARTNER NOTIFICATION LOG SHEET

Partner #	Email Date	Website Name	Partner’s Email	Email Script #	AM Logon Time	PM Logon Time	Notes/Outcomes of Activity	Disease Verification 3rd Email)	Closure Code

Initiation Date: OP Id # and Disease

5.1 DATA SECURITY AND CONFIDENTIALITY

Standards for the collection, sharing, and security of HIV, viral hepatitis, STD, and TB data can be found in the National Center for HIV, viral Hepatitis, STD and TB (NCHHSTP) Data Security and Confidentiality Guidelines. https://www.cdc.gov/nchhstp/programintegration/docs/pcsidatasecurityguidelines.pdf
These guidelines require that all newly hired staff sign confidentiality agreements before being given access to identifiable information and require annual renewal of that agreement.³⁹ Examples of confidentiality agreements can be found in Appendix H.

Because screen names, email addresses, and telephone numbers are considered to be personal identifying information (2 C.F.R. § 200.79), and are to be held to the same levels of confidentiality as a patient’s first name and surname, programs offering IPS typically require DIS to sign additional agreements regarding the specific use of technology. For example, programs create policies describing the acceptable use of technology for PS, including what websites, apps, and social networking sites may be used for PS, what information can be shared or discussed over these mediums, expected professional and behavioral standards, and consequences for violations of the policy. These policies may also indicate whether staff may conduct IPS from personal computers and devices and other structural limitations. For an example of an acceptable use agreement, see below and Appendix I.

Example of Acceptable Use for Accessing Restricted Websites

Example- Maryland Department of Health and Mental Hygiene

Agreement for Accessing Restricted Websites

Purpose

Staff members that perform Internet Partner Services (IPS) are often required to access websites that are restricted and may contain adult oriented material during the course of their normal job duties. This agreement has been developed to establish clear expectations when restricted sites are to be accessed while performing IPS, and the consequences for acting outside of these expectations. All staff performing IPS must sign this agreement prior to being granted access to restricted websites

Agreement

gree to comply with this agreement.

___________________________ _________________
Employee Name Date

Some programs may require a non-networked computer, one with Internet access that is separate from the internal network, in order to protect servers from viruses or unauthorized intrusions, and may alleviate IT networking concerns about security breaches.

5.2 EVALUATION

5.2.1 Logic Models

Logic models are an effective way to describe the inputs, outputs, and desired outcomes of IPS and may help in developing evaluation plans.

Example Logic Model: 2014 STD-AAPPS Logic Model
Assurance: Partner services and linkage to care

Activities

Increase the provision of targeted and effective health department Disease Intervention Specialists partner services for:
1. Primary and secondary syphilis cases
2. HIV co-infected GC and syphilis cases
3. GC cases with possible GC treatment failure or suspected/probable cephalosporin-resistant GC
Link partners who have not been diagnosed previously with HIV who test positive for HIV to care
Within state law, increase the provision of EPT for CT and GC according to CDC guidelines
Increase the provision of effective partner services through social media web sites and other digital or communication technologies (e.g., IPS)
Link newly identified HIV-infected individuals in STD clinics to HIV care
Link uninsured or uninsured partners to safety net services

Outputs

Partner services

#, timeliness, & quality of PS provided (to OP and partners), including counseling/education
Formalization & use of a PS triage/ prioritization plan
IPS utilization
# EPT distributed, by provider/mode
% of eligible who were offered; who got

Linkage

#/% linkages made
Linkage protocols in place, provider connections maintained/ developed

Key outcomes

Partner services

Short-term

Identification of new cases/infections of HIV and others STDs
Reduction in sexual risk behaviors of OP and partners
Decreases in reinfection of OP and partners

Long-term

Decreased morbidity/mortality related to STD reduction (e.g., infant mortality due to CS, PID)
Decreased incidence, prevalence, cost

Linkage activities

Short-term

Increases in people receive care for HIV
Increases in people receiving care for other health issue and prevention care (eg., PreP & diabetes screening)
Increases in people receiving needed social services

Long-term

Reduced morbidity, mortality, and costs
Reduced HIV transmission
Increases in quality of life, improved social determinants of health

An example of an IPS logic model and a blank logic model for your use can also be found in Appendix J.

Once the logic model is complete, programs can consider whether to conduct a process or an outcome evaluation, or both.

Examples of possible evaluation questions and associated indictors

Both process and outcomes evaluations can provide valuable insight into the effectiveness of a program. Process evaluations can assess whether IPS activities have been implemented as intended. For example, process measures can capture whether patients are being asked about technology-based, sex-seeking, or if attempts have been made by DIS to contact online or mobile-based partners. Outcome evaluations determine the effects of the intervention on the targeted population and may include numbers of partners notified, tested, infected, and treated.

The following is a sample of PS and IPS-specific process and outcome evaluation questions and possible indicators a program may choose to consider.

iPartner is defined as any partner met via the Internet or other digital technology (e.g. mobile app). Sometimes the only means of contacting an iPartner is through these digital venues.

5.3 EXAMPLES OF PROCESS & OUTCOME EVALUATION QUESTIONS AND ASSOCIATED INDICATORS

Process Evaluation Questions	Indicator/Measure
How productive are interviews in terms of eliciting iPartners for field investigation?	Total number of interviews
	Total number of partners, regardless of locating info
	Number of patients asked about iPartners
	Number of iPartners for which some contact information was obtained (e.g. profile name, email address, phone number)
	Number of iPartners initiated for field investigation
Outcome Evaluation Question examples	Indicator/Measure
How productive are field investigations of iPartners in terms of notification of exposure?	Number of iPartners sent notification¹ of their possible exposure
	Number of iPartners contacted² about their possible exposure
	Number of iPartners for whom traditional contacting information is obtained (telephone number or physical address)
	Number of iPartners notified³ of their possible exposure
How productive are field investigations of iPartners in terms of testing/examination?	Number of iPartners previously diagnosed with HIV
	Number of iPartners tested/examined (STD or HIV)
How productive are field investigations of iPartners in terms of identifying new infections?	Number of iPartners preventively treated for STD
	Number of iPartners infected, brought to treatment, for STD
	Number of iPartners newly-diagnosed with HIV
How effective are efforts to link newly diagnosed HIV + and previously HIV + partners to HIV care?	Number of HIV+ partners linked to HIV care (includes newly diagnosed partners and previously positive partners who have fallen out of care)

¹ – The term notification refers to a message being sent to an iPartner by email, text message via a social/sexual networking site, or similar format. It does not imply that any response was received back to the DIS from the iPartner.

² – Contact refers to a message that has been sent to an iPartner and for which there is some indicator that the iPartner received and read the message, for example, the delivery of a “read receipt” back to the DIS or direct contact by the iPartner.

³ – Notify refers to a face to face or telephone conversation between the DIS and an iPartner notifying the iPartner of their possible exposure.

Some of the above questions can be further analyzed for more nuanced detail that may reveal useful insight. For example, partner notification methods can be compared to determine the most effective means of reaching partners, i.e., a field visit versus a telephone call versus an email versus a text message.

Measures can be monitored over time to track the status or performance of IPS activities.

As with any evaluation effort, the findings are most useful when used to inform, modify or further develop program planning and activities and are an integral part of the quality improvement process.

For example, evaluation results can be used to

Understand the effectiveness of IPS components (e.g., notification via email versus social/sexual networking site);
Identify technical assistance and training needs (e.g., cultural competency training);
Allocate program resources (e.g., more DIS allowed to conduct IPS); or
Identify gaps that could be addressed by increased resources (staff, equipment, funding)
Identify virtual communities in need of additional services.

5.4 SUPPORTING RESOURCES FOR PROGRAM EVALUATION

There are online resources that programs may find useful related to program evaluation. These resources are comprehensive and serve as a good reference for evaluating any STD/HIV intervention or activity, including Partner Services (PS).