R U Up 4 It? Collecting Data via Texting: Development and Testing of the Youth Ecological Momentary Assessment System (YEMAS)
Carolyn Garcia
Gyu Kwon
Rachel Hardeman
Therese Genis
Sonya S. Brady
Bonnie Klimes-Dougan
Why Data Collection via Text Messaging?
Adolescents text more than they talk. And no one communicates by text messaging more than adolescent females, who average 4,050 texts per month. Boys of the same age average 2,539 and to give context to this magnitude, the next highest texting average is 1,630 texts per month among 18-24 year olds (Nielsen, 2010). Numerous interventions have capitalized on this behavior trend, testing health promotion text messaging to teens on topics such as sexual health, teen parenting, diabetes care, and physical activity (See www.texting4health.org for many study examples; Levine. et al, 2008). For example, Hookup is a new California state-wide initiative designed to provide adolescent subscribers with weekly sexual health information that includes a zip-code clinic referral mechanism to facilitate timely access to care (Braun, 2010). Text messaging thus has great promise to enhance traditional ways interventions are delivered.
In addition to being an intervention delivery mechanism, text messaging is growing in popularity as a strategy for collecting ecological momentary assessment data. Ecological momentary assessments are useful in obtaining real-time data about emotions, mood, and behaviors from adolescents in their natural settings (Shiffman, Stone, & Hufford, 2008; Stone, Shiffman, Atienza & Nebeling, 2007). These data can provide insights with respect to the range of emotions and moods that occur over a period of time during the real day-to-day experiences of adolescents. The challenges associated with recall-dependent data collection methods are well known yet few valid and reliable alternative approaches have been available to date. Those in existence until recently (e.g., direct observation, hand-written diaries) have been costly, time consuming, susceptible to deceptive reporting (i.e. falsified dates of when diaries were completed), and often unsustainable beyond the grant-funded project.
With technological advances, researchers have collected ecological momentary assessment data via electronic diaries completed with hand-held computers (i.e., a Palm Pilot) and more recently, cell phones. Interestingly, most researchers have used cell phones to merely collect and store momentary data but have relied on manual, in-person uploading of data from the phone rather than real-time data transfer. For example, Shrier, Shih, Hacker, and de Moor (2007) collected momentary assessments of sexual health behaviors of adolescents using cell phones that were locked and programmed to probe the teen at random times over a week. The data were stored and later uploaded to a computer when the phone was returned to researchers.
More recently, Dunton, Liao, Intille, Spruijt-Metz, and Pentz (in press) used cell phones to capture real time data from youth about moods and physical activity with responses stored until the phone was returned. Dunton (personal communication, 2010) stated future projects undertaken by her team will most likely use immediate data transfer protocols (rather than stored and uploaded at the end of data collection) because among other reasons, this approach provides immediate awareness of responses as well as indicators of compliance, which can trigger necessary follow-up or reminder intervention. Building on this concept, one team of researchers has begun using cell phones and text messaging technology to collect and transmit ecological momentary assessments about Australian adolescents’ alcohol use via real-time text messaging (Kauer, Reid, Sanci & Patton, 2009). Similarly, Collins, Kashdan, & Gollnisch (2003) have demonstrated the feasibility and advantages of using cell phones to collect real-time ecological momentary assessment data, also regarding alcohol consumption.
Using this available information in the literature, our team undertook a study to pilot an automated system for collecting ecological momentary assessment data from adolescents four times a day for two weeks at a time. This required us to custom build a system within the University setting to securely and automatically send unique surveys to the participants. Our study was the first, to our knowledge, to custom build a short messaging service (SMS) aka texting assessment delivery system for use with adolescents to collect and transfer real-time data about individual- and social-level factors that influence mental well-being. Benefits of this system include ease of data exchange, automation thereby reducing errors related to human effort, real-time understanding of health behaviors, and ability to capture data across numerous data points for longitudinal and trajectory data analysis.
Chapter Purpose
In this chapter we describe the collaborative development of the Youth Ecological Momentary Assessment System (YEMAS), and the specific features, including linkage to a cloud communication service provider, that foster efficient data collection, management, and storage. We highlight the broad ways in which this system can be used and applied in research with diverse populations across the U.S. and globally. We conclude with our vision for future directions, including next steps and advice for future users of our system.
Development of YEMAS
Our multidisciplinary team was comprised of academic faculty (i.e., nursing, public health, psychology), information technology (IT) professionals, and graduate students. Desirable features of the system, from a researcher perspective, were balanced with the realities of technological capabilities, shared by the IT experts. We explored options for maintaining all aspects of the system within the University structure and options that involved a non-academic third party supporting the transfer of texted data. Pros and cons of each scenario were deliberated and carefully considered in the context of security concerns and priorities for protecting research study participants and the data they share via text.
As part of our exploration, we contacted a researcher in the U.S. who was concurrently piloting real-time data collection via SMS with adolescents; his research team outsourced everything to a non-University private entity. Creating a full blown SMS Delivery System requires tremendous financial investment and resources including purchasing the hardware, software systems, and phone numbers in addition to the time intensive development of the application. By utilizing a cloud communication service such as Twilio, a system like YEMAS can offer a very cost efficient way to achieve the primary goal of delivering and receiving content via SMS. YEMAS was developed to leverage this cost-effective strategy. Also, with a University-based system, this enables us to capitalize on internal security and server features, establish the web-based system of Assessment Management System on a university web address, and use secure university log-in systems to control access.
Through numerous planning meetings, the team identified key system features and worked through challenges that were encountered. For example, an early idea was to use the email infrastructure to deliver and receive text messages to study participants; however, it became clear that the scope of data being collected and the limited capacity of data management and organization via email necessitated an alternative approach. This led to identification of Twilio (www.twilio.com), a cloud communication service provider that could facilitate both the text messaging and a more efficient mechanism for managing and exporting data for analysis purposes. Thus, YEMAS was developed as a Web-based survey application that facilitates the sending and receiving of SMS texts by integrating with a web-service application programming interface (API) solution, Twilio.
YEMAS is a custom built SMS Assessment Delivery System which operates in a Windows 2008 Server environment utilizing Oracle as the database and ColdFusion as programming language. The YEMAS Web System is comprised of five components:
- Questions: This is where all survey questions and corresponding answer lists are entered and stored.
- Surveys: This section houses all of the surveys created from the question bank. Each survey can be configured with the following parameters (See Figures 1 and 2 below):
-
- Delivery type (daily or weekly)
- Delivery time schedule (in 5 minute increments)
- Question order (random or predetermined)
- Messages: This section has three parts:
-
- SMS Queue where surveys are sent once they are assigned a day and time for release and are made active.
- SMS Results captures each of the individual survey responses, from whom each response came, and the time each response was sent.
- SMS log displays the real-time delivery log and is connected directly to Twilio.
- Participants: The researcher enters the name/ID, email address and phone number of each survey participant in this section. Participants are assigned to surveys so that the research team can select which surveys are received by some or all study participants.
- Admin: Once a survey is completed, the researcher uses this section to export the data into an Excel spreadsheet.
Figure 1. Screenshot of YEMAS Survey List for Example Project
Figure 2. Screenshot of YEMAS Survey Formatting Screen of Example Project
How YEMAS Works
The functional purpose of YEMAS is to send real-time study questions to participants in an automated, scheduled manner and to enable participants to post SMS text messages back to the system (see Figure 3).
Figure 3. YEMAS Process
The idea of the process is straightforward: The administrator constructs a survey with a list of questions that are in a SMS friendly format and schedules the survey for a specific date and time to be delivered. Each survey will be assigned to a “rented” SMS number obtained from the Twilio Service (see Figure 4 for an example survey, and Figure 5 for an example survey schedule). Once survey questions are scheduled in the system by the researcher, the YEMAS system completes a server-side schedule task that builds a daily delivery queue automatically. When the survey is due to be delivered, the system sends each question as a SMS text message to the participant’s phone number via the Twilio SMS API. The API is a set of programming standards for accessing a Web-based software application or web services. It is a software-to-software interface, not a user interface. With APIs, applications talk to each other without any user knowledge or intervention. In this way, the YEMAS system interacts with Twilio to effectively send and receive text messages with numerous study participants simultaneously.
When the participant sends an answer as an SMS text message, Twilio takes the SMS text message and posts it via HTTP to the Coldfusion-powered SMS end point in YEMAS. The system then looks at the form data to confirm that it can determine the participant’s phone number (ID number) and the assigned SMS number (which is unique to the survey delivered at a specific time of day and day of the week). Once both phone numbers are confirmed, the SMS end point stores the returned answer to the database. The collected data can be viewed to observe actual responses, response rates and trends, and then can be exported to a spreadsheet file for further analysis.
Figure 4. Example Survey
- How HAPPY were u feeling just before u got this txt?
- How TIRED were u feeling just before u got this txt?
- How ENERGETIC were u feeling just before u got this txt?
- WHERE were u just before u got this txt?
- How SAFE do u feel where u r right now?
- Were u ALONE just before u got this txt?
- Were u with ur MOM or DAD just before u got this txt?
- Describe something stressful u had to cope with
Figure 5. Example Survey Schedule
Uses for YEMAS
There are numerous uses for the YEMAS system as a research tool. From descriptive studies exploring new phenomena to randomized controlled intervention trials, ecological momentary assessment data collected via text messaging have potential to provide new and complementary insights. As a complement to traditional pre- and post-intervention self-report survey data, ecological momentary assessment data offer real-time insights that can be efficiently collected across multiple time points. Prospectively, these data could be useful in describing day-to-day fluctuations in characteristics or variables that might not be as effectively captured in traditional survey approaches, such as one’s mood, or feelings. Indeed, momentary sampling is more appropriate for those factors that one would not expect to be as constant over time, and therefore a system such as YEMAS should be used to collect data on those factors, independently, or as a complement to data collected using traditional methods.
Although YEMAS was designed initially for use with adolescent participants, the structure and system that has been created can be used with any research population. The survey questions and frequency of their delivery are factors to be determined by the research team and the research questions being asked. Because texting is so commonly used to communicate, among those of all ages and ethnicities, the YEMAS system in essence, is a web-based tool that can facilitate data collected from any target study population anywhere across the globe.
Future Directions
In developing this web-based system for collecting real-time data via SMS, our project team learned many valuable lessons that have already led to refining the system. As a work in progress, the system will continue to be improved as more researchers use it to conduct ecological momentary assessments with their participants.
Most adults and many adolescents own a cell phone in the U.S., and rates are similarly high in many countries on all continents. With many people opting to have unlimited texting, a SMS-based research methodology is a cost-effective approach (in our pilot, approximately 10% of the participants required us to provide a phone, and no one requested financial support to upgrade their plan to unlimited texting because that was what those with phones subscribed to).
Future steps for our development team include conducting a larger scale descriptive study to document the day-to-day shifts in emotions, moods, and experiences of adolescents and young adults. We plan to use this system to complement data collected in self-report surveys so that we can optimize opportunities for longitudinal and trajectory data analyses.
With additional resources, we will continue to improve the usability of the system and incorporate additional features that optimize data collected. Future steps include exploration of capacity to sync with geographical information system (GIS) data, and to adapt the system for use in an application format via smart phones or social media outlets. Technology advances in the 21st century necessitate similar advances among researchers in the ways they collect, use, and share data. YEMAS is one of many systems guaranteed to dramatically influence advances in knowledge across disciplines and professions. R u ready?
Acknowledgements
Garcia received funding support that make this project possible: a 2012 Midwest Nursing Research Society Seed Grant and a 2011 University of Minnesota Grant-in-Aid award (co-I’s included Brady and Klimes-Dougan).
References
Braun, R., Howard, H. & Levine, D. (2010). Got the Hookup: Implementing a Text-Message-Based Sexual Health Information and Clinic Referral Service for Youth. Oral presentation at the American Public Health Association Annual Meeting, Denver, Colorado, November 8, 2010.
Collins, R.L., Kashdan, T.B., & Gollnisch, G. (2003). The feasibility of using cellular phones to collect ecological momentary assessment data: application to alcohol consumption. Experimental and Clinical Psychopharmacology, 11(1), 73-78.
Dunton, G. (2010). Personal Communication by telephone on November 12, 2010.
Dunton, G., Liao, Y., Intille, S., Spruijt-Metz, D., & Pentz, M. (in press). Physical and Social Contextual Influences on Children’s Leisure-Time Physical Activity: An Ecological Momentary Assessment Study. Journal of Physical Activity and Health.
Kauer, S., Reid, S., Sanci, L., & Patton, G. (2009). Investigating the utility of mobile phones for collecting data about adolescent alcohol use and related mood, stress and coping behaviors: lessons and recommendations. Drug and Alcohol Review, 28, 25-30.
Levine, D., McCright, J., Dobkin, L., et al. (2008). SEXINFO: a sexual health text messaging service for San Francisco youth. American Journal of Public Health, 98(3), 393-5.
Shiffman, S., Stone, A., & Hufford, M. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1-32.
Shrier, L.A., Shih, M., Hacker, L., & de Moor, C. (2007). A momentary sampling study of the affective experience following coital events in adolescents. Journal of Adolescent Health, 40, 357.e1-3357.e8.
Stone, A., Shiffman, S., Atienza, A.A., & Nebeling, L. (2007). Historical roots and rationale of Ecological Momentary Assessment (EMA). In: Stone, A., Shiffman, S., Atienza, A.A., & Nebeling, L., eds. The Science of Real-Time Data Capture. New York: Oxford University Press, 3-10.
Carolyn Garcia, PhD, MPH <garcia@umn.edu>
Carolyn is Associate Professor in the School of Nursing at the University of Minnesota, with an adjunct faculty appointment in the School of Public Health. Dr. Garcia had the initial idea for this SMS project and oversaw implementation; she is committed to advancing technological tools that facilitate the understanding and promoting of health and well-being, particularly among the most vulnerable in society.
|
|
Gyu Kwon <gyu@umn.edu>
Gyu is a Systems Architect and Application Development Coordinator in Academic Health Center Information Systems, University of Minnesota. Mr. Kwon served as application developer for the YEMAS system including system design, development, and implementation of the business logic.
|
|
Rachel Hardeman, MPH <hard0222@umn.edu>
Rachel is a PhD Candidate in Health Services Research Policy and Administration at the University of Minnesota School of Public Health. Ms. Hardeman was responsible for the structural functioning and set-up of YEMAS in the pilot study, including data management; she is interested in applying her mixed method expertise in future research that informs policy changes to benefit the health of populations.
|
|
Therese Genis, BA <geni0010@umn.edu>
Therese is an Environmental Public Health Graduate student and research assistant in the Department of Nursing at the University of Minnesota. Ms. Genis was the project participant recruiter and outreach person and would like to see texting utilized more with adolescents as a role in public health promotion and risk behavior assessment in the future.
|
|
Sonya S. Brady, PhD <ssbrady@umn.edu>
Sonya is Assistant Professor in the School of Public Health at the University of Minnesota. Dr. Brady contributed to YEMAS survey content and other aspects of project design; she is interested in the use of texting and other technology-based tools to collect data and promote health among youth.
|
|
Bonnie Klimes-Dougan, Ph.D., L.P. <klimes@umn.edu>
Bonnie is a Assistant Professor in the Departments of Psychology and Psychiatry at the University of Minnesota. Dr. Klimes-Dougan conducts research with adolescents and has been collaborating on the content of the assessment and in this project.
|