As nurse educators, we are invested in ensuring that our students emerge from the educational process as competent professionals, ready to take on the complexities of the modern health care system and serve as effective and productive members of the profession. But how do we know that we have done our jobs well? Is our program effective? Did our students learn what we intended? Are they able to apply learned concepts in practice? Equally important are questions related to whether our graduates feel well prepared to become contributing members of the profession and were satisfied with the educational process. Finding the answers to these questions is not an easy task, but finding answers and acting on the information gathered is an expected component of ongoing nursing program assessment for accreditation for schools of nursing (Accreditation Commission for Education in Nursing, 2016; Commission on Collegiate Nursing Education, 2013; National League for Nursing Commission for Nursing Education Accreditation, 2016).

The process of assessing student learning can serve two purposes that should not be viewed as mutually exclusive: assessing for accountability and assessing for improvement (Ewell, 2009). In general, assessment can be defined as "the process of gathering and discussing information from multiple and diverse sources in order to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences: the process culminates when assessment results are used to improve subsequent learning" (Huba & Freed, 2000, p. 8). The mid-1980s saw a shift in focus among institutions of higher education with increased emphasis on validating educational excellence. Expectations that education be assessed and quantified in a meaningful manner in efforts to hold institutions accountable for the quality of student learning and the value of education resulted in nationwide reform.

In considering how assessment is used in nursing education, programs are required by accrediting agencies, educational institutions, and State Boards of Nursing to consider program outcomes such as graduation rates, completion rates, program satisfaction, and first-time pass rates for licensure exams, as well as student learning outcomes that may be identified in course, unit, or level objectives. These data are systematically reviewed, and curricula are revised or supported based on the findings. The use of multiple and diverse sources of information adds to the richness and usefulness of the data generated by ongoing assessment. Currently, the primary accrediting agencies for nursing education at all levels require input from alumni in the assessment process.

THE ALUMNI PERSPECTIVE

The incorporation of the alumni perspective after graduation serves to help the evaluator more fully understand how education is being used in practice, the level of preparedness of the entry-level practitioner, and the lingering perceptions of the educational process. However, tracking alumni as they enter the workforce or proceed to advanced education is a difficult, sometimes impossible task. Often, as graduates leave their home educational institutions and move on to the next phase of their careers, they also leave behind "permanent" residences and "permanent" email addresses.

This natural and expected process makes the collection of assessment data from alumni one of the most challenging aspects of program assessment, as we experienced over the years in our associate degree and RN-BSN programs. Low response rates to alumni surveys can have a significant negative impact on the statistical and practical value of the information obtained. Although a significant body of literature exists to support what needs to be assessed in higher education, little has been written about how to best access the alumni population for the purpose of program assessment. Finding novel and creative ways of maintaining connections with alumni is imperative for improving the quality of information obtained for the purpose of program evaluation - and that's exactly what we have done.

During our 2012 site visit from the Accreditation Commission for Education in Nursing, we received a recommendation from the program evaluators to explore ways to improve return rates for the graduate survey. Relying primarily on the US Postal Service for the initial and follow-up mailings of the graduate survey, return rates have been consistent with the low rate of return characteristic of this method of assessment. In an effort to improve return rates, we engaged in discussion with students at the time of graduation about when to expect the program satisfaction survey and the importance of their feedback as a critical piece to grow and develop our programs. We developed a graduate survey that did not involve more than 10 to 15 minutes of the graduate's time to complete and provided a self-addressed stamped envelope for easy return.

Despite these efforts, we continued to struggle with low return rates. Some surveys were completed and returned, but others came back with a "Return to Sender" sticker. Graduates had moved on and left no forwarding address with the university, even though we had updated addresses at the time of graduation. What we learned was that the US Postal Service was not the preferred method of doing business with the millennial generation.

THE PHENOMENON OF SOCIAL MEDIA

With the rise in freely accessible and widely used social media applications among young people, we observed that many of our students engaged constantly in such forms of communication. In time this spread to their engagement with nurse faculty members. Where the predominate method of communication with faculty had traditionally been face-to-face in the faculty office, it moved first to sending email messages and eventually to text messaging about course-related matters. Furthermore, students connected with one another online through Facebook and texting applications. As we reflected on how technology was changing the ways that we communicated and reached our constituents, it became clear that the methods we used to reach our tech-savvy graduates were out of sync with their current means of communication.

Recognizing the availability of electronic survey tools for ease in collection of data, we discussed the potential use of an online survey for graduate feedback about program satisfaction. This would eliminate the time and mailing of a paper survey and would be more convenient and time efficient for graduates. Working with our webmaster, we used the online forms application of the content management system of our website to revise our satisfaction surveys for electronic formats. We also created a private link for graduates to access the survey.

With the electronic surveys in place, we advised students to retain their university email addresses after graduation as a means for regular communication. We also secured our graduates' personal email addresses and telephone numbers and advised graduates to have their university email forwarded to their personal email. For each program, we identified a class representative prior to graduation who would assist our efforts by reaching out to graduates through email and via Facebook and encouraging the completion of the program satisfaction survey.

One year after graduation, graduates were emailed a letter and link to the website inviting them to complete the online survey. We also placed reminders about the annual online program surveys on our university Facebook page and in our biannual electronic program newsletter, Nursing Notes.

KEEPING UP WITH TECHNOLOGY

With these efforts in place, we observed some changes in the return rates, although they were not as high as we would have liked. The use of technology among our students and graduates was changing faster than our newly implemented efforts.

We then went back to our webmaster to discuss the possibility of sending the letter and survey link via text messaging to a mobile phone device. With our email and Facebook efforts in place, we added a mobile-friendly version using the online forms application of our website and sent the private link through a text message to our graduates to access the survey.

The addition of the text message led to significant and almost instantaneous increases in return rates for both programs. ASN return rates improved from 24 percent to 67 percent and RN-BSN return rates improved from 56 percent to 100 percent, as we transitioned from the paper survey via the US Postal Service to an electronic survey using email, Facebook, and mobile phone applications. Not only did the addition of text messaging increase return rates, the survey response from graduates was almost instantaneous, unlike the eight weeks or more needed for initial and follow-up mailings using the traditional paper surveys.

Surveying graduates about the effectiveness of educational programs is a necessary and beneficial process to the ongoing development and improvement of nursing curricula. The challenge for nurse educators is how to best reach and encourage graduates to complete and return satisfaction surveys. Using an incremental process with a variety of ways to reach graduates proved to be key for us in improving our return rates and securing valuable data at this point in our program development.

Although our strategy included only one form of social media, Facebook, the use of alternative social media formats such as Twitter or LinkedIn may also have a positive impact. As technologies and students continue to change, nurse educators must be open to and engage in the ways in which technologies are being used to communicate. This has the potential to lead to a more time-efficient and fruitful program evaluation process.

REFERENCES