Delivery of Student Services

 

Sub-Competencies for Competency #4

    1. Utilize research and assessment data to identify needs and establish learning outcomes for the development of programs and services
    2. Design and implement assessment tools and interpret data to inform future programs
    3. Apply knowledge of diverse audiences in the development, communication, delivery, and assessment of programs, curricula, and services
    4. Identify target populations and use appropriate marketing strategies to maximize program effectiveness
    5. Implement accessible and engaging programs incorporating innovative techniques and technology to meet the needs of a diverse audience

 

Introduction

Data can tell us about the past, present, and future in terms of statistical data for programs, demographic data about who is a part of that program and it can give us clues to where improvements need to be made. Assessment is an essential part of student services. Limited budgets cause an increasing demand in the decision-making process to decide what parts of programming is kept staying authentic to that unit’s mission and future programming must consider and assess the collected data from past programs to make decisions for how new programs should be developed. Prior to my entering the CSSA program, I was unaware of how research and data makes such a significant impact on a higher education campus. While I have always been a fan of assessment as a tool, the advanced technology and programing now available on campuses also makes me a tad nervous, as sets of data can often be manipulated, both intentionally and unintentionally as well as misinterpreted and misrepresented. A few of the bigger questions I continue to ask myself are: How are we effectively training the students, faculty and staff that will have access to data and do those trainings include FERPA protocols? Is each campus staying FERPA compliant as access to data across campus in different units becomes easier and the number of those that have access increases? On the opposite side of that, I also wonder if people are given enough data to do their jobs well when supporting students. Lastly, how are we gaining an understanding of the effects of the of data collection often over tracked marginalized communities? In the following section I will discuss how my career experience, internships and class work have addressed this competency.

 

Class Experiences

‘Research & Assessment in Higher Education’ (CSSA 513) gave me a first look at several different topics, but also provided me with a better understanding of how research and assessment works within higher education. Prior to this class I did not have much experience with data, research, and assessment other than taking a few classes during my undergraduate program. In the CSSA 513 class, I was taught the importance of accessible learning outcomes and how well written outcomes can help determine if there is a correlation between an intentional intervention and a desired outcome. If one can demonstrate that intentional intervention can increase the likelihood of a desired outcome through the assessment of learning outcomes, it is far more likely to justify the cost of the intervention. Learning outcomes need to be measurable. Hence, when one is creating learning outcomes, they must always ask themselves how such learning outcomes can be assessed (Schuh, 2009).

This class taught me the challenges and advantages of using different assessment tools. Schuh (2009) discusses quantitative and qualitative assessment tools and what validates them while also highlighting that quantitative tools are more often used for the assessment of larger samples of a population where qualitative tools are ideal for smaller sample sizes. Quantitative methods are believed by some to be more accurate descriptors of common actions, beliefs, and problems held by a large population but that those methods only scratch the surface. On the other hand, qualitative research goes deeper into issues, but lacks the capability to be applicable in large numbers (Schuh, 2009).

Quantitative assessment offers a myriad of data collection tools including structured interviews, questionnaires, and tests. In the higher education setting, this type of design is found in many nationally employed assessment tools (e.g., National Survey of Student Engagement or Community College Survey of Student Engagement) but can also be locally developed and used to assess more specific campus needs and student learning outcomes such as examining campus living or new student orientation programming. Qualitative research is narrow in scope, applicable to specific situations and experiences, and is not intended for generalization to broad situations. Different from quantitative research, it employs the researcher as the primary means of data collection (e.g., interviews, focus groups, and observations). Within higher education, qualitative research could be used to interview students to gain feedback on student health services and if there were any changes those students would make.

In higher education, part of communicating more effectively with a diverse audience is understanding our own biases and how our experiences and values shape the lens through which we view our world. I cannot assume that others share my view of the world. This misassumption creates a disconnect between us and our audience, and it can sometimes be seen as ethnocentric. We need to study our audiences and inspect our words and gestures carefully to ensure nothing suggests that we assume our way is superior to how another culture might operate. In ‘Multicultural Issues in Higher Education’ (CSSA 520), we were taught about Transdisciplinary Applied Social Justice (TASJ) which is an Afrocentric, praxis-oriented, theoretical, and methodological approach for addressing the marginalization, exclusion, and disenfranchisement of people of color, and women of color (Pratt-Clarke, 2012). TASJ reveals that projects should be informed by recognition of the power of institutions, conceptual schemes and their texts (Pratt-Clark, 2012). Being a social justice lens, I believe it can help institutions examine what they are attempting to communicate to diverse communities before doing so. During ‘Principles and Theories of Development I’ (CSSA 552), we discussed Critical Race Theory (CRT) which highlights how seeing only one social identity in someone ignores other parts that make up that person’s individuality (Patton et al., 2016); I believe CRT to be beneficial when we are thinking about how to communicate more effectively with a diverse audience because assumptions of someone’s identity, experiences or values are not welcome, so we must think about how we connect to audiences which have multiple identities present.

 

OSUAA Internship

One of the first opportunities that I had with using data to identify needs and make improvements to a program was during my internship with the OSU Alumni Association (OSUAA) in Summer 2019.For this internship, I worked with OSUAA’s career services director and director of alumni diversity and affinity groups to conduct a diversity assessment for their career programming (Artifact I). As part of this assessment, I completed an annotated bibliography with the focus of diversity and inclusion on college campuses and campus partners as it relates to career programming. I also conducted interviews with Oregon State alumni, OSU faculty/staff and representatives of other higher education institutions. In addition, we also conducted a demographic data analysis of a random sampling of alumni served by OSUAA during the 2018-19 fiscal year, as well as those who volunteered as career alumni ambassadors.

The alumni interviews taught me that during their time as students at Oregon State, they felt unsupported and/or not welcome by the university and they had to establish their own networks to find support. It was made clear no one on campus had asked questions about what they needed or what was missing from services for diverse or underrepresented alumni. Those I interviewed had encouraged the alumni association to continue to be proactive about diversity, to ask questions, to find out what was missing and what is needed and stressed how such conversations will make the most impact for diverse and underrepresented alumni. They also suggested to continue working with campus partners that students may already have a personal connection with such as the cultural resource centers or Educational Opportunities Program (EOP), so that such campus partners can help connect students who graduate to alumni career services. Interviewees also encouraged to have more representation of underrepresented alumni in the career ambassadors program, as panelists in career workshops and webinars, and other programs organized by the alumni association.

To better understand the population served by OSU alumni career services, 100 of 1,000 alumni engaged by the OSUAA during the 2018-201 fiscal year were randomly selected for demographic data analysis. The results revealed the following:

  • OSU class years represented were: 1967-2019
  • Colleges: 9 of the 11 colleges at OSU were represented including the graduate school
  • Location: 12 states were represented, and 1 other country (not including the United States) was represented as well
  • Gender: 51 were male, 49 female
  • Ethnicity: 6 were non-white, 43 undecided and 51 white 
    • Ethnicity was determined based off LinkedIn or Facebook pictures; undecided was chosen if there was a possibility of multiple races, a picture/profile could not be found or if we were unsure what they would identify with. 

As noted above, a data analysis was also conducted for the Alumni Career Ambassadors. These volunteers are alumni or friends of the university who are invested in the career growth and development of alumni and students. They are there to help with career goals, making connections to other people, giving advice to others and/or sharing their experience in their industry/region. During the 2018-19 fiscal year, OSUAA had 50 career ambassadors as follows:

  • Class years represented were: 1973-2018
  • Colleges: 8 of the 11 colleges were represented including the graduate school
  • Location: 8 states represented and 1 country outside of the United States
  • Gender: 22 are male, 28 are female
  • Ethnicity: 8 are non-white, 42 white 
    • Ethnicity was determined based off LinkedIn or Facebook pictures

Determining ethnicity from a picture of off social media (if it could be found) was not the ideal technique. I would not have chosen it myself and typically I would look to our Customer Relationship Manager (CRM) system for this information, but unfortunately ethnicities are currently a work in process within the Alumni Association’s system so not each person’s record has them available yet, so this method was picked as to act as a band aid to this situation. Pitfalls to using this method is it reduced an individual’s identity to be based on their looks and it misrepresents how these individuals identify in terms of ethnicity. This does question the reliability of my data for this project. Looking back, if I could do it over, I think it would be best to either wait to attempt this assessment until ethnicities were entered in the CRM or I could have sent a survey to these random participants to confirm their demographic information.

At the end of the assessment, I found out OSUAA career services continues to provide more services with less resources (in terms of staffing and budget) than other higher institutions’ career services programs. They work to meet the needs and wishes of our 200,000 living alumni as much as possible. At the time, OSU was one of the first universities to be working towards specific career programming for underrepresented and diverse alumni. Yes, there are areas for improvement: more representation among the Alumni Career Ambassadors and reaching more diverse and underrepresented alumni in their programs to start with. This organization is working to make progress towards their goals though; starting with conducting this benchmarking and being aware and transparent where they could improve. Having difficult conversations, asking questions, and listening to feedback from not only alumni and faculty, but colleagues at other institutions is a well-rounded approach to truly comprehend where OSUAA is in this process of better understanding diversity.

 

Ecampus Internship

During summer term 2020, I had the opportunity to do an internship with the Ecampus student success team in which I carried out an assessment of the Ecampus orientation and the first-year student experience (Artifact J) by administering a student survey through Qualtrics, conducting student success coach interviews and completing an annotated bibliography about first year student experiences within an online program. To provide a bit of background, the Ecampus online orientation is a required enrollment step for every newly admitted undergraduate, degree-seeking student. Speaking with an Ecampus success coach at a midpoint during the student’s onboarding was added to the orientation. Ecampus was curious to identify ways to improve the experience for all students by collecting feedback and other forms of data to inform improvements.

During my assessment, I learned Ecampus provides 35+ undergraduate degrees and 40+ undergraduate minor and certificate programs and has 6,975 undergraduate students. 115 current Ecampus students completed the Qualtrics survey and provided feedback about the orientation experience, and I took a closer look at the demographics of the students that provided feedback in our survey. Most represented demographics for those that took the survey:

  • 71 Female (65%), 34 Male (30%) and 5 Non-Binary (5%)
  • Ages 26-34
  • 60+ credits brought to Oregon State
  • Starting term: Summer 2020

As the Ecampus orientation moved to Canvas, Oregon State’s learning management system, it was important to check to see how incoming students liked completing their orientation using the system they would be using for classes. It was also imperative to find out how important it is to the students that topics such as online readiness, course expectations, academic success services, degree requirements, funding their education, registration, connecting with their advisor and speaking with a student success coach) are introduced during the orientation to a new university. I then asked how satisfied they were by the information provided on these topics during their Ecampus orientation. While I raised concerns about survey fatigue when it comes to online students, unfortunately, we could not come up with a better way to reach a substantial number of students in a short amount of time other than a survey.

  • Importance of Topics Being Addressed in an Orientation
    • Understanding Degree Requirements: 93.75%
    • Course Expectations: 91.23%
    • Degree Planning: 89.48%
    • Academic Success Services: 84.21%
    • Connecting with an Advisor: 84.21%
    • Funding Your Education: 78.07%
    • Online Readiness: 66.67%
    • Speaking with an Ecampus Success Coach: 47.37%
  • Satisfaction of Topics Being Addressed in the Ecampus Orientation:
    • Connecting with an Advisor: 91.15%
    • Degree Planning: 88.59%
    • Understanding Degree Requirements: 87.72% 
    • Academic Success Services: 87.61%
    • Online Readiness: 86.85%
    • Course Expectations: 85.84%
    • Funding Your Education: 72.81%
    • Speaking with an Ecampus Success Coach: 71.06%
  • 85% of students felt their time and effort were well spent during the Ecampus Orientation
  • 90% of those that completed the survey feel aware of resources and contacts introduced in the Ecampus Orientation that they can use when needed
  • Should be a modified orientation for transfer students coming from Corvallis campus – repeated information

This survey provided a great window into the minds of Ecampus students and what is important to them during an orientation process and how satisfied they are by the topics being addressed. In my opinion, it incredibly important for Ecampus to have their own specific orientation for their distanced students as their education experience likely will look different than those who are on campus in Corvallis or Bend. I was not necessarily surprised by any of the results of the survey. From conducting this survey, I learned that students prefer more information than less information particularly during an orientation, but they prefer that information to be easily accessible and organized in one place, making it easy to find. To do this, I recommended that Ecampus include a student services section in the orientation to know what services are offered to them, as it is not always clear which services Ecampus students have access to since they are distanced learners.

From this assessment, I was able to determine that Ecampus continues to be a driving force for accessible online education. Indeed, OSU’s Ecampus was ranked No. 4 in the nation by U.S. News & World Report, making it Ecampus’ seventh straight year in the top 10 (Ecampus, 2021). The Ecampus new student orientation proves to be well-rounded and close to what the students are expecting in terms of topics and satisfaction of those topics being addressed. Students like the idea of success coaching but capturing what it is can be difficult. Students and success coaches would like to see more dedicated career and financial aid resources. A case management system would also make it easier for the coaches to measure and track those they are meeting with.

 

Conclusion

Assessment is important to higher education. During my internship experiences, I had the opportunity to connect with students, faculty/staff, and alumni through qualitative and quantitative methods as I conducted assessments, but there are still improvements to be made for when I conduct projects like this. I plan on continuing to develop the skill of using research and assessment data when developing programs and learning outcomes. During the CSSA program, I have come to know that one of the best ways to help create inclusivity in communication and programming for diverse audiences is to involve those who are traditionally excluded, especially those who do not identify with the majority. It is incredibly important to me to always ensure inclusion is happening within my own interactions. Staying up to date on current research and discourse related to developing programs and communication methods that are inclusive is another goal of mine. I am by no means an expert on assessment and delivering student services. In fact, I would say this is the competency I am most interested in continuing my education with.

 


 

Resources:

Ecampus, O. S. U. (2021, January). Top-ranked in the Nation – About Ecampus | Oregon State Ecampus | OSU Degrees Online. https://ecampus.oregonstate.edu/about/top-ranked.htm#:~:text=Oregon%20State%20University%20Ecampus%20is,year%20in%20the%20top%2010.

Patton, L. D., Renn, K. A., Guido, F. M., & Quaye, S.J. (2016). Student development in college: Theory, research, and practice. San Francisco, CA: Jossey-Bass.

Pratt-Clarke, M. (2012). A black woman’s search for the transdisciplinary applied social justice model: Encounters with critical race feminism black feminism, and Africana studies. The Journal of Pan African Studies. 5(1), 83-101.

Schuh, J.H. (2009). Assessment methods for student affairs. San Francisco: Jossey-Bass.

Artifacts: