Running head: EVALUATION PLAN 1

Evaluation Plan: Dominican University Family SOAR

Meghan Coletta, Tanner Strong, and Elizabeth Thomas

Loyola University Chicago

EVALUATION PLAN 2

Table of Contents
Introduction 3
Program Context 3
Statement of the Problem/ Purpose of Evaluation 6
Program and Evaluation Stakeholders 7
Logic Model Description 9
Quantitative Approach 12
Research Design 12
Sampling Strategy and Validity 13
Survey Instrument 13
Implementation 14
Analysis of Data 15
Limitations 16
Qualitative Approach 16
Participants 17
Phone Interview Procedure 18
Positionality Statement 18
Implementation Plan 19
Data Analysis 20
Coding Procedures 20
Validity and Ethics Considerations 20
Timeline 21
Budget 21
Next Steps 22
References 24
Appendices
Appendix A: Family SOAR Session Presentation-English 25
Appendix B: Logic Model 42
Appendix C: 2015 Family SOAR Evaluation 43
Appendix D: 2017 Family SOAR Evaluation 44
Appendix E: Phone Interview E-mail Invitation 46
Appendix F: Phone Interview Procedure 47
Appendix G: Budget 49
Appendix H: Evaluation Plan Timeline 50
EVALUATION PLAN 3

Evaluation Plan for Family Student Orientation and Registration at Dominican University

Dominican University is a small liberal arts catholic institution that is located in River

Forest, IL. Dominican has about 2,000 undergraduates and about 1,500 graduates along with a

rigorous academic life that continues to engage students constantly in social justice

conversations. This institution was founded in 1901 and was originally called Rosary College

that only women could attend. By the 1950s it began to switch to a co-ed institution to engage

with more students that desired a social change in the world. In the 1990s it switched the name

Dominican University to make the institution more appealing for both men and women.

The Family Programming or Family Student Orientation and Registration (SOAR), at

Dominican University was designed to provide information to the family members of students.

By collaborating with various departments that piece together valuable information to the

participants, such as financial aid, academic and campus life, and other area functions, this half

day session allows participants to learn their next steps in helping their students on campus.

These sessions happen five times throughout the summer. Participants can sign up for these

sessions online and happen on the first day of SOAR. Family members have the option to attend

one of the four to five sessions offered during the summer. Since this program is still fairly new

it is still being evaluated for effectiveness through surveys and word of mouth and how it can

continue to update and be more prepared for next years Family SOAR.

Program Context

Dominican University provides a two-day program for their students during SOAR that

allows them to stay on campus overnight to connect to the community. In past years, on day one

of SOAR, parents and other family members would drop off their students on campus and attend
EVALUATION PLAN 4

a small session, Coffee and Conversation, that would last for an hour. This conversation would

be directed by the Associate Dean of Students, Norah Collins, an admission counselor, and also

by the Dean of College and Arts and Sciences. At this time, Coffee and Conversation, was

attended by about 10 individuals per session, or in some cases less than 10. In most cases family

members were told to attend in the fall for a Welcome Weekend program that housed a rotation

of sessions on next steps and what the student would be doing throughout their two to four years

at Dominican University. The program happens the weekend before classes and while family

members are in these sessions the students have the opportunity to attend their own session to

learn more about the institution.

The idea for new programming came after gathering initial feedback from family

members who had attended SOAR and specifically the Coffee and Conversations session.

Questions from participants who attended the session and did not primarily speak English would

be directed to another staff member, who could be from the Wellness Center, Academic

Advising, or any office that employed a willing staff member who was fluent in Spanish.

Because Latinx students are the majority ethnic group on Dominicans campus, questions from

Spanish-speaking families were occurring frequently during each orientation session throughout

the summer. This left the staff helping with orientation feeling exploited or overused for their

Spanish-speaking abilities, especially when family members would attempt to have all of their

questions about the institution and their students transition answered by this one staff member.

As a result of this feedback from staff and supporters, Trudi Goggin and other leadership from

the Dean of Students office concluded that there needed to be a more concrete experience created

for Spanish-speaking families and supporters and in 2014 set out to give them an orientation

identical to the program provided for English-speaking supporters.

EVALUATION PLAN 5

However, the program lacked the resources in providing enough information to the

family members, and the presentations were themed with different topic ideas that remained

surface-level. This conclusion was reached by talking to other professional staff on campus as

well as talking to the students of family members. One of the major omissions in was the lack of

information in Spanish for those who identified as Latinx. Dominican Universitys highest

population consists of Latinx students and a majority of the students identify as first generation.

The second highest population on campus are those who identify as White. It became highly

noted that there was a need for Spanish speaking faculty members and also the desire to provide

more information to those families (T. Strong, personal communication, August 24, 2016).

The Family Program has evolved over the past two years with the Dean of Students

Office from a simple coffee and conversation that was provided on day one of SOAR and also

from Welcome Weekend that happens once a year at the beginning of the academic semester to a

program that is provided five times over the summer on day one of SOAR. Since then the

program has allowed family members to attend these sessions in either Spanish or English. Each

presentation (See Appendix A) contains information on academics, campus life, wellness center,

financial aid, student accounts, and other key functional areas that allow the family participants

to learn about their role in helping their student. For those attending the presentation in Spanish,

the Dean of Students, Trudi Goggin, compiled a list of people who would be able to help. This

list consists of people from various departments who can help bring knowledge for the

presentations and resources family members and their students might need. Currently,

Dominican University lacks resources in having enough faculty members who speak Spanish.

After taking note about how the university lacked in the presentation for those who are

bilingual and speak Spanish it was evident that there needed to be people who could help
EVALUATION PLAN 6

translate. The presentations in Spanish are translated by a hired interpreter and also by John

DeCostanza from University Ministry, Rosa Varela, the Assistant Director of Admissions, and

Ricardo Aguilera from Academic Advising. The interpreter, Carol Bernard, attends the sessions

and provides simultaneous translation. Carol was hired onto the team from a company called

InterpreNet which is housed in downtown Chicago, IL. Founded in 2003, InterpreNet is one of

Chicagos top on-site interpreter service with accreditations and approvals from Better Business

Bureaus, The Association of Language Companies, International Medical Interpreters

Association, and the U.S. General Services Administration.

In order for participants to get to these sessions on campus from various spots around

campus they are guided by Torch Leaders. Torch Leaders are known as the Orientation Leaders

on campus. The name Torch Leader comes within the history of the institution. The story is

described as St. Dominics mother having a dream while she was pregnant with him. In this

dream, he was a dog carrying a torch that enlightened the world. The name Torch Leader derives

from student leaders guiding the incoming students to find their truth on campus. This also ties

into the universitys mission statement: caritas et veritas that translates to love and truth. Torch

Leaders are student leaders on campus who are trained to guide the incoming class with

knowledge of the university. During Orientation, two bilingual students guide Spanish-speaking

family members and two other students guide English-speakers to their respective sessions.

During their campus traveling the Torch Leaders answer questions and give a mini-tour as well.

Statement of the Problem/ Purpose of Evaluation

The purpose of this evaluation is to estimate the knowledge attendees are gaining as a

result of attending the bilingual orientation. As stated before this is needed because Dominican

University has not had a bilingual family session created until recently. This entire population of
EVALUATION PLAN 7

family members have not been fully acknowledged. Because the program is only a few years

old, we will be using a formative approach to assess whether or not Spanish-speaking Latinx

families are being adequately supported throughout this portion of their students transition to

Dominican University.

Program and Evaluation Stakeholders

Evaluation of stakeholders (Table 1) is key to creating a useful and valuable evaluation

that can be used for change by those involved at all levels. Bryson and Patton (2010) define

stakeholders as, individuals, groups, or organizations that affect or are affected by an evaluation

process or its findings. The definition is purposefully broad so that the full range of possible

stakeholders is considered (2010, p. 31). This open definition allows our stakeholders to be

more open to those who take part within these sessions either on the administrative side to the

attendees. The evaluation from the stakeholders allows this program to flourish and move

forward. The players, meaning those who participate within this program being either the actual

attendees or administrative staff, have a high interest and power that give them great control over

program content and provided to the attendees that attend the session (Appendix A). The key

players for this program are the administrator staff members that engage in each orientation

session from various campus units. These include, Financial Aid, Student Accounts, Admissions,

University Ministry, Office of Student Involvement (OSI), Dean of College of Arts and Sciences.

Each department brings a representative or two from a variety of job titles from Vice President to

entry level staff workers to ensure a plethora of ideas brought together from different

experiences. They provide specific information to the attendees of the sessions that they create

based on their own knowledge of what incoming students supporters and families want and need

to know the most. Trudi Goggin, Dean of Students, has been involved in the evolution of the
 EVALUATION PLAN 8

program since its inception. She began the original version with Coffee and Conversations and

through incorporation of new knowledge and attendee evaluations the program continues to

change each year. The main subjects were identified as new students that were in the SOAR

program, those who attend the sessions, and torch leaders that are working with both of these

groups. Their high interest is why the program is in existence but they have low power in making

large changes to the program. The crowd has both low power and interest which in this case are

the other faculty and staff at Dominican University as well as the English-speaking students and

those who are not engaged in the program. The context-setter is the Board of Trustees at

Dominican University. They hold high power over the program based on the recommendations

that they make for Dominican University as whole and budgets that they approve.

Table1:

Evaluation of Stakeholders
Subjects: Interest(High)- Power (Low) Players: Interest (High)- Power (High)
New Students in the program Trudi Goggin, Dean of Students
Participants that go through the program Carol Bernard, Interpreter
Torch Leaders (SOAR Program Orientation Campus Representatives that present at sessions
Leaders) from:
Dean of Students student workers Admissions
Residence Life
Office of Student Involvement (OSI)
Financial Aid
Student Accounts
University Ministry
Dean of College Arts and Sciences
Disability Support Services (DSS) -
Housed in Dean of Students Office
(DOS)
Wellness Center
The Crowd: Interest (Low)- Power (Low) Context-Setters: Power (High)- Interest (Low)
 EVALUATION PLAN 9

Other Faculty and Staff not in the program Board of Trustees

Other Dominican University students not in the President
program

Logic Model

The logic model (Appendix B) was generated in order to have a more complete

understanding of all of the vital pieces of the program and the university for our evaluation.

McLaughlin and Jordan (2010) define a logic model as, a plausible and sensible model of how a

program will work under certain environmental conditions to solve identified problems. The

elements of the logic models are resources, activities, outputs, short-term outcomes, intermediate

outcomes, and long-term outcomes, (2010, p.56). It allowed us to see how certain inputs and

outputs affect the expected short, medium and long term outcomes of the program.

The model outlines the inputs for the program which include staffing, finances, and

resources. The outcomes for the program are the activities that occur through the academic year

and are the key pieces that will be evaluated for the program. The participation identified ranges

from players identified in the stakeholder analysis to the crowd-setters. All have various

functions but are necessary in the facilitation of the program.

The short term outcomes align with the immediate functions of the program and the

knowledge attendees leave a session with and that they should know they have obtained valuable

information. Those who engage in this session will develop a working knowledge of the

following units and services: how financial aid works/applies to their student(s), what it means to

support their student through college, and being aware of the institution's mission statement: a

more just and humane world. The medium outcomes are about the outcomes that will occur

during the entire academic year. The hope is to maintain the same staff from various campus
EVALUATION PLAN 10

units to present during a session and also to continue consistent attendance for the family SOAR

program. By having the same departments or even adding in more future collaborations, it will

allow family members, faculty, and staff to assist with one another in helping students from the

beginning of their journey to their graduation day. This includes the long term goal of wanting

the students to graduate on time and have a successful and impactful relationship with the

institution. This program will allow us to continue to work on building a solid positive

relationship with both students and family members by continuing to get feedback from this

Family SOAR session and to help us find better ways to provide for both communities.

Assumptions that can occur with this program include the idea of family members

wanting a bilingual session. Those who attend might feel uncomfortable with the idea of a

different sessions exist for the same program. Without the knowledge that the programs are

exactly the same in content, they might feel that the university is doing them a disservice by

giving the option of a bilingual session. Also, what is presented during these sessions may lend

itself to drawing assumptions that families might have of Dominican University and university

life. Understanding that some of the participants might have never attended college and that their

student is a first-generation student while preparing and presenting the sessions will ensure that

multiple identified outcomes will be met. Being cognizant of prior knowledge a participant

brings to the session is of vital importance. Since the session is open to multiple people who

identify as a family member or supporter it brings in a big assumption of what should be

supplied for each presentation from the departments. One of the many examples would be a

parent participating who has another student in college while a different participant identifying

as a parent is having their first child attend college now, with no knowledge of college since they

did not attend. The information provided would be a set on two end extremes. How do we
EVALUATION PLAN 11

include information to keep those who already have students in college engaged? How much do

we supply information to first timers and continue to encourage them to make appointments with

other departments for follow up questions? These two end extremes can meet a balanced middle-

ground.

Another aspect that could affect the outcome of the program are the language barriers. As

of now there is two specific languages used in presentations: English and Spanish. Majority of

faculty and staff on campus only speak English as their language. The percentage of Spanish

speakers has been increasing, but it is not enough. Trudi Goggin, Dean of Students at Dominican

University hired an interpreter and also asked for help from other various staff in other

departments on campus to help translate and present. However, it has been proven difficult for

her to connect to other family members by her not being able to speak Spanish fluently. Because

of this, she has the interpreter translate her conversations back and forth to have a better

connection to participants. At the same time this can cause a small disconnection of being able to

truly connect to a participant. It might even make the participant feel disconnected from the

institution because not many people speak their language. This also connects to those who might

speak Polish.

Finally, recognizing external factors that can affect the existence of the program and the

budget necessary for it can continue is vitally important. With a new program being able to make

a compelling argument for why the program must continue to be financed and how it fits with

the needs of the University is key. We want to use this evaluation of the program to provide data

to shows the value of it for all populations involved.

Quantitative Approach
EVALUATION PLAN 12

The quantitative approach will allow us to gain data about how effective the participants

feel the SOAR program is. The survey will show us if there is an increased understanding of life

at Dominican and ways to support students during their undergraduate career. This will lead to

our qualitative approach which will be conducting phone interviews that will be used to add to

our findings from the quantitative data. Our main strategy is our quantitative approach because

we will yield the most data from the survey we are administering based on the convenience

sampling that we are employing which is defined as, contacting population members who are

easily located and willing to participate, (Wholey et al., 2010, pp. 276).The next sections will

cover our research design, survey, followed by the limitations and final report.

Research Design

A pre-experimental one shot case study design which is a study of a single group with an

intervention during the experiment (Creswell, 2009, pp.158), for the assessment that will yield

results that will help us to determine how well the program is fulfilling its intended learning

outcomes as defined in our logic model (Appendix B). This assessment will allow us to obtain

evaluations at the time that the information is still fresh in the participants minds. We believe

that this approach will give us the most honest feedback about the content and information

presented in the SOAR presentation. This design does not have a control group compared with

the experiment group, (Creswell, 2009, p.158). We will not be implementing a control group

because we want as many people as possible to get information that is given in the session.

Based on the structure of the program, doing a pre and post survey would be difficult because the

participants are only on campus for a day, which makes a one group pre-test and post-test design

improbable to obtain the same numbers of evaluations to analysis.

Sampling Strategy and Validity

EVALUATION PLAN 13

Our participants for the survey instrument will be gathered using a convenience sampling

which is defined as, contacting population members who are easily located and willing to

participate, (Wholey et al., 2010, pp. 276). This method seems the most applicable for our

intended population of participants because we will be able to access them at the orientation

session required for their students. While our participants will not be necessarily chosen,

everyone who attends the bi-lingual orientation will have the opportunity to complete the survey

and take part in our sampling. We will invite them to take the survey at the end of the session

and the purpose of the evaluation will be introduced and explained by the presenter. We will

target everyone who attends the sessions but they will have the ability to opt out of evaluation by

not completing the survey. We will target all attendees to align with our pre-experimental single

shot design. The data collection instrument of a survey will be included in the folder of resources

attendees will receive at the beginning of the session. They will not be required to provide their

name on the survey so their confidentially.

Survey Instrument

Our chosen survey instrument has been revised from an original questionnaire which was

written and provided by the Dean of Students Office at Dominican University. The purpose of

the original questionnaire was to gain feedback from attendees about the program during the first

two summers of the programs implementation. It allowed the staff in charge of the program to

modify and make changes to the new program. We made intentional changes to the questionnaire

so it is more applicable to our quantitative evaluation. We have chosen to adapt the Dominicans

existing survey to include more questions because we felt the original survey failed to ask the

participants questions regarding the orientation resource folder and the specific offices

represented at the session.Our survey is meant to assess how well the learning outcomes that are
EVALUATION PLAN 14

defined in our logic model (Appendix B) of the program are met through three sets of questions

based on a Likert scale (those three sets being one survey translated into both English and

Spanish). The first set of questions aims to gain information about the program in general and the

different populations of people that attendees interact with during the program. This will allow

the program to make changes if certain populations are shown to be ineffective by the data. The

second section of the survey asks participants to rate different sections of the presentation based

on their ability to increase their understanding of life at Dominican University. This will give us

data about if certain sections are giving attendees the information that they deem important or if

the information is not necessary. The last piece of the survey evaluates the effectiveness of the

resources provided in the folder that participants receive upon arrival at the presentation and if

they feel there is value in them or if they are not necessary. Each of the resources will be

provided in both English and Spanish. The surveys will be administered in both English and

Spanish based on the session that a participant attends. This plan only includes in the revised

evaluation in English but will be translated by our partners at InterpreNet, the on-site interpreter

company.

This may be one of the first surveys that supporters/attendees of new students are asked

to complete, so through specific instructions and an explanation of the purpose of the survey the

participants will be honest and candid in their answers. Also, assuring participants that there will

no way to connect them to their responses so they should feel free to be honest. Since this is a

revision of the universitys original survey, the completed instrument will be shared with the

Dean of Students who can may then choose to incorporate our instrument into their orientation

assessment.

Implementation
EVALUATION PLAN 15

The survey implemented will be used to gain feedback from the students families who

support the latter through college. The questions asked will allow faculty and staff to engage in

areas of improvement and to move forward in creating more inclusive presentations. This survey

also allows comments to be added about other area functions or departments that should be

included, making it more beneficial in the long run for family members.

The survey is distributed at the end of each individual Family SOAR session before the

participants leave. Each participant is given an orientation folder at the presentation Within each

participant's folder there is information on resources in the institution, an article, and the

evaluation. Participants are asked to fill out the evaluation in person in order to have direct and

immediate feedback to the presenters. The survey will take approximately five minutes to

complete and will be provided in the folder they receive at the beginning of the session. The

presenter will explain that feedback gained from the surveys will be used to enhance the content

presentation and session. The presenter will also give a designated five minutes at the end of the

session for participants to complete it while also distributing writing utensils. If participants

choose to leave without filling out the survey, that is their choice. A follow-up survey is not a

part of our evaluation plan, but we will be including post-orientation phone interviews in our

qualitative research plans.

Analysis of Data

We will be utilizing chi-squared tests to obtain descriptive statistic data to have an

effective analysis of the survey. This will allow the survey and analysis to show how the

participants of the session could change over time with their knowledge of college and the

institution at Dominican University. The importance of assessing the data collected from this

survey is for the departments to be prepared with what information needs to be provided to help
EVALUATION PLAN 16

the family members who attend. Since the program has only been live for two years and is being

further developed for its third year, the information about the data is still new and evolving.

Already the numbers for participants has grown stronger with a variety of family members

identifying their students as first generation, which is not a surprise, since the population of

Dominican University has increased and is now identifying with a high Latinx student

population. The variables that we will use to begin to answer our research question will be the

data of the ratings by participants regarding whether each piece of the session was helpful or

worthwhile. This could potentially lead to revising certain pieces of the presentation to either

make them shorter or lengthening them to give more information. We will also be utilizing the

data that we gain from our third section of questions in the survey regarding contents of the

folder to assess the effectiveness of helping participants feel that they are adequately prepared to

help their students acclimate to Dominican and university life.

Limitations

The greatest limitation of our evaluation is our inability to do a pre and post test survey

because of the limits that the program presents. The participants are only on campus for a short

period of time and the session is only an hour long. We want to allow our participants to be able

to reflect more on what they have gained during the program and how it is effective for them to

support their students. We are going to be able to gain that important data through our qualitative

phone interviews.

Qualitative Approach

The main methodological strategy we will be using in this evaluation is our quantitative

approach, a survey. However, in order to accommodate different learning styles, encourage

participants to share their experience in a more private setting, and recognize that participants
EVALUATION PLAN 17

may need time to process their experience, we will be conducting phone interviews as our

qualitative approach. This qualitative piece serves to add to our data and increase the validity of

our findings. In the survey, participants will be able to give immediate feedback with the SOAR

sessions fresh on their minds. In contrast, the phone interviews will give participants a chance to

reflect on their experience following the session and give feedback on how the experience may

have been beneficial to their students transition (as well as their transition to being a family

member of a college student). This approach allows us to interact with the participant in a non-

threatening environment separate from their student and other families of college students. It is

our hope that this method allows us to further our relationship with the participant and create a

rapport of trust that allows the participant to speak freely, minimizing fear of judgement or

power dynamics.

Participants

The Dean of Students Office will send electronic invitations to participants that attended

SOAR sessions throughout the summer. It will be at the participant's discretion whether they

wish to partake in this phone questionnaire follow-up. No monetary incentives will be used to

gain participants. The hope is that participants will see the inherent value in continuing to be

engaged with the university community through the phone interview with the SOAR program.

The interviews will be conducted by the student workers and Torch Leaders who work

through the Dean of Students Office. They will conduct these interviews during their work shifts

at scheduled times with the participants. The student workers and Torch Leaders will be given a

script that will prompt them to provide information as to why they are calling and have a series

of questions prepared for those who wish to engage. For participants who only speak in Spanish,

the Dean of Students Office will also provide student workers and Torch Leaders that are
EVALUATION PLAN 18

bilingual who can speak with these families. It will be the same procedure, but it will allow it to

be inclusive and socially justice driven in providing a preference for the participant to speak in

their own dominant language.

Phone Interview Procedure

The phone interview procedure will consist of a series of steps to ensure it is a well-

rounded interview and interaction between the institution and the participant is consistent. The

interviewers, student workers and Torch Leaders, will be trained about the procedure for

conducting phone interviews by the Dean of Students office. The student workers and Torch

Leaders will be divided into two groups: English and Spanish speaking. They will each be

assigned pre-scheduled interviews to conduct based on their work schedule. Participants will

provide contact information on the survey that they fill out at the end of each Family SOAR

summer session. This information form is located in their folders attached with their surveys and

there will be designated time for participants to fill this out during the session. The Dean of

Students office will work to schedule specific interview times with participants in order to

eliminate cold calling people.

Positionality Statement

In this portion of the research, our role as evaluators will be to give SOAR families the

opportunity to share their experience after attending the SOAR program. However, it is

important to note how our potential biases as evaluators may inform the data we collect. All

three of the evaluators identify as white native English speakers, and we must be careful to not

place stereotypes upon participants in this study. To do this, we will ask for feedback from our

interpreters on how our survey and interview questions translate into Spanish, making sure that

the our words are not too intrusive and encourage participants to speak freely. While we will not
EVALUATION PLAN 19

be necessarily hoping for the interviewees to have had negative experiences, we will be looking

very closely at participants answers to make sure their needs are being met during sessions.

Because none of the three evaluators is proficient in Spanish, it is imperative that we effectively

evaluate if the needs of non-native English speakers are being met during bilingual SOAR

session. So, in our quest to accurately measure if needs are being met, we must be careful to

present open-ended questions to our participants and be sure to avoid language that sways their

reflection of their experience one way or another. Carol Bernard, who is a outside translation

consultant hired from a ty Dominican, will be working with us to translate our instrument into

Spanish, as well as work with the Torch Leaders and student workers who will be conducting the

interviews with Spanish-speaking families. Because of the previous capacity that Carol has

worked with the Dean of Students in, we are confident that she will be a trustworthy partner in

this evaluation effort.

Implementation Plan

As stated earlier, the student workers will be trained beforehand. This training includes a

one day session with customer care practices such as answering phones, questions, and being

able to transfer phone calls to other departments on campus. During this training the student

workers will be tested on how to create a welcoming environment for the interviewees, how to

handle conflict/hostile family members, and how to follow up appropriately when asking

questions to build a more solid and harmonious relationship with the institution and family

member(s). It is crucial that students know when to ask the right questions and how to record this

information. Students will type all interview transcriptions into a file on the computer and send it

to a private, password-protected email address that will be moderated and accessed by the

evaluators only. Once the original file has been confirmed of receipt by the evaluators, the
EVALUATION PLAN 20

student will delete the original file off of the computer they used to conduct the interview.

Additionally, bilingual staff members who have been trained along with the student workers will

be on-call if the need for translation or transferring of calls is necessary. Students conducting the

interviews will have the phone interview script and will not be allowed to deviate from it (See

Appendix B).

Data Analysis

The Torch Leaders, also known as the student orientation leaders, will be conducting the

majority of the phone interviews. However, the transcribing and data analysis will be done by the

research team. We will then work to define themes and sub-categories to allow us to code all of

the data. We are not utilizing our logic model because we hope to gain new information from

these interviews that might not be reflected in our current logic model.

Coding Procedures

We the researchers will be taking on the role of coding all the of the information

ourselves. Compiling full transcripts of each interview will be the first part of the process and

beginning to identify meta themes that are obvious and continue that process until we are able to

identify sub-categories that appear within the larger themes. We will using multiple sources of

data to gather and organize our data during coding (Creswell, 2009). The data will be assembled

and along with our quantitative data it will be used to make program recommendations to the

Dean of Students Office.

Validity and Ethics Considerations

As a reminder, all phone conversations are confidential and will only be shared with the

key stakeholders who facilitate Family SOAR. As a Sinsinawa Dominican-sponsored institution,

Dominican University prepares students to pursue truth, give compassionate service, and to
EVALUATION PLAN 21

participate in the creation of a more just and humane world (motto of institution). We believe

our interview questions are open-ended enough to encourage participants to share what they are

thinking without feeling that we may be looking for a specific answer. We believe our questions

also give participants the opportunity to reflect on several factors that may be affecting their

student- academics, financial aid, social life challenges- and will provide us with the opportunity

to see which specific session benefitted them and their student. By taking part in this survey we

believe that each participant is created equal and will help us grow our program into a more

informational and engaging opportunity for those who interact with their student and the

institution.

Timeline

The provided timeline reflects the estimated dates of how the evaluations will be

executed, training for phone conversations, and the next steps in updating the Family SOAR

session presentation and survey. After the presentation and evaluation review has been done it

will move forward to future collaboration and the update of both presentation slides and

evaluation questions that are provided. Another function of this timeline is that it will allow

Trudi and the rest of the administration to see what else should be placed in folders. This

estimated timeline will take about a year to reflect meetings, suggested changes to presentations

and evaluations, and future collaborations. Since this timeline is only a recommendation it can be

rearranged by Trudi and her team if they desire. The overall outcome goal of this timeline is to

allow the team working on this Family SOAR presentations to become aware of when

appropriate deadlines should be created.

Budget
EVALUATION PLAN 22

In addition to providing a timeline to stakeholders, we will need to provide a breakdown

of perceived costs. Please see Appendix G for the estimated costs associated with one full phase

of evaluation.

In order to encourage participation from families and supporters of new Dominican

students, we will be offering one (1) gift card at the cost of $75 to be used for dinner and two (2)

tickets to Dominican student performance, valued at $50. All interview and survey names will be

written and placed into a bag. We will then shake the bag and select one name to receive the

prize. They will then be notified by email of their prize and will be allowed to elect one person to

attend dinner and the show with them. While stakeholders may be concerned about the cost of

the prize, we believe this can be justified through the following reasoning: since the selected

winner will be a family member of someone attending Dominican, this prize allows them to

experience an aspect of the Dominican community that they may not previously had exposure

too. The prize entices them to visit the campus again, and gives them a chance to even take their

student to the dinner and show, if they choose. So, while this prize is an added cost to the

evaluation budget, we do believe that it serves our purpose as evaluators and the overall mission

of the Dean of Students.

The rest of our spending will be used to compensate the student workers through during

the one-day training period and cover printing materials for the survey portion of our qualitative

approach. Our third-party translation consultant is already compensated through her contract

with the Dean of Students office, so her participation in this evaluation will be covered by her

contract.

Next Steps
EVALUATION PLAN 23

The data collected in this evaluation will allow the SOAR Family program to continue to

evolve and meet the needs of the participants and the students that they support. Both the

quantitative and qualitative data will give the staff and stakeholders specific themes to pay

attention to when considering modifications to the program. There could be the potential for new

topics to cover in the presentation or new resources that need to be included in the folder

provided to participants. There may be a need to assess staff or personnel involved in the

program and hire more staff for the program. Continuing to evaluate this program will ensure its

longevity and that it will continue to align with Dominican University mission and values.
EVALUATION PLAN 24

References

Bryson, J. M., & Patton, M. Q. (2010). Analyzing and engaging stakeholders.

In J.S. Wholey, K.E. Newcomer, & T. Triplett (Eds.), Handbook of practical program

evaluation (31). Jossey-Bass: San Francisco.

Creswell, J. (2009). Quantitative Methods, Research Design: Qualitative, quantitative, and

mixed methods approaches. (pp. 145-171). Thousand Oaks, CA: SAGE Publications.

Jordan, G. B., & McLaughlin, J. A. (2010). Using logic models In J.S. Wholey, K.E.

Newcomer, & T. Triplett (Eds.), Handbook of practical program evaluation (56). Jossey-

Bass: San Francisco.

Newcomer, K. E., & Triplett, T. (2010). Using surveys. In J.S. Wholey, K.E. Newcomer, & T.

Triplett (Eds.), Handbook of practical program evaluation (pp. 262-297). Jossey-Bass:

San Francisco.

Schuh, J. H & Associates. (2009). Assessment methods for student affairs. San Fransisco, CA:

John Wiley & Sons, Inc.

Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Analyzing and Engaging

Stakeholders. Handbook of practical program evaluation (30-54). Jossey-Bass: San

Francisco.

Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Using Logic Models. Handbook of

practical program evaluation (55-80). Jossey-Bass: San Francisco.

EVALUATION PLAN 25

Appendix A:
Family SOAR Session Presentation-English

Family SOAR

Introduction
SOAR Orientation Part 1
Welcome Weekend Part 2 August 26-28
August 26th- Move in
August 27th- Parent Workshops, Mass and
Reception
August 28th- DU Fest for Students
August 29th First day of class
EVALUATION PLAN 26

Financial Aid

Dominican Financial Aid

Awards
Federal, State and Dominican Grants
Merit Award
Student Employment
Stafford Loans
EVALUATION PLAN 27

Grants
Federal
Pell Grant
Supplemental Education Opportunity Grant (SEOG)
State of Illinois
Monetary Award Program (MAP)
Dominican
Based on need

Merit Awards
Based on academic record (fresh-senior years)
Traditionally directly related to GPA and
test score
PRESIDENTIAL
DEANS
FOUNDERS
ACHIEVEMENT
RECOGNITION
OPPORTUNITY
EVALUATION PLAN 28

Student Employment
Hourly waged jobs that assist in paying for college,
as well as work experience. Students receive a
paycheck, funds are not posted directly to the
students account.
Federal Work Study - FWS
Dominican Employment DE

Student Employment
Applying for a job at Dominican
June 1st postings with new positions going up all
summer
Posted online at www.dom.edu/jobs
Students with FWS/DE can start applying as of July
15
New hires can start working after August 1
New students without FWS/DE can start applying
September 1.
EVALUATION PLAN 29

Student Loans
2016-2017
RATE
AMOUNT GRACE

$5,500 for
Direct Stafford Loans: first year, up to
3.76%
Subsidized (need-based) and $3500 of that 6 months
fixed
Unsubsidized may be
subsidized

Must be requested
Up to Cost of
otherwise
6.31% Attendance
Direct Parent PLUS repayment begins
fixed minus other aid
once the loan is
received
fully disbursed

Private Varies Varies Varies

Student Loan Checklist

Complete the following steps:
Online Loan Acknowledgment Form found on the
www.dom.edu/financialaid
Must be completed even if you are declining all or a portion of the
loan

Entrance Counseling at www.studentloans.gov

Sign your Master Promissory Note (MPN) at
www.studentloans.gov
Items needed to complete your MPN
1. FSA ID Number
2. Drivers license or state ID
3. Names and addresses of two references
EVALUATION PLAN 30

The Financial Aid Award

Commuter Student
Cost of Attendance: $36,720
Direct Cost: $31,720

Resident Student
Cost of Attendance: $43,912
Direct Cost: $41,412

Determining Out-of-Pocket Expense

Direct Cost Gift Aid Loans = Out-of-Pocket Expense

Verification Process
The Department of Education randomly selects students
for verification. The financial aid office may request some
or all of the following in order to complete that process:
IRS Tax Transcripts
Copies of W2 forms
Verification Worksheet
Copy of Social Security Card
Naturalization/Citizenship Documentation
Selective Service Card
EVALUATION PLAN 31

Financial Responsibility
Student Account
Tuition due by the first day of class
Payment Plans
Student Account Hold

Financial Aid (FAFSA) complete annually

and on time. The 2017 2018 FAFSA will
be available October 1, 2016.

Financial Aid Questions?

Email us at:
finaid@dom.edu

Call us at:
708-524-6809
800-828-8475 outside of the Chicago area
EVALUATION PLAN 32

Student Account Questions?

Email us at:
bursar@dom.edu

Call us at:
708-524-6791

Campus Life
EVALUATION PLAN 33

Benefits of Campus Engagement

Student Organizations
University Ministry
Lectures
Community Based Learning
Performances
Value of living on campus

Rhythm of College Life

Challenges and Trends

Questions to Ask
Strategies for Family Involvement
EVALUATION PLAN 34

The Discussion Doesnt Stop Here

Welcome Weekend Program
Family Programs
DU Mobile App
Parent Website
Dom.edu/parents
Email Communication
Emergency Notification

Contact: deanofstudents@dom.edu

Academic Life
EVALUATION PLAN 35

Key Word: Time

Being a student is a full time job (40 hrs. a
week)
Key points associated with time are study
and preparation.
Discipline: need to put in the work and
develop a study plan.

Academics
Full-time Status is 12 18 credit hours
Two hours outside the classroom for each
hour in the class (Time Management)
Course Schedule
Monday/Wednesday/Friday
Tuesday/Thursday
Evening Classes
EVALUATION PLAN 36

Academics
Full-time Status is 12 18 credit hours
Two hours outside the classroom for each
hour in the class (Time Management)
Course Schedule
Monday/Wednesday/Friday
Tuesday/Thursday
Evening Classes

Sample 2yr Schedule

BUSINESS ADMINISTRATION
FIRST YEAR SECOND YEAR
FALL Credits SPRING Credits FALL Credits SPRING Credits
LAS 3 Area Studies - 1 3 LAS/Area Studies 3 LAS/Area Studies-3 3
ENGL 101 3 ENGL 102 3 ECON 191 3 BAD 250 3
MATH 130 or 170 3 CIS 120 3 ACCT 101 3 ACCT 102 3
BAD 240 3 ECON 190 (SS) 3 Area Studies - 4 3 Area Studies - 5 3
LANG 4 LANG 4 Elective 3 Elective 3
Total 16 Total 16 Total 15 Total 15
EVALUATION PLAN 37

Academics
Syllabi Course expectations and
requirements
Professor Office Hours
No Mid-term Grades
Undergraduate Warning Notices
Freshman Seminar - Advisor
ADA/504 Accommodations

Student Responsibility
FERPA Federal Educational Rights and
Privacy Act

Dominican email is the official form of

communication (bills, university notices,
deadlines, professional development)

Self-Advocacy (IDEA vs. ADA)

EVALUATION PLAN 38

Key Word: Success

You are successful, because you made it
to college, however we want you to be
successful in college and career.
We will help you develop a clear academic
and career plan.
Make sure that you speak with your
advisor!

Student Success Resources

Academic Enrichment Center
Tutoring/Literacy and Learning Resources
Disability Support Services (DSS)
Career Development

Workshops and Classes

ID 092 Integration and Success
ID 110 Career Planning Major Exploration
ID 111 Personal Finance
EVALUATION PLAN 39

Key Word: Career

Exploration: you will find many interests
and passions.
Take advantage of exploring while you are
in college.
Major to career.

Student Support Services

RCAS Academic Advising

Peer Advisors
Freshman Seminars
RCAS Academic Success Services
Student Affairs
Wellness Center (Health and Counseling
Services)
EVALUATION PLAN
ADVISING WORKSHEET BUSINESS ADMINISTRATION
Name _______________________________________
CORE CURRICULUM BUSINESS ADMINISTRATION
FOUNDATIONS
ENGL 102 /190 ___________
MATH 130 or 170 _________________
LANG 102 or 192 Elementary Lang II ______
CIS 120 Intro to Comp Apps _____________
Information Access Workshop ____________
(Required only of those not taking ENGL 102 or
190 at Dominican University.)
LAS SEMINARS (4 courses/12 hours)
Freshman Seminar _________________
Sophomore Seminar _______________
Junior Seminar ____________________
Senior Seminar ____________________
AREA STUDIES (7 courses/21 hours)
History (HI) _______________________
Literature (LT) ____________________
Fine Arts (FA) _____________________
Natural Science (NS) _______________
Philosophy (PH) ___________________
Social Science (SS) ________________
Theology (TH) _____________________
MULTICULTURAL (MC) (1 course/3 hours)
Multicultural (MC) _________________
Email us at:
rcas@dom.edu
Call us at:
708-524-6814
Advising Worksheet
Economics Courses (3 courses/9 hours)
ECON 190 Microeconomics _____________
ECON 191 Macroeconomics ____________
ECON 260 Statistics for Bus and Econ. ____
Accounting Courses (2 courses/6 hours)
ACCT 101 Principles of Accounting I ______
ACCT 102 Principles of Account II _________
Business Courses (8 courses/ 24 hours)
BAD 240 Business Law __________
BAD 250 Marketing _____________
BAD 255 Intl Business __________
BAD 335 Business Ethics ________
BAD 345 Management __________
BAD 350 Managerial Finance ____
BAD 380 Operations Mgt.________
BAD 490 Business Policies _______
*Please check prerequisites for all
courses
NOTE: All GSB courses require
acceptance into the BA/MBA program.
Students may enroll in a maximum of 2
graduate courses as an undergraduate.
**Special topic courses may be taken
more than once if the content of each
class is different (BAD 491, 492 or
493)
RCAS Questions?
40
2016-2017
ID #______________
AREAS OF CONCENTRATION ELECTIVES
Number of open electives varies depending upon
Finance (BS) (5 courses/15 hours) student.
ECON 367 Fin. MKTS & Inst.________ 1) ___________________________________
BAD 370 Investments _____________ 2) ___________________________________
** Three (3) courses chosen from: 3) ___________________________________
BAD 365,375, 491; ECON 376 or GSB 4) ___________________________________
624, 723, 754, or 782; Math 250;
5) ___________________________________
CPSC 140 or 155
6) ___________________________________
1. _____________________
7) ___________________________________
2. _____________________
3. _____________________ 8) ___________________________________
9) ___________________________________
Management (BA)(5 courses/15 hours)
10) __________________________________
BAD 245 Human Resources and Career
11) __________________________________
Development _______________________
12) __________________________________
BAD 275 Entrepreneurship ____________
** Three (3) courses chosen from:
ACCT 210, BAD 492, CAS 320, CAS 321,
ECON 367, PSYC 360 or GSB 624, 723,
Courses not applicable to the degree
754 or 782
1) ______________________________
1. ______________________
2) ______________________________
2. ______________________
3. ______________________
Dominican Credits Earned ______
Marketing (BA) (5 courses/15 hours)
Transfer Earned _ ___
BAD 351 Intl. Marketing ____________
BAD 352 Marketing Research ________ TOTAL for Graduation 124
** Three (3) courses chosen from:
APRL 271, 281; BAD 275, 353, 365, 493;
CAS 269,274, 275, 321 or 350; GSB 626,
755, 764, 771, 772, 774 or 782
1. _______________________
2. _______________________
3. _______________________
EVALUATION PLAN 41

Questions?
EVALUATION PLAN 42

Appendix B:
Logic Model
EVALUATION PLAN 43

Appendix C:
DOMINICAN UNIVERSITY
2015 Family SOAR Evaluation

Thanks for joining us for the Family SOAR program. To help us plan for next years Family
SOAR, please complete the following form. Rate with number 5 being the highest rating and
number 1 being a poor score. Please circle N/A if you did not experience or attend one of the
programs or events.

Please circle the appropriate response:

STRONGLY DISAGREE STRONGLY AGREE
1. The Family SOAR check-in process was efficient. 1 2 3 4 5 N/A
2. The information provided in the Family Folder was valuable. 1 2 3 4 5 N/A
3. I feel better about my student attending Dominican 1 2 3 4 5 N/A
University because I attended Family SOAR.
4. The TORCH Leaders were very helpful and knowledgeable. 1 2 3 4 5 N/A
5. The available faculty and staff were knowledgeable 1 2 3 4 5 N/A
and helpful.
6. Overall, the SOAR Family program was worthwhile. 1 2 3 4 5 N/A

Please rate the sessions below based on the extent to which you agree or disagree that the
following experiences were helpful and/or worthwhile:
STRONGLY DISAGREE STRONGLY AGREE
7. Academic Life 1 2 3 4 5 N/A
8. Campus Life 1 2 3 4 5 N/A
9. Financial Life 1 2 3 4 5 N/A
10. Family Prayer 1 2 3 4 5 N/A

Comments & Suggestions:

Thank you for completing the evaluation & Welcome to the Dominican University
Community!
EVALUATION PLAN 44

Appendix D:
DOMINICAN UNIVERSITY
2017 Family SOAR Evaluation

Thanks for joining us for the Family SOAR program. To help us plan for next years Family
SOAR, please complete the following form. Rate with number 5 being the Strongly Agree
rating and number 1 the Strongly Disagree score. Please circle N/A if you did not experience or
attend one of the programs or events.

Please circle the appropriate response:

STRONGLY DISAGREE STRONGLY AGREE
1. The Family SOAR check-in process was efficient. 1 2 3 4 5 N/A
2. The information provided in the Family Folder was valuable. 1 2 3 4 5 N/A
3. As a result of attending orientation, I feel more prepared for my 1 2 3 4 5 N/A
student to attend Dominican University.
4. The TORCH Leaders were very helpful and knowledgeable. 1 2 3 4 5 N/A
5. The available faculty and staff were knowledgeable 1 2 3 4 5 N/A
and helpful.
6. Overall, the SOAR Family program was worthwhile. 1 2 3 4 5 N/A

Please rate the sessions below based on the extent to which you agree or disagree that the
following experiences were helpful in increasing your understanding of life at Dominican
University:
STRONGLY DISAGREE STRONGLY
AGREE
1. Academic Life 1 2 3 4 5 N/A
2. Campus Life 1 2 3 4 5 N/A
3. Financial Aid 1 2 3 4 5 N/A
4. Family Prayer 1 2 3 4 5 N/A

Are there any other topics that you felt needed to be covered in the session?:
______________________________________________________________________________
_____________________________________________________________________________

I found the following contents within the resource folder combined with the information from
the session furthered my understanding of:
STRONGLY DISAGREE STRONGLY AGREE
5. Financial Aid Worksheet 1 2 3 4 5 N/A
6. Campus Life Packet 1 2 3 4 5 N/A
7. Family Article 1 2 3 4 5 N/A
8. Family Prayer 1 2 3 4 5 N/A

Continue on page 2..

EVALUATION PLAN 45

Are there other handout or worksheets that you feel need to be included in the folder?:
______________________________________________________________________________
__________________________________________________________________________

Comments & Suggestions:

______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________

Thank you for completing the evaluation & Welcome to the Dominican University
Community!
EVALUATION PLAN 46

Appendix E:
Phone Interview Invitation

Subject: Invitation to Research Study: Call for Research Participants: Bilingual Attendees of
SOAR Sessions

Thank you for attending Dominican Universitys Family Student Orientation and Registration. It
was a pleasure to have you and your students on campus.

At the end of your orientation session, you completed our written survey offering feedback on
your experience during the session. Additionally, our team is recruiting attendees to participate
in a study focused on bilingual experiences in the Family SOAR experience.

Families and guests of students who attended SOAR may participate in this study. Being a
participant in this study will require you to be interviewed by our staff over the phone about how
the Spanish and English sessions during SOAR added to your orientation experience. The
interview should last between 30 and 45 minutes. This study is completely confidential. The
interview will be audio-recorded, therefore your voice may identify you, but your name will
remain confidential and every effort will be made to protect your identity.

During the interview, you can share whatever you wish with the interviewer and are free to
choose not to participate in any or all part of this study. If you would rather not respond to a
particular question, just let the interviewer know. At any time you can excuse yourself from the
interview without consequences.

If you are interested in participating in this study or have any questions, please feel free to reach
out to us via email at soar@dominican.edu or via phone at 512-939-2424. Please state your
interest in the study, provide either your name or email, and a time of day preference for the
interview (morning, afternoon, or evening). We will contact you with further instructions.

Thank you for your consideration.

Best,
Elizabeth Thomas
Researcher on behalf of:
Dean of Students
Dominican University
7900 West Division Street
River Forest, IL 60305
EVALUATION PLAN 47

Appendix F:
Phone Interview Procedure

Introduction
Thank you for agreeing to tell us about your experience with Dominicans SOAR program. As a
reminder, our team is recruiting attendees to participate in a study focused on bilingual
experiences in the Family SOAR experience. My name is ___________ and I will be conducting
your interview. The purpose of this phone interview is to learn more about your experience at
the Family SOAR program and how you felt about the presentation. We are evaluating this
program in order to ensure that it is meets the needs of those who participate in SOAR. Your
answers will help us to improve the program. With your permission I will record the interview,
but the recording will not be shared with anyone outside the evaluation team. All your responses
will be completely confidential.

This interview will take between 30-45 minutes and I will ask you a series of questions.

This phone interview is being conducted for the Family SOAR program evaluation on [date] by
[ moderator]. The start time is [time].

Please know that all phone conversations are confidential and will only be shared with the few
staff members who organize the SOAR sessions. These staff members will only use this
information to better improve orientation for future students and families. As a Sinsinawa
Dominican-sponsored institution, Dominican University prepares students to pursue truth, to
give compassionate service and to participate in the creation of a more just and humane world.

Questions
I would like to start with some background questions about your student.

1. Why did he/she decide to enroll at Dominican?

2. How was the application process for you and your student?
3. What is primary language spoken in your home?
4. Is he/she the first of your children to attend college? If not, was the presentation helpful
to you? If yes, was the presentation valuable for you and how so?
5. A a result of attending orientation, how confident do you feel in helping your student to
navigate the following topics:
Financial Aid:
Academics:
Student Life:
6. What were your expectations for the orientation, if any? To what extent did the
orientation meet those expectations?
Probes: If not, how did it not meet your expectations?
7. What can we do to improve future experiences for attendees?
Probes: Any specific topic that wasnt covered that you felt should have been included?
8. Did you feel your needs were met during the session?
EVALUATION PLAN 48

9. Is there anything you wish had been covered during orientation, but wasnt? If so, could
you tell me about that?

Conclusion
Thank you for participating in this phone interview. Your feedback will assist us in evaluating
the Family SOAR program at Dominican University and where enhancements can be made.
We appreciate your willingness to be a part of this interview. If you have any questions or
concerns after the conclusion of this interview, please feel free to reach out to our office via
phone or email.
EVALUATION PLAN 49

Appendix G:

Evaluation Plan Budget

Materials Cost Hospitality/Incentives Cost

Printing materials for $50.00 Gift Card for dinner $75.00

survey

Office supplies: pens $50.00 Tickets (2) to $50.00 ($25 each)

for participants to Dominican
complete survey performance

Pizza for student $100.00

worker training*

Water & beverages $25.00

$100.00 $250.00

*Cost of pizza is estimated according to the following information: we will have ten (10)

students workers who need to be fed during the day-long training. We would like for everyone

participating to have at least two (2) slices of pizza, as well for as our three (3) evaluators. This

equals 26 slices of pizza; each pizza is served in roughly eight (8) pieces, so four (4) pizzas at a

cost of $20 each equals a total of $100.00 for pizza.

 EVALUATION PLAN 50

Appendix H:

Timeline

2017 2018

Task Aug. Sept. Oct. Nov. Jan. Feb. Mar. Apr. May Jun
e

Evaluation 31st 1st

Review

Presentation 6th
Review

Training of 12th
Student
Workers
(Phone Calls)

Eval Follow- 18th

Up (Phone -
Calls) 21st

Evaluation Month
Findings of Oct.

Presentation
Findings Month
of
Nov.

Future End of Early

Collab. January Feb.

Development Last two

for Next weeks
Steps (19th-30th)

Updated Due X X
Pres. and by the
Eval 27th