Improvement Process: © Pascal Metrics 2016

IMPROVEMENT PROCESS
© Pascal Metrics 2016

INTRODUCTION
Summary
Congratulations on a successful survey administration! You now have high quality survey data that
can help provide insight into specific topics about your organization. The question you may be
asking at this time is, “Now what?”
Over the last several years of working with client partners throughout the United States and
internationally, Pascal Metrics has developed a recommended process for reviewing and acting on
survey data in a robust and structured way. This process is the key to realizing improvement in
your organization over time, and can be used in conjunction with other improvement tools and
strategies you may already be utilizing.
The Pascal Metrics Improvement Process is multiple steps and requires engagement at many
levels of the organization. It is important for the Survey Leader to take accountability for this
process, to involve leadership early on, and to clearly communicate about it throughout the
organization. For tips on where to begin, use the Getting Started Checklist in Appendix A.
The Pascal Metrics Improvement Process includes the following steps, which will be described in
detail in each section of this guide.
Discussing and Using Improvement Monitor and Evaluate

Review the Data Survey Data Planning Progress Review Progress
(Section 1) (Section 3)
(Section 2) (Section 3) (Section 3)
Pascal Metrics makes the following recommendations in order for this process to be successful:
1. As mentioned above, this process will require much effort and involvement at different
levels of your organization. Pascal will provide helpful resources and trainings; however, this
document outlines a recommended process to help you and the members of your
organization take action on your recently collected data.
2. Access to HealthBench should be provided for all Work Setting Managers and Feedback
Session Facilitators so they may review the data as needed.
3. The Survey Leader will serve as the central point of contact for the Feedback Sessions, as
well as communication back to leadership.
4. The Survey Leader is someone who is in a position to hold Work Setting Managers
accountable for the Improvement Planning process.
5. The leadership of your organization must set expectations for participation in the process.
A detailed, recommended timeline can be found in Appendix B.

© Pascal Metrics 2016 | 2

CONTENTS
Roles and Definitions .............................................................................................................................................4
Section 1: Understanding Your Survey Data ...................................................................................................6
• 1.1 Purpose ...................................................................................................................................................................... 7
• 1.2 Basic Survey Information .............................................................................................................................. 7
• 1.3 Culture Survey Data Tip Sheet ................................................................................................................ 11
• 1.4 Conclusion ............................................................................................................................................................ 13
Section 2: Discussing and Using Survey Data .............................................................................................. 14
• 2.1 Using this Guide ................................................................................................................................................ 15
• 2.2 For the Survey Leader .................................................................................................................................... 17
• 2.3 For the Work Setting Manager ................................................................................................................ 21
• 2.4 For the Feedback Session Facilitator ...................................................................................................23
• 2.5 Frequently Asked Questions .....................................................................................................................28
Section 3: Improvement Planning .................................................................................................................... 31
• 3.1 Moving Forward with Improvement Planning ..............................................................................32
Appendices ............................................................................................................................................................ 36


ROLES AND DEFINITIONS

The Improvement Process is multiple steps and requires participation at many different levels of
your organization. As you think about the process and next steps, think also about the individuals
in your organization who would be able to fulfill the responsibilities and be the best fit for each role.
Roles
Survey Leader: The Survey Leader has played a key role throughout the survey administration and
will lead the Improvement Process. This person serves as a liaison between the leadership,
managers and Feedback Session Facilitators. The Survey Leader will work with the Pascal Metrics
Client Services Partner to coordinate trainings, provide data or access to data to others engaged in
improvement within the organization, and communicate with leadership and peers throughout the
process. This person is generally from the Quality and/or Safety department of the organization,
but depending on the type of survey used, this role may be filled by a multidisciplinary team
including members from Quality and Safety, Education and/or Human Resources. This team
should have the ability to coordinate within and across multiple levels in the organization, from
Executive Leadership to Work Setting Managers. Responsibilities of the Survey Leader include:
• Connect with the Pascal Metrics Client Services Partner for the Improvement Foundation
Call
• Schedule, coordinate, and invite participants to the HealthBench User Trainings, Executive
Briefing, and Using Survey Data Series
• Identify individuals to fill key roles in the process such as the Feedback Session Facilitators
• Communicate to leadership, Work Setting Managers and Feedback Session Facilitators
about the Improvement Process and next steps after the survey administration is complete
• Hold the Work Setting Managers accountable for Improvement Planning and improvement
efforts through regular progress reviews
Executive Leadership: The role of leadership in the survey administration and improvement
process is to help set expectations for the survey administration and participation, as well as create
accountability for the Improvement Process. When we refer to leadership, we are referring to C-
Suite level leadership, but may also include directors and VPs depending on how your specific
organization is structured. Leadership should be aware and involved in the following ways:
• Review the survey data once it has been collected and published. This will be done through
an Executive Briefing prepared by Pascal Metrics, as well as communications and reporting
prepared by the Survey Leader.
• Review and respond to staff feedback provided by the Survey Leader.
• Engage with staff in the organization, as needed, to hold the work settings accountable for
the Improvement Process
Work Setting Manager: The Work Setting Manager is responsible for communicating initial
survey results back to their staff and setting expectations for participation in the Feedback
Sessions that will follow. This person generally has staff reporting directly to them in the area and


is responsible for the operations of the work area. While the manager will not participate in the
Feedback Sessions, their role is vital in reviewing the feedback notes, identifying areas for
immediate action they can take, and scheduling Improvement Planning sessions with their staff.
The outcome of the Improvement Planning session should be 1-3 action items for improvement on
which progress will be monitored by the manager. Specific tasks the Work Setting Manager may be
asked to complete include:
• Attending HealthBench User Training

• Attend the Using Survey Data Series training sessions to learn how to understand data and
use the information collected during Feedback Sessions for Improvement Planning.
• Schedule and lead Improvement Planning sessions with their work setting, and hold the
team accountable for actions
• Provide updates to the Survey Leader regarding improvement progress and next steps
Feedback Session Facilitator (“Facilitator”): Pascal Metrics strongly recommends that Feedback
Sessions be led by a neutral third party, and not the Work Setting Manager. Our experience
demonstrates that discussions are more open and productive when a person who is neutral to the
unit leads each session, regardless of the relationship the manager has with their staff. Most often
this is because someone unrelated to the work setting does not have preconceived opinions
regarding the topics that come up – they may ask more questions that get to the root of the topic
and help identify some easy actions that can be taken to make improvements. A Facilitator can be
from a wide variety of areas within the organization including Quality Management/Safety/Risk
departments, nurse educators, human resources personnel, or others who will be perceived as
‘neutral’ by work setting staff. Work Setting Managers can serve as a Facilitator for work settings
other than their own. Specific tasks the Feedback Session Facilitators will complete include:
• Attend the Using Survey Data Series training sessions to learn how to understand data and
lead Feedback Sessions.
• Use work setting data to lead Feedback Sessions with frontline staff
• Summarize notes taken during the Feedback Sessions and provide them to both the Survey
Leader and Work Setting Manager
• Share any concerns that come up from the conversations with the Survey Leader


UNDERSTANDING YOUR SURVEY DATA


PURPOSE OF THIS SECTION

The purpose of this section is to provide the basic information you need to begin using your high
quality survey data to drive improvement in your organization. Pascal Metrics offers a variety of
survey domains linked to Safety Culture. While there may be specific approaches to help dig
deeper into each area of interest, this guide will provide direction on how to review and understand
the survey data you have collected with Pascal Metrics. For help understanding particular domains
you have used on your survey use this guide along with the Domain Score Calculation documents
in Appendix C.
As you begin to explore your survey results, you should understand that there are two types of data
available following a survey administration: quantitative and qualitative. The quantitative data are
the numerical item and domain scores, while the qualitative data are the comments respondents
provided (optional) when they took the survey. This section will focus on how to interpret the
quantitative results of your survey. The quantitative results help you learn about challenges and
issues respondents experience in their work area. The qualitative results are also critically
important in providing context to the quantitative results and to help you understand “why” people
responded in a certain way. All results should be given careful consideration since they have the
potential to have great impact on culture when improvement activities result from them. For more
information about reviewing the qualitative data, refer to Section 3.
BASIC SURVEY INFORMATION

What You Need to Know About Quantitative Survey Data
Survey data:
• Presents a snapshot of respondents’ perceptions and beliefs at one point in time
• Generates more questions than it provides answers
• Identifies issues for further exploration and attention, but does not explain them
• Identifies opportunities for improvement.
Surveys are validated instruments that include statements to which individuals are asked to
respond, often referred to as survey items. Related survey items are grouped into domains.
Quantitative data is reported and compared at both the item and domain level. This section
provides guidance on how interpret these results.
Survey Items
Individuals completing the survey are asked to respond to each of the items using a specific
response scale. In general, the number you see on reports is the percentage of favorable responses
to the item.


Following are examples of scoring from two different surveys from which you might have included
items.
Example 1: Item Score Calculation - SAQ

The item, “I would feel safe being treated here as a patient” from the Safety Attitudes Questionnaire
(SAQ), uses the following response bank:
1 = Strongly Disagree 2 = Disagree Slightly 3 = Neutral 4 = Agree Slightly 5 = Strongly Agree

N/A=Not Applicable
Scores for this item only take into account favorable responses. In other words, the score reflects
the percentage of “Agree Slightly” (4) or “Strongly Agree” (5) responses to that item. In this
example, a higher score is better.
Below is an illustration of how an item score is calculated:
Responses to a single item from a group of 10 people that took the SAQ:
Response to Item Response is Agree
Slightly (4) or Strongly
Agree (5)
Anne 4 *
Brad N/A
Charlie 4 *
Devon 5 *
Elaine 2
Fred
Greg 1
Harry 3
Irene 4 *
Jennifer 5 *
Total Response: 8 5
When calculating an item score, individuals who did not answer the item or who indicated the item
was not applicable (N/A) are disregarded. Note that Brad answered “N/A” and Fred skipped this
question altogether; therefore, only 8 of the 10 respondents are included in the item score
calculation. Five respondents Agreed Slightly (4) or Strongly Agreed (5), as indicated by the *, so
the item score includes those respondents, divided by the number of eligible respondents (in this
case 8).
Number of people who agreed slightly (4) or strongly agreed (5)

Number of eligible respondents
5
= .625 = 63%
8
In this example, the item score is 63%, which means that 63% of respondents, on average, responded
favorably to this item.


The SAQ also includes a few negatively worded items. Similarly, the scoring for these items reflects
the respondents who chose a 4 or a 5 on the survey, but the goal is for people to disagree with these
items, so a lower score is better. If your survey included any negatively worded items, they will be
separated to highlight a lower score is desirable for these items.
Example 2: Item Score Calculation - HSOPS

The item, “Staff are afraid to ask questions when something does not seem right” from the Hospital
Survey on Patient Safety (HSOPS), uses the following response bank:
1 = Never 2 = Rarely 3 = Sometimes 4 = Most of the time 5 = Always
Because this is a negatively worded item from the HSOPS, scores for this item only take into
account the more positive responses to this item, given its context. In other words, the score reflects
the percentage of respondents who chose “Rarely” (2) or “Never” (1) for that item. In this example, a
higher score is better, because the desirable response is for people to be unafraid to ask questions
when something does not seem right.
As you can see item scores are calculated differently for each survey instrument. However, a
response of “N/A” (SAQ only) or a skipped item on any survey is removed from item score
calculations. For more specifics related to scoring the items included on your survey, please refer to
Appendix C.
Survey Domains
Items that ask about aspects of a specific topic are grouped together to form “domains.” A domain
score is the combined result of the items it includes. For example, the Safety Attitudes
Questionnaire has items focused on coordination, conflict resolution, and communication in the
work setting. Taken together, these items represent the broader concept of teamwork and form the
Teamwork Climate Domain. By looking at your domain scores, you can begin to identify areas of
strength and areas of opportunity within your organization as a starting point for digging deeper
into specific items.
You may notice that your domain scores for the SAQ domains seem low compared to the item
scores within it, and compared to the other domain scores from your survey. This is due to the
methodology used to calculate domain scores.
SAQ Domains AHRQ Domains

Methodology Domain score represents the proportion Domain score represents the proportion of
of respondents who, on average, positive responses to total responses for
responded favorably to the majority of items within a given domain
items within a domain
Key Differences SAQ methodology requires that AHRQ methodology is looking at how
someone respond favorably across the many positive responses there are,
majority of items within a domain regardless of how favorably someone feels
across items in the domain


For detailed examples and explanations regarding the two methodologies, please see the Domain
Score Calculation documents included in Appendix C, and also available in the online help section
of HealthBench.
Frequently Asked Questions

What is the Overall Culture Score?
The Overall Culture Score is the average of all domain scores for your survey.
What is a Percent Positive Score?

To represent respondents' perceptions across a work setting, a percent positive score is calculated.
For item scores, this is the percentage of respondents that marked a 4 (agree slightly) or greater.
For domain scores, see explanation above.
My domain scores on the SAQ look drastically lower than the individual item scores. How are
the SAQ Domain Scores calculated?
SAQ domain scores represent the proportion of respondents who, on average, responded favorably
to all items within a domain. For additional information on the SAQ Domain Score calculations see
the SAQ Domain Score Calculation Explanation in Appendix C
My domain scores on the HSOPS look higher than my domain scores from the SAQ. How are the
AHRQ Domain Scores calculated?
The HSOPS is scored on a different scale than the SAQ, as described in this section. For additional
information regarding the AHRQ Domain Score calculations, see Appendix C.
What is the difference between Coarse Position and Position Type?

Another way to look at your data is by position type. The Coarse Position is simply a roll up of the
more granular types. For example, rather than reporting on physician specialties, the coarse
position would provide a score for physicians in general. This is most useful for facilities that have
administered the survey electronically, because the list of position types available for a respondent
to select is quite extensive, and the Coarse Position type option allows a higher level grouping so
that there are fewer respondents in the "Groups with less than five respondents" group on the
position type report.
What are Risk Levels for Work Setting Types in the Scatterplot?
The risk levels assigned to each work setting type in HealthBench are a five-point system meant to
indicate the level of risk that a work setting poses to the broader organization. It takes into
account: (1) the type of patients that the work setting typically treats; (2) the type of procedures that
are typically performed within that work setting; and (3) the overall risk that a serious medical
event will occur within that work setting.


CULTURE SURVEY DATA TIP SHEET

Here are a few key tips for interpreting culture survey data.
Domain Scores:
• When looking at domain scores compare them to one another to find out what strengths and
weaknesses exist. Also look at the size of the domain scores to determine how strong or
weak they are.
• The higher the domain score, the better. As a general rule of thumb, you want domain scores
to be a minimum of 60% and scores above 80% should be recognized as achievements.
Item Scores:
• When interpreting item scores look at (1) the size of the score, (2) if there are discrepancies
between position types, and (3) understand the context surrounding the item.
• A good score for negatively worded items is 20% or less (SAQ only), a good score for the rest
of the items on SAQ and SOPS is 80% or above.
• All items that are below 60% (above 40% for negatively worded SAQ items) should be
flagged as topics that require further investigation and attention.
• Reading the qualitative comments (if available) helps to provide additional contextual
information that can be used to interpret the item scores. Talking directly to staff is another
best practice for understanding the survey scores.
• Look for patterns in survey items that can help explain the scores.

Historical Trends
• Examine if changes in scores over time reflect an actual change in the culture or if it reflects
bias in the measurement process, such as having different people complete the survey at
different time periods.
o If there has been relatively low turnover between survey administrations and your
response rates are similarly high over time, it is more likely that changes are reflective of
true cultural change.
o Note: having similar response rates across survey administrations does not a guarantee
that the same people responded to each survey, it simply means that the same
PORTION of people took the survey at different times.
o The best way to determine if changes in survey scores is caused by changes in culture is
to ask frontline care providers
• The amount of change you can expect to see in your culture survey scores over time
depends on: 1) the amount of time lapsed between survey administrations, 2) what
improvement actions have taken place between survey administrations, 3) if the
improvement actions were related to the concepts measured by the survey, 4) what level of
data you are looking at (work setting, facility, system), and 5) if you are looking at domain
scores or item scores.


• Consider the practical and statistical significance of changes in culture survey scores over
time. Statistical significance tells you the probability that the change in score is not simply
due to chance. Practical significance tells how important and meaningful the change is.
• As a general rule of thumb, for a work setting, consider an improvement of 10-15 percent in
an item score and 3-5 percent in a domain score in one year a success. For a hospital or
facility, consider an improvement of 7-10 percent in an item score and 2-3 percent in a
domain score over the course of a year a success.
Benchmarking
• The benchmark reports compare your data to a national database. For SAQ, your results are
compared to the data from more than 22,000 clinical units from >1,100 hospitals; for HSOPS
items, your data is compared to the AHRQ database. The report displays data in quartiles:
green (higher than 75% percentile); yellow (between 50th and 75th percentile); orange
(between 25th and 50th percentile); and red (below the 25th percentile).
• If you are looking at an entire facility’s data, the usual distribution you would expect to see is
a relatively even amount of green, yellow, orange, and red, indicating that approximately ¼
of each of your units falls into each quartile. If you are only looking at one work setting of
data, the color of the bar (green, yellow, orange, or red) would indicate which quartile the
work setting is in compared to the national database.
• When looking at benchmarking data, it is important to consider both how you compare to
the benchmark (e.g., how many work settings are above the 50th percentile) AND what the
benchmark percentiles are. Being comparable to the 75th percentile when the 75th percentile
is a low score (e.g., 45%) means that it is still an area that requires improvement.
Comparing Survey Data Across Multiple Work Settings

• Compare work settings by domain and item scores. Look for work settings that are
consistently the lowest and highest across domains and items.


CONCLUSION
Surveys are best used to identify issues for further detailed exploration. The results of these
explorations can yield specific improvement opportunities. As such, the survey data and the
questions that arise from the interpretation process should be widely discussed with leaders and
frontline staff. It is this combination of quantitative survey data and qualitative comments that
produce a comprehensive view of culture and will best inform improvement actions.
The next step in the survey process is to follow up with staff through Feedback Sessions to present
the survey results, and to ask for additional insights into the questions the survey results have
produced. Note that your evaluation of the survey domains and items will inevitably generate a list
of questions that must be answered to give you useful information on which to base improvement
activities. The survey results may generate a list of patient safety issues that need to be addressed;
the follow up sessions are likely to inform this list and expand it. The following documentation will
provide guidelines for holding these sessions and how to move forward with Improvement
Planning.


DISCUSSING AND USING SURVEY DATA

A GUIDE FOR SURVEY LEADERS, WORK SETTING MANAGERS,
AND FEEDBACK SESSION FACILITATORS


USING THIS GUIDE

A critical piece of the Improvement Process is communication. Now that you have reviewed your
data, it is time to communicate those initial quantitative results, gather additional qualitative
feedback, and begin the Improvement Process. Reviewing culture data with front-line staff and
using their feedback to develop actionable improvement plans are essential parts of the survey
process.
This guide is designed to assist the Survey Leader, the Work Setting Manager, and the Feedback
Session Facilitator in understanding how to communicate your organization’s survey results – both
quantitative and qualitative.
Why Robust Communication of Survey Results is Critical

Broad communication of results – both ‘up’ to hospital leadership and ‘down’ to frontline staff – is a
critical step in promoting transparency and driving improvement. People who took the time to
participate in the survey deserve to know that their voice was heard and that the facility is taking
the results seriously. Senior leaders need to understand where in the organization there are
challenges that may require more resources and where there are ‘bright spots’ that can help to
inform improvement in other areas.
Everyone has a role to play in communication, but the individuals filling each of the following roles
will be especially critical.
The Survey Leader will need to communicate results to ensure that Work Setting Managers and
Feedback Session Facilitators (“Facilitators”) have access to HealthBench or will need to provide
the data to them. The Survey Leader will also manage the Improvement Process.
Work Setting Managers need to communicate initial survey results to their work setting’s frontline
staff, explain the Improvement Process and ensure participation. Once the Feedback Sessions are
complete, the Work Setting Manager will review the feedback provided and lead Improvement
Planning in the work setting.
The Feedback Session Facilitator shares data with frontline staff members during the Feedback
Session(s), elicits ideas and examples from staff, and shares this feedback with the Survey Leader
and the Work Setting Manager, who use the comments to inform Improvement Planning.
An outline of the flow of communication is included in Figure 1.


Figure 1: Improvement Process
Quantitative vs Qualitative Data

The quantitative data from a survey reveal general perceptions of what it is like to work in a
particular work setting, and provide guidance on where to focus energy and resources to improve
the culture within an organization. This data often generates more questions than it answers. The
qualitative comments from staff members, obtained during the culture survey or Feedback
Sessions, provide important context about specific issues, which allow for a more comprehensive
understanding of a unit’s culture and challenges to improvement. Both the quantitative and
qualitative data together provide a picture of the perceived culture within the organization and
some of the key opportunities for improving it.
Qualitative comments offer information that may not be discovered through other means. An
anonymous survey or well-moderated Feedback Session may be the only opportunity for staff to
express concerns and frustrations about issues that they do not know how to address or have not
had the opportunity to bring to the attention of leadership. Each comment should be carefully
considered, especially the measured and thoughtful observations. In general, staff who take the
time to write in comments on an online survey are engaged and motivated.


In addition to observations about the culture, the comments may also include information about
the physical working environment, equipment and supplies, as well as constructive suggestions for
improving safety, process efficiency, and overall working conditions. By encouraging participation
in a culture survey, there is a tacit agreement that the institution is interested in uncovering and
understanding staff concerns and is willing to address them. Failure to do so is likely to jeopardize
trust in the institution’s leadership and commitment to a culture of safety.
FOR THE SURVEY LEADER

As your facility’s Survey Leader, the preparation and administration of the survey were just the first
steps in the process. At the end of the survey administration, you have two sets of data that need to
be analyzed and disseminated: the quantitative results and the qualitative results from the
qualitative comments collected in an electronic survey as well as the Feedback Sessions. Your role
is to communicate this information both to Leadership and to Work Setting Managers as well as set
the stage for the Improvement Process.
Sharing Survey Data With Leadership

As your facility’s Survey Leader, an important part of your role is to ensure that the survey results
are shared as widely as possible with front line staff and managers, as well as Executive Leaders.
Before the Executive Briefing it is important for the Survey Leader to share preliminary results with
leadership in preparation for this session. Preliminary results to show include the Quantitative data
and themes apparent in the comments filled out when the survey was taken. Timely
communication of survey results to Executive Leaders is important to generate support for the
work that will follow. The data you present to them must be clear about areas of risk and
opportunity as well as the bright spots in your organization. This will help Executive Leadership to
understand the concentration of resources that may be needed to address opportunities across the
facility.
Through the Feedback Process, you will gain additional insight from the comments gathered
during the Feedback Sessions. There are often many similarities between the Qualitative
Comments entered during the survey and those from the Feedback Sessions. Regardless of the
source of the comments, they are truly the ‘voice’ of staff members. These comments are vital to a
complete understanding of the quantitative results and they should be communicated to leadership
as well.
Of course, communication with Executive Leaders should begin long before the survey results are
available. Leadership should be engaged before, during and after the survey is complete so they
understand the commitment the facility is undertaking, the need for adequate resources, and how
they can help set expectations for a successful process. Leadership support in linking the survey to
organizational initiatives will go a long way toward gaining an understanding of your
organization’s culture.


Quantitative Results
There are numerous tools on HealthBench that you can use to identify which information to share
with Executive Leadership. Here are some recommendations:
Reports
• Dashboard
• Current Year Report (by Work Setting)
Interactive Analytics
Scatterplot
Pick two Domains/Items that:
• Show high scores across the facility (highlights)
• Show a broad range of scores (some work settings in lower left quadrant [below 60%]
ranging to work settings in upper right quadrant [above 80%])
• Show Areas of Greatest Risk: Pay close attention to work settings that are inherently higher
risk (Emergency Departments, Intensive Care Units, Operating Rooms, Labor and Delivery,
other procedural areas). If they are below 60% on particular domains or items of interest, the
Senior Leadership should be aware of that so they may devote appropriate resources to
understanding the issues and improving performance. The Scatterplot is one of the most
effective ways of displaying this data.
• Relate to a facility-wide initiative or focus
Recommended domains for comparison:

SAQ: Teamwork Climate and Safety Climate
HSOPS: Teamwork within Hospital Units and Overall Perceptions of Safety
Dashboard
Use the Dashboard to show:
• Highest scoring and Lowest scoring items
• Overall Culture Score
• Highest and lowest scoring work settings/facilities
• Most improved/declined work settings/facilities
Bar Charts
The interactive Bar Charts are helpful for displaying data by the facility, work setting, or by role
type.
Qualitative Results
The comments entered by respondents during the survey are the preliminary qualitative results
you may present to leadership. These comments often provide important context to the
quantitative results and you may quickly identify themes that would be valuable to provide to
Executive Leadership.


Knowing what to share with Executive Leadership can be difficult; the data sets may be large and
the comments broad. The best approach is to group similar comments together, by theme or how
they relate to individual items/domains of the survey. It may be helpful to think of the attributes of
a positive culture – transparency, psychological safety, teamwork, leadership, communication – and
group representative comments according to those themes. Another option is to share comments
that speak to certain items on the survey that you have brought to the Executive Leadership’s
attention.
For more information regarding how to share feedback results with leadership, see Appendix E.
Sharing Survey Data With Work Setting Managers

Work Setting Managers should have access to HealthBench and should be accountable for
reviewing their Work Setting’s survey data using the Work Setting Dashboard, Current Year
Report, and Culture Change reports if there are multiple years of data available. These reports can
also be downloaded and distributed.
Some ideas and best practices for sharing the data include:
• If your facility has a computer lab (for training), host several sessions where Work Setting
Managers can all log into HealthBench, view and download their results at the same time.
This is helpful because you, the Survey Leader, will be there to answer questions about
accessing results and capture initial impressions.
• Schedule a ‘brown-bag lunch’ for Work Setting Managers. They bring their lunch and a copy
of their results which will encourage discussion and ensure that people have viewed their
results!
• Meet with small groups of Work Setting Managers (for example: all work settings in the
Department of Medicine; all outpatient areas; etc) to review results.
Encourage Work Setting Managers to share the following with their front-line staff members:
• Dashboard (by Work Setting)
• The Current Year Report (for a Work Setting by Role Type)
(Work Setting Managers may want to put these results in a binder and put it in the break room, or
post on a wall. Encourage them to do this as soon as the results are available -- it is an excellent way
to draw attention to the results and encourage participation in the discussion during the Feedback
Sessions.)
Qualitative Comments and Feedback Sessions
If your facility took the survey electronically, each Work Setting may also have Qualitative
Comments. Because these comments are “unfiltered,” careful consideration should be given to how
they are used. It is vitally important that they are discussed in a constructive manner, especially if
they are negative. The Qualitative Comments entered into the survey will often be very similar to
what will come up during the Feedback Sessions and a skilled Facilitator can often ‘drill down’
based upon their knowledge of what was entered anonymously during the survey. The conversation


during the Feedback Session also provides additional context to the comments shared on the
survey.
Dealing With Negative Comments About Management

If the qualitative comments indicate that multiple staff find the work setting management to be
ineffective, punitive, fear-based, disorganized, abusive or unresponsive to staff needs, it is unlikely
that performance improvement efforts will be successful. If morale is suffering, so will teamwork,
safety, engagement and other aspects of work life.
Negative comments should not be used as an automatic indictment of the staff member; rather,
they provide an opportunity for a conversation with the manager about substantive issues
identified during the survey. Many times, managers are unaware of how they are perceived by staff,
or staff may have made erroneous assumptions about the manager’s intent or personal style.
The Survey Leader and/or the manager’s Director should discuss them with the Work Setting
Manager and refer to the Feedback Session section for guidance on how to initiate the
conversation.
Feedback Sessions
In addition to sharing the information collected during the survey administration, an important
part of the Improvement Process is collecting additional information from staff through Feedback
Sessions.
Remember to provide the Qualitative Comments to the Feedback Session Facilitator for review
prior to the sessions as well. These can be used to develop discussion topics/questions to help
guide the discussion.
What Happens During Feedback Sessions?
During a Feedback Session, a neutral facilitator:

• Shares the results of the survey
• Answers questions staff may have about the survey
• Elicits staff insights about the survey responses and their perceptions of the culture of the
work setting
• Helps staff identify an item(s) or issue(s) that concerns them most
The Facilitator writes up his/her notes from each Feedback Session using the guidance in
Important Things to Note in Your Feedback Session Write-up section for Facilitators. After all of the
Feedback Sessions have been conducted for the work area, the Manager will have a chance to read
through all the material before conducting the Improvement Planning Session with the staff.

During the Improvement Planning Session, the Work Setting Manager:
• Reviews the common issues and desired improvements that arose during the Feedback
Sessions
• Identifies (with the staff) actionable, sustainable, measureable improvements to work on
The Manager can use the mechanism used by your organization to create and track improvements,
or can use the templates and tools provided by Pascal to guide the improvement work in their area.


FOR THE WORK SETTING MANAGER

Introduction
As Work Setting Manager, you play a vital role in the collection and use of survey data. At the end
of the survey administration, you will have access to the quantitative data (available on
HealthBench), and if your facility administered the survey electronically, you may also have
qualitative comments from respondents. The next step is to share the results and prepare for the
Feedback Sessions. Your role is to explain this process to staff, assist the Facilitator with setting up
Feedback Sessions, and to use the qualitative results entered during the survey and from the
Feedback Sessions to drive improvement at the work setting level, through the Improvement
Planning Session with the staff.
Reviewing Your Survey Results

When the Survey Leader notifies you that your results are available, you should login to
HealthBench to review them. Guidance on how to get the most value from your survey results is
available in Section 1 and HealthBench User Training.
Sharing Data With Your Work Setting’s Staff

Once you have had a chance to review and interpret the results, you should share the following
with your work setting’s staff at your earliest convenience (a staff meeting is a good venue for this).
Print out the following reports:
• Dashboard (by Work Setting)
• The Current Year Report (for a Work Setting by Role Type)
When sharing results with staff, you should thank them for taking the survey and share the
response rate for the work setting. You may also want to share highlights or scores of interest to
you when you reviewed the results. The full report should be available for staff to review at their
convenience, perhaps by putting the results in a binder and leaving it in the break room.
Be sure to tell staff about the next steps: the Feedback Session and Improvement Process. Having
feedback sessions about the data is important because it allows the people who took the survey to
provide context to the quantitative results. There is no way to truly understand what the survey
respondents were thinking about or how they interpreted the questions without going to the source
– the survey respondents themselves. You might think of the Feedback Session as a type of ‘focus
group’ during which a Facilitator shares the culture survey results for a particular work setting with
the staff and allows them to explain what they were thinking about when they answered the
questions. This dialogue allows the Facilitator to drill down on items of interest or concern.
Explain to the staff that once the Feedback Sessions are over, you will receive a write-up of the
discussion, which will serve as a tool as you begin working on improvement activities as a group.


Work Setting Manager’s Role in the Improvement Process

Reviewing culture data with front-line staff and using their feedback to develop actionable
improvement plans are essential parts of the process. These instructions align with the Pascal
Metrics methodology for a two-part Feedback and Improvement Process, which consists of a
Feedback Session and an Improvement Planning Session. It is based on industry best practices and
how Pascal Metrics has partnered with clients to build improvements into an institution’s culture of
safety.
Two Steps in the Process
The Improvement Process at the work setting level consists of two steps: the Feedback Session(s),
which is facilitated by a “neutral” person from outside of your work setting; and the Improvement
Planning Session, which you will lead. The Survey Leader for your facility can provide more
information about who the Facilitator will be for your work setting.
Why a two-step process?

It is far too difficult to share data, elicit feedback, and come up with an improvement plan all in one
session. It is also common for different shifts and groups to decide on variable plans of action,
which is challenging to manage. The approach we recommend is for you to review the data from all
Feedback Sessions and assist your staff with aligning their improvement efforts. A dedicated
session for this, the Improvement Planning Session, is a good forum for you and your staff to agree
on a unified plan of improvement. It also allows you an opportunity to respond to comments or
questions that arose during the Feedback Session. The product of these two sessions is information
that will drive improvement.
Why Am I Not Facilitating the Feedback Session?

Our experience has shown that discussions are more open and productive when a person unrelated
to the work setting facilitates the Feedback Session. In many cases, it is helpful simply because
someone unrelated to the work setting is, in essence, a ‘blank slate.’ They do not have preconceived
notions of past issues or challenges and are completely objective about concerns raised during
Feedback Sessions. This process is not intended to ‘exclude’ managers but rather to provide them
with the most robust and productive feedback possible.

Reviewing the Write-ups from the Feedback Sessions
When you review the notes from the Feedback Session Facilitator, perhaps the most important
thing to keep in mind is that the comments from your staff are not meant to be taken personally.
That can be tough to do – after all, you work hard to make the work setting the best it can be, but
the staff may reflect that it is still falling short. Keep in mind that staff are sharing opinions and
perceptions, which are valid and useful, but can occasionally reflect misinformation that has
become the ‘truth’ in the work setting.
Take the information in the spirit in which it was elicited: as a way to learn more about the work
setting, to identify issues that may not get talked about (and you may not even have known about),


and to determine areas for improvement. The Feedback Sessions often elicit issues (and
assumptions!) that would otherwise not be discussed – this is your opportunity to clarify
misunderstandings, understand assumptions, and learn about challenges that may never have
come to your attention before.

FOR THE FEEDBACK SESSION FACILITATOR

Introduction
This guide is designed to help you successfully facilitate work-setting level Feedback Sessions for
your institution’s survey results. The suggestions and tips are based on years of experience and
best practice at institutions nationwide; however, your Survey Leader (usually the Director of
Safety/Quality or equivalent) may wish to modify these instructions to best suit your institution.
The Feedback and Improvement Process

The Feedback Sessions you will be leading are the first step in a two-step work setting level
Improvement Process. Once you have completed the Feedback Session and written up your notes
(refer to Important Things to Note in Your Feedback Session Write-up below for guidance), the
Work Setting Manager will meet with the staff to begin Improvement Planning.

How to Conduct a Successful Feedback Session
1. If possible, arrange the group in a circle. While waiting for everyone to arrive, engage the group
in small talk about how their day is going, if the unit is busy, etc., as a way to break the ice and put
them at ease.
2. Introduce yourself to the group and explain that you are conducting the Feedback Session on
behalf of the _____________(Patient Safety Office/Dept of Quality/etc). Your purpose is to:
• Share the results of the survey
• Answer questions they may have about the survey
• Elicit their insights about the survey responses and their perceptions of the culture on the
unit
• Help them identify issue(s) that they would like to work on
3. Ask if the staff members present are all from the same shift (if applicable). If not, be sure to ask
staff if perceptions are the same for different shifts.
4. State that everything that is said is anonymous – you will be taking notes and the Feedback
Session write-ups will go to the work setting manager and higher leadership. However, you will not
record anything that would be identifiable to a particular person/case.
5. Ask the group if they remember taking the culture survey (sometimes they get it confused with
other surveys). If they are not sure, describe it in a general manner (“It asked about your


perceptions of teamwork, the safety of patient care, if management is supportive, etc. You took it in
(month)”). Regardless of whether staff completed the survey, they should be invited to participate
in the feedback sessions.
6. Show the group the one-page reports for the facility domain scores and their work setting domain
scores. Tell the unit their participation rate (be sure to congratulate them on good response rates!).
7. Review the work setting’s data, highlighting good scores and pausing to ‘drill down’ on lower-
scoring items of interest. You may need to initiate the conversation by asking open-ended
questions:
Example: “The score on item x was quite low (showing them the bar chart). ” What do you
think respondents in this work setting were thinking about when they answered this
question”
Or relate your question to the item: “Can you give me an example of a time when you felt your
input was not well-received?” or “Why do you feel that teamwork between nurses and
physicians is not as good as it could be?”
If only one staff member answers, you can ask the others whether they agree or disagree to
get them engaged. If no one responds, don’t be afraid of a little bit of silence. Move on to the
next question; very often you can tie back to previous questions. Once staff is comfortable
speaking, they are often very vocal!
8. Listen for “absolutes” (“the MDs never tell us the plan of care”; “the blood pressure cuffs are
always missing”). Ask for more specific estimates – how many times per week or day or shift does
this happen? Something that is measurable presents a ripe opportunity for improvement.
9. Don’t be concerned if the session turns into more of a conversation and less of a data-sharing
presentation. Going through every graph is less important than hearing and documenting
concerns, issues, and examples.
10. Remember to thank the staff for their time and input and reiterate that their input will only be
shared in an anonymous manner. Their Work Setting Manager will follow-up regarding next steps
with the improvement plan.
11. Remind them that if they have any other questions or concerns about the culture survey or
safety issues in general they are always welcome to contact _________.
12. Write up the notes you took during the Session and send this document to the Survey Leader so
he/she may review them in aggregate to identify issues and concerns that transcend individual
units. See Important Things to Note section below.
Important Things to Note in Your Feedback Session Write-up
1. Time and date of Feedback Session.


2. # of staff members in attendance; role breakdown (RN, tech, etc.). DO NOT take down names. If a
large group you can note: “Approx. 15 RNs, 6 PCTs, 3 MAs.” If a small group (and individuals would
be identifiable), simply note: “Total of 7 staff attendees; mix of RN, PCT, MA.”
3. Who the group perceived as Local Management and who as Senior Management (SAQ only).
You can record names or roles (i.e. “the charge RN or nurse manager” or “the Executive team”)
4. If applicable, the general mood of the group, i.e. “the staff seemed fearful of answering questions
about management.”
5. Unanimity or dissent:
One staff member stated that she feels “respected” by Attendings on the unit, but Residents don’t
include her in the plan of care for her patients and “blatantly disregard” Unit policies. The rest of the
group was in complete agreement. One night shift nurse stated that she feels the Residents “run wild”
when there are no Attendings or Fellows around.
Or
A patient care tech described feeling “overwhelmed” when she works on day shift. Several other techs
did not share that concern, though they stated patient acuity seems higher than previously.
6. The template in Appendix D may be a helpful way for you to organize this information. However,
you may find there are different categories (such as “Process” or “Equipment”) that you would like
to use, depending on the notes you took.
7. If staff are particularly adamant about something, underline, italicize or otherwise emphasize the
statement or issues to draw attention to them.
8. Keep in mind that staff are sharing opinions and perceptions, which are valid and useful, but can
occasionally reflect misinformation that has become the ‘truth’ in the work setting. The Feedback
Session helps elicit issues (and assumptions!) that would otherwise not be discussed, and this will
provide valuable information back to the Manager so he/she can clarify misunderstandings and
erroneous assumptions.
9. When thinking about how to write up comments, keep in mind that constructive comments
should be shown to the manager, but judgment should be used in filtering unconstructive
comments.
a. Not appropriate to share verbatim: “We all think he’s a jerk.” “I can’t stand working
with her.”
b. Appropriate to share verbatim: “She is difficult to approach and does not have an
open door policy.” “He discourages us from submitting event reports; if you do, you
get written up.”
When you have finished the Feedback Session Write-ups, send them simultaneously to the Work
Setting Manager AND the Survey Leader


Frequently Asked Questions for Facilitators

What should I do if no one shows up to a Feedback Session or participation is really low?
Due to unforeseen clinical situations or staffing issues, it is possible that people may not be able to
attend a scheduled session. Check with the manager about rescheduling and ask them to remind
staff about the importance of participating in a Feedback Session.
What if no one talks during the session?

While it is not uncommon for a group to be quiet initially, once one person starts talking, you can
usually draw others into the conversation. It may be uncomfortable, but keep the conversation
going. Maybe the first few items you ask about don’t really resonate with people or they can’t think
of specific examples to give. You can ask broad questions to draw people out like “What is a
frustration you often have in this work setting” or “Tell me what you like most/least about working
here.”
What do I do if there is one person constantly talking and not letting others talk?
You may have to (politely) interrupt if one person is talking nonstop. You can say “Thanks for your
input – let’s let someone else provide some examples” or “We have such limited time during this
session that it is important everybody has a chance to share their ideas.” If you see another
attendee nodding or shaking their head, point to them and say “You look like you agree/disagree –
can you tell me more?”
What happens if the manager insists on being part of the session when that was not planned?
If the manager gets there before the session, you can remind him/her that the Feedback Sessions
are usually conducted without them there. If they insist on staying, or arrive once the session has
started, there is not much you can do at that point without it being awkward. Be sure to let the
Survey Leader know that the manager participated.
What do I do if the manager attends the session and does the majority of talking?
As with any other session member who monopolizes the conversation, you may need to interrupt
the manager in a polite way. Be sure to redirect the conversation to other attendees by saying “I’d
really like to hear some examples from your staff” or something similar.
What do I do if I learn about an unreported serious safety event during a session?

Please encourage staff to report the event and note it in your write-up. You may also wish to alert
the Survey Leader as soon as possible so it doesn’t get buried in the write-up.
What do I do if the conversation turns into a complaining session?

There are a couple of strategies that you can use to help keep the conversation constructive, even if
staff have a lot of negative comments. Don’t allow the group to “beat a dead horse” – once everyone
has weighed in on an issue, you can say “Ok, I’ve recorded your comments, let’s move on to another
item.” If people keep complaining about something, ask them for suggestions to fix it. You can ask
the meeting attendees to identify positive aspects of the work environment. Sometimes people get
bogged down in a specific example or incident that has occurred (usually recently) – ask them to
think more broadly. Again, once you have exhausted an issue or discussion of a particular incident,
move on.


What does the Facilitator do if the comments are really negative?

The qualitative comments from staff members obtained during the Feedback Sessions provide
important context about specific issues in the work setting and information that may not be
discovered through other means. In fact, the Feedback Session may be the only opportunity for
staff to express concerns and frustrations about issues that they do not know how to address or
have not had the opportunity to bring to the attention of leadership. The Sessions may also uncover
assumptions and misunderstandings about various issues.
As a Facilitator, it may feel uncomfortable to document the concerns that people express, especially
if they are very negative about management or leadership. You may feel that you are in an awkward
position having to record the staff’s dissatisfaction; after all, you don’t want anyone to get in trouble
from something you have written!
Keep in mind that what you are recording are perceptions and, often, assumptions. How a staff
member feels or perceives things is absolutely valid, but it may not reflect the absolute truth about
what is going on in the work setting. Your job is not to fix problems, make judgments, or resolve
issues – it is simply to record the concerns of staff as accurately as you can. You can maintain a
neutral tone in your report by using phrases like “Staff perceived abc…” or “Staff stated that xyz…”
Negative comments about a manager, for example, are not an automatic indictment of the staff
member; rather, they provide an opportunity for a conversation about substantive issues identified
during the survey. Many times, managers are unaware of how they are perceived by staff, or staff
may have made erroneous assumptions about the manager’s intent or personal style. Frustrations
that staff share with you may be completely unknown to the manager, even if they have persisted
for a long time (and people have made the assumption that of course the manager has known about
it, but in reality nobody has ever told them).
Negative comments are always difficult for people to hear, but fear of hurt feelings should not get
in the way of improving a culture of safety. Sharing staff comments should be viewed as a learning
opportunity for all parties, with a view to improving the communication and functioning of the
work setting.
What do I do if I forget to take notes during the session?

It can be tough to facilitate an interesting conversation and take notes at the same time! However,
please try to piece together at least some of the conversation. It is likely that if you got so absorbed
you forgot to take notes, it was because the session was very intense. Can you remember specific
examples or statements that really grabbed your attention? It may be helpful to arrange a scribe, in
advance, to record the conversation during the Feedback Session.

FREQUENTLY ASKED QUESTIONS
1. If not the manager, who should conduct Feedback Sessions?
A wide variety of your facility’s staff can be successful facilitators of Feedback Sessions. Facilitators
may be members of the Quality Management/Safety/Risk departments, nurse educators, human
resources personnel, or others who will be perceived as ‘neutral’ by unit staff. It is helpful to have
Facilitators who are familiar with clinical processes and a general understanding of clinical
operations, but they do not need to have a clinical background. High-acuity areas of a hospital
(ORs, ICUs, ED, etc), for example, should have their session(s) conducted by someone with a
sophisticated understanding of clinical processes relative to those units.
A Work Setting Manager should not facilitate or attend the Feedback Session in his/her own area.
If necessary, Work Setting Managers can be Facilitators, as long as they are assigned to a work
setting with which they have had little or no contact.
2. Do certain areas get priority?

High-risk/high-acuity work settings with low scores should be the first priority. These may include
the Emergency Department, Labor & Delivery, perioperative services, the Intensive Care Unit, and
the Pharmacy. After that, lower-risk but low-scoring areas should be among the next wave of areas
to debrief, continuing with other low-scoring areas. Work setting size may also be a determining
factor in developing the priority list.
The Scatterplot tool in the Reporting & Analytics section of HealthBench provides an excellent
graphical representation of acuity, size, and scores for each work setting in your institution. The
scatterplot below shows Teamwork Climate scores on the x axis and Safety Climate (from the SAQ)
scores on the y axis. Your initial focus should be on the higher-risk (red and orange circles) units in
the lower left quadrant of the graph (i.e. those that have scored under 60% in those domains). If
using the HSOPS tool, you may choose Teamwork within Hospital Units and Overall Perceptions
of Safety.

3. What percentage of respondents in a work setting should participate in the Feedback Session?
It is recommended that at least 50% of a work setting’s staff members participate in a Feedback
Session, so that resulting action items are supported by the majority.
4. Should there be multiple sessions per work setting?

Feedback Sessions will need to take place for each shift (day, evening, night, weekend) to ensure a
representative sample.
5. Should Feedback Sessions be multi-disciplinary or should only one provider type participate
at a time?
The group can be multi-disciplinary or limited to one role type (e.g. RNs, Patient Care Technicians,
etc) at a time. If the results show a large discrepancy between position types, it is best to do the
Feedback Sessions by role type. Regardless of the strategy, all members of a work setting who were
included in the survey should have the opportunity to participate.
6. What materials or information should the Facilitator have for conducting the Feedback
Sessions?
It is helpful to put together a “packet” for each Facilitator that contains:
• Dashboard
• The Work Setting Current Year Report OR Culture Change Report (if more than one year of
data is available) for (Work Setting)
• Instructions for Conducting the Feedback Session
Additionally, you should ensure that the Work Setting Manager has access to HealthBench so
he/she can explore the data in more detail, as needed, and to input the results of Improvement
Planning Session.
7. What needs to be done prior to the Feedback Sessions?

Three main tasks prior to the Feedback Sessions:
• Managers should review your work setting’s data, noting items of concern or interest that
you would like the Facilitator to ‘drill down’ on during the Feedback Session.
• Facilitator contacts Work Setting Manager to arrange the times, dates, and locations for the
Feedback Sessions.
• Work Setting Manager informs staff of the Feedback Sessions!
8. Where and when should Feedback Sessions be held?

Ideally, Feedback Sessions should take place in a quiet area away from the distractions of the work
setting. If it becomes difficult to get enough staff away from the work setting to participate in the
Feedback Session, a break room on or near the work setting is often the best choice. Feedback
Sessions should not be conducted at a nurse’s station or in another ‘public’ area; there are too many
distractions and the environment is not conducive to ensuring that people will feel comfortable
speaking up.
Conducting the Feedback Session(s) during a regularly-scheduled staff meeting is often a good
way to ensure attendance. It is also helpful to schedule near shift change so that you may speak
with both off-going and on-coming staff.

Allow an hour for each session, although sometimes it is easier to do a few half hour sessions with a
smaller number of staff. Ideally, there should be no more than 12 staff per session, as it makes it
difficult for everyone to share their opinions.
9. What is the involvement of the Work Setting Manager in the Feedback Sessions?
The Facilitator should ask the Work Setting Manager if he/she has any particular concerns or
questions that they would like you to ask. If they have not yet reviewed their data, encourage them
to do so and get back to you prior to the Feedback Session. Remind the manager that the Feedback
Session is conducted without him/her present. After the Feedback Sessions are completed, the
Work Setting Manager will move forward with the next phase of the process: the Improvement
Planning Session.
At least one day prior to your scheduled session, please contact the Work Setting Manager to
ensure they have informed staff about the session and that a representative sample will be able to
attend. Sometimes there are unavoidable conflicts or emergencies in clinical areas, so please be
flexible if they need to reschedule.

IMPROVEMENT PLANNING
A GUIDE FOR WORK SETTING MANAGERS

MOVING FORWARD WITH IMPROVEMENT PLANNING
The Improvement Planning Process is the responsibility of the Work Setting Manager. Culture is
local and so is improvement. The issues that staff identified through the survey or during Feedback
Sessions relate to perceptions of their local environment. While the Survey Leader maintains
oversight over the Improvement Planning being conducted across work settings, Work Setting
Managers are responsible for Improvement Planning in their work setting.
Categorize issues Assemble team, set

Review issues using Improvement Meet with staff to goal and begin Monitor, evaluate and
identified during prioritize issues to adjust intervention,
Action Tool or similar working on
Feedback Sessions address as needed
tool improvement
Setting a Plan
The value in any evaluation comes from the information gained from it and how it is put to use.
Until now, you have spent a great deal of time reviewing your survey results, considering the areas
of opportunity, and discussing them with your staff. Information gathered during the Feedback
Sessions, along with the Qualitative Comments that staff shared during the survey will help you
learn what the staff in your work setting feel is important.
In the Feedback Session write-up the Feedback Session Facilitator groups the comments according
to themes. This provides you [the Work Setting Manager] with the information needed to begin
Improvement Planning. The tools included in Appendix G (Improvement Planning Template) are
intended to serve as a guide when you’re not sure where to start, as well as a complement to the
many improvement tools that are already in use. If your organization has a preferred improvement
tool, you may choose to use that instead.
Categorize Issues
To begin taking action on the feedback from staff, categorize the issues into groups based on how
and/or by whom they can be addressed. Doing this will help you and your staff determine priorities
and needed resources. We recommend using these three categories:
1) Issues the Manager can address on his/her own: There will be issues from both the Qualitative
Comments and the Feedback Sessions that the Manager may be able to act on independently. Very
often, these include misunderstandings or misperceptions and equipment/environment issues.
While misperceptions and misunderstandings are frustrating for most managers to hear, it is far
better for the work environment when these are shared openly so they can be addressed! It is
common to hear about misunderstandings about policies or procedures, or assumptions that have
become an accepted truth, but aren’t actually correct! If there seem to be a lot of these, it may reflect
the need for more frequent communication with staff through huddles, staff meetings, or posting

information in public places (like a break room) or the use of different communication tools so that
everyone is hearing the same thing.
Sometimes minor equipment or supply issues create major headaches and/or dangerous
workarounds for staff. You may not be aware of these issues, such as a shortage of blood pressure
cuffs or a printer that keeps breaking down, but these are low hanging fruit - an opportunity to
make a big difference in a short amount of time. Quickly correcting these is a big ‘win’ for the
manager! Be sure to involve staff as needed in any decisions which affect their environment or
work process, and be sure to keep staff ‘in the loop’ about the changes that are being made (posting
a list of things that are getting fixed is a great way of doing this).
2) Issues that can be addressed by peer members of the work setting: These can include teamwork
issues among staff; suggestions about new ways to do things; communication problems in the work
setting, etc. Addressing these issues will require input from the staff on their preferences and
involve testing new processes or communication strategies to determine what works best. Staff
should address the issues that fall into this category since any interventions that result will directly
impact them.
3) Issues/Items that require outside assistance: Some issues will require the manager to elicit
assistance from people outside of the work setting, such as managers of other work settings or
service areas (such as Human Resources), as well as leadership. These issues might include
difficult patient handoffs from one unit to another, disruptive behavior by physicians or other staff,
difficulty getting supplies, test results, or medications in a reasonable time, major
equipment/environment issues, or staff concerns about the organization or executive leadership.
Meet with Staff to Prioritize Issues

A good place to begin the Improvement Planning is at a staff meeting using the issues you
categorized. At the beginning of the meeting share with staff how you plan to address issues under
your control (category 1). Some of them will be things that you as a manager will need to take care
of or delegate, but there are other issues where your role will be to advocate for staff and remove
barriers where possible. If you have limited resources, ask your staff to help you determine the
highest priority issues or items; the ones that are making their daily work most difficult. Next you
will need to prioritize with staff issues that can be addressed within the work setting (category 2).
For these items your role will be helping the staff develop a plan and stay on track with goals
outlined on the Improvement Planning Template (Appendix F). The Improvements decided on
should be posted in a break room so that everyone in the work setting knows what is being
addressed! Doing this will make the changes underway visible which will go a long way in gaining
the respect and engagement of staff.

Begin Work On Action(s) Using the Improvement Planning Template
Once you have selected an issue(s) to work on, use the Improvement Planning Template to help
you stay organized, track progress, and identify barriers to success. For each issue identified, the
staff impacted should engage in discussing possible causes and set an improvement goal. The goal
should be both specific and measurable including both a numerical target (by X%) and a timeframe
(by a specific date). We recommend a team of individuals willing to work towards that goal be
assembled with leadership from the manager or a safety champion within the same work setting.
Sustaining improvement is dependent on the involvement of staff whose daily work is most
affected. Since they will share responsibility and ownership for the goal, staff should be considered
while assembling the team.
The strategy outlined by the team will occur in three phases: the Planning, Action and Evaluation
Phases. The Planning Phase is an opportunity for the team to assemble, meet and discuss the issue
identified as well as potential solutions. This phase is essentially when a plan is mapped out.
During this phase the team should also consider what change they would consider to be
improvement. This may include identifying metrics for evaluation, some of which already may be
measured on an ongoing basis and others that would require some additional data collection.
During the Action Phase, the team will identify strategies to implement and how to engage the
related staff to execute the plan. The last phase is Evaluation and ongoing monitoring of the
action(s) implemented. This is often where teams tend to taper their efforts, but it is the most
critical element to sustain any improvements that were gained. If changes were not apparent or
moving in the right direction, this is also the time when the team would re-evaluate the plan and
make adjustments. Evaluations are recommended at intervals of 1, 2, 3, 4, 8, 12, and 16 weeks to
maintain a focus on achieving the goal and sustaining improvement. As the team works through
each phase, the Improvement Strategy Tool can be used to track who has agreed to take
responsibility for each action and the deadline that has been set. It can also serve as a guide for
team meetings and following up on the status of each task in working toward the goal.
Communication is essential throughout the entire process. The team’s efforts should be shared and
recognized so the staff is aware that their concerns are being heard and addressed. These activities
should continue until the goal is achieved and/or the action implemented becomes part of the
standard process.
The IMPROVEMENT PLANNING TEMPLATE in Appendix F can help guide the Improvement
Process.
Sustain and Spread

The Improvement Strategy Tool can be used to organize a plan for each issue to be addressed.
While the team assembled to work on each issue may differ, the process is the same for planning,
acting on, and evaluating each intervention. There are often many improvement activities
occurring simultaneously throughout an organization, so communicating with all staff the
intervention will reach throughout each of the phases will make it much more likely to be adopted
and spread. You may consider whether there are any forums in your organization where you could
share new strategies or approaches and share what has been learned.

In addition to communicating with staff, it is also critical for the Work Setting Manager to monitor
and communicate progress to the Survey Leader. We recommend that the Work Setting Manager
review progress every 6 months to continuously monitor the interventions and results. Often it is
difficult to attribute changes in survey scores to a particular intervention, but tracking those that
take place between survey administrations will help facilitate learning about the impact on culture.
Completed WORK SETTING ACTIONS TOOL in Appendix G can be used to share progress
updates with your Survey Leader.
Additional Resources:
Defining Measures
National Quality Measures Clearinghouse: http://www.qualitymeasures.ahrq.gov/
National Quality Forum-endorsed Measures: http://www.qualityforum.org/QPS/QPSTool.aspx
Implementation Guides
Implementation Guide for Improving Patient Safety:
http://www.ahrq.gov/professionals/quality-patient-safety/patient-safety-
resources/resources/pstools/index.html#guides
Leadership Engagement:
http://www.ihi.org/resources/Pages/Tools/StrategiesforLeadershipHospitalExecutivesandTheirRo
leinPatientSafety.aspx
Leadership Guide:
http://www.ihi.org/resources/Pages/IHIWhitePapers/LeadershipGuidetoPatientSafetyWhitePaper
.aspx

APPENDICES

Appendicies
Appendix A: Getting Started Checklist
Appendix B: Post-Survey Timeline
Appendix C: Domain Guides
Appendix D: Survey Leader Checklist: Post-Survey Communications
Appendix E: Key Themes from Feedback Sessions
Appendix F: Improvement Planning Template
Appendix G: Feedback to Survey Leader

Appendix A
IMPROVEMENT PROCESS
Getting Started Checklist
Congratulations! You have completed the survey administration phase and now the fun begins. It’s
time to look at the data and begin to take action. Before you get started, there are several things to
think about. Below is a checklist of steps you will need to complete in order to get started on the
Improvement Process.
o Outline and Fill Key Roles

As the Survey Leader, there are certain key roles you need to fill before this process can begin.
Identifying the appropriate people to fill these roles will be critical to the success of the
Improvement Process.
1) Feedback Session Facilitators will lead Feedback Sessions and provide notes back to the
Work Setting Managers. You need enough Facilitators to perform Feedback Sessions for
each Work Setting participating in the Improvement Process.
2) Leadership will need to provide support for the Improvement process and encourage
compliance with the outlined plan.
3) Managers will play a critical role in carrying out the actions planned, manage the day-to-
day actions, and monitor progress.
o Identify HealthBench Users

Pascal Metrics strongly recommends that Facilitators and Managers be given access to
HealthBench to review data on their own. Facilitators and Managers who have HealthBench
access will benefit from access to tools which allow them to interact with the data as well as and
utilize tools to being improvement planning. In the absence of HealthBench access granted,
data will need to be provided to Facilitators and Managers and a process for communicating it
will need to be devised.
o Schedule HealthBench User Training

All designated HealthBench Users will be expected to attend this training.
o Plan Using Survey Data Series

Survey Leader prepares those holding key roles for the trainings, and invites them to the
session. At a minimum, all Feedback Session Facilitators and Work Setting Managers should
attend these trainings to learn how to interpret data and facilitate Feedback Sessions.
o Accountability for Improvement Planning

Survey Leader provides timeline and expectations for the feedback sessions, and determines
check in points.
Appendix B!
IMPROVEMENT PROCESS TIMELINE
IMPROVEMENT FOUNDATION CALL TRAINING: HEALTHBENCH USERS

During this call, a Pascal Metrics Client Services Partner will review a This training takes a “train the trainer” approach to teaching
recommended process for utilizing survey results to initiate HealthBench users how to download reports and use the views and tools
improvement.! available for reviewing survey data.!
! !
Client! Client!
• Identify leaders and supporters for survey evaluation and improvement to participate! • Coordinate date for HealthBench User Training with Pascal Metrics !
• Coordinate a date for the Improvement Foundation Call with Pascal Metrics ! • Identify and invite individuals who you’ve granted access to the survey results to
! participate !
Pascal Metrics! !
• Coordinate a date for the Improvement Foundation Call ! Pascal Metrics!
• Connect through the Improvement Foundation Call to discuss a recommended • Coordinate date for HealthBench User Training!
improvement process ! • Host the web-based training!
! • Answer any questions participants may have !
Outcome! !
• A guide for designing and executing an improvement process using survey data ! Outcome!
! • HealthBench users who are trained and ready to use the system to access and utilize
Timeframe: immediately following survey completion ! survey results !
Format: ½ hour call scheduled while the data is being processed by Pascal Metrics.!
!
!
Timeframe: within two weeks of survey completion!
Format: 1 hour webinar scheduled the day or week before survey data is available on HealthBench!
!
© Pascal Metrics 2016!

EXECUTIVE BRIEFING SESSION TRAINING: UNDERSTANDING DATA

This session engages organizational leaders in a conversation about the Training for managers and debriefing facilitators to understand how to
survey results, what they show, what they mean and how they can be used view and interpret data at the most granular level. (Part 1 of 3 part Using
to drive improvement.! Survey Data Series)!
! !
Client! Client!
• Survey Leader prepares to brief leadership using templates and recommendations • Coordinate Understanding Data Training with Pascal Metrics!
provided by Pascal Metrics! • Identify and invite managers and facilitators to attend !
• Engage leaders to participate in this review of survey results! !
• If Pascal Metrics-led session is included in SOW, invite leaders to the Leadership Briefing Pascal Metrics!
session hosted by Pascal Metrics ! • Coordinate date for Understanding Data Training !
! • Host the web-based training !
Pascal Metrics! !
• Provide recommended templates for reporting data back to leaders! Outcome!
• If Pascal Metrics-led session is included in SOW, ! • Attendees equipped with the tools to review and understand survey data as well as move
- coordinate date for the Leadership Briefing! forward with the improvement process !
- prepare an overview of survey results, highlighting common themes and opportunities ! !
! Timeframe: within 4 weeks of HealthBench User Training !
Outcome! Format: 1 hour webinar!
• Organizational leaders walk away with an understanding of key opportunities and some !
recommendations on how to create accountability for change, knowledge of the Pascal
Metrics recommended improvement process and buy-in from leadership for that process !
!
Timeframe: within 2 weeks of HealthBench UserTraining!
Format: 1 hour webinar !
!

REVIEW DATA TRAINING: LEADING A FEEDBACK SESSION

Once the survey results have been published, spend time reviewing them Provides background and guidance for Feedback Session Facilitators
at a local level looking for patterns and significance. ! regarding how to lead sessions to collect quality information and gain
! cultural insight. (Part 2 of the 3 part Using Survey Data Series)!
Client! !
• Managers and Facilitators review and understand data for their areas, identify key items of Client!
focus! • Coordinate Debriefing Survey Data Training date with Pascal Metrics!
• Survey Leader reviews data to identify themes and trends within their organization! • Identify and invite debriefing facilitators to attend!
! !
Pascal Metrics! Pascal Metrics!
• Provide support to HealthBench users as they access and review survey results ! • Coordinate date for Debriefing Survey Data Training !
! • Host the web-based training!
Outcome! !
• An understanding of the survey results and begin identifying opportunities for Outcome!
improvements ! • After this training, facilitators will be ready to begin debriefing survey data within
! respondent groups!
Timeframe: begin within 2 weeks of HealthBench User Training! !
!
Timeframe: within 8 weeks of HealthBench User Training !
Format: 1 hour webinar!

COMMUNICATE RESULTS COMMUNICATE FEEDBACK TO LEADERSHIP

Develop a plan to communicate survey results with all leaders and staff During this step, the Survey Leader evaluates themes from the feedback
who were engaged in the survey throughout the organization." sessions and identifies items to share with management."
" "
Client! Client!
• Survey Leader shares local results with Manager/ Manager accesses results on • Survey Leader identifies feedback to communicate to senior leadership "
HealthBench" "
"Time: within 2 weeks of HealthBench User Training" Pascal Metrics!
• Manager shares results with frontline staff, explains debriefing process and schedules • Provide guidance on identifying and communicating key themes to leadership"
meeting" "
"Time: within 4 weeks of HealthBench User Training" Outcome!
• Facilitator conducts feedback session and shares results with Survey Leader and Manager" • Common themes identified to begin action planning"
"Time: within 8-10 weeks of HealthBench User Training!
"
!
Timeframe: within 12 weeks of HealthBench User Training"
Pascal Metrics!
• Provide support and guidance for communicating survey results"
• Provide recommended training tools and templates for feedback sessions "
!
Outcome!
• Survey results are distributed and discussed throughout the organization "
"
Timeframe: linked to each step in the process"

IMPROVEMENT PLANNING TRAINING: ACTION PLANNING

Up to this point you’ve been reviewing and communicating your survey The third part of the 3 part Using Survey Data Series, this session is
results. Now it is time to plan how you will use them to improve your focused on learning how to utilize the survey data and feedback to
work environment. ! identify and create action plans.!
! !
Client! Client!
• Manager identifies actions to take based on input from feedback sessions and arranges • Work with feedback session facilitators and managers to identify the best timeframe in
action planning meeting! which to hold this session.!
!Time: within 12 weeks of HealthBench User Training! • Coordinate date of session with Pascal Metrics!
• Conduct action planning meeting to identify unit-level priorities and follow-up strategy! !
!Time: within 14 weeks of HealthBench User Training! Pascal Metrics!
• Manager checks in regularly with team on progress! • Coordinate date and topic of session!
! • Host session!
Pascal Metrics! !
• Provide guidance and tools to identify and set goals, assign ownership for tasks, as well as Outcome!
define timeframes and measures of success! • Additional training and input regarding the improvement process and steps to continue
! moving forward.!
Outcome!
!
• A plan of action based on a few measureable goals supported by a team of individuals Timeframe: within 14 weeks of HealthBench User Training!
committed to leading improvement! Format: 1 hour webinar!
!
Timeframe: linked to each step in the process!

Appendix C
DOMAIN SPECIFIC GUIDES
The Safety Attitudes Questionnaire (SAQ) and Survey on Patient Safety (SOPS) are validated
survey instruments that have been widely used in the healthcare industry. Each includes a number
of statements, which are referred to as survey “items”. Items are grouped into topics called
“Domains”. This appendix outlines the scoring for each of these tools in detail.
Safety Attitudes Questionnaire (SAQ)

My domain scores looks drastically lower than any of the items that make up the domain. How can
this be?
It is common for domain scores to be lower than the scores for items that make up the domain.
Indeed, it is difficult to achieve high domain scores because a domain score takes into account
people’s perceptions across all of the smaller concepts that make up the domain – for example for
teamwork climate, things like cooperation, feedback, conflict resolution. Often, people feel very
positively about some of these smaller concepts, but may feel very negatively about one or more.
Because the domain score looks across all concepts, it is a high bar to be “positive” on average
across all of these.
Often, an example using some sample data can help show how and why domain scores might be
lower than the scores for items that make up the domain. Consider the following sample data for a
5-item domain from a clinical area with 10 people:
Item 1 Item 2 Item 3 Item 4 Item 5

(neg)
Tom 4 3 4 4 3
Jim 3 4 5 5 4
Mary 4 2 4 5 4
Bill 4 2 3 3 4
Jen 3 1 5 5 3
Cliff 5 1 5 5 5
Ryan 5 3 3 4 4
Will 4 2 4 4 3
Gina 4 2 4 4 4
Julia 3 1 4 3 4
1 = “Strongly Disagree” 2 = “Disagree Slightly” 3 = “Neutral” 4 = “Agree Slightly” 5 = “Strongly Agree”
Calculating the domain score for these 5 items for this clinical area requires following a number of
steps. (Shortcut note: STEP 4 IS CRITICAL FOR UNDERSTANDING WHY DOMAIN SCORES
ARE LOWER THAN ITEM SCORES.)

MONITOR & EVALUATE PROGRESS PROGRESS REVIEW

During this step, the teams that convened to action plan will monitor This step is intended to close the feedback loop with the Survey Leader
progress toward the goals they set and re-evaluate the strategy as needed. ! on work being done by staff at the local level.!
! !
Client! Client!
• Team monitors progress based on identified metrics! • Manager facilitates unit-level progress review every 6 months!
• Manager and team re-evaluate plan of action and adjust as needed! !Time: within 6 months of action planning being initiated!
• Regular team check in with Manager - every week for first 4 weeks and monthly thereafter! • Manager communicates progress to Survey Leader!
(timeframe from start of action planning: weeks 1, 2, 3, 4, 8, 12, 16)! !Time: following Progress Review!
• Celebrate small successes ! !
! Pascal Metrics!
Pascal Metrics! • Provide guidance on facilitation of a progress review and communication tools!
• Provide tools to monitor progress toward goals! !
! Outcome!
Outcome! • Survey leader is aware of the goals that have been achieved and work that is still in
• Ongoing evaluation of progress toward goals and shared responsibility between the teams progress !
and management ! • Survey Leader connects with Pascal Metrics Client Services Partner to discuss
! improvement progress and plan to re-survey!
Timeframe: ongoing after action planning is complete! !
Timeframe: linked to each step in the process!

1. Reverse score any negatively worded items. A negatively worded item (e.g., “In this
clinical area it is difficult to speak up if I perceive a problem with patient care.”) is an item
for which a response of “Agree Slightly” or “Strongly Agree” is bad. So, we first flip the
scoring for any such items, making 5 = “Strongly Disagree”, 4 = “Disagree Slightly”, 3 =
“Neutral”, 2 = “Agree Slightly”, and 1 = “Strongly Agree”. Now, a high score means that
someone felt good or positively about the item. Completing this step alters our sample data,
changing Item 2 only:

(neg)
Tom 4 3 4 4 3
Jim 3 2 5 5 4
Mary 4 4 4 5 4
Bill 4 4 3 3 4
Jen 3 5 5 5 3
Cliff 5 5 5 5 5
Ryan 5 3 3 4 4
Will 4 4 4 4 3
Gina 4 4 4 4 4
Julia 3 5 4 3 4
2. Calculate the “Scale Score” as the mean across the items that compose the domain.
Consistent with standard psychometric practice, items are combined using the average
(specifically, the mean) across items. By taking the mean of the items, each item contributes
equally to the domain score and the domain score represents how, on average, someone felt
about this specific domain.
Item 1 Item 2 Item 3 Item 4 Item 5 Scale Score

(neg)
Tom 4 3 4 4 3 3.6
Jim 3 2 5 5 4 3.8
Mary 4 4 4 5 4 4.2
Bill 4 4 3 3 4 3.6
Jen 3 5 5 5 3 4.2
Cliff 5 5 5 5 5 5.0
Ryan 5 3 3 4 4 3.8
Will 4 4 4 4 3 3.8
Gina 4 4 4 4 4 4.0
Julia 3 5 4 3 4 3.8
3. Change the scores from a 1 to 5 scale to a 0 to 100 scale. Some research indicates that
practitioners prefer viewing scores on a 0 to 100 scale rather than a 1 to 5 scale. As such, one
can at this point transform the scores so that a 1 = 0, 2 = 25, 3 = 50, 4 = 75, 5 = 100. This does not

change the meaning of the score, it simply changes the anchors of the scale. To do so, you
can easily subtract 1 from each score and multiply by 25.

(neg)
Tom 75 50 75 75 50 65
Jim 50 25 100 100 75 70
Mary 75 75 75 100 75 80
Bill 75 75 50 50 75 65
Jen 50 100 100 100 50 80
Cliff 100 100 100 100 100 100
Ryan 100 50 50 75 75 70
Will 75 75 75 75 50 70
Gina 75 75 75 75 75 75
Julia 50 100 75 50 75 70
4. Aggregate data to the clinical area level. At this point, there are 5 item scores and a scale
score for each member of this clinical area.
a. To represent people’s perceptions across the entire work setting for item scores,
the “Agree” is calculated. This is the percent of people who marked “Agree Slightly”
or “Strongly Agree” on the survey form for the given item.
b. To represent people’s perceptions across the entire work setting for the domain
score, the “Positive” is calculated. KEY POINT: This is the percent of people who,
on average across items, marked a 4. That is, it is the percent of people who have a
scale score of 4.0 (or 75 on the 0 to 100 scale). This is the percent of people who on
average perceived the domain positively.
(NOTE: In the current example, since we rescaled items to 0 to 100 rather than 1 to 5, we count
below the number of people who have a score greater than or equal to 75. On the original 1 to 5
scale, we would count the number of people who have a score greater than or equal to 4.)
After we have this count, we divide it by the number of people in the area who responded to the
item. In the sample data, there were 10 responses for each survey item.

(neg)
Tom 75 50 75 75 50 65
Jim 50 25 100 100 75 70
Mary 75 75 75 100 75 80
Bill 75 75 50 50 75 65
Jen 50 100 100 100 50 80
Cliff 100 100 100 100 100 100
Ryan 100 50 50 75 75 70
Will 75 75 75 75 50 70
Gina 75 75 75 75 75 75
Julia 50 100 75 50 75 70

Summary
So, the reason your work setting’s domain score may be lower all of the item scores is that the scale
score for an individual can be brought below an average “Agree” response if that person marks one
or more other items as “Disagree” or even “Neutral”.
The domain score is a high bar – someone must feel positively across the items that go into the
domain.

Survey on Patient Safety (SOPS)
My domain scores on the Survey of Patient Safety Culture (SOPS) look higher than my domain
scores from the Safety Attitudes Questionnaire (SAQ). Why does this appear to be the case?
Scores for the SOPS domains are calculated using the SOPS algorithm, which is different than the
SAQ algorithm for calculating domain scores. The key difference between the two is that the SOPS
domain score represents the proportion of positive responses to total responses for a given domain.
The SAQ domain score, on the other hand, measures the proportion of respondents who on average
answered questions within a domain positively.
In order to go into more depth regarding their differences, let us consider a specific example. A
separate memo outlines the SAQ methodology for calculating domain scores. Let us consider the
same sample data for a 5-item domain from a clinical area with 10 people and apply the SOPS
methodology for calculating domain scores.

Tom 4 3 4 4 3
Jim 3 4 5 5 4
Mary 4 2 4 5 4
Bill 4 2 3 3 4
Jen 3 1 5 5 3
Cliff 5 1 5 5 5
Ryan 5 3 3 4 4
Will 4 2 4 4 3
Gina 4 2 4 4 4
Julia 3 1 4 3 4
1 = “Strongly Disagree” 2 = “Disagree” 3 = “Neutral” 4 = “Agree” 5 = “Strongly Agree”
Calculating the domain score for these 5 items for this clinical area requires following several steps:
1. Count the number of positive responses to each item in the dimension. “Agree” or
“Strongly Agree” are positive responses for positively worded items. For negatively
worded items, disagreement indicates a positive response, so count the number of
“Disagree” or “Strongly Disagree” responses:

Tom 4 3 4 4 3
Jim 3 4 5 5 4
Mary 4 2 4 5 4
Bill 4 2 3 3 4
Jen 3 1 5 5 3
Cliff 5 1 5 5 5
Ryan 5 3 3 4 4
Will 4 2 4 4 3
Gina 4 2 4 4 4
Julia 3 1 4 3 4
% of Positive 7 7 8 8 7
Responses
2. Count the total number of responses for the items in the dimension (this excludes
missing data).

Tom 4 3 4 4 3
Jim 3 4 5 5 4
Mary 4 2 4 5 4
Bill 4 2 3 3 4
Jen 3 1 5 5 3
Cliff 5 1 5 5 5
Ryan 5 3 3 4 4
Will 4 2 4 4 3
Gina 4 2 4 4 4
Julia 3 1 4 3 4
% of Positive 7 7 8 8 7
Responses
Total # of 10 10 10 10 10
Responses

3. Divide the number of positive responses to the items (answer from step 2) by the total
number of responses (answer from step 3).

Item 1 Item 2 Item 3 Item 4 Item 5 Total
Tom 4 3 4 4 3
Jim 3 4 5 5 4
Mary 4 2 4 5 4
Bill 4 2 3 3 4
Jen 3 1 5 5 3
Cliff 5 1 5 5 5
Ryan 5 3 3 4 4
Will 4 2 4 4 3
Gina 4 2 4 4 4
Julia 3 1 4 3 4
% of Positive 7 7 8 8 7 37
Responses
Total # of 10 10 10 10 10 50
Responses
Domain Score = Number of Positive Responses = 37 = 74%

Total Number of Responses 50
Summary:
Whereas the scale score for this domain utilizing the SAQ methodology was 40 , it is instead 74
when we utilize the SOPS methodology on the exact same data set. The key difference is that the
SAQ methodology requires that someone feel positively across all of the items that go into a
domain. The SOPS methodology, on the other hand, calculates the proportion of positive
responses to total responses for a given domain.

Appendix D
SURVEY LEADER CHECKLIST

Post Survey Communications
Immediately Following the Improvement Foundation Call

o Discuss with survey team and leadership the details of the Improvement Process
o Identify staff members to act as Feedback Session Facilitators
o Determine which staff members will be included in the HealthBench User Training
o Schedule HealthBench User Training
o Begin identifying dates for Executive Briefing and determine who will attend
Immediately Following HealthBench Training

o Confirm Executive Briefing Dates
o Schedule Understanding Data and Leading a Feedback Session training webinars
(required attendees: Work Setting Managers, Feedback Session Facilitators)
o Begin reviewing quantitative reports and qualitative comments (if available)
o Communicate the role of the Managers and Feedback Session Facilitators, and the
importance of their participation.
o Provide access to reports / copies of reports to Work Setting Managers / Feedback
Session Facilitators in preparation for the Understanding Data Training.
o Provide Work Setting Managers and Feedback Session Facilitators a timeline and
expectations for the Feedback Sessions, and determine check in points for accountability
Feedback Sessions
o Review Feedback Session notes and identify themes and information that should be
reported back to leadership
o Ensure Feedback Session notes are provided to the appropriate Work Setting Manager
o Use your previously decided upon method to ensure follow-up and improvement
planning by the Work Setting Manager
o Set the intervals and dates to review progress with the Work Setting Managers

Appendix E
KEY THEMES FROM FEEDBACK SESSIONS

Once you have received the notes from all of the Feedback Sessions, you will want to identify key
themes throughout the facility that leadership should be aware of, so they can remove barriers to
fixing issues.
At first, it may seem like an overwhelming task to identify issues from the Feedback Session notes
to bring to leadership’s attention. We recommend that you use a similar process that the individual
Work Setting Managers used – categorizing issues into groups. Below are some common themes
to consider:
Facility-wide Equipment Issues

When staff in many locations cites the same lack of common/shared equipment, such as
wheelchairs or infusion pumps, this is an issue that needs to be brought to the attention of Senior
Leadership; these are not problems that a Work Setting Manager can solve. There may also be
concerns about the maintenance, upkeep, or cleaning of shared equipment, and someone at the
leadership level will need to ensure that the issue is resolved.
Facility-wide Process Issues

Facility-wide process issues might include numerous work settings reporting problems receiving
timely lab results, or difficulties with delivery of medications or blood products. Sometimes a
process issue, like patient transport delays, may be related to an equipment issue (lack of
stretchers) or staffing issues which will require leadership support to resolve.
Local Management Issues

Feedback Sessions are likely to reveal a broad array of work setting management styles. Senior
Leadership should know about work settings with poor internal communication or concerns about
local leadership so that they can assist with education or mentoring.
Policy Misperceptions and Concerns

Concerns about facility-wide policies (vacation, medical benefits, sick leave, education, overtime,
etc) may come up during Feedback Sessions. This is important feedback for Senior Leadership and
Human Resources.
Human Resources Concerns and Issues

In addition to policy and procedural concerns, it may become apparent that bullying or disruptive
behavior is occurring in some areas. Even if this seems isolated, it needs to
be dealt with swiftly and decisively, and will likely require assistance from
Human Resources and Senior Leadership.
Senior Leadership Issues
Of course, Senior Leadership will be eager to hear how they are perceived across the facility! There
are few other ways that they are going to get this feedback, so it is important to be honest and
forthright about staff perceptions captured during the survey and through the Feedback Sessions
process.

Appendix F

Appendix G
PROVIDING FEEDBACK TO THE SURVEY LEADER:

WORK SETTING ACTIONS
A critical component of an Improvement Action Plan is communication. Once the issues from the
Qualitative Comments on the survey and notes from all of the Feedback Sessions have been
categorized (on the Improvement Action Tool), the Survey Leader should be made aware of the
items you’ve chosen to work on. Your facility’s Survey Leader plays a critical role in the
Improvement Process by tracking issues being addressed at the work setting level and identifying
common issues that are affecting multiple areas which may need their help to resolve. Below is a
template that may be a helpful way for you to share this information with your Survey Leader
including both what you are working on now and what you plan to address in the future.
If you foresee or run into any barriers or problems, this will also be a good opportunity to provide
that information to the Survey Leader who may be able to help remove any obstacles that would
prevent further action.
Work Setting: 3 South Date: April 1, 2015

Manager: Jane Smith
Issues Currently Being Addressed or In Process:

Issue being Addressed Status since [DATE of last Who is Addressing the Issue
report]
Blood pressure cuffs not Blood pressure cuffs on Manager

available in all rooms backorder – follow-up mid-
December
Breakroom is unorganized Manager
Workstations on Wheels Clinical IS investigating Manager
losing signal at far end of
hallway, affects med scanning
PCT handoffs poor New PCT-PCT shift handoff Improvement Team in the
tool in development Work Setting
Staff do not know one another Bulletin board in breakroom Improvement Team in the
well now includes pictures and Work Setting
names of all staff and an
icebreaker was introduced at
the last staff meeting
Rumors about staff cuts HR liaison is coming to next Human Resources
concerning staff meeting to address
rumors
Residents do not address Resident MDs are being Residency Director
nurses and techs by name encouraged to learn and use
names of nurses and techs to
improve
communication/morale
Challenges Encountered:
CIS not sure why signal is being lost – may require re-wiring/new router. Not in current budget.
Work around being implemented: Team to identify exact point at which signal becomes
interrupted and discuss an alternate location where the work station can be moved on the unit to
avoid the interruption.
Issues to be Addressed in the Future
Issue to be Addressed Responsible Party
Need new signage on unit doors to direct Manager

visitors appropriately
Need more prioritization of tasks by Nurses Improvement Team in the Work Setting
Transport assistance: 1) need more help getting Outside Assistance (Representatives of depts.
pts to Radiology for studies and 2) need that may be included: Transport, Operations,
advanced notice when they are coming to pick Radiology)
up a patient
Challenges Likely to Encounter:

Can’t order new signage until construction of new visitor area is completed next year
Issues Resolved from mm/dd/yy to mm/dd/yy:
Issue Addressed Outcome Staff feedback

Staff do not know one another Bulletin board and icebreaker Both solutions were simple,
well introduced and staff who did but they gave us opportunities
not previously interact are now to learn something about one
communicating another we didn’t previously
know

Improvement Process: © Pascal Metrics 2016

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Improvement Process: © Pascal Metrics 2016

Uploaded by

Copyright:

Available Formats

IMPROVEMENT PROCESS

© Pascal Metrics 2016

Discussing and Using Improvement Monitor and Evaluate

A detailed, recommended timeline can be found in Appendix B.

Section 1: Understanding Your Survey Data ...................................................................................................6

• 1.1 Purpose ...................................................................................................................................................................... 7

• 1.2 Basic Survey Information .............................................................................................................................. 7

• 1.3 Culture Survey Data Tip Sheet ................................................................................................................ 11

• 1.4 Conclusion ............................................................................................................................................................ 13

Section 2: Discussing and Using Survey Data .............................................................................................. 14

• 2.1 Using this Guide ................................................................................................................................................ 15

• 2.2 For the Survey Leader .................................................................................................................................... 17

• 2.3 For the Work Setting Manager ................................................................................................................ 21

• 2.4 For the Feedback Session Facilitator ...................................................................................................23

• 2.5 Frequently Asked Questions .....................................................................................................................28

Section 3: Improvement Planning .................................................................................................................... 31

• 3.1 Moving Forward with Improvement Planning ..............................................................................32

ROLES AND DEFINITIONS

• Attending HealthBench User Training

UNDERSTANDING YOUR SURVEY DATA

PURPOSE OF THIS SECTION

BASIC SURVEY INFORMATION

Example 1: Item Score Calculation - SAQ

1 = Strongly Disagree 2 = Disagree Slightly 3 = Neutral 4 = Agree Slightly 5 = Strongly Agree

Below is an illustration of how an item score is calculated:

Number of people who agreed slightly (4) or strongly agreed (5)

Example 2: Item Score Calculation - HSOPS

1 = Never 2 = Rarely 3 = Sometimes 4 = Most of the time 5 = Always

SAQ Domains AHRQ Domains

Frequently Asked Questions

What is a Percent Positive Score?

What is the difference between Coarse Position and Position Type?

CULTURE SURVEY DATA TIP SHEET

Comparing Survey Data Across Multiple Work Settings

DISCUSSING AND USING SURVEY DATA

USING THIS GUIDE

Why Robust Communication of Survey Results is Critical

An outline of the flow of communication is included in Figure 1.

Figure 1: Improvement Process

Quantitative vs Qualitative Data

FOR THE SURVEY LEADER

Sharing Survey Data With Leadership

Recommended domains for comparison:

Sharing Survey Data With Work Setting Managers

Qualitative Comments and Feedback Sessions

Dealing With Negative Comments About Management

What Happens During Feedback Sessions?

During a Feedback Session, a neutral facilitator:

FOR THE WORK SETTING MANAGER

Reviewing Your Survey Results

Sharing Data With Your Work Setting’s Staff

Work Setting Manager’s Role in the Improvement Process

Two Steps in the Process

Why a two-step process?

Why Am I Not Facilitating the Feedback Session?

FOR THE FEEDBACK SESSION FACILITATOR

The Feedback and Improvement Process

How to Conduct a Successful Feedback Session

Important Things to Note in Your Feedback Session Write-up

1. Time and date of Feedback Session.

Frequently Asked Questions for Facilitators

What if no one talks during the session?

What do I do if I learn about an unreported serious safety event during a session?