Introduction
Quality improvement is a formal approach to the analysis of performance and systematic efforts to improve it. Documenting quality improvement (QI) projects involves preparing written descriptions of purpose, methods, results, and next steps. Individuals who lead QI projects should consider how this information would be useful to others if shared locally, shared with others elsewhere, or might become a scholarly contribution to improving care.
The information below will help leaders of QI projects develop project reports, presentations (including posters), and manuscripts for publication. The information is organized into the sections outlined below. Consider scanning the content to understand what is covered, then as you need more specific information return to relevant sections and go through them in detail.
Several general considerations are involved in deciding what to report about a QI project. Some important ones are:
a. General expectations for health science presentations and publications. The development of reports, presentations, and scholarly manuscripts involves a wide range of factual knowledge and technical skills. Most leaders of QI projects at an academic institution are aware of much of that knowledge and have many of those skills, but they "don't know what they don't know."
For a broad overview of considerations, review the table of contents of a general reference. For example, SEE THE DETAILED TABLE OF CONTENTS (8 pages!) for Lang, TA. How to Write, Publish, & Present in the Health Sciences. Philadelphia, PA: American College of Physicians, 2010.
b. Specific expectations for QI publications (SQUIRE 2.0 guidelines). The Standards for Quality Improvement Reporting Excellence (“SQUIRE 2.0 guidelines”) outline the recommended content for publications describing quality improvement efforts (see http://www.squire-statement.org/guidelines). Several journals use these guidelines in assessing QI manuscripts. The guidelines outline and explain 18 elements of information to include in a scholarly manuscript about a QI initiative (Table 1):
Title and Abstract
1. Title
Introduction 3. Problem description Methods 7. Context |
Results
13. Results Discussion 14. Summary Other Information 18. Funding |
c. Consider your audience and their interests. Who is your audience and what interests them? Most audiences are not fundamentally interested in what you did. They are primarily interested in implications for their specific issues/concerns. Understand your audience’s interests and focus on information that addresses their interests. Accept that many of the things that were interesting to you personally may not be as interesting to your audience and therefore not included in your communications.
To whom do you want to describe the QI project and what are their interests? The typical audiences are summarized in Table 2 along with a communication’s usual relevant purpose, format, and content. More detail is provided below concerning reports for local leaders, presentations to colleagues outside the institution, and publication/scholarly contributions for external audience.
Audience |
Purpose |
Format |
Content |
Local clinical leaders |
Communicate activities and impact in context of local clinical priorities
|
Administrative report (typically 1-2 pages) |
Written summary highlighting purpose, rationale, interventions, results, and next steps |
Colleagues with similar problem at other institutions |
Share what was done so others can identify aspects they want to implement |
Poster or verbal presentation (limited space/time to communicate)
|
Highlights of urpose, rationale, methods including interventions, results, limitations, and recommendations |
Others with similar problem interested in what is unique about the project, how the methods and results can be adapted to improve care elsewhere and used to develop future innovations. |
Scholarly sharing with others: what was done and how it advances the field |
Manuscript for journal article |
Description of current evidence concerning the problem and how it has been previously addressed, rationale for study, methods including interventions, results, limitations, implications for practice and for research. |
When preparing a report for local leaders, consider the following.
Focus and brevity. A one or two-page summary plus a data table or figure often provides sufficient information for local clinical leaders to understand key activities and develop broader plans and priorities for future local QI initiatives. More detailed information may be placed in appendices.
Content. In general, the report will describe the problem and the sequence of a data-guided cycle of improvement: purpose/importance, current state and problem (process and performance), goals underlying causes, countermeasures, implementation, results, and conclusions/next steps. (Baseline and post-intervention results may be presented in one or two tables or figures.) .
Statistical tests may be optional. Formal statistical tests of the probability that results occurred by chance are helpful. However, they likely will not be needed if results demonstrate practically meaningful change sufficient for administrative decisions. (Leaders do not have evidence of > 95% probability for most decisions.)
Recommendations. Recommendations for action should be clear, including how the change will be sustained, remaining causes that should be addressed next, and where similar problems exist in the organization that could benefit through “spreading” the activities in this project.
Example. Click here to see one example of a report of a QI project that was designed for clinical leaders. It is a one-page executive summary plus a figure showing the main results. The full report that presents step-by-step activities and results in more detail is appended if a leader is interested in more information. The executive summary is organized by several headings commonly used in reporting research results, headings that clinical leaders are likely to easily recognize and follow.
When making a poster or short verbal presentation, consider the following.
Brevity and highlights. The constraints of poster space and of time for verbal presentations limit the content to highlights of important information and activities.
Audience and their practice context. The audience of colleagues will typically include people practicing in other settings and institutions, so throughout the presentation consider what should be addressed regarding differences between the local practice setting and typical practice settings of others.
Content. The main content will describe the problem and the sequence of a data-guided cycle of improvement: purpose/importance, current state and problem (process and performance measures), goals underlying causes, countermeasures, implementation, results, and conclusions/next steps. Alternatively, the information may be grouped under headings more frequently used for a research report: introduction, methods, results, discussion - see Table 1. (Baseline and post-intervention results may be presented in one or two tables or figures.)
A possible addition is a few key referens to previous work relevant to the problem, the methods, and the results.
Statistical tests. Formal statistical tests are often performed to demonstrate that results had a very low probability of occurring by chance. However, formal statistics may be less relevant for pilot studies that do not have enough subjects/observations —“power”— to determine the probability is < .05 for change of the expected magnitude.)
Limitations. A few key limitations of the project should be noted in the discussion, including important methodological limitations and factors in the local setting (providers, patients, operational context) that may enhance or limit the generalizability of the findings to other settings.
Implications/recommendations. Based on the study results, include recommendations for changes to practice in settings in which the audience provides care. Implications for future studies may also be relevant.
In addition to the Squire 2 guidelines (above), a helpful resource is “Wong BM, Sullivan GM. How to write up your quality improvement initiatives for publication, Journal of Graduate Medical Education, 2016, 8(2):128-133” available here.
When preparing a manuscript reporting a QI activity as a scholarly contribution, in addition to information in the above resources, pay special attention to the following.
Journal specifications. Review the journal’s information for authors, including the readership, types of manuscripts accepted, and manuscript format and length.
Readers. Who are the likely readers - the audience - and what is important to them? Readers are fundamentally interested in learning from you what evidence-based changes they can implement to improve their practice and research settings.
State of the field. Is this an advance or innovation of interest to others? Or does it simply replicate improvement activities already shared?
Content. Follow the SQUIRE 2 guidelines for the content of publications of QI (see Table 1). This is a formal and rigorous list of information to include. The introduction explains why the quality improvement initiative is of interest to others and an advance to the field. The methods (e.g., important local contextual factors, meaningful differences, and sample size/power to detect them) should demonstrate the internal validity of what was performed. The results should clarify impact. The discussion should explain the advance to current knowledge, limitations to generalizing the findings, and implications for others (practitioners and researchers).
Length. Whatever length is allowed, you know enough about your project to write more. Keep in mind what is most important to individuals in the audience and how much information is adequate.
(Lark Speyer, Exempt/Non-Regulated Coordinator for IRBMED helped write this section.)
Individuals performing innovations and improvements in clinical care should understand how Institutional Review Boards (IRBs) differentiate non-regulated quality improvement (QI) and regulated research. Three key points are:
“Research” or “quality improvement?” IRB approval for involving human subjects is not needed to perform QI activities based on existing knowledge or to present or publish QI results internally or externally. However, federal regulations define “QI activities” more narrowly than the term QI is sometimes used. You should understand those technical definitions and be sure your project does not include sufficient “research” components that IRB approval is needed before you initiate the project.
Formal documentation that a QI activity is not regulated. Journals sometimes ask for formal documentation from an IRB that a QI activity described in a manuscript did not require IRB review. If you expect to publish, consider likely journals and their requirements regarding documentation that a QI project is not regulated. If documentation of “not regulated” status may eventually be needed, request a determination letter from the IRB prior to initiating the project. The early request assures “no surprises,” which may occur if you conduct the project without confirmation from the IRB.
HIPAA. HIPAA requirements regarding patient privacy apply whether an activity is research or quality improvement.
Each of these topics is addressed below.
“Research” or “quality improvement?” “Research” must meet several federal regulations for human subjects involvement, which IRBs assure. “QI” activities are “non-regulated” and do not require IRB oversight. However, federal regulations make technical distinctions between “research” and “quality improvement” that are not always reflected in common uses of these terms. In summary, federal definitions are:
Research is designed “to test a hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge (expressed, for example, in theories, principles, and statements of relationships).” Some features likely to characterize research are: the purpose of testing issues beyond current science and experience, randomization of subjects, fixed protocols, and researchers with no ongoing commitment to improvement of the local care situation.
Quality improvement involves interventions designed solely to enhance the well-being of patients and have a reasonable expectation of success. Some features likely to characterize QI are: applying known science to improve care, no unusual grouping of patients, ongoing adjustments/improvements to local activities, and personnel involved in ongoing improvement in the local situation.
The intent to publish (share descriptions/results with interested others) is not a criterion for determining whether an activity is “research” or “QI.” Descriptions of local QI activities can be published as examples of what worked in one local context. Publications of research share generalized knowledge and recommendations regarding what should occur more broadly.
Two ways in which something referred to as “QI” in common language may better fit the definition of research are:
In both instances, the efforts go beyond the narrow focus on doing “local QI” to performing “research” about how to go about doing QI.
If you are unsure whether your QI activity involves elements of research, our local IRB (IRBMED) suggests using the following resources to guide your assessment of your activity:
IRBMED written guidance: https://az.research.umich.edu/medschool/guidance/quality-assurance-and-quality-improvement-qaqi-projects (The drop-down menus provide operational detail and local information.)
HHS QI activities FAQs: https://www.hhs.gov/ohrp/regulations-and-policy/guidance/faq/quality-improvement-activities/index.html (General national guidance)
QI or Research Worksheet: https://www.med.umich.edu/moc-qi/pdf/QI-Research_Worksheet.pdf (Functionally oriented descriptions distinguishing QI and Research.)
Contact IRBMED for guidance. Read the IRBMED information (see above) in advance so that you can explain your situation and questions in their terms.
You can also contact IRBMED for guidance. Read their information in advance so that you can explain your situation and questions in their terms.
Documentation that a QI activity is not regulated. Some journals may accept an author’s statement that the described QI activity required no IRB review, while other journals may want more formal documentation. Both informal and formal options exist for documenting that a QI activity requires no IRB review. Check journals to which you are likely to submit a manuscript to determine informal determination is adequate for formal determination is needed.
Informal -self-determination. Review the above information and links to confirm your understanding that it is “QI” and do the project without any eResearch application to IRBMED.
Formal - IRB determination. Fill out an abbreviated eResearch application, “Activities Not Regulated...” application type, submit it to IRBMED, and receive a formal “Not Reguated” determination. This often involves a clarification request from an IRB representative.
For further general information on filling out an eResearch application, see
https://az.research.umich.edu/medschool/guidance/eresearch-regulatory-management-errm (general research regulatory management for human subjects)
https://its.umich.edu/academics-research/research/eresearch/regulatory-management/support/pi-study-team (regulatory management support for principal investigators and study teams)
Health Insurance Portability and Accountability Act (HIPAA). HIPAA requirements concerning patient privacy are separate from the regulation of research. QI activities involving patient information must follow HIPAA requirements. External presentations and publications need to maintain the privacy of individual patients involved in the project.
Suggestions for the following common problems are provided below
a. What is an “advance in the field”? The maturity of a field affects what information is an advance. An area of study typically evolves through publications focusing on:
Review current literature regarding the content and process you are addressing. Determine the current state of knowledge in that area. What contribution will your manuscript make to what is currently known? Organize your manuscript around its significance to advancing what is known. Early in the evolution of a field, case reports may be adequate. After several case studies have been published, another “me too” case report makes little contribution to the field (although doing the replication locally may be relevant for local purposes and reports). As a field matures, increasing effort is involved in highlighting relevant past work and putting the results in context.
b. Identify your aspirations early.When developing your QI project, consider the audience(s) likely to be interested in the results and your personal interests and resources. What reports, presentations, or publications do you expect to prepare? Considering this early is important because it affects some aspects of how the project is carried out. For example, will formal statistical tests of probability be needed and are sample sizes appropriate to detect the expected amount of change? Deciding early is also important for determining the extent and type of records and documentation that will be relevant for the audience and type of communication.
Your plans regarding reports, presentations, and publications may change as the project evolves. However, adapting a plan is usually easier than having no plan. If you have no initial plan, at the end of the project you may have overlooked a crucial aspect or you may have to spend appreciable time and resources recovering information.
c. Identify the needed skills. Identifying your aspirations early will also help you identify relevant skills that may be needed to carry out the project or to develop appropriate communications. The project lead may have ready access to skills needed to prepare a simple local report. More complex communications may involve assistance with literature searches, developing measures, statistical analysis, graphic design, or technical writing. Plan ahead to assure that needed skills will be available.
d. Selecting venues to submit presentation proposals and manuscripts. Project leaders are already likely to be aware of local institutional leaders who would be interested in a report on the project and its results.
Selecting venues to submit proposals for presentations (verbal or poster) and manuscripts for publication is often more difficult. The large number of regional and national meetings and the variety of journals offer many options.
One approach to narrowing the options is deciding whether your project's strength is in:
Innovations frequently apply existing improvement processes to new clinical contexts. This type of advance increases the likelihood of acceptance in meetings and journals related to the new relevant clinical area. However, improvements addressing clinical areas that are national priorities (e.g., value-based performance measures, Joint Commission core measures) may be of interest both to clinical specialists engaged those areas and to QI professionals. Meetings and journals of either group may be considered
Project leaders are likely to know from their own clinical backgrounds the meetings and journals most relevant to the clinical area addressed by the project. Project leaders are less likely to be aware of journals focusing on improvement in clinical care and patient safety. For examples of these journals, see:
Although the lists overlap in including journals that publish on QI in general, they focus on different detailed content and vary in the level of information presented about QI.
If a journal rejects a manuscript, reconsider the selection of journals. Does a rejection letter indicate the manuscript is fatally flawed or is the emphasis on it not fitting the interests/priorities of the journal? If the issue is interest/priority, consider other journals for which the content is relevant and edit the manuscript to reflect their priorities. Published manuscripts may have been through 2 to 4 previous rejections, with the review process helping shape and match the content a relevant journal.
e. Communication skills: writing and presenting. No matter how important the project, if information about it is not communicated in ways that are clear and easy for the audience to understand, your communications will be ignored.
A few illustrative recommendations include:
The TABLE OF CONTENTS for How to Write, Publish, & Present in the Health Sciences details many more points. Recommendations range from using effective words, sentences, and paragraphs to choosing the layout, fonts, and color for posters and for slides.
f. Formative feedback from representative audience members. Authors who are creating communications are often unaware that they both know too much and know too little.
These problems can be identified only if project leads share drafts of communications with representative members of the intended audience for feedback.
g. Peer review and common errors by authors. You can greatly improve your likelihood of acceptance by understanding how the review process works, the “audience” of reviewers, and common errors in manuscripts.
A GOOD OVERVIEW OF THE PEER-REVIEW PROCESS is presented in Shea JA, Caelleigh AS, Pangaro L, Steinecke A. Review process and publication decision. Academic Medicine, 2001; 76:911-921.
Common REASONS THAT REVIEWERS REJECT AND ACCEPT MANUSCRIPTS are summarized in Bordage G. Reasons reviewers reject and accept manuscripts: The strengths and weaknesses in medical education reports. Academic Medicine, 2001; 76:889-896. The reasons apply to all types of manuscripts and include:
For manuscripts reporting QI projects, failure to include information relevant to elements of the SQUIRE guidelines increases the likelihood of rejection.
When considering comments from reviewers, if they misunderstood or overlooked information in your manuscript, the underlying cause is that you did not present the information clearly for the reader. For an elaboration of this point, see Eva KW. THE REVIEWER IS ALWAYS RIGHT: peer review of research in medical education. Medical Education, 2009; 43:2-4.
h. Identify resources for guidance. Suggestions from individuals who have developed successful communications often provide excellent advice and mentoring. In seeking guidance:
For reports to leadership, who understands the priorities of leaders and can provide models of formats for reports that the leaders are used to seeing?
For presentations (poster or verbal), who has had previous proposals accepted at the relevant meeting and what advice can they provide regarding increasing the likelihood of acceptance?
For publications, who has had manuscripts accepted by the targeted journal and what advice can they provide regarding the focus and presentation of your planned manuscript?
UMHS QI MOC Program personnel can provide lists of possibly relevant individuals:
Watch for more general resources for guidance, including:
Live sessions offered locally or at national meetings on developing more effective presentations and on how to write for publication.
Online sites providing guidance for publishing QI studies
Technical writers retained by some departments to assist with manuscript preparation.