Shortly after sharing the Agile Guideline Development Framework on this blog, I attended the 2022 Guideline International Network meeting in Toronto. One of the conference themes was “equity”. The first plenary session featured speakers sharing how they’ve considered equity in recommendations for covid vaccines, building equity practices into the entire guideline development process, and finally, the importance of incorporating social determinants of health into guideline development.
The last speaker, Sir. Michael Marmot, reflected on his experience working with politicians in the UK to improve equity in their national healthcare system. He shared an anecdote of once being asked how to improve health equity when the government was focused on maximizing efficiency. His response to this question was simply, “You don’t.” In his view, it seemed, actions that maximize efficiency are mutually exclusive of efforts to improve equity.
As someone tasked with leading a guideline development program, I couldn’t shake the draining feeling that trying to improve an already-sisyphean task was at odds with, perhaps slightly anathema to, the values of equity.
During the Q&A session, I shared my concern with this apparent polarity between improving efficiency and equity. I asked the panel, each in leadership positions of diverse organizations, what type of capabilities guideline developing organizations should build to accomplish this feat, as impossible as it might appear to be.
“We need data to measure whether we’re being equitable.” replied Sir Marmot.
Reflecting on this insight after the meeting, I considered how Sir Marmot’s observation, that measurement precedes intentional improvement, applies to improving the efficiency of the guideline development process.
Unfortunately, it quickly occurred to me that I have no meaningful metrics for guideline efficiency. As I considered this reality, and thought of how best to move forward, I came to the realization that I was not yet able to describe what those metrics should be.
Defining “Efficiency”
Measuring efficiency can mean different things, depending on which lens we use to view our work. Therefore, managing efficiency can look different in practice, so we must carefully consider which “efficiency” we care about measuring.
When using the lens of the guideline development process, efficiency often refers to how long a particular step takes to complete. This perspective often focuses our efforts on improving a particular task, often through the use of new tools and/or by restructuring the development team. We often end up generating new lists of tasks and timelines that we believe will improve the efficiency of the development process, and carry on with the arduous task of completing that work before time runs out (e.g., our lit searches are outdated and the work has to be repeated!).
Figure 1 - Timelines & Task Lists are the most commonly used tools to track workflows.
However, when considering the entire work-flow at a programmatic level, the type of efficiency we’re concerned with is how well work continues to flow through the entire development process. Measuring efficiency in this way focuses improvement efforts on ensuring each change further optimizes the full development process; changes that improve a single step while leading to an overall reduction in efficiency are naturally excluded.
Using this lens, we can quantify the time required to move a PICO through a “cycle” of the process (i.e., turning a PICO question into a clinical practice recommendation) and begin evaluating the efficiency of that process. By considering variables such as project scope, team size, and methodologic complexity, we can begin generating estimates for potential projects.
Value Stream Mapping
Value stream mapping (VSM) is a technique used to visualize the complete sequence of activities that must be completed to deliver a customer request.(1) The resultant maps can also capture how information flows between activities, identify contingencies and alternative paths through the sequence, and express how quickly and smoothly work flows between activities.
In guideline development, the VSM techniques can be used to map the entire development process and modelize the resource requirements of a project (e.g., people, time). By capturing time requirements for each step in the development process, and assessing the quality of each steps’ deliverable outputs, we can begin identifying opportunities to improve the efficiency of the development process.
Figure 2 - Key details captured by Value Stream Mapping.
For guideline development organizations, applying the VSM techniques will most likely start with mapping the processes that are followed during guideline development. These processes will describe the activities that occur during each step of guideline development, according to the standards and policies of the organization.
Therefore, unlike the lists of standardized methodological practices, the resultant map of a VSM exercise will be unique to each organization. Furthermore, if an organization delivers a variety of products using differing development methodologies, the maps for each methodology will be unique.
Figure 3 - A sample mapping for a simple value stream.
As a highly complex, resource-intensive process, guideline development won’t improve in meaningful ways without clear and precise metrics. Guideline developing organizations can use tools like value stream mapping to begin mapping and measuring the efficiency of their own development processes. Finally, once the current development workflow is mapped, organizations can use the VSM techniques to ideate and design new workflows that improve the efficiency of work, and develop process-improvement plans to move towards that future-state design.
References:
Martin, K., Osterling, M. Value Stream Mapping: How to Visualize Work and Align Leadership for Organizational Transformation. McGraw Hill. 2013.
Opmerkingen