of Individual Attitudes Towards, and Known Organizational Barriers to, Operationalizing Living Guidelines
Summary:
During a webinar introducing Agile Guideline concepts and management framework(1), an informal survey of attendees was conducted. Fifteen of the twenty-one attendees responded to the survey; twelve respondents completed the survey. The overall results suggest that most guideline developers are not yet ready to adopt living guidelines due to significant uncertainty about the model, mismatched organizational management strategies, and other significant barriers including capacity, dissemination strategies and financial resources.
Purpose:
The primary purpose of this survey was as an audience engagement tool during a 20-minute introductory presentation to Agile Guideline concepts and the Agile Guideline Development Framework. In addition to increasing enjoyment of the webinar, the questions were meant to gauge audience "readiness" to accepting innovative ideas for developing living guidelines. The results of the survey are presented here for informational purposes...and because they're kinda interesting.
Survey Format:
The survey was conducted informally via a Zoom webinar, as audience-response questions interspersed within the first 15 minutes of a 20-minute presentation. Respondents were asked a total of 3 questions, each of which included 5 pre-selected answers: the first two questions were single code; the third question was multi-code (i.e., "select all that apply").
Respondents entered their answers into the Zoom chat feature, which was viewable by all participants. The responses were recorded automatically by the Zoom recording feature, and were downloaded after the webinar for analysis.
Analysis:
Simple pie chart analyses were conducted to identify trends among the survey responses. For the purposes of this analysis, responses that fell outside the pre-selected answers were coded to the most relevant answer. Results are presented below as simple pie charts, with the most relevant descriptive statistics summarized in narrative form.
Results:
Sample Characteristics:
The webinar was attended by 21 individuals. Most attendees (95%) were from the US, with one attendee (5%) from Saudi Arabia. (Note: the webinar was held at 12pm ET.)
Fifteen of the attendees responded to the 3-question survey; 12 respondents completed all 3 questions. Twelve of the respondents were female (80%); 3 respondents were male.
All survey respondents were US-based and remained on the webinar for an average of 58 minutes. Twelve respondents work for medical specialty societies; 2 are independent guideline developers; 1 respondent is employed by a large academic university/hospital system.
Analysis Results:
Fifteen respondents answered the question, “How do you feel about living guidelines?” (see Figure 1) Three respondents (20%) indicated they like living guidelines; 7 respondents (47%) indicated they were unsure how they felt about living guidelines; 4 respondents (27%) indicated living guidelines were worrisome; 1 respondent (7%) indicated they hated living guidelines. No respondent indicated a love for living guidelines.
Figure 1
Twelve respondents answered the question, “How does your organization define ‘workload’?” (See Figure 2) The majority of respondents, 10 (83%), indicated their organization defines workload according to the total number of guidelines being worked on, while 2 respondents (17%) indicated their organization defines workload according to the total number of PICOs being worked on.
Figure 2
Twelve respondents answered the question, “What do you see as the biggest challenges to operationalizing living guidelines?” (See Figure 3) This question allowed respondents to select as many answers as they felt applied, resulting in a total of 29 responses. The highest ranked challenge, capacity of staff & volunteers, was identified by 10 respondents (83%). Six respondents (50%) indicated that financial resources and dissemination strategies were significant challenges. Five respondents (42%) indicated that policy & process obstacles were a challenge, while only 2 respondents (17%) indicated that methodology challenges were a significant barrier.
Figure 3
Limitations:
The most significant limitation of these results is likely the they are a convenience sample, and therefore not necessarily representative of all US-based guideline developers in medical specialty societies. Most respondents were personally known to the webinar host through professional connections. Therefore, there is a slight risk this may have biased the responses. However, the familiarity between the respondents and the host could also have encouraged honest feedback, making these results more reliable when viewed according to their inherent limitations.
Additionally, the survey questions were informal and have not been validated. The third question, in particular, addresses complex issues that require more rigorous study to identify the relationships between these barriers, and which barriers are important to address first. As a result, these analyses are intended only as a pilot test to indicate future research directions.
Discussion:
The overall results of the survey indicate that most guideline developers are not yet ready to adopt a living guidelines model due to significant uncertainty about the model, mismatched management strategies, and significant organizational barriers including capacity, dissemination strategies and financial resources.
The audience responses indicating their uncertain feelings towards living guidelines suggests that most guideline developers are at the “precontemplation” stage of the Transtheoretical Model of change (2). A sizable minority of developers are progressing towards the “contemplation” stage -possibly even towards “determination” to produce living guidelines- while a larger minority of developers are likely to reject living guidelines due to negative feelings towards them.
Ideas presented within the Agile Guidelines Development Framework, such as Single PICO flow and matching workload with capacity, may enable guideline developing organizations to begin addressing the numerous challenges, both structural and attitudinal, that limit their capabilities to produce living guidelines.
Comentarios