Staggers Et Al 2002 A Delphi Study Informatics Competencies
Staggers Et Al 2002 A Delphi Study Informatics Competencies
e.g., systems analysis, or system selection techniques, are ciency in the use of computer hardware and software. Com-
outlined in only two publications (Staggers, 1994; Gassert puter skills, while not high level, are considered a compo-
& McDowell, 1995). However, the knowledge and skills nent of informatics competencies.
required by highly experienced and/or educated informat- Other elements in the framework include informatics
ics nurses (e.g., great depth of expertise), were absent. knowledge and skills, a set of competencies beyond just
Except for Staggers, authors did not address specific com- learning how to manipulate computer technology. As the
petencies across several levels of nurses. initial list of items were examined, they inductively sepa-
The available publications built less upon each other rated into the categories of informatics skills and knowl-
than was anticipated (Staggers, et al., 2001). For example, edge. Informatics knowledge is the theoretical and con-
the validated competencies from Armstrong (1986) and ceptual basis for the specialty, while informatics skills are
Bryson (1991) have been available for many years, but the use of methods, tools and techniques particular to
their adoption into curricula is not yet evident. More informatics. For example, informatics skills include tech-
important, these informatics competencies are seemingly niques and tools in systems analysis and project manage-
absent from competency determinations within work set- ment. Informatics knowledge includes familiarity with
tings. In summary, then, there is a critical need for a cur- nursing taxonomies and reasons for systems slowness. The
rent, research-based and specific list of informatics compe- larger construct of information management includes
tencies for nurses at various levels of practice. skills in cognitive information processing capabilities.
However, this study focuses upon informatics competen-
cies and the antecedent categories represented in the con-
Framework for the Study ceptual framework.
The authors developed a conceptual framework to guide the
study (Figure 1). The terms and concepts in the framework
were derived from a synthesis of nursing and informatics
Methods
sources. The skills and knowledge concepts were basic con- To validate the competency statements and level of perfor-
cepts mentioned in literature about competencies in nursing; mance, the study included five steps: (a) competencies were
these two concepts are foundational for competency devel- extracted from the literature, (b) unique competencies were
opment. The informatics terms evolved from categorizing listed, (c) competencies were leveled and expanded by an
the 1,159 database items extracted from the literature. To be expert panel, (d) pilot test was administered, and (e) three
consistent with the competency literature, the authors chose Delphi rounds were conducted. The initial competency
the term computer skills to represent basic functioning with development in steps (a) through (c) is described elsewhere
technology. This term corresponds to others used in the lit- (Staggers, et al., 2001). The levels of nurses are defined in
erature i.e., computer literacy or information technology Table 1. After step (c), the list of competencies included
skills. Computer skills is defined in this study as the profi- 304 statements in 39 categories (Table 2).
386 A Study to Determine Competencies Nursing Research November/December 2002 Vol 51, No 6
Beginning nurses (Level 1) have fundamental information management and computer technology
skills and use existing information systems and available information to manage their practice.
Experienced nurses (Level 2) have proficiency in their domain of interest (e.g., public health,
education, administration). These nurses are highly skilled in using information management and
computer technology skills to support their major area of practice. They see relationships among data
elements, and make judgments based on trends and patterns within these data. Experienced nurses
use current information systems but collaborate with the informatics nurse specialist to suggest
system improvements.
Informatics specialists (Level 3) are registered nurses prepared at least at the baccalaureate level
who possess additional knowledge and skills specific to information management and computer
technology. They focus on information needs for the practice of nursing, which includes education,
administration, research and clinical practice. Informatics specialists’ practices are built on the
integration and application of information science, computer science and nursing science. In their
practice, informatics specialists use the tools of critical thinking, process skills, data management
skills (includes identifying, acquiring, preserving, retrieving, aggregating, analyzing, and transmitting
data), systems development life cycle, and computer skills.
Informatics innovators (Level 4) are educationally prepared to conduct informatics research and to
generate informatics theory. These nurses lead the advancement of informatics practice and research
because they have a vision of what is possible, and a keen sense of timing to make things happen.
Innovators function with an ongoing healthy skepticism of existing data management practices and
are creative in developing solutions. Innovators possess a sophisticated level of understanding and
skills in information management and computer technology. They understand the interdependence of
systems, disciplines, and outcomes, and can finesse situations to maximize outcomes.
experience in NI practice, and (c) visibility within the spe- appropriate level data were given for all items within each
cialty through presentations, publications, or an officer in category. Seventy subjects (97%) responded to Round 2.
informatics organizations. Expert panel members suggested The third and final round of surveys was sent in Febru-
potential study subjects and supplied their e-mail addresses. ary 2001. The data presentation format was the same as
The final “Informatics Competencies for Nurses” ques- Round 2. Because the Round 3 questionnaire contained only
tionnaire for Round 1 of the Delphi study was 19 pages 33 items and was 6 pages long, the 70 subjects were given an
long and included 305 items in 39 subcategories within the option of receiving either an electronic or hardcopy version
3 broad categories of computer skills, informatics knowl- of the questionnaire. Most (65) subjects preferred an elec-
edge, and informatics skills. Because of the length of the tronic copy. Sixty-five (93%) respondents returned Round 3
questionnaire, 110 potential subjects were contacted by e- surveys; all but one returned the survey electronically. An
mail to determine their willingness to participate. In overall retention rate of 87% from the beginning to the end
December 1999, the questionnaire was mailed to the 82 of the Delphi study reflects the interest and enthusiasm of the
nursing informatics specialists who agreed to participate in Delphi study participants in identifying and differentiating
the study. After survey receipt, three subjects did not meet informatics competencies for nursing (Table 3).
the requirements for participation. Of 79 eligible subjects,
72 (91%) returned usable responses. Results of the Delphi Study
In July 2000, 72 respondents received Round 2 sur- By the end of Round 3, 281 items (92%) of the original
veys. Only the 88 items that failed to meet 80% consensus 305 competencies were accepted as valid competencies for
were included in the 9-page questionnaire. The question- both the importance of the item and the appropriateness of
naire contained three sections of data for each item: (a) the the level (Table 3). Table 4 lists the percent agreement by
total participant response (in percentages) by response cat- level of practitioner while Table 5 depicts the number of
egory for every item on Round 1, (b) that participant’s competencies validated by level of practice. The master list
individual responses for each item, and (c) space for the of validated competencies is available at http://www.
individual’s Round 2 responses. Both item importance and nurs.utah.edu/informatics/competencies.htm.
a
Percentages represent the % agreement within the round and are not cumulative across rounds.
388 A Study to Determine Competencies Nursing Research November/December 2002 Vol 51, No 6
Of the 24 competencies not validated, only 5 were Other items with comments had only 1 or 2 comments
rejected (Table 6), 2 competencies from the experienced each. In general, comments varied from “don’t think this is
nurse level and 3 from the informatics specialist level of a NI competency” or “only if in an administrative role,” to
practice. The competency “writes an original computer recommending wording changes, especially verb changes,
program and modifies it” was rejected early at Round 1. (e.g., circled “manage” and changed to “analyze”).
The other 4 competencies did not reach an 80% agreement Twenty-one (30%) participants made comments on
to discard them until Round 3. round 2. Only 2 items had 3 or more comments. About 20
Six competencies were determined to be valid compe- comments on this round occurred at the bottom of pages
tencies but no agreement was reached about their practice and were not related to specific competencies. Examples
level. Only 1 of these 6 competencies achieved the 80% include “remain concerned that there is no advanced nurs-
threshold of agreement that it was at the wrong level, but ing content here,” “many of the elements identified here
subsequently, respondents could not reach agreement on a are appropriate for selected jobs but are not core compe-
correct level (Table 7). No competencies changed levels of tencies for all nurses at level 3,” and “believe all of these
practice as a result of the Delphi study. are computer literacy and not informatics.”
Thirteen items did not reach agreement as valid com- There were 25 comments from 16 respondents (25%)
petencies. Interestingly, these fell into 2 distinct practice on round 3. General comments included:
levels: the beginning nurse and the informatics specialist.
All of these beginning nurse competencies came from the 1. Matching the appropriate level of research activity
computer skills category. For the informatics specialist and the informatics level. Specifically, that level 3
level, which clearly had the largest number of competen- nurses should apply and use research but that level
cies by level of practice, items that did not achieve consen- 4 nurses should generate research (3 comments).
sus came from all three major categories. 2. The perspective each individual used to respond to
Only the experienced nurse level achieved a 100% con- the survey (i.e., several individuals felt that responses
sensus on listed competencies by the end of Round 3 for changed as their jobs changed) (3 comments).
importance and level. The nurse innovator level reached 3. The informatics specialist nurses “modifying” soft-
100% agreement for item importance but 2 items ware, which should be left to programmers (2 com-
(“teaches informatics competencies required for specific ments).
role functions for the practicing nurse, the nurse adminis- 4. Basic computer skills such as spreadsheets and pre-
trator” and “evaluates applications supporting clinical sentation software being a pre-nursing (and pre-col-
care (including decision support), education, administra- lege) requirement (2 comments).
tion, and/or research”) did not achieve agreement about 5. The competency “evaluates network capacity” being
appropriate practice level. too technical, but that the knowledge component
On all three rounds, participants were given the oppor- should be included as a competency (2 comments).
tunity to write in comments or additional competencies. 6. A number of respondents commented favorably
The research team a priori set a criterion that 10% of the about receiving the survey via e-mail.
participants had to introduce the new competency or com-
ment on the same competency for those items to be
included in the next round. Fifty-four percent of the par-
Discussion
ticipants (39 individuals) made comments on Round 1. The high response rates of 91% in Round 1, 97% in
Comments involved 85 items (28% of the total items). Round 2 and 93% in Round 3 demonstrated the great level
Only 7 items had 3 or more comments and none met of interest and need for a comprehensive list of informatics
the minimum criteria of 7 comments. One item received 5 competencies. Despite the 19-pages in the initial question-
comments. However, this item had a printing error when naire, a majority of participants completed the form, and
sent out. Of the 7 items, 1 item, “Has the ability to inte- the study had a high response rate overall.
grate different applications or programs” was in the com- The vast majority (92%) of competencies and their
puter skills category and the other 6 were in the informat- associated levels were validated. Out of 305 competencies
ics skills category. and their associated levels, only 24 items were not vali-
Nursing Research November/December 2002 Vol 51, No 6 A Study to Determine Competencies 389
dated for both importance and level. Five additional items The IMIA provided broad guidelines such as knowledge
were deemed important competencies but consensus could about general characteristics of health information systems.
not be reached about an appropriate level. This study outlines the specifics of those characteristics.
Respondents commented during each round about There are limitations to this work. While the expert
whether computer skills should be considered informatics panel added competencies to the nurse innovator level, the
competencies. One respondent even crossed out all the Delphi participants added no new competencies. Thus, the
items related to computer skills during Round 1 and said current list may not be exhaustive. The research team set a
that none of these skills should be considered part of infor- threshold of 10% for the addition of a new competency.
matics competencies. In the literature, the terms computer Future research should state the required threshold on the
skills and computer literacy are ill defined and often used survey as well as provide a summarized overview of com-
interchangeably. As per the information management ments made regardless of the frequency of the comment.
framework used in this study, computer skills are one set Since the creation and validation of competencies is a
of tools within the larger category of informatics compe- time-consuming process, additional competencies should
tencies that are ultimately needed to manage information. be added to the master list. New competencies should also
Participants clearly did not view computer program- be added to the master list as the specialty evolves and as
ming as a required competency for informatics nurse spe- more nurses develop expertise in a specific subspecialty
cialists. The competency directly speaking to programming within informatics (e.g., database design).
(i.e., “writes an original computer program and modifies Competencies were validated only by informatics nurses
it”) was rejected in Round 1. Similar competencies such as and not by beginning and experienced nurses. The research
“identifies the more common programming languages in team specifically chose only informatics specialist or inno-
use today,” did not reach consensus after 3 rounds. Com- vator level nurses because non-informatics nurses may not
ments were made that these competencies applied to “pro- know the competencies required at their level of practice.
grammers” and not informatics nurse specialists. However, one participant in Round 2 commented that
The results of this Delphi study created a master list of he/she “remained concerned that there is no advanced nurs-
informatics competencies for nurses at four levels of prac- ing content.” Interestingly, the competency “uses applica-
tice. In the past, Armstrong (1986), Bryson (1991), and tions for diagnostic coding” was eliminated from the expe-
Staggers (1994) used research techniques to create lists of rienced nurse level on Round 3. Perhaps if advanced practice
competencies for nurses. Armstrong studied the “computer nurses (i.e., master’s prepared nurse practitioners, had par-
competence” needed for nursing practice and teaching in the ticipated) they would have retained the competency above
early-1980s, while Bryson created a list of skills needed for item and generated additional items for review.
computer training in baccalaureate programs. Staggers This study provides a current, research-based list of
(1994) created a list of skills and knowledge for nurses in the informatics competencies. Few competencies were elimi-
early 1990s. Despite numerous discussions about the need nated or failed to meet consensus and the resulting list spans
for informatics competencies, no research-based competen- many topics, showing the content diversity of informatics.
cies were created since the early 1990s. Therefore, this study The last research-based work in this area was in the
answers the need for valid informatics competencies, and it early 1990s, did not include advanced informatics content,
is the first study to span four levels of nurses, create compe- and it did not include four levels of nurses. Follow up to
tencies for both entry-level and experienced informatics this specific research should define the core competencies
nurse specialists, and examine the categories of computer for nurses by level of practice and job type and to create
skills, informatics knowledge and informatics skills. valid and reliable tools to evaluate informatics competency
The study results complement the work performed by levels. Other research could concentrate on explicating the
the International Medical Informatics Association (1999). full set of competencies needed within the framework cre-
390 A Study to Determine Competencies Nursing Research November/December 2002 Vol 51, No 6
ated for this study (i.e., adding the competencies for Grobe, S. J. (1989). Nursing informatics competencies. Methods of
human information processing to informatics competen- Information in Medicine, 28(4), 267-269.
cies). Then, the full suite of competencies would be expli- Hebert, M. (2000). A national educational strategy to develop
cated for nurses’ management of information. ▼ nursing informatics competencies. Canadian Journal of Nursing
Leadership, 13(2), 11-14.
International Medical Informatics Association. (1999). Recommen-
Accepted for publication August 18, 2002. dations of the International Medical Informatics Association
The authors thank the expert panel members and the experts par- (IMIA) on education in health and medical informatics: Interna-
ticipating as pilot subjects: Barbara Carty, EdD, RN; Sue Grobe, tional Medical Informatics Association. ttp://www.imia.org.
PhD, RN, FAAN; Mary McAlindon, PhD, RN; Ramona Nelson, Lewis, D., & Watson, J. E. (1997). Nursing faculty concerns
PhD, RN; Rita Snyder-Halpern, PhD, RN, C; and Cheryl B. regarding the adoption of computer technology. Computers in
Thompson, PhD, RN, and especially Melody Rydgren, MS, RN, Nursing, 15(2), 71-76.
whose participation in this work and obvious dedication to nurs-
ing informatics was remarkable. Peterson, H., & Gerdin-Jelger, V. (1988). Preparing nurses for using
information systems: Recommended informatics competencies.
The views expressed in this article are solely those of the authors New York: NLN Publications.
and not necessarily those of the Health Resources and Services
Administration, Department of Health and Human Services. Pew. (1998). Recreating health professional practice for a new cen-
tury: The fourth report of the Pew Health Professions Commis-
Corresponding author: Nancy Staggers, PhD, RN, FAAN, Infor- sion. San Francisco, CA: Pew Health Professions Commission.
mation Technology Services, University of Utah, 421 Wakara
Way, Suite 204, Salt Lake City, UT 84108 (e-mail: nancy.stag- Riley, J. B. (1996). Educational applications. In K. A. McCormick
[email protected]). (Ed.), Essentials of computers for nurses (2nd ed.). New York:
McGraw Hill.
Staggers, N., Gassert, C. A., & Curran, C. (2001). Informatics
References competencies for nurses at four levels of practice. Journal of
American Association of Colleges of Nursing. (1997). A vision of Nursing Education, 4(7), 303-313.
baccalaureate and graduate nursing education: The next Staggers, N., Gassert, C. A., & Skiba, D. J. (2000). Health pro-
decade. Washington, DC: Author. fessionals’ views of informatics education: findings from the
American Association of Colleges of Nursing. (1998). Essentials AMIA 1999 spring conference. Journal of the American Med-
of baccalaureate education for professional nursing practice. ical Informatics Association, 7(6), 550-558.
Washington, DC: Author. Staggers, N. (1994). The Staggers Nursing Computer Experience
Armstrong, M. L. (1986). Computer competence for nurse edu- Questionnaire. Applied Nursing Research, 7(2), 97-106.
cators. Image Journal of Nursing Scholarship, 18(4), 155-160. Travis, L., & Flatley Brennan, P. (1998). Information science for
Austin, S. I. (1999). Baccalaureate nursing faculty performance of the future: an innovative nursing informatics curriculum. Jour-
nursing computer literacy skills and curriculum integration of nal of Nursing Education, 37(4), 162-168.
these skills through teaching practice. Journal of Nursing Edu- Vanderbeek, J., & Beery, T. A. (1998). A blueprint for an under-
cation, 38(5), 261-266. graduate healthcare informatics course. Nurse Educator, 23(1),
Bryson, D. M. (1991). The computer-literate nurse. Computers in 15-19.
Nursing, 9(3), 100-107. Verhey, M. P. (1999). Information literacy in an undergraduate
Carter, B. E., & Axford, R. L. (1993). Assessment of computer nursing curriculum: Development, implementation, and evalu-
learning needs and priorities of registered nurses practicing in ation. Journal of Nursing Education, 38(5), 252-259.
hospitals. Computers in Nursing, 11(3), 122-126. Walker, P. H., & Walker, J. M. (1994). Informatics for nurse
Gassert, C. A. (1998). The challenge of meetings patients’ needs managers: integrating clinical expertise, business applica-
with a national nursing informatics agenda. Journal of the tions, and technology. Seminars in Nursing Management,
American Medical Informatics Association, 5(3), 263-268. 2(2), 63-71.
Gassert, C. A., & McDowell, D. (1995). Evaluating graduate and Walker, P. H., & Walker, J. M. (1995). Nursing informatics:
undergraduate nursing students’ computer skills to determine Opportunities for administrators, clinicians, educators, and
the need to continue teaching computer literacy. Medinfo, 8(Pt researchers. Journal of the American Psychiatric Nurses Asso-
2), 1370. ciation, 1(1), 22-29.