van Velzen et al.

BMC Medical Informatics and

BMC Medical Informatics and Decision Making (2023) 23:279
https://doi.org/10.1186/s12911-023-02372-4 Decision Making

REVIEW Open Access

21st century (clinical) decision support

in nursing and allied healthcare. Developing
a learning health system: a reasoned design
of a theoretical framework
Mark van Velzen1,2*†, Helen I. de Graaf‑Waar1,2†, Tanja Ubert3, Robert F. van der Willigen3, Lotte Muilwijk1,3,
Maarten A. Schmitt1, Mark C. Scheper1,2,4 and Nico L. U. van Meeteren2,5

Abstract
In this paper, we present a framework for developing a Learning Health System (LHS) to provide means to a comput‑
erized clinical decision support system for allied healthcare and/or nursing professionals. LHSs are well suited to trans‑
form healthcare systems in a mission-oriented approach, and is being adopted by an increasing number of countries.
Our theoretical framework provides a blueprint for organizing such a transformation with help of evidence based
state of the art methodologies and techniques to eventually optimize personalized health and healthcare. Learning
via health information technologies using LHS enables users to learn both individually and collectively, and independ‑
ent of their location. These developments demand healthcare innovations beyond a disease focused orientation
since clinical decision making in allied healthcare and nursing is mainly based on aspects of individuals’ functioning,
wellbeing and (dis)abilities. Developing LHSs depends heavily on intertwined social and technological innovation,
and research and development. Crucial factors may be the transformation of the Internet of Things into the Internet
of FAIR data & services. However, Electronic Health Record (EHR) data is in up to 80% unstructured including free text
narratives and stored in various inaccessible data warehouses. Enabling the use of data as a driver for learning is chal‑
lenged by interoperability and reusability.
To address technical needs, key enabling technologies are suitable to convert relevant health data into machine
actionable data and to develop algorithms for computerized decision support. To enable data conversions, existing
classification and terminology systems serve as definition providers for natural language processing through (un)
supervised learning.
To facilitate clinical reasoning and personalized healthcare using LHSs, the development of personomics and func‑
tionomics are useful in allied healthcare and nursing. Developing these omics will be determined via text and data
mining. This will focus on the relationships between social, psychological, cultural, behavioral and economic determi‑
nants, and human functioning.

†
M. van Velzen & H.I. de Graaf- Waar are joint first author and contributed
equally to this work.
*Correspondence:
Mark van Velzen
[email protected]
Full list of author information is available at the end of the article

© The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit http://​creat​iveco​mmons.​org/​licen​ses/​by/4.​0/. The Creative Commons Public Domain Dedication waiver (http://​creat​iveco​
mmons.​org/​publi​cdoma​in/​zero/1.​0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 2 of 11

Furthermore, multiparty collaboration is crucial to develop LHSs, and man-machine interaction studies are required
to develop a functional design and prototype. During development, validation and maintenance of the LHS continu‑
ous attention for challenges like data-drift, ethical, technical and practical implementation difficulties is required.
Keywords Learning health system, Clinical decision support system, Experience based evidence, Allied healthcare,
Nursing, Functionomics, Personomics, Key enabling technologies, Key enabling methodologies

Introduction focus on the LHS, codesign, learning communities, eth-

Transforming health information technologies is critical ics, organization structures, patient outcomes, informa-
to safeguard and advance healthcare in a dynamic world. tion technology, security, science, data and performance
We describe our design for a learning health system [16–21]. Here we present a next stage theoretical frame-
(LHS) to aid decision-making in allied health care and work as a mission map and in conjunction with the FAIR
nursing. This article is to be viewed as the presentation principles, key enabling technologies and key enabling
of a basic theoretical framework that serves as a starting methodologies. In our framework we considered each
point of a program for the practical design, development of these components. In general the development of
and deployment of the LHS for health and healthcare LHS are rapidly evolving though adoption remains diffi-
and, in parallel, for the start of a dialogue amongst rel- cult [20]. Strong partnership between academic, citizens
evant stakeholders in order to strengthen the framework (patients and relatives), clinical, technical and as well as
during this program. We start by drawing attention to involving administrative stakeholders in codesign is pre-
the scale of the challenge before discussing the actual sented as an important success factor for adoption and
development. implementation of an LHS, whereafter development can
With the global challenges and their urgency of the start [16, 20–22]. On the other hand organizational cul-
United Nations Sustainable Development Goals [1] ture, adequate data systems and data sharing policies,
in mind, many countries are adopting mission-driven limited skilled persons, funding and competing priorities
approaches [2, 3]. Missions concerning transformative remain challenging [18] and, what is more, to be validated
actions depend on intertwined social and technological in the next steps of our program of design, development
innovation and research and development [4]. Trans- and deployment. Our LHS framework focusses on com-
forming from the Internet of Things (IoT) to the Internet puterized clinical decision support system (cCDSS)
of FAIR (Findable, Accessible, Interoperable, Reusable) for allied healthcare and/or nursing professionals. We
data & services (IoFAIRaS) is a key factor [5]. This trans- explain LHSs in more detail and their importance for
formation is supported by key enabling technologies [4, the usability of the transformation of health, and health-
6, 7] such as Life-Science Technologies, Security & Con- care professionals that are embedded in the health sys-
nectivity, Artificial Intelligence, and Foundation Models tem. We also highlight the challenges of using data and
[8] that were recently put forward. These technologies data-driven approaches in this context. These challenges
can be combined in with key enabling methodologies [9] might (partly) be overcome by using federated learning
like Critical Design, Fieldlabs and Learning Communi- data-principles [23–25]. This requires the “FAIRification”
ties, and Transition Design [4, 9]. of data as this is often inaccessible and unstructured data
The Dutch government introduced a mission-driven formats, like in EHR [26, 27].
approach in 2019 [10]. ‘Health and healthcare’ is one
of four nationwide transformative challenges, inspired
by five missions of the ministry of health [10–12]. The
Development of learning health system, social,
intended health and healthcare transformation acceler-
technological and scientific context
LHSs were introduced as a potential solution to support
ates by the IoFAIRaS-transformation [5] as one of the
health and healthcare users and professionals knowledge
technological ingredients [4, 7] and Fieldlabs i.e. LHSs
discovery through learning from clinical data [13, 14, 19].
[13, 14] as crucial social ingredient to improve personal-
The learning cycle (Fig. 1, section I) represents an itera-
ized health and healthcare [15].
tive process, that consists of several stages. First; improv-
We present a framework that schematically represents
ing users knowledge discovery [28], based on existing
the crucial reuse of health and healthcare data to develop
data (data to knowledge). For instance, by reflecting on
a Learning Health System (LHS). A theoretical frame-
the impact of care delivery or by giving insights in quality
work is deemed necessary to be designed, developed and
of care or cost-effectiveness. Second, learning from data
deployed a LHS in a solid and state of the art program
implies the option to utilize the data to improve indi-
[16]. Common components in LHS frameworks are the
viduals performance (knowledge to performance) [28],
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 3 of 11

Fig. 1 Learning cycle in a Learning Health System

organizations or systems. The third stage is when the communication portals [27]. EHR contain a variety of
improved performance generates new data (performance nomenclature and languages, abbreviations and defini-
to data) itself [28]. This accumulation of new data then tions and this occurs within and between individuals and
leads to new knowledge; as a gradual buildup of ‘experi- within and between health and healthcare disciplines [26,
ence based evidence’ [29–31]. LHS enables users to learn 33, 34]. Using structured (meta)data, standardized termi-
individually and collectively, by reflecting on their own nologies and classifications improves the interoperability
decisions and performances, and on top of this by reflect- and reusability of data. This extends the learning cycle by
ing on data gathered by others, independent of their using multiple data silos (Fig. 1, IIB) and consequently,
location. the global success of LHSs may depend heavily on FAIRi-
To optimize health-related decision making a number fication of health and health(care) related data (Fig. 1,
of factors are vital, Evidence should be available to the IIC). To be able to learn from data collected by others
right person, and in the right format, and through the [28], irrespective of location or profession and from mul-
correct channel (e.g. EHR), and at the right time in the tiple decentralized data-silos, data must be FAIR [35–37]
workflow [32] using routinely collected and research data preventing numerous amounts of health data exchange
[13, 14, 28]. Developing LHSs to optimize health related between research databases. It should contain not only
decision making is made harder due to issues of the acces- research and public data, but routinely collected health
sibility and interoperability of data held in so-called ‘data data as well [13, 14, 28, 38]. The reuse of health data, as
silos’ (Fig. 1, section II). Data in an EHR is considered as a the ultimate goal of FAIR, requires a set/system of agree-
single data silo, holding both structured and unstructured ments concerning: standardization of data, metadata,
data formats [26, 27, 33] including free text. This results unique identifiers, authentication & authorization, licens-
in locking in the data which restricts the potential learn- ing and key infrastructures [35–37].
ing cycle (Fig. 1, II A). A systematic review of systematic Developing LHSs for allied health care and nursing
reviews [27] found that EHRs data comprises up to 80% demands healthcare insights and innovations that go
as unstructured including free text narratives. Healthcare beyond a disease focused orientation [39]. Clinical rea-
registration has become more and more required from soning by these professionals, is driven by the apprecia-
clinical and legislation perspectives, and is also paral- tion of patient preferences [40, 41] and interrelationship
leled by an exponential increase in digital communica- between personal, psychological, social, and environ-
tion between patients and healthcare providers via online mental determinants [15] and their variability over time,
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 4 of 11

to understand the patients’ functioning and (dis)ability determinants, are available in structured format for sci-
[42, 43]. Besides, these determinants should be the focus entific research by Statistics Netherlands (CBS) [54] and
in shared decision making, as means to come to person- the National Institute for Public Health and the Envi-
alized healthcare [15, 44]. ronment (RIVM)[55]). In addition to linking public and
In the following section we present a framework to EHR data and transforming routinely collected data into
construct cCDSS in LHSs taking into account these machine actionable data, these procedures should also be
challenges. performed for empirical research data and wearable sen-
sor data. This IoFAIRaS-transformation, by applying the
Framework FAIR principles, maximizes the clinical meaningful reuse
Constructing computerized clinical decision support of health and healthcare data [35–37] in order to develop
in learning health systems multicenter multidisciplinary LHSs as represented in
The proposed framework (Fig. 2) uses the Cross Indus- Fig. 1C. Besides the reuse of research data, health data
try Standard Process for Data Mining (CRISP-DM) exchange acts [56] demands to put forward health data
Extension for Medical Domain [45], in every stage of the exchange between health information systems (HIS). The
development and research. The CRISP-DM is charac- Fast Healthcare Interoperability Resources (FHIR) is the
terized by its iterative nature, where the depth of details standard to put forward health data exchange between
of these processes described increases with every cycle HIS and could speed up the FAIRification of EHR data
[45]. Although multiple data mining models are avail- [38, 57, 58] and data from medical devices as well [59].
able, CRISP-DM is feasible and the most commonly used With huge amounts of unstructured data collected in
model in the medical domain [46]. Development is not EHRs [26], technical and, especially, semantic interoper-
a linear process, but for the sake of clarity in the con- ability remains challenging [60, 61]. Semantic interoper-
ceptual description, we present only basic information, ability, defined as the unambiguous representation of
divided into technological, healthcare, and research and clinical concepts [61], is complicated by heterogeneity
development aspects.1 In addition, we present in supple- of data quality and the recognition of concepts of con-
ment 1. in multiple steps (Fig. S1.) the detailed flow for cern in free text narratives suitable for allied healthcare
technical development. professionals and nurses [60]. To develop technical and
semantic interoperable data, all input data, including
Technologies free text narratives, must be mapped to existing ter-
Key enabling technologies (KET) [4, 6, 7] to address tech- minology or classification systems using named entity
nical needs are suitable to convert relevant health-related recognition (NER) [62]. Hereto, the International Clas-
data, from different sources, in machine actionable data sification of Function, Disabilities and Health (ICF) [43,
[35–37, 47] suitable for clinical meaningful exchange and 62], NANDA International classification of nursing diag-
federated learning [25]. noses (NANDA-I) [60], Nursing Outcome Classification
To develop machine actionable data, input data must be (NOC), SNOMED-CT [63–65] and International Clas-
transformed into FAIR data [35–37] (Fig. 2-I). Relevant sification of Diseases (ICD-11) [66] (Fig. 2-I) serve as
and useful input data is stored in different (in)accessible definition providers as these contain meaningful repre-
data silos like EHR systems, public databases, research sentations of clinical concepts for allied healthcare pro-
databases and wearables and sensors. Public databases fessionals and nurses.
contain potential useful data for clinical decision mak- When developing and maintaining a LHS with cCDSS,
ing on specific, e.g. environmental, determinants that according to data mining models, the data needs to be
are not documented in encounters with healthcare pro- prepared and modelled [45, 46]. Free text data must be
fessionals. For example, several studies have shown that validated, cleaned, repaired and abbreviations must be
environmental determinants are potentially relevant handled. Subsequently, both structured and unstruc-
determinants of health [39, 48–53]. Automatically link- tured EHR data can be extracted and processed using
ing public data to the EHR is preferable to expecting natural language processing techniques to map them to
healthcare professionals to gather this themselves (e.g. the classification terminologies [64]. Both unsupervised
In the Netherlands, public data containing clinical use- and supervised learning (i.e. machine learning or deep
ful information on social, environmental and economic learning) would be suitable for this (Fig. 2-I). The selec-
tion of techniques can be aided by Responsible Technol-
ogy frameworks like Fundamental Rights and Algorithms
1
Research data, developed ontologies and developed algorithms are consid- Impact Assessment [67].
ered as open science and therefor will be published in scientific literature, Respecting the FAIR principles and to prevent trans-
ontology databases like Bioportal, and all algorithms will be made available mission of huge amounts of data between silos, the data
to EHR providers, other researchers and developers.
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 5 of 11

Fig. 2 Development of Learning Health system; a mission map

van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 6 of 11

remains stored in a machine-readable format in its origi- and enables the transformation from a disease focused to
nal location [36, 37]. Using Federated learning or Multi a personalized approach.
Party Computation [23, 24] algorithms are sent to the Transforming the health and healthcare system, in
data without full access to these data (Fig. 2-I). Only the this case, by developing a LHS, requires not only key
results of processed algorithms are collected preserving enabling technologies (KET) but key enabling method-
the optimum data privacy [23–25]. ologies (KEM) as well. KET have been proven as interna-
tional concepts [7, 86], while KEM are limited to national
concepts and contain eight methodologies which are
Key premises in healthcare encounters currently further developed [4, 9]. A reflection on used
Some systematic reviews [68, 69] have assessed the bar- KEM will be performed in a later phase of this project.
riers and factors influencing the implementation of The presence of a LHS with cCDSS, is considered a cru-
cCDSS. The included studies were limited to technology, cial social ingredient to enable the fulfilment of the mis-
organization and healthcare provider perspectives. Using sions of the Dutch Ministry of Health to improve health
cCDSS affects the primary process of care and, more and healthcare quality by learning via clinical data. This
importantly impacts patients (Fig. 2-II) [70–73]. Recom- evolution affects not only healthcare encounters, but also
mendations generated by cCDSS aim to improve patient EHR developers and healthcare organizations [13, 14,
relevant outcomes and therefore facilitate evidence based 19]. All relevant stakeholders such as; patients, health-
practice when healthcare professionals discuss these rec- care professionals, data scientists, data engineers, EHR
ommendations with their patients [40, 70]. vendors and healthcare organizations must collaborate to
Research has shown that social, functional, environ- identify clinical and technical needs and barriers. Code-
mental and personal determinants for decision making sign is a crucial element in KEMs [4, 9] and is vital to
by allied healthcare professionals and nurses [60, 74] are develop a functional design followed by prototype of a
mostly recorded in the unstructured free text areas of LHS with cCDSS [16, 17, 20–22, 68, 69, 87].
EHRs [26]. Within clinical reasoning of allied healthcare
professionals and nurses, the ICF [43] and NANDA-I Future Research & Development
[60] are often used as theoretical knowledge based classi- Before deployment in clinical practice, several scientific
fications. These classifications contain social, functional, methods are executed to develop, test and maintain a
environmental and personal determinants as elements working LHS with cCDSS (Fig. 2- III). At each stage of
and can be combined with reasoning frameworks like development the data is trained and tested on independ-
the hypothesis-oriented algorithm for clinicians II [75, ent datasets until acceptable performance is achieved.
76], or the nursing process model [77]. While these clas- Processes are executed with historical data followed by
sifications are useful to describe, clinical concepts are not the validation of the results by healthcare professionals
widely implemented in EHR systems for documentation and patients before implementation in a real time EHR
[62]. environment. Research using text and data mining, e,g.
For data supported personalized healthcare and pre- natural language processing or deep learning, will be
cision medicine, development of new, or deployment of performed to determine the interactions between social,
existing ontologies are crucial as prerequisite for machine psychological, cultural, behavioral and economic deter-
readable data [15, 19, 73, 78, 79]. Personomics [15] and minants, and human functioning to develop personomics
functionomics [42, 80] (Fig. 2-II) in addition to biologi- and functionomics.
cal omics [81–83] (e.g. genomics, proteomics, metabo- Man-machine interaction studies are crucial to develop
lomics, etc.) may provide for this [15, 80, 84]. the functional design followed by the prototype of a
The variety and sequencing of omics is not fully devel- LHS with cCDSS [88–90]. Supervised learning will be
oped and does not cover all domains in health [15]. Inter- performed for prediction analyses using decision trees,
actions between social, psychological, cultural, behavioral regression analysis and neural networks as analytical
and economic factors affecting the patients’ health beliefs tools [83, 91–98]. This lays the framework to develop
and illness approach within the medical system are algorithms suitable for computerized decision support in
described as personomics [15]. Studying the complex a LHS. These algorithms, decision rules and the results
structure and associations in human functioning has of the man-machine interaction studies are stepping
been defined as human functionomics [42, 80]. Person- stones to develop the prototype. It is then essential to
omics and functionomics are suitable for the domain of assess how well the prototype performs before deploy-
allied healthcare and nursing, and assisting personalized ment in clinical research as this saves costs and time
healthcare provision by these professions [42, 78, 80, 85]. [89]. When testing a non-operational system, healthcare
This expands the body of knowledge for decision making, professionals enter clinical data into the prototype, test
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 7 of 11

the feasibility, and evaluate whether the cCDSS recom- time data are processed, giving rise to the (im)possibil-
mendation is consistent with their clinical expertise and ity of informed consent and so approval of medical eth-
scientific knowledge [99–101]. If the prototype performs ics committees is crucial.
acceptably, then an impact analysis of the system pre- Considering these needs and demands, the FAIRifica-
cedes implementation in clinical practice. Impact analysis tion of health and research data needs to be accelerated.
could be done using cluster randomized controlled trials In the era of smart devices and internet of things (IoT)
[102–105] or retrospective cohort, pre-post and prospec- data are a source of information [59] about context,
tive cohort designs, using a single or multicenter setting history, physiology, functioning and behavior. Consid-
[106]. These have been shown to be suitable to evaluate ering the potential to link data from EHRs, empirical
the impact of a cCDSS [102–106]. Multiple baseline stud- research, public data, smart devices and IoT, the inter-
ies or interrupted-time-series are also appropriate ways net of FAIR Data & Services facilitates the optimal use
to analyze the impact [107, 108]. of life science technologies and artificial intelligence as
key enabling technologies [5–7, 9, 35, 47].
Deliverables While there are many possible advantages, domain
If the processes we have described are followed then experts, developers and data scientists should be aware
EHR providers would be able to convert their data into of disadvantages. They need to consider aspects like data
structured and standardized data. This would make EHR drift and technical and practical implementation difficul-
data machine actionable so it can be reused for other ties [119]. First, to overcome these challenges, the data
purposes. This could be data extraction for quality indi- and processed algorithms need to be maintained and
cators, or computerized clinical decision support, as tested regularly [120–123]. Second, early multi-stake-
described in the literature [34, 109–113]. holder dialogue and collaboration in a learning commu-
nity [21] and continuing evaluation of our framework is
General considerations vital to successfully develop and deploy in clinical care
To achieve the health and healthcare transformation [114, 124, 125]. Third, data sovereignty versus data soli-
envisaged by the Dutch nationwide transformative chal- darity [126] will have to be studied. Fourth, beside code-
lenges we presented a framework to develop a cCDSS signing via learning communities educational institutes
as part of a LHS for allied healthcare and nursing. Mul- should considerably educate agile health professionals in
tiparty collaboration will be crucial to develop, validate an agile manner [127].
and maintain a working LHS [21, 114]. The proposed the- Patients, nurses and allied healthcare profession-
oretical framework can also serve as a key enabling meth- als could benefit greatly if we develop and implement
odology [9] to develop and deploy LHSs in other health learning health systems together. This would improve
and healthcare domains and thereafter to be extensively healthcare and the healthcare system. This roadmap
validated and adjusted where necessary. As so, this paper provides guidance on how we could achieve the Dutch
opens up dialogue amongst experts to strengthen our ini- and project missions of personalized healthcare via a
tial thoughts and that of others before and during devel- learning health system.
opment of this methodology. Artificial intelligence is a
key enabling technology [4, 6, 7] which will be used to Supplementary Information
The online version contains supplementary material available at https://​doi.​
develop algorithms for clinical decision support in daily org/​10.​1186/​s12911-​023-​02372-4.
practice. A working LHS with cCDSS could enable per-
sonalized healthcare by expanding the learning cycle. Additional file 1. Development of a Learning Health System; technical
The LHS follows the principles of evidence based prac- flow.
tice [40] to optimize safe and efficient healthcare provi-
sion (knowledge to performance), and enlarge experience Acknowledgements
based evidence (performance to data) [28–31]. We would also like to thank Dr. Adam Weir who was hired from personal fund‑
ings to provide feedback on the scientific English writing of the manuscript.
Reusing routinely collected health data could
(in accordance with Dutch Electronic Health Data Authors’ contributions
Exchange Act) [56] decrease administrative burden M. van Velzen: Literature review, conception of design, analysis and drafting
the manuscript. Shared first authorship with H.I. de Graaf-Waar.
and prevent harmful care [115, 116]. Access to empiri- H.I. de Graaf-Waar: Literature review, conception of design, analysis and draft‑
cal research data or routinely collected health data is ing the manuscript. Shared first authorship with M. van Velzen.
impeded by the European General Data Protection T. Ubert: Contribution to the conception of design, review manuscript.
R.F. van der Willigen: Contribution tot the technical parts of conception of
Regulation [117, 118]. The development and research design, review manuscript.
of LHSs faces the challenges of data privacy, informed L. Muilwijk Contribution to the conception of design, review manuscript.
consent and medical ethical approval. Historical or real M.A. Schmitt: Contribution to the conception of design, review manuscript.
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 8 of 11

M.C. Scheper: Conception of design and drafting the manuscript. 20Dut​ch%​20Mis​sion-​orien​ted%​20Top​sector%​20and%​20Inn​ovati​on%​

N.L.U. van Meeteren: Conception of design and drafting the manuscript. 20Pol​icy.​pdf.
12. Larrue P. The design and implementation of mission-oriented innova‑
Funding tion policies: a new systemic policy approach to address societal chal‑
Not applicable. lenges; 2021.
13. Etheredge LM. A rapid-learning health system. Health Aff (Millwood).
Availability of data and materials 2007;26(2):w107–18.
Not applicable. 14. Friedman CP, Wong AK, Blumenthal D. Achieving a nationwide learning
health system. Sci Transl Med. 2010;2(57):57cm29.
15. Ziegelstein RC. Personomics and precision medicine. Trans Am Clin
Declarations Climatol Assoc. 2017;128:160–8.
16. Anderson JL, Mugavero MJ, Ivankova NV, Reamey RA, Varley AL, Samuel
Ethics approval and consent to participate SE, Cherrington AL. Adapting an interdisciplinary learning health
Not applicable. system framework for academic health centers: a scoping review. Acad
Med. 2022;97(10):1564–72.
Consent for publication 17. Rosenthal GE, McClain DA, High KP, Easterling D, Sharkey A, Wagen‑
Not applicable. knecht LE, O’Byrne C, Woodside R, Houston TK. The academic learning
health system: a framework for integrating the multiple missions of
Competing interests academic medical centers. Acad Med. 2023;98(9):1002–7.
The authors declare no competing interests. 18. Taylor YJ, Kowalkowski M, Spencer MD, Evans SM, Hall MN, Rissmiller S,
Shrestha R, McWilliams A. Realizing a learning health system through
Author details process, rigor and culture change. Healthc (Amst). 2021;8(Suppl
1
Data Supported Healthcare: Data‑Science unit, Research Center Innovations 1):100478.
in care, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands. 19. Pomare C, Mahmoud Z, Vedovi A, Ellis LA, Knaggs G, Smith CL, Zurynski
2
Department of Anesthesiology, Erasmus Medical Center, Rotterdam, the Y, Braithwaite J. Learning health systems: a review of key topic areas and
Netherlands. 3 Institute for Communication, media and information Technol‑ bibliometric trends. Learn Health Syst. 2022;6(1):e10265.
ogy, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands. 20. Platt JE, Raj M, Wienroth M. An analysis of the learning health system
4
Allied Health professions, faculty of medicine and science, Macquarrie Univer‑ in its first decade in practice: scoping review. J Med Internet Res.
sity, Sydney, Australia. 5 Top Sector Life Sciences and Health (Health~Holland), 2020;22(3):e17026.
The Hague, the Netherlands. 21. Foley T, Vale L. A framework for understanding, designing, devel‑
oping and evaluating learning health systems. Learn Health Syst.
Received: 23 June 2023 Accepted: 9 November 2023 2023;7(1):e10315.
22. Reid RJ, Greene SM. Gathering speed and countering tensions in the
rapid learning health system. Learn Health Syst. 2023;7(3):e10358.
23. Bogetoft P, Christensen DL, Damgård I, Geisler M, Jakobsen T, Krøigaard
M, Nielsen JD, Nielsen JB, Nielsen K, Pagter J. Secure multiparty compu‑
References tation goes live. Cryptology EPrint Archive, Report 2008/068; 2008.
1. Transforming Our World, the 2030 Agenda for Sustainable Develop‑ 24. Chaum D, Crépeau C, Damgard I. Multiparty unconditionally secure
ment [https://​sdgs.​un.​org/​2030a​genda]. protocols. In: Proceedings of the twentieth annual ACM symposium
2. Mission-Oriented Innovation: Tackling society’s grand challenges on Theory of computing; 1988. p. 11–9. https://​doi.​org/​10.​1145/​62212.​
[https://​oecd-​opsi.​org/​proje​cts/​missi​on-​orien​ted-​innov​ation/]. 62214.
3. European Commission, Mazzucato M. Mission-oriented research & 25. Ghavamipour AR, Turkmen F, Jiang X. Privacy-preserving logistic regres‑
innovation in the European Union : a problem-solving approach to fuel sion with secret sharing. BMC Med Inform Decis Mak. 2022;22(1):89.
innovation-led growth. Publications Office; 2018. 26. Cannon J, Lucci S. Transcription and EHRs. Benefits of a blended
4. te Velde R, den Hertog P, Ysebaert W. Over KEMs, KETs en Maatschap‑ approach. J AHIMA. 2010;81(2):36–40.
pelijke Uitdagingen. Position paper over bruikbaarheid van het concept 27. Negro-Calduch E, Azzopardi-Muscat N, Krishnamurthy RS, Novillo-Ortiz
Key Enabling Methodologies (KEMs) als complement van Key Enabling D. Technological progress in electronic health record system optimiza‑
Technologies (KETs) voorhet programmeren van missie-georiënteerde tion: systematic review of systematic literature reviews. Int J Med
R&D programma’s. Dialogic, innovatie * interactieVrije Universiteit Brus‑ Inform. 2021;152:104507.
sel; 2019. 28. Friedman CP, Rubin JC, Sullivan KJ. Toward an information infrastructure
5. The Internet of FAIR Data & Services [https://​www.​go-​fair.​org/​resou​ for Global Health improvement. Yearb Med Inform. 2017;26(1):16–23.
rces/​inter​net-​fair-​data-​servi​ces/]. 29. Tian S, Yang W, Grange JML, Wang P, Huang W, Ye Z. Smart healthcare:
6. European Commission. Re-finding industry : defining innovation. Publi‑ making medical care more intelligent. Global Health J. 2019;3(3):62–5.
cations Office; 2018. 30. Thompson C. Clinical experience as evidence in evidence-based prac‑
7. Lyakh AV, Swain A. Modernization of industry based on key ena‑ tice. J Adv Nurs. 2003;43(3):230–7.
bling technologies: overview of foreign experience. Econ Indust. 31. Yang J, Xiao L, Li K. Modelling clinical experience data as an evidence
2019;3(87):34–58. for patient-oriented decision support. BMC Med Inform Decis Mak.
8. Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, Bern‑ 2020;20(Suppl 3):138.
stein MS, Bohg J, Bosselut A, Brunskill E: On the opportunities and risks 32. Osheroff JA: Improving Medication Use and Outcomes with Clinical
of foundation models. https://​arxiv.​org/​abs/​2108.​07258 2021. Decision Support:: A Step by Step Guide. In: 2009: HIMSS; 2009.
9. Alonso MB, van der Bijl-Brouwer M, Hekkert P, Hummels C, Kraal J, Krul 33. Doan S. CM, Phuong T.M.., Ohno-Machado L. : Natural language
K, Ludden G, van der Horst T, Rindertsma L, Rutten P. Sleutelmethod‑ processing in biomedicine; a unified system architecture overview. In:
ologieën (KEM’s) voor missiegedreven innovatie. 2020. https://​kems-​en.​ Clinical bioinformatics methods in molecular biology. Edited by R. T, vol.
click​nl.​nl/. 1168. New York, NY: Humana Press; 2014: 275–294.
10. Conway R, Clinton N, Bellinson R, Von Burgsdorff LK, Cooke A, Van 34. Meystre SM, Lovis C, Burkle T, Tognola G, Budrionis A, Lehmann CU.
Spronsen K, Groen H, Smith R, Cerezo F, Thompson C. Mission-oriented Clinical data reuse or secondary use: current status and potential future
innovation in action: 2021 casebook. 2022. https://​apo.​org.​au/​node/​ Progress. Yearb Med Inform. 2017;26(1):38–52.
316722. 35. Collins S, Genova F, Harrower N, Hodson S, Jones S, Laaksonen L,
11. Janssen M. Post-commencement analysis of the Dutch ‘Mission-ori‑ Mietchen D, Petrauskaitė R, Wittenburg P: Turning FAIR into reality: final
ented Topsector and Innovation Policy’strategy. 2020. https://​www.​uu.​ report and action plan from the European Commission expert group
nl/​sites/​defau​lt/​files/​Post-​comme​nceme​nt%​20ana​lysis%​20of%​20the%​
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 9 of 11

on FAIR data. In.: Luxembourg: Publications Office of the European 57. Lete SA, Cavero C, Lustrek M, Kyriazis D, Kiourtis A, Mantas J, Montandon
Union; 2018. L. Interoperability techniques in CrowdHEALTH project: the terminol‑
36. Mons B, Neylon C, Velterop J, Dumontier M, da Silva Santos LOB, ogy service. Acta Inform Med. 2019;27(5):355–61.
Wilkinson MD. Cloudy, increasingly FAIR; revisiting the FAIR data 58. Rosenau L, Majeed RW, Ingenerf J, Kiel A, Kroll B, Kohler T, Prokosch HU,
guiding principles for the European Open Science cloud. Inf Serv Use. Gruendner J. Generation of a fast healthcare interoperability resources
2017;37(1):49–56. (FHIR)-based ontology for federated feasibility queries in the context of
37. Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak COVID-19: feasibility study. JMIR Med Inform. 2022;10(4):e35789.
A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE, et al. The FAIR 59. Mavrogiorgou A, Kiourtis A, Perakis K, Pitsios S, Kyriazis D. IoT in health‑
guiding principles for scientific data management and stewardship. Sci care: achieving interoperability of High-quality data acquired by IoT
Data. 2016;3(1):160018. medical devices. Sensors (Basel). 2019;19(9):1978.
38. Vorisek CN, Lehne M, Klopfenstein SAI, Mayer PJ, Bartschke A, Haese T, 60. De Groot K, De Veer AJE, Paans W, Francke AL. Use of electronic health
Thun S. Fast healthcare interoperability resources (FHIR) for inter‑ records and standardized terminologies: a nationwide survey of nurs‑
operability in Health Research: systematic review. JMIR Med Inform. ing staff experiences. Int J Nurs Stud. 2020;104:103523.
2022;10(7):e35724. 61. Stellmach C, Muzoora MR, Thun S. Digitalization of health data: inter‑
39. Cieza A, Sabariego C, Bickenbach J, Chatterji S. Rethinking Disability. operability of the proposed European health data space. Stud Health
BMC Med. 2018;16(1):14. Technol Inform. 2022;298:132–6.
40. Sackett DL, Rosenberg WM, Gray MJ, Haynes BR, Richardson SW. 62. Maritz R, Aronsky D, Prodinger B. The international classification of
Evidence based medicine: what it is and what it isn’t. Br Med J. functioning, disability and health (ICF) in electronic health records. A
1996;312:71–2. systematic literature review. Appl Clin. Inform. 2017;8(3):964–80.
41. van der Sluis G, Jager J, Punt I, Goldbohm A, Meinders MJ, Bimmel R, 63. Cornet R, Van Eldik A, De Keizer N. Inventory of tools for Dutch clinical
van Meeteren NLU. Nijhuis-van Der Sanden MWG, Hoogeboom TJ: cur‑ language processing. Stud Health Technol Inform. 2012;180:245–9.
rent status and future prospects for shared decision making before and 64. Gaudet-Blavignac C, Foufi V, Wehrli E, Lovis C. Automatic annotation
after Total knee replacement surgery—a scoping review. Int J Environ of French medical narratives with SNOMED CT concepts. Stud Health
Res Public Health. 2021;18(2):668. Technol Inform. 2018;247:710–4.
42. Janssen ER, Punt IM, van Soest J, Heerkens Y, Stallinga HA, ten Napel H, 65. Minarro-Gimenez JA, Martinez-Costa C, Karlsson D, Schulz S, Goeg KR.
van Rhijn LW, Mons B, Dekker A, Willems PC, et al. Operationalizing and Qualitative analysis of manual annotations of clinical text with SNOMED
digitizing person-centered daily functioning: a case for ‘functionomics’. CT. PLoS One. 2018;13(12):e0209547.
The Optimal P. 139 66. Dorjbal D, Cieza A, Gmunder HP, Scheel-Sailer A, Stucki G, Ustun TB,
43. Heerkens YF, de Weerd M, Huber M, de Brouwer CPM, van der Veen Prodinger B. Strengthening quality of care through standardized report‑
S, Perenboom RJM, van Gool CH, Ten Napel H, van Bon-Martens M, ing based on the World Health Organization’s reference classifications.
Stallinga HA, et al. Reconsideration of the scheme of the international Int J Qual Health Care. 2016;28(5):626–33.
classification of functioning, disability and health: incentives from the 67. Gerards J, Schäfer MT, Muis I, Vankan A. Fundamental Rights and Algo‑
Netherlands for a global debate. Disabil Rehabil. 2018;40(5):603–11. rithms Impact Assessment (FRAIA). 2022. https://​dspace.​libra​r y.​uu.​nl/​
44. Hoffmann TC, Lewis J, Maher CG. Shared decision making should be an handle/​1874/​420552.
integral part of physiotherapy practice. Physiotherapy. 2020;107:43–9. 68. Kilsdonk E, Peute L, Jaspers M. Factors influencing implementation suc‑
45. Niaksu O. CRISP data mining methodology extension for medical cess of aGuideline-based clinical decision support systems: a systematic
domain. Baltic J Modern Computing. 2015;3:92–109. review and gaps analysis. System Rev Gaps Anal. 2017;98:56–64.
46. Martínez-Plumed F, Contreras-Ochando L, Ferri C, Hernández-Orallo 69. Westerbeek L, Ploegmakers KJ, de Bruijn GJ, Linn AJ, van Weert JCM,
J, Kull M, Lachiche N, Ramirez-Quintana MJ, Flach P. CRISP-DM twenty Daams JG, van der Velde N, van Weert HC, Abu-Hanna A, Medlock S.
years later: from data mining processes to data science trajectories. IEEE Barriers and facilitators influencing medication-related CDSS accept‑
Trans Knowl Data Eng. 2019;33(8):3048–61. ance according to clinicians: a systematic review. Int J Med Inform.
47. Mons B. FAIR science for social machines: Let’s share metadata 2021;152:104506.
Knowlets in the internet of FAIR data and services. Data Intelligence. 70. Bezemer T, de Groot MC, Blasse E, Ten Berg MJ, Kappen TH, Bredenoord
2019;1(1):22–42. AL, van Solinge WW, Hoefer IE, Haitjema S. A human(e) factor in clinical
48. Ellaway A, Benzeval M, Green M, Leyland A, Macintyre S. "getting sicker decision support systems. J Med Internet Res. 2019;21(3):e11732.
quicker": does living in a more deprived neighbourhood mean your 71. Pombo N, Araujo P, Viana J. Knowledge discovery in clinical decision
health deteriorates faster? Health Place. 2012;18(2):132–7. support systems for pain management: a systematic review. Artif Intell
49. Leidelmeijer K, Marlet G, Ponds R, Schulenberg R, Woerkens Cv. Leef‑ Med. 2014;60(1):1–11.
baarheidsbarometer 2.0: instrumentontwikkeling. 2014. https://​doc.​ 72. Richardson JE, Middleton B, Platt JE, Blumenfeld BH. Building and
leefb​aarom​eter.​nl/​resou​rces/​Leefb​aarom​eter+2.​0+​Instr​ument​ontwi​ maintaining trust in clinical decision support: recommendations
kkeli​ng.​pdf. from the patient-centered CDS learning network. Learn Health Syst.
50. Pickett KE, Pearl M. Multilevel analyses of neighbourhood socioeco‑ 2020;4(2):e10208.
nomic context and health outcomes: a critical review. J Epidemiol Com‑ 73. Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year
munity Health. 2001;55(2):111–22. retrospective and a 25 year vision. Yearb Med Inform. 2016;25:S103–16.
51. Riva M, Gauvin L, Barnett TA. Toward the next generation of research 74. Paans W, Sermeus W, Nieweg RM, van der Schans CP. Prevalence
into small area effects on health: a synthesis of multilevel investiga‑ of accurate nursing documentation in patient records. J Adv Nurs.
tions published since July 1998. J Epidemiol Community Health. 2010;66(11):2481–9.
2007;61(10):853–61. 75. Echternach JL, Rothstein JM. Hypothesis-oriented algorithms. Phys Ther.
52. Putrik P, de Vries NK, Mujakovic S, van Amelsvoort L, Kant I, Kunst 1989;69(7):559–64.
AE, van Oers H, Jansen M. Living environment matters: relationships 76. Thoomes EJ, Schmitt MS. Practical use of the HOAC II for clinical deci‑
between neighborhood characteristics and health of the residents in a sion making and subsequent therapeutic interventions in an elite ath‑
Dutch municipality. J Community Health. 2015;40(1):47–56. lete with low back pain. J Orthop Sports Phys Ther. 2011;41(2):108–17.
53. Vermeulen R, Schymanski EL, Barabasi AL, Miller GW. The exposome and 77. Muller-Staub M, de Graaf-Waar H, Paans W. An internationally
health: where chemistry meets biology. Science. 2020;367(6476):392–6. consented standard for nursing process-clinical decision sup‑
54. https://​www.​cbs.​nl/​nl-​nl/​onze-​diens​ten/​open-​data/​statl​ine-​als-​open-​ port Systems in Electronic Health Records. Comput Inform Nurs.
data. Accessed 30 May 2023. 2016;34(11):493–502.
55. https://​data.​rivm.​nl/​meta/​srv/​dut/​catal​og.​searc​h#/​home. Accessed 30 78. Haendel MA, Chute CG, Robinson PN. Classification, ontology, and
May 2023. precision medicine. N Engl J Med. 2018;379(15):1452–62.
56. Ark Tv: Wetsvoorstel elektronische gegevensuitwisseling in de zorg. In. 79. Dissanayake PI, Colicchio TK, Cimino JJ. Using clinical reasoning ontolo‑
Edited by Ministerie van Volksgezondheid WeS. Den Haag: Rijkshover‑ gies to make smarter clinical decision support systems: a systematic
heid; 2021. review and data synthesis. J Am Med Inform Assoc. 2020;27(1):159–74.
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 10 of 11

80. Stucki G. Olle hook lectureship 2015: the World Health Organization’s practice, usability, and reduce the risk of litigation. J Perianesth Nurs.
paradigm shift and implementation of the international classification 2022;37(6):778–80.
of functioning, disability and health in rehabilitation. J Rehabil Med. 102. Feldstein DA, Hess R, McGinn T, Mishuris RG, McCullagh L, Smith PD,
2016;48(6):486–93. Flynn M, Palmisano J, Doros G, Mann D. Design and implementation
81. Chuong KH, Mack DR, Stintzi A, O’Doherty KC. Human microbiome of electronic health record integrated clinical prediction rules (iCPR):
and learning healthcare systems: integrating research and precision a randomized trial in diverse primary care settings. Implement Sci.
medicine for inflammatory bowel disease. OMICS. 2018;22(2):119–26. 2017;12(1):37.
82. Hirsch BR, Abernethy AP. Leveraging informatics, mobile health 103. Figueiras A, Lopez-Vazquez P, Gonzalez-Gonzalez C, Vazquez-Lago JM,
technologies and biobanks to treat each patient right. Per Med. Pineiro-Lamas M, Lopez-Duran A, Sanchez C, Herdeiro MT, Zapata-
2012;9(8):849–57. Cachafeiro M, Group G. Impact of a multifaceted intervention to
83. McNutt TR, Benedict SH, Low DA, Moore K, Shpitser I, Jiang W, Laksh‑ improve antibiotic prescribing: a pragmatic cluster-randomised con‑
minarayanan P, Cheng Z, Han P, Hui X, et al. Using big data analytics trolled trial. Antimicrob Resist Infect Control. 2020;9(1):195.
to advance precision radiation oncology. Int J Radiat Oncol Biol Phys. 104. Murphy ME, McSharry J, Byrne M, Boland F, Corrigan D, Gillespie P,
2018;101(2):285–91. Fahey T, Smith SM. Supporting care for suboptimally controlled type
84. Johnson KB, Wei WQ, Weeraratne D, Frisse ME, Misulis K, Rhee K, Zhao 2 diabetes mellitus in general practice with a clinical decision support
J, Snowdon JL. Precision medicine, AI, and the future of personalized system: a mixed methods pilot cluster randomised trial. BMJ Open.
health care. Clin Transl Sci. 2021;14(1):86–93. 2020;10(2):e032594.
85. Ottes L. In: Regeringsbeleid WR, editor. Big Data in de zorg, vol. 19. Den 105. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting adher‑
Haag: Wetenschappelijke Raad voor het Regeringsbeleid; 2016. p. 5–73. ence to influenza vaccination recommendations in pediatric practice. J
86. European Commission: a European strategy for key enabling technolo‑ Prim Care Community Health. 2019;10:2150132719853061.
gies – a bridge to growth and jobs. Brussels. 2012. https://​eur-​lex.​ 106. Klarenbeek SE, Weekenstroo HHA, Sedelaar JPM, Futterer JJ, Prokop M,
europa.​eu/​LexUr​iServ/​LexUr​iServ.​do?​uri=​COM:​2012:​0341:​FIN:​EN:​PDF. Tummers M. The effect of higher level computerized clinical decision
87. Kaddoura T, Vadlamudi K, Kumar S, Bobhate P, Guo L, Jain S, Elgendi support systems on oncology care: a systematic review. Cancers (Basel).
M, Coe JY, Kim D, Taylor D, et al. Acoustic diagnosis of pulmonary 2020;12(4):1032.
hypertension: automated speech- recognition-inspired classification 107. Holland WC, Nath B, Li F, Maciejewski K, Paek H, Dziura J, Rajeevan H,
algorithm outperforms physicians. Sci Rep. 2016;6:33182. Lu CC, Katsovich L, D’Onofrio G, et al. Interrupted time series of user-
88. Forrest CB, Margolis P, Seid M, Colletti RB. PEDSnet: how a prototype centered clinical decision support implementation for emergency
pediatric learning health system is being expanded into a national department-initiated buprenorphine for opioid use disorder. Acad
network. Health Aff (Millwood). 2014;33(7):1171–7. Emerg Med. 2020;27(8):753–63.
89. Pflanzl-Knizacek L, Bergmoser K, Mattersdorfer K, Schilcher G, 108. Tao L, Zhang C, Zeng L, Zhu S, Li N, Li W, Zhang H, Zhao Y, Zhan S, Ji H.
Baumgartner C. Development of a clinical decision support system in Accuracy and effects of clinical decision support systems integrated
intensive care. Stud Health Technol Inform. 2018;248:247–54. with BMJ best practice-aided diagnosis: interrupted time series study.
90. Soyiri IN, Sheikh A, Reis S, Kavanagh K, Vieno M, Clemens T, Carnell EJ, JMIR Med Inform. 2020;8(1):e16912.
Pan J, King A, Beck RC, et al. Improving predictive asthma algorithms 109. AMIA. Secondary use and re-uses of healthcare data: taxonoy for policy
with modelled environment data for Scotland: an observational cohort formulation and planning. Amarican Medical Informatics Association;
study protocol. BMJ Open. 2018;8(5):e023289. 2007. p. 1–4.
91. Caliebe A, Scherag A, Strech D, Mansmann U. Scientific and ethical 110. Cimino JJ. Collect once, use many: enabling the reuse of clinical data
evaluation of projects in data-driven medicine. Bundesgesundheitsblatt through controlled terminologies. J AHIMA. 2007;78(2):24–9.
Gesundheitsforschung Gesundheitsschutz. 2019;62(6):765–72. 111. Opondo D, Visscher S, Eslami S, Medlock S, Verheij R, Korevaar JC, Abu-
92. Chen L, Gu Y, Ji X, Sun Z, Li H, Gao Y, Huang Y. Extracting medications Hanna A. Feasibility of automatic evaluation of clinical rules in general
and associated adverse drug events using a natural language process‑ practice. Int J Med Inform. 2017;100:190–4.
ing system combining knowledge base and deep learning. J Am Med 112. Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang
Inform Assoc. 2020;27(1):56–64. PC, Detmer DE, Expert P. Toward a national framework for the second‑
93. Forsyth AW, Barzilay R, Hughes KS, Lui D, Lorenz KA, Enzinger A, Tulsky ary use of health data: an American medical informatics association
JA, Lindvall C. Machine learning methods to extract documentation of White paper. J Am Med Inform Assoc. 2007;14(1):1–9.
breast Cancer symptoms from electronic health records. J Pain Symp‑ 113. Hackl WO, Ammenwerth E. SPIRIT: systematic planning of intelligent
tom Manag. 2018;55(6):1492–9. reuse of integrated clinical routine data. A conceptual best-practice
94. Garcelon N, Burgun A, Salomon R, Neuraz A. Electronic health records framework and procedure model. Methods Inf Med. 2016;55(2):114–24.
for the diagnosis of rare diseases. Kidney Int. 2020;97(4):676–86. 114. Seid M, Hartley DM, Margolis PA. A science of collaborative learning
95. Gehrmann S, Dernoncourt F, Li Y, Carlson ET, Wu JT, Welt J, Foote J Jr, health systems. Learn Health Syst. 2021;5(3):e10278.
Moseley ET, Grant DW, Tyler PD, et al. Comparing deep learning and 115. VWS: Rijksoverheid. 2019. https://​www.​rijks​overh​eid.​nl/​onder​werpen/​
concept extraction based methods for patient phenotyping from clini‑ digit​ale-​gegev​ens-​in-​de-​zorg.
cal narratives. PLoS One. 2018;13(2):e0192360. 116. Schippers EI. In: van Volksgezondheid M, Den Haag W, editors. Beleids‑
96. Qiu JX, Yoon HJ, Fearn PA, Tourassi GD. Deep learning for automated doelstellingen op het gebied van Volksgezondheid, Welzijn en Sport.
extraction of primary sites from Cancer pathology reports. IEEE J Rijksoverheid; 2013. p. 1–10.
Biomed Health Inform. 2018;22(1):244–51. 117. Haug CJ. Turning the tables - the new European general data protec‑
97. Sendak MP, Ratliff W, Sarro D, Alderton E, Futoma J, Gao M, Nichols M, tion regulation. N Engl J Med. 2018;379(3):207–9.
Revoir M, Yashar F, Miller C, et al. Real-world integration of a Sepsis deep 118. European Parliament. Regulation (EU) 2016/679 of the European Parlia‑
learning technology into routine clinical care: implementation study. ment and of the Council of 27 April 2016 on the protection of natural
JMIR Med Inform. 2020;8(7):e15182. persons with regard to the processing of personal data and on the free
98. Subbiah V. The next generation of evidence-based medicine. Nat Med. movement of such data, and repealing Directive 95/46/EC (General
2023;29(1):49–58. Data Protection Regulation). In: Official Journal of the European Union,
99. Damoiseaux-Volman BA, Medlock S, van der Meulen DM, de Boer J, vol. 59. European Union; 2016.
Romijn JA, van der Velde N, Abu-Hanna A. Clinical validation of clinical 119. Duckworth C, Chmiel FP, Burns DK, Zlatev ZD, White NM, Daniels
decision support systems for medication review: a scoping review. Br J TWV, Kiuber M, Boniface MJ. Using explainable machine learning to
Clin Pharmacol. 2022;88(5):2035–51. characterise data drift and detect emergent health risks for emergency
100. Kouladjian L, Gnjidic D, Chen TF, Hilmer SN. Development, validation department admissions during COVID-19. Sci Rep. 2021;11(1):23017.
and evaluation of an electronic pharmacological tool: the drug burden 120. Keleko AT, Kamsu-Foguem B, Ngouna RH. Tongne a: artificial intel‑
index calculator(c). Res Social Adm Pharm. 2016;12(6):865–75. ligence and real-time predictive maintenance in industry 4.0: a biblio‑
101. Lyerla F, Danks J, Hajdini H, Henderson R. Embedding policy and metric analysis. AI and Ethics. 2022;2(4):553–77.
procedure hyperlinks into the electronic health record to improve
van Velzen et al. BMC Medical Informatics and Decision Making (2023) 23:279 Page 11 of 11

121. Wellsandt S, Klein K, Hribernik K, Lewandowski M, Bousdekis A,

Mentzas G, Thoben K-D. Hybrid-augmented intelligence in predic‑
tive maintenance with digital intelligent assistants. Annu Rev Control.
2022;53:382–90.
122. Crossnohere NL, Elsaid M, Paskett J, Bose-Brill S, Bridges JFP. Guidelines
for artificial intelligence in medicine: literature review and content
analysis of frameworks. J Med Internet Res. 2022;24(8):e36823.
123. Badnjevic A. Evidence-based maintenance of medical devices: cur‑
rent shortage and pathway towards solution. Technol Health Care.
2023;31(1):293–305.
124. Peeters LM, Parciak T, Kalra D, Moreau Y, Kasilingam E, van Galen P, Thal‑
heim C, Uitdehaag B, Vermersch P, Hellings N, et al. Multiple sclerosis
data Alliance - a global multi-stakeholder collaboration to scale-up real
world data research. Mult Scler Relat Disord. 2021;47:102634.
125. Seid M, Hartley DM, Dellal G, Myers S, Margolis PA. Organizing for col‑
laboration: an actor-oriented architecture in ImproveCareNow. Learn
Health Syst. 2020;4(1):e10205.
126. Hummel P, Braun M. Just data? Solidarity and justice in data-driven
medicine. Life Sci Soc Policy. 2020;16(1):8.
127. Adams LV, Wagner CM, Nutt CT, Binagwaho A. The future of global
health education: training for equity in global health. BMC Med Educ.
2016;16(1):296.

Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in pub‑
lished maps and institutional affiliations.

Ready to submit your research ? Choose BMC and benefit from:

• fast, convenient online submission

• thorough peer review by experienced researchers in your field
• rapid publication on acceptance
• support for research data, including large and complex data types
• gold Open Access which fosters wider collaboration and increased citations
• maximum visibility for your research: over 100M website views per year

At BMC, research is always in progress.

Learn more biomedcentral.com/submissions