J Digit Imaging

DOI 10.1007/s10278-013-9671-y

Towards a Theory of PACS Deployment: An Integrative PACS

Maturity Framework
Rogier van de Wetering & Ronald Batenburg

# Society for Imaging Informatics in Medicine 2014

Abstract Owing to large financial investments that go along performance. This result is robust and stable for various sub-
with the picture archiving and communication system (PACS) samples and segments. This paper presents a conceptual model
deployments and inconsistent PACS performance evaluations, that explains how alignment in deploying PACS in hospitals is
there is a pressing need for a better understanding of the positively related to the perceived performance of PACS. The
implications of PACS deployment in hospitals. We claim that conceptual model is extended with tools as checklists to sys-
there is a gap in the research field, both theoretically and tematically identify the improvement areas for hospitals in the
empirically, to explain the success of the PACS deployment PACS domain. The holistic approach towards PACS alignment
and maturity in hospitals. Theoretical principles are relevant to and maturity provides a framework for clinical practice.
the PACS performance; maturity and alignment are reviewed
from a system and complexity perspective. A conceptual Keywords Picture archiving and communication systems .
model to explain the PACS performance and a set of testable PACS maturity model . Performance . Complexity theory .
hypotheses are then developed. Then, structural equation Strategic planning . Structural equation modeling
modeling (SEM), i.e. causal modeling, is applied to validate
the model and hypotheses based on a research sample of 64
hospitals that use PACS, i.e. 70 % of all hospitals in the Introduction
Netherlands. Outcomes of the SEM analyses substantiate that
the measurements of all constructs are reliable and valid. The After nearly 30 years of picture archiving and communication
PACS alignment—modeled as a higher-order construct of five system (PACS) technology development and evolution, PACS
complementary organizational dimensions and maturity has become an integrated component of today’s health care
levels—has a significant positive impact on the PACS delivery system [1]. Nowadays, more extensive, efficient,
cost-effective, scalable and vendor-independent infrastructure
R. van de Wetering (*) : R. Batenburg PACS solutions (e.g. using DICOM) are available, overcom-
Department of Information and Computing Sciences, ing the inherent technical and practical limitations of earlier
Faculty of Sciences, Utrecht University,
PACS deployments. Many hospitals are strategically planning
Fruinplantsoen 77, 3571 PT Utrecht, The Netherlands
e-mail: [email protected] and preparing for future radiology needs by re-evaluating their
radiology systems and looking to replace (or upgrade) their
R. Wetering original imaging networks with state-of-the-art equipment to
e-mail: [email protected]
improve the overall system performance [2].
R. Batenburg In this respect, evaluation methods have proven valuable to
e-mail: [email protected]
assess the impacts of PACS on (radiological) workflow, al-
R. Batenburg though it has been argued that PACS benefits for hospitals
Netherlands Institute for Health Services Research (NIVEL), should be evaluated from different angles and that the inclu-
PO Box 1568, 3500 BN Utrecht, The Netherlands sion of clinical and not-for-profit goals makes the evaluations
R. van de Wetering
more relevant [3].
Deloitte Consulting Strategy & Operations, Still, little scientific knowledge is available about the
PO Box 3502, Utrecht, GD, The Netherlands mechanisms that govern the PACS performance and
 ROGIER ET AL.

deployment success in hospitals. Owing to the large financial These are the following:
expenses that go along with PACS, there is a pressing need for
models or frameworks that are adequate to rigorously assess 1. PACS Infrastructure
and evaluate the performance of PACS, so that improvement This initial maturity level is concerned with the basic
guidelines for strategic planning and optimization plans and and unstructured implementation and usage of image
future investments can be systematically derived. acquisition, storage, distribution and display.
As we will argue in this paper, the PACS maturity model 2. PACS Process
(PMM), which departs from the notion that PACS deployment At the PACS process level, most of initial pitfalls have
is a stepwise process from an immature stage of growth/ been covered by the so-called ‘second’ generation–more
maturity towards the next maturity level, can be enriched with advanced–PACS deployments. The general focus on this
other theories into a conceptual model that is both extended level is on effective process redesign/re-engineering, op-
and sparse enough to explain and understand PACS perfor- timizing manual workflow in radiology and initiating
mance variations in hospitals. transparent PACS processes outside radiology. This re-
Therefore, the main goal of this study is to develop an quires a high level of integration of the various imaging
integrative model to empirically assess, on the one hand, the information systems and hospital information system
maturity and organizational alignment of PACS and, on the (HIS) and radiology information system (RIS).
other hand, their impact on PACS performance. This implies 3. Clinical Process Capability
that performance is defined as having multifactorial impacts This third level is represented by the evolution of
and benefits, as produced by the application of PACS in terms PACS towards a system that can cope with operational
of hospital efficiency (and service) and clinical effectiveness. workflow and patient management, hospital-wide PACS
We depart from the notion that theories from the IS/IT field distribution, communication and image-based clinical ac-
provide new perspectives to understand how key elements in tion. The evolution to this level requires important alter-
clinical practice can be achieved using PACS [3]. ations in terms of PACS processes, extending the scope
The validation of the proposed conceptual model for PACS beyond imaging data and the level of integration of health
performance is essential given the intangible nature of PACS information systems like HIS, RIS and PACS.
performance as the central explanandum at stake. We first 4. Integrated Managed Innovation
present how theoretical concepts of maturity and business The integrated managed innovation level can be char-
alignment coincide with covariation (or co-alignment) [4] as acterized by the initial integration of PACS into the elec-
an operationalized statistical scheme within structural equa- tronic patient record (ePR) (or electronic medical record
tion modeling (SEM). The empirical part of this paper is (EMR)) and cross-enterprise exchange of digital imaging
dedicated to assessing the impact of PACS maturity and data (XDS-i) and supporting material. Basically, this level
alignment on the multifactorial nature of the PACS perfor- forms a bridge between the optimization of internal clin-
mance using a primary data collected among 64 hospitals in ical PACS processes and the wider adoption within an
the Netherlands. The main objective of this part of the paper is ePR/EMR and enterprise PACS chain(s).
to empirically validate the proposed integrative PACS perfor- 5. Optimized Enterprise PACS Chain
mance model. Based on these analyses, the third and final step Finally, level five is the ‘optimized enterprise PACS
is to derive improvement guidelines for strategic planning and chain’. At this level, and with PACS fully integrated into
optimization plans of PACS maturity and performance within the wider ePR, PACS can be maximized for efficiency
hospitals. purposes and clinical effectiveness. Thus, the key process
characteristics at this developmental stage include the fol-
lowing: large system integrations, PACS and web-based
Theoretical Background technology and image distribution though web-based ePR.

Starting Point: The PACS Maturity Model With the evolution of PACS towards higher levels of
maturity, workflow efficiency (medical), IS/IT-integration,
Maturity models have been developed to measure, plan and and effective qualitative care expand. It should be noted,
monitor the evolution of IS/IT in various organizations and however, that high quality service, efficiency and clinical
markets. Within this field, Nolan and Gibson [5] are consid- effectiveness using a PACS can only be achieved if PACS is
ered the founders of the IS/IT stage-based maturity perspec- integrated within a wider ePR of the hospital. This integration
tive, although it has been further extended by others. For is one of the most expensive and time-consuming projects but
digital radiology and PACS, Van de Wetering and Batenburg yields many benefits. The integration of PACS within an ePR
developed the PMM [6]. In their study, they defined five enables a consistent work environment within the hospital for
levels of PACS maturity that hospitals can achieve. radiologists, referring clinicians, nurses, staff and management.
J Digit Imaging

Furthermore, it provides opportunities for effective the SAM and other IS/IT alignment approaches perceive and
(e)consultation, retrieval of more timely and accurate patient operationalize the alignment as a linear (static) mechanism.
information, allowing for real-time diagnosis, decision support, This neglects the fact that mechanisms are multidirectional and
inter-disciplinary processes (intelligent), data mining activities, that change in one organizational domain has multilevel effects
continuous clinical optimization and so on. on other domains. Organizational performance is, in fact, a
The PMM is descriptive and partly normative and has been non-linear, emergent and partly unintended outcome, which
developed as a guideline for assessment and strategic plan- cannot be approximated by any linear form [13].
ning. In that respect, the PMM can be used for strategic To turn this perspective into a conceptual model, a system-
planning, incorporating growth paths towards achieving atic agenda are required, linking theory development with
higher levels of PACS maturity. An important omission of mathematical or computational model development that does
the model is, however, that the development through the not follow the concepts of equilibrium-based mathematical
maturity model might differ by organizational domains and approaches (i.e. that rely on linearity, attractors, fixed points
that maximizing maturity might not be effective or ‘optimal’ and the like [14]). This is addressed in the next section.
in all circumstances. For this reason, we involve another
theoretical perspective, as shown in the next section.
An Integrative PACS Performance Model
Complementarity and Alignment Theories
Based on the previous analyses, we develop a model that
The theory of complementarity was introduced by Edgeworth, combines three concepts: (1) PACS maturity as the concept
who defined activities as complements ‘if doing (more of) any to define PACS and its elements (i.e. classifying PACS sys-
one of them increases the returns to doing (more of) the others’ tems according to their stage of development and evolutionary
[7]. Complementarity theory assumes that the individual ele- plateau of process improvement), (2) PACS alignment as the
ments of a strategic planning process (i.e. the variables) cannot concept to complement the organizational dimensions of
be individually optimized to achieve a better performance [8]. In PACS (i.e. investments made in organizational dimension
the business and strategic management literature, complementar- related to PACS should be balanced out in the organization
ity is often labeled as ‘fit’ [4] or strategic alignment. Strategic in order to obtain synergizing benefits), and (3) PACS perfor-
alignment refers to applying IS/IT in a structural and stable way, mance as the added value of PACS within hospitals.
in harmony with business strategies, goals and needs. The stra- Using the PMM as a starting point, we suggest measuring
tegic alignment model (SAM) of Henderson and Venkatraman is maturity and alignment (as independent variables) by the
the most cited concept within this field [9, 10]. Their model degree to which hospitals score and differ on five organiza-
implies that a systematic process is required to govern continuous tional dimensions (see ‘Complementarity and Alignment
alignment between business and IS/IT domains, i.e. to achieve Theories’ section). For each of these five dimensions, distinc-
‘strategic fit’ as well as ‘functional integration’. The SAM has tive maturity levels have previously been defined by the PMM
been extended by theorists, industry and consulting [10], who [11]. These accompanying maturity levels can be successively
have all defined ‘fit’ as the balance or equilibrium of different labeled for S&P3, S&P4 and S&P5, O&P3, O&P4 and
organizational dimensions and ‘external fit’ as the strategic de- O&P5, and so on. Maturity levels 1 and 2—as defined by
velopment that is based on environmental trends and changes. the PMM—are omitted for practical reasons, which will be
However, the SAM is not able to monitor or measure matu- elaborated upon in the ‘Results’ section. In addition, we define
rity and/or performance. This was improved by Scheper [11], PACS performance as a multifactorial (dependent) variable to
who extended the SAM by combining it with the be measured in terms of hospital efficiency (i.e. organizational
Massachusetts Institute of Technology (MIT) 1990s model construct containing the patient service, end-user service and
[12]—developed as part of a major business IT research—and organizational efficiency perspectives) and clinical perfor-
defining five key organizational domains that are essential to be mance (i.e. subdivided into diagnostic efficacy and commu-
aligned: (1) strategy and policy (S&P), (2) organization and nication efficacy) [3, 15].
processes (O&P), (3) monitoring and control (M&C), (4) in- Our conceptual model contains higher-order
formation technology (IT) and (5) people and culture (P&C). In (multidimensional) latent constructs within the context of
contrast to the SAM, Scheper also defined levels of incremental simultaneous equation systems [16]. These ‘latent constructs’
maturity for each of the five domains. Hence, he claimed that cannot be observed directly because their meanings are ob-
alignment could be practically measured and assessed by the tained by measuring the manifest variables. In interconnecting
comparative levels of maturity on each of the five dimensions. the three key concepts of PACS maturity, PACS alignment
Probably for reasons of complexity, the co-evolutionary and and PACS performance, we propose a reflective construct
emergent nature of alignment has rarely been taken into con- model, through which the manifest variables are affected by
sideration in IS/IT alignment research [13]. In the same vein, the latent variables (in contrast to the formative constructs).
 ROGIER ET AL.

We apply a multistep approach using path modeling to who provided the project team with input and advice on key
hierarchically construct latent variables as the independent concepts in diagnostic imaging. This initial survey was then
part (i.e. PACS alignment) of the conceptual model and latent discussed with industry consultants and a PACS R&D man-
variables as the dependent part (i.e. PACS performance) of the ager during a focus group meeting, thereby redefining some of
conceptual model. (see Appendix for a detailed description of the questions in the survey. The topics in the survey were
the constructs and their relation). subsequently validated in several individual validation ses-
Based on the above mentioned, the main hypothesis to be sions (using the ‘Delphi method’) with PACS experts (three
empirically tested by the conceptual model can be formulated as radiologists, a neurologist, a technologist and medical infor-
matics researcher) representing four hospitals in four different
‘The alignment of PACS, as represented by the multifac- geographical areas in the Netherlands. The outcomes were
torial nature of five organizational domains and their used to improve our survey statements on validity, reliability
related maturity levels, has a positive relationship on and empirical application (e.g. the size of the survey and
PACS performance, as represented by the multifactorial tooling).
nature in terms of hospital efficiency and clinical effec- Taking considerable comments into account, this initial
tiveness and their related items.’ survey was extended and applied in a pilot with two hospitals
of different sizes and operating regions that were actively
Alignment of PACS is defined as the pattern of internal
involved in optimizing their PACS deployments. At each
consistency among the two sets of underlying constructs.
hospital, two radiologists (including heads of department),
More specifically, PACS alignment is modeled as a third-
the head of radiological technologists and a PACS adminis-
order latent construct, whereas the second-order constructs
trator completed an online survey within a secure web envi-
represent the organizational domains to be co-aligned and
ronment. These informants were most familiar with the sub-
the first-order constructs represent the maturity levels. This
ject of PACS maturity and performance, making intra-
modeling of PACS alignment is statistically appropriately
institutional validity likely. Including multiple stakeholders
captured by a pattern of covariation, which coincides with
from the radiology department also reduces common source
the concept of (co-)alignment [4].
variance associated with sampling from the same source [17],
Our conceptual model follows the central concept of internal
excluding face validity issues. Respondents completed the
logic among the various dimensions, since it is in accordance
survey separately to avoid systematic bias and any peer pres-
with the theories of complexity and CAS outlined previously.
sure to give particular answers.
SEM techniques are specifically suited for the modeling of
The pilot offered good opportunities to improve the con-
complex processes to serve both theory and practice.
tents of the survey and improve the clarity of the statements.
Therefore, SEM is the appropriate method to validate our
Finally the questionnaire was extended to 42 statements1,
conceptual model to capture the complex entanglement of
covering most intersections of our framework.
PACS deployment and performance in hospitals. The appli-
For each organizational dimension, the items were formu-
cation of SEM (and latent variable modeling) fits a mode of
lated according to a cumulative order. Our questionnaire ex-
integrative thinking about theory construction, measurement
plicitly addressed a hierarchical order (i.e. increasing com-
problems and data analysis. It enables stating the theory more
plexity’) of survey items along the maturity scale, communal-
exactly, testing the theory more precisely and yielding a more
ity and interrelationship of stages of maturity, so that we
thorough modeling/understanding of the empirical data about
avoided common pitfalls in survey instruments and case re-
complex phenomena and relationships [17].
search. All questions were assessed using a seven-point Likert
Figure 1 displays the SEM notation of our conceptual mod-
scale for each statement from strongly disagree to strongly
el, capturing the theorized relationships between organizational
agree.
domains (i.e. second-order construct) and PACS alignment (i.e.
Furthermore, the statements were phrased in the present
third-order construct), on the one hand, and its impact on PACS
tense, but respondents were asked to provide answers for both
performance (i.e. third-order construct), on the other.
the current and future/preferred situations of their hospitals.
The survey also contained some general questions (e.g.
name, function, years of experience using PACS, etc.).
Material and Methods Finally, PACS performance was measured using 12

Instrument Development Process

1
Statements for maturity levels 1 and 2 were omitted for practical reasons
An initial survey was developed based on the literature, field
and because all Dutch hospitals have implemented the initial maturity
experience and valuable suggestions by two PACS experts (a level. Level 2 could be deducted from the assigned scores to level 3
professor of radiology and the head of a radiology department) statements.
J Digit Imaging

Fig. 1 Theoretical SEM notation for the PACS alignment model (β=estimated value for the path relationship in the structural model)

performance statements on how well the system contributes to reminder mail after 5 weeks) to all the heads of radiology in
efficiency and effectiveness [3, 15]. the Netherlands by a recognized radiologist in the field.
Five weeks after, follow-up phone calls were made to all
radiology departments that had not yet returned a single
Data and Sample Collection Procedure questionnaire.
In total, 82 questionnaires were either filled in online or
A survey was conducted targeting all general and top clinical returned in the post. Representatives from 12 hospitals filled
hospitals (i.e. non-university teaching hospitals) and universi- in at least one questionnaire, resulting in an overall response
ty medical centres in the Netherlands (N=91). The question- from the 64 participating hospitals. This percentage is remark-
naire was sent to (1) the heads of the radiology departments ably high in comparison with common survey response rates.
(and radiologists), (2) the heads of technologists and/or de- All questionnaires were included into the analysis subject to
partment managers and (3) the PACS/RIS administrators of all quality criteria (e.g. no missing answers). Table 1 provides the
radiology departments. Contact details were obtained from the demographics of our obtained sample. Participating hospi-
secretaries of each individual radiology department. tals—which all had their own radiology departments—could
Respondents were asked to fill up the survey either online be divided into three categories: general hospitals, top clini-
or by returning the provided printed version to the university. cal—large educational hospitals providing highly specialized
In parallel, invitations to participate were sent via mail (and a medical care—and academic medical centres.
 ROGIER ET AL.

Table 1 Sample demographics

Sample descriptives

Total Percentage of total AVG beds FTE radiologist AVG exams Total

Hospital type Respondents

General hospital 37 (56) 66 353 6.4 95,900 Radiologist 29 (35 %)
Top clinical 21 (27) 78 657 10 160,000 Head technologists/manager 26 (32 %)
Academic 6 (8) 75 944 27.2 193,500 PACS administrator 27 (33 %)

AVG average, FTE full-time equivalent)

As can be seen from Table 1, our sample contains 75 % of SEM techniques are specifically suited for the modeling of
the academic hospitals, 78 % of the top clinical hospitals and complex processes to serve both theory and practice.
66 % of the general hospitals in the Netherlands. This is a total Therefore, SEM is the appropriate method to validate our
response rate of 70 % of the targeted hospitals. Therefore, the conceptual model to capture the complex entanglement of
obtained sample is representative of hospitals in the PACS deployment and performance in hospitals. It enables
Netherlands with regard to size. stating the theory more exactly, testing the theory more pre-
Also, Table 2 includes a distribution of the PACS vendors cisely and yielding a more thorough modeling/understanding
currently involved in our sample. As can be seen from our of empirical data about complex phenomena and relationships
sample, there are currently eight vendors active in the Dutch [17].
market. Currently, many Dutch hospitals re-evaluated their Since the interpretation of parameter outcomes in SEM is
current PACS systems and are looking to replace their original not straightforward, we adopted the validation procedures
imaging networks. Also, hospitals are planning for major outlined by Marcoulides and Saunders [19] to assess the
upgrades, and this can change the PACS vendor landscape ‘outer’ (measurement) and ‘inner’ model (structural). This
considerably in the coming years. was to:

1. Propose a model that is consistent with all currently

Instrument Validation available theoretical knowledge and collect data to test
that theory;
The operationalization of our conceptual model can be per- 2. Perform data screening (including the accuracy of inputs
formed most accurately using SEM [18]. SEM (or ‘causal and outliers.). We first performed tests on the data nor-
modeling’) is typically used to simultaneously validate multi- mality distribution of all manifest variables (MV1–MV42)
faceted phenomena in terms of tentative cause and effect using SPSS version 18.0. As a general rule of thumb, the
variables, including causal effects. In doing so, it simulta- absolute values of the ratio of skewness to its standard
neously examines the measurement model (factor model or error (SE) and of kurtosis to its SE should be between
outer model) and the structural model (inner model or path −2≤×≤2; higher values indicate greater asymmetry and
model). deviation from normality;
3. Examine the ‘psychometric properties’ (i.e. measurement
Table 2 PACS vendor distribution (in percentages)
model) of all variables. To demonstrate that the psycho-
metric properties had satisfactory levels of validity and
Sample vendor descriptives reliability, the measurement model was assessed for first-
order constructs. Composite reliabilities2 (CRs; [20]) and
Vendor Percentage in sample (%)
average variance extracted (AVE; [20])—i.e. the average
Agfa 26 variance of measures accounted by the latent construct—
Carestream 9 were computed. As a general rule of thumb, variables with
Delft Diagnostic Imaging (Rogan) 10 a loading less than 0.6 should be removed from the
Fuji 9 sample [21]. Discriminant validity was assessed by
GE 5
Philips 31 2
Composite reliability is similar to Cronbach’s alpha without the assump-
Sectra 5 tion of the equal weighting of variables. Its mathematical formula (with
Siemens 5 the assumption that the factor variance=1; standardized indicators) is
ρ=(Σλi)2/((Σλi)2 +Σ1−(λi)2).
J Digit Imaging

verifying (1) whether indicators loaded more strongly on The current study has a sample size of N=64. Given the
their corresponding (first-order) constructs than they did rationale above (and in ‘Material and Methods’ section) and
on the other constructs and (2) that the square root of the the fact that our data are not normally distributed, we chose a
AVEs should be larger than the inter-construct correla- PLS approach—which is robust for moderate sample sizes—
tions (see entries in bold in Table 3 along the matrix over the use of covariance-based structures to validate our
diagonal). The off-diagonal elements are correlations be- model. Hence, our main focus was on explaining (and
tween latent variables as calculated by the partial least predicting) the endogenous construct ‘PACS performance’ in
squares (PLS) algorithm. See ‘Assessment of which R2 and the significant relationships among constructs
Discriminant Validity’ section for a brief explanation on indicated how well our model performed. Therefore, variance-
the entries in Table 3. based methods were preferred.
4. Examine the magnitude of the relationships (i.e. structural Within PLS-SEM, higher-order constructs can be con-
model) and effects between the variables being consid- structed using repeated indicators (i.e. the hierarchical com-
ered in the proposed model. We accounted for possible ponent model). That is, all indicators of the first-order con-
moderating effects (i.e. interaction effects) within our data structs are reassigned to the second-order construct, as second-
through a multisample/group approach [22]. In doing so, order models are a special type of PLS path modeling that use
we equally divided our research sample into two groups manifest variables twice for model estimation. The same
based on the amount of beds of each hospital. Hence, patterns are applicable to subsequent higher-order constructs.
group 1 (≤450 beds) was assigned to 30 hospitals and the A prerequisite for this model approach is that all manifest
second group (>450 beds) to 34 hospitals. The model’s variables of the first-order and higher-order constructs should
path coefficients were subsequently estimated separately be reflective [25]. As such, indicators share a common theme
for each group using the SEs obtained from and are manifestations of the key constructs. In addition, any
bootstrapping. Likewise, we divided our sample of ques- changes in constructs cause changes in the indicators. Thus,
tionnaires into three disjoint groups, based on respondent all constructs within our model were configured as reflective
category (i.e. radiologist, PACS administrator and head indicators and are considered exogenous variables. Therefore,
technologists/manager). Also, we assessed various proper all constructs in our PLS model were configured like this.
measures for model fit including (a) the Goodness-of-fit
index, [20], defined as the geometric mean of the average
communality of all constructs with multiple indica- Results
tors and the average R2 (for endogenous constructs),
(b) R2—the coefficient of determination, (c) Q2 of Data Screening
our endogenous constructs (using Stone–Geisser’s test
[20]) to assess the quality of each structural equation Outcomes of the data screening suggest that our data slightly
measured by the cross-validated redundancy and commu- deviate from normality (AVG skew = |2,2|; AVG kurto-
nality index (using the blindfolding procedure in sis=|1,4|). Additional support for non-normal distribution came
SmartPLS) and to evaluate the predictive relevance for from a Kolmogorov–Smirnov test (Kolmogorov–Smirnov–
the model constructs; Lilliefors test) for normality. All variables demonstrated signif-
5. As a final step, assess and report the power of the study. icant values; thereby, we rejected the null hypotheses that our
We used G*Power [23]—a general standalone program— data were not significantly different from normal distributions.
for statistical tests. Power (1−β) of statistical tests can be
defined as the probability of falsely retaining an incorrect Assessment of the Measurement Model (the ‘outer’ model)
H0 [24].
Table 4 includes loadings (λ) for all the items (MVi) of each
To perform this multistep approach and estimate the pa- organizational domain and maturity levels. λ can be best
rameters in the inner and outer models, we used SmartPLS understood in terms of factor loadings (e.g. as a result of factor
version 2.0 M3, which is a SEM application using PLS. We analysis). All loadings exceeded 0.7 except MV1, MV16,
applied the path weighting scheme available within SmartPLS MV24 and MV25. Considering that these values were close
in addition to centroid and factor schemes with the knowledge to the threshold, these items were retained in the original
that the choice among each scheme has a minor impact on the model. As can be seen from Table 4, all CR values were well
final result [20]. In addition, we applied a non-parametric above 0.7. Likewise, all AVE values exceeded the cut-off
bootstrapping [20], as implemented into the SmartPLS appli- value indicating sufficient convergent validity. Table 5 in-
cation, to compute the level of the significance of the regres- cludes five performance dimensions, measurements and indi-
sion coefficients, with 500 replications to interpret their sig- cators and the psychometric properties (i.e. AVE, CR and λ) of
nificance and to obtain stable results. the dependent construct (i.e. endogenous construct). All the
Table 3 Inter-correlations of first-order constructs (N=64)

First-order construct 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

1. IT3 0.86
2. IT4 0.38 0.81
3. IT5 0.24 0.32 0.72
4. MC3 0.13 0.34 0.21 0.79
5. MC4 0.14 0.30 0.27 0.50 0.77
6. MC5 0.31 0.28 0.25 0.43 0.59 0.81
7. OP3 0.17 0.19 0.38 0.40 0.32 0.39 0.81
8. OP4 0.06 0.21 0.13 0.23 0.52 0.52 0.24 0.79
9. OP5 0.19 0.06 0.24 0.07 0.27 0.19 0.03 0.42 0.81
10. PC3 0.42 0.46 0.22 0.40 0.23 0.37 0.23 0.17 0.19 0.78
11. PC4 0.28 0.10 −0.04 0.29 0.15 0.28 0.05 −0.05 0.14 0.51 0.90
12. PC5 0.23 0.42 0.21 0.38 0.45 0.31 0.24 0.21 0.33 0.33 0.28 0.82
13. SP3 0.07 −0.03 −0.02 −0.20 0.15 0.30 −0.11 0.19 0.04 0.09 0.05 −0.07 0.73
14. SP4 0.17 0.23 0.47 0.31 0.32 0.35 0.22 0.24 0.36 0.23 −0.11 0.18 0.25 0.77
15. SP5 0.03 0.20 0.26 0.39 0.32 0.42 0.44 0.18 0.21 0.27 0.23 0.23 0.19 0.43 0.83
16. Patient service 0.27 0.15 0.31 0.11 0.09 0.34 0.19 0.16 −0.08 0.14 0.23 0.00 0.20 −0.01 0.14 0.74
17. End-user service 0.44 0.36 0.34 0.46 0.45 0.53 0.40 0.20 0.07 0.39 0.16 0.39 −0.02 0.27 0.10 0.20 0.76
18. Organizational efficiency −0.13 −0.21 −0.08 0.02 −0.05 −0.02 0.23 −0.18 −0.28 −0.10 −0.18 0.09 −0.18 0.00 −0.04 0.04 0.20 0.79
19. Diagnostic efficacy 0.08 0.13 0.28 0.19 0.32 0.30 0.23 0.21 0.08 0.01 0.07 0.17 0.01 0.08 0.17 0.12 0.30 −0.02 0.81
20. Communication efficacy 0.49 0.31 0.22 0.30 0.10 0.18 0.20 0.06 0.30 0.38 0.24 0.36 −0.09 0.08 0.20 0.26 0.28 -0.12 0.09 0.79

Entries in bold along the matrix diagonal are the square roots of the AVE
ROGIER ET AL.
J Digit Imaging

Table 4 Estimates for the psychometric properties of the first-order constructs for PACS alignment

PACS alignment construct

Domain Maturity Indicators (i.e. shortened survey statements that persons responded to) λ AVE CR
level

Strategy and policy 3 Primary interpretation by radiologists using uncompressed images (MV1) 0.67 0.54 0.70
Emphasis is on the direct display of images from the archive (MV2) 0.79
4 PACS integration with the ePR is an important strategic objective (MV3) 0.76 0.59 0.77
Alignment of investment plans between radiology and other departments/wards (MV4) 0.77
5 Inquiry of the external environment for new developments and products to optimize PACS 0.84 0.69 0.83
functionality (MV5)
Strategic and operational (multiyear) plans contain impact and opportunities for chain partners 0.82
(MV6)
Organization and 3 Active improvement of service levels using quality standards and measures for digital PACS 0.86 0.66 0.79
processes workflow (MV7)
Every image is instantly available on any workstation in the hospital for every user at any time 0.76
(MV8)
4 All diagnostic images from other departments are stored into one central PACS archive (MV9) 0.81 0.62 0.77
Dedicated workspace has all required patient information and integrated 2D/3D reconstruction 0.77
tools (MV10)
5 PACS real-time data with chain partners using standard exchange protocols (XDS-i) if necessary 0.80 0.66 0.80
(MV11)
Hospital-wide requests and planning radiology exams using an electronic order-entry system 0.83
(MV12)
Monitoring and 3 Recurrent prognosis concerning the amount of radiology exams and required storage capacity 0.84 0.62 0.77
control (MV13)
Measurement and monitoring of financial and non-financial PACS data (MV14) 0.73
4 Service level agreements with PACS vendors are periodically evaluated (MV15) 0.90 0.60 0.74
PACS generates comprehensive management information that is always on time (MV16) 0.61
5 The hospital confronts PACS vendors if service level agreements are not (or partially) achieved 0.86 0.64 0.78
(MV17)
An accurate overview of the contribution of PACS to overall cost prices per radiology exam 0.73
(MV18)
Information 3 PACS is compatible with current international standards and classifications (HL7 and DICOM) 0.85 0.73 0.85
technology (MV19)
PACS exchanges information with the RIS and HIS without any complications (MV20) 0.86
4 Adoption of standard ‘off-the-shelf’—vendor-independent—hardware and software (MV21) 0.78 0.66 0.79
Impact prognosis on storage capacity because of modality upgrades or newly acquired devices 0.83
(MV22)
5 Application of reagent (security) protocols in preserving the privacy of patient data, PACS data 0.77 0.52 0.70
security and backup (MV23)
PACS is an integral part in the hospitals’ ePRs (MV24) 0.67
People and culture 3 The hospital actively involves the users of PACS in the development of customizable user 0.64 0.60 0.75
interfaces (MV25)
PACS process and procedure knowledge are extensively applied by clinicians and technologists 0.89
(MV26)
4 End-users of PACS affect the decision-making process in selecting a specific PACS vendor 0.90 0.81 0.89
(MV27)
End-users affect digital PACS workflow and functionality improvements (MV28) 0.90
5 Radiologist awareness of PACS has a potential to influence the competitive position of the 0.77 0.67 0.80
hospital and service delivery (MV29)
Innovative solutions with PACS are discussed during clinico-radiological meetings (MV30) 0.87

loadings of the dependent construct exceeded acceptable indicated that the dependent constructs were well defined
thresholds. Manifest variables MV37 and MV38 both had and unidimensional.
negative loadings and had to be removed from the PLS Next to the assessment of first-order constructs, the higher-
program to obtain reliable outcomes. Since MV33 and order constructs exceeded the average threshold values for CR
MV40 had loadings close to the threshold, these items were (i.e. CR≥0.7). Table 6 includes loadings (i.e. factor loading
retained in the original model. Once again, all measures coefficients, γi) for both second-order and third-order
 ROGIER ET AL.

Table 5 Estimates for the psychometric properties of the first-order constructs for PACS performance

PACS performance construct

Measurement construct Indicator λ AVE CR

1. Patient service
Patient waiting time Elapsed time between a patients’ arrival at radiology (on appointment) and subsequent 0.71 0.55 0.71
exam (MV31)
Patient satisfaction Satisfaction of patients on service delivery (MV32) 0.77
2. End-user service
Physician satisfaction Satisfaction of referring clinicians on availability of imaging data and associated 0.62 0.58 0.73
reports (MV33)
User satisfaction User satisfaction on the current user interface and functionality of PACS (MV34) 0.88
3. Organizational efficiency
Report turnaround time Sum of time after execution, reporting and the availability of imaging exams’ 0.81 0.62 0.77
finalized report of CT exams (MV35)
Radiologist productivity The amount of yearly radiology exams per FTE (MV36) 0.77
Budget ratio Percentage (over) expenditures of allocated PACS budgets (MV37) −0.62
4. Diagnostic efficacy
Interpretation time Time to process a series of CT exams (MV38) −0.14 0.65 0.79
Diagnostic accuracy Sufficiency rate of current radiology workspaces for image interpretation (MV39) 0.93
Clinical capability Workstations capability of displaying uncompressed CT studies (avg. 1,500–2,000 images) 0.67
without delay (MV40)
5. Communication efficacy
Patient management Contribution of PACS towards decision-making in diagnostic processes or treatment 0.82 0.62 0.76
plans of patients (MV41)
Communication efficacy PACS contribution towards the communication of critical findings and interdepartmental 0.76
collaboration (MV42)

endogenous and exogenous reflective constructs (see also stipulated thresholds and, thereby, supported the third-order
Appendix for a detailed description of the constructs and their hierarchical model of PACS alignment and its measurement
relation). model.
All higher-order ‘factor’ loadings of the independent part Thus, looking at the exogenous part of the model, we see
of the model provided a satisfactory fit to the data, meeting that loadings of the first-order latent variables (i.e. maturity

Table 6 Factor loadings for

higher-order constructs Second-order construct
Factor loading coefficients (γi)
Exogenous Endogenous
S&P O&P M&C IT P&C OC CP
Maturity level 3 0.48 0.55 0.80 0.78 0.79
Maturity level 4 0.80 0.83 0.82 0.85 0.80
Maturity level 5 0.84 0.71 0.67 0.83 0.61
Patient service 0.62
End-user service 0.82
Organizational efficiency 0.50
Diagnostic efficacy 0.73
Communication efficacy 0.74
Third-order construct
Factor loading coefficients (γi)
Exogenous constructs Endogenous
S&P O&P M&C IT P&C OC CP
PACS alignment 0.64 0.70 0.70 0.85 0.70
PACS performance 0.84 0.85
J Digit Imaging

levels 1, 2 and 3) on the second-order factors (i.e. the five No significant difference between the structural models for
organizational domains, S&P, O&P, M&C, IT and P&C) each of the group comparisons was found (radiology-admin-
exceed the threshold values. istrator group t= 0.01, p< 0.99; head technologist-
In analogy to the exogenous constructs, the loadings for the administrator group t= 0.55, p< 0.60; head technologist-
endogenous higher-order constructs (i.e. PACS performance) radiology group t=0.46, p<0.61). These outcomes imply that
had a significant meaning, indicating a strong goodness of fit the impact of PACS alignment on the PACS performance
and supporting the reflective PACS performance construct construct (i.e. the hypothesized relationship) is stable for
and its manifests. subsamples, i.e. the different respondent groups.

Assessment of Discriminant Validity

Goodness-of-Fit Measurement
As can be seen from Table 3, all square root scores of
Although PLS modeling does not include a proper sin-
the AVEs are higher than the shared variances of the
gle goodness-of-fit measure, the variance explained by
constructs with other constructs in the model. This
the model (R2)—the coefficient of determination—values
means that the square root of the AVE (i.e. entries in
of the endogenous constructs can be used to assess this
bold in Table 3 along the matrix diagonal) exceeds the
model fit. R2 accounted for by PACS performance was
intercorrelations of the construct with the other con-
0.37. In accordance with R2 effect size categorizations (i.e.
structs in the proposed model. Thus, evidence of ade-
0.02, 0.13 and 0.26 [24]), we concluded that the explanatory
quate convergent and discriminant validity was found
power was large.
for all constructs in the proposed conceptual framework.
We also calculated the Q2 of our endogenous constructs. Q2
Further evidence of discriminant validity was obtained
measures how well the observed values are reproduced by the
using cross loadings as quality criteria [26]. These find-
model and its parameter estimates by using the cross valida-
ings indicate that the loadings for each indicator were
tion [20]. As such, Q2 values larger than 0 imply the model’s
greater than the cross loading on other latent variables
predictive relevance; values less than 0 suggest the model’s
in the model.
lack of predictive relevance. In this study, all Q2 values were
In summary, the outcomes of the measurement model
above the threshold value of zero, thereby, indicating the
suggest that the PLS model construct is valid and reliable
overall model’s predictive relevance.
and that the estimates of the structural model (i.e. inner path
A global measure of fit (i.e. goodness-of-fit index, GoF)
model estimates) can thereby be evaluated.
has also recently been suggested [20]. The GoF, defined as the
geometric mean of the average communality of all constructs
Assessment of the Structural Model (the ‘Inner’ Model) with multiple indicators and the average R2 (for endogenous
constructs), represents an operational solution for an index
We found support for our main hypothesis. There was a validating the PLS model globally. Since communality equals
significant positive impact of PACS alignment on PACS per- AVE in PLS [26], the cut-off is set to 0.5. Subsequently, taking
formance (β=0.62; t=4.01; p<0.0001) after validating the small, medium and large effect sizes for R2 (i.e. 0.02, 0.13 and
outer model. 0.26) into account, GoF criteria for small, medium and large
As outlined, we accounted for moderating the effects with- effect sizes can be obtained: 0.10, 0.25 and 0.36. For our
in our data by dividing our research sample into two groups model, a GoF value of 0.45 was obtained, thereby, exceeding
based on the amount of hospital beds. Group 1 (≤450 beds) the cut-off value for large effects (GoF=0.36) [24], which
was assigned to 30 hospitals and the second group (>450 directs towards the conclusion that our model performs well
beds) to 34 hospitals. compared with the base values and was a good fit of the model
The model’s path coefficients were subsequently estimated to the data.
separately for each group using the SEs obtained from
bootstrapping, and no significant difference between the struc-
tural models could be detected (t=0.36, p<0.72). Also, the Estimation of Power
model and its path coefficients were subsequently estimated
for three disjoint groups, based on the respondent category Power calculations indicated that the power for all the param-
(i.e. radiologist, PACS administrator and head technologists/ eters in our model exceeded 0.96. This level of power
manager). The radiology group contains 29 cases (i.e. the 29 indicates a high probability that the analytical tests will
questionnaires that were completed by them), the PACS ad- yield statistically significant results if the phenomena
ministrator group 27 cases, and the head technologist/manager truly exist and thereby a high probability of successfully
group 26 cases. rejecting H0 [24].
 ROGIER ET AL.

Discussion and Conclusion 1. Depart from PMM (that includes a checklist for evolving
onto the next maturity level) and assess the current matu-
Principle Findings and Conclusions rity state of PACS (‘as is’), and also a ‘to be’ situation
should be determined involving multiple stakeholders for
This study presented and validated an integrative model to well-balanced and objective perspectives.
determine PACS alignment and performance in hospitals, 2. Second is a fit–gap analysis that allows to assess whether
adopting theories and perspectives from the field of informa- the current PACS maturity level is either a precursor for
tion system research and complexity theory. Our aim was to the ‘to be’ situation or the desired maturity level ‘leaps’
overcome the limitations of most approaches in the field that over intermediary stages. Now, decision-makers need to
do not focus on the synergizing, complementarity and inte- decide which road and enhancement plan are most suit-
grative effects of PACS in relation to performance. able for the hospital. Important is that the plan need to be
Based on reliable and empirically valid data collected in aligned with the context of hospital strategies.
Dutch hospitals, it is empirically validated that PACS align- 3. As a final step, we suggest to organize improvement
ment has a significant impact on the performance of PACS in projects that take into account the risks involved, invest-
terms of efficiency and effectiveness. This implies that ment costs, critical success factors and benefits. The ex-
hospital-specific resources, capabilities and the use of PACS tensively outlined alignment perspective in this paper
are strongly interrelated, and integrative management is es- needs to be applied in managing similarities, overlap
sential to optimize the added value of PACS. and synergy between the improvement projects in order
From a practical point of view, operational and technical to realize strategic, objectives and optimal deployment of
improvement opportunities can be identified based on our PACS.
model and alignment perspective on the PACS domain. It
should be recognized, however, that improvements often im- In practice, hospitals often apply improvement routes either
ply change of existing processes, organization structures and evolutionary (routes that develop logically in subsequent
touch the interest of stakeholders. As with any (IT) change stages), revolutionary (routes that take a more radical ap-
project in organizations, this cannot be prepared or executed in proach in that it takes strategic ‘leaps’ in evolving towards
a vacuum; hence, internal stakeholders and potential (and higher levels of PACS maturity) or both as a combined strat-
current) vendor should collaborate during a replacement (or egy. For each enhancement path, critical success factors are
upgrade) process of original imaging networks. the involvement of multidisciplinary teams consisting of phy-
To the best of our knowledge, this study is the first that sicians, technicians and engineers and project commitment at
empirically applies the concept of alignment, maturity and all levels within the hospital.
complexity science and theory to the research domain of These steps follow the logic of an ‘intended’ PACS strate-
PACS and medical informatics. We believe that the outcomes gy. Complementary to this deliberate (i.e. conscious) planning
of this study will support hospital decision-makers. process, hospitals also need to plan PACS maturity, alignment
and performance as goals that ‘co-evolve’ within hospitals.
Strategic Guidelines for PACS Deployment This complex task can only be achieved by mobilizing the
diversity of interactions among all organizational agents in-
Based on the above mentioned, we believe that hospitals volved in the deployment of PACS in clinical and IT practice.
should follow a dual strategic PACS maturity planning per-
spective that drives a continuous process of change and adap- Limitations and Venues for Future Research
tation as well as the co-evolvement and alignment of PACS.
Adaptability and changeability should be the integral proper- Despite its attractiveness, our study and integrative framework
ties, next to traditional and deliberate PACS strategic have several limitations. These limitations are largely related
planning. because of our research sample. First, although sufficiently
To do so, we suggest that hospitals who want to take their large to achieve acceptable levels of statistical significance
PACS systems to the next (maturity) level and advance PACS given all the quality criteria for the inner model and outer
performance should explicitly identify and execute improve- model, our sample is limited to hospitals in the Netherlands,
ment activities on each of the five organizational dimensions thereby, limiting generalizability.
of the PMM. To guide decision-makers in deciding how to Although we believe that our framework provides an as-
travel and mature PACS in a certain direction (for any hospi- sessment framework for hospitals worldwide to evaluate the
tal, large or small, public or private that wants to integrate its triangular construct of PACS maturity, alignment and perfor-
strategic objectives for growth and maturity in terms of PACS, mance, we expect that our model can also be used to describe
including ePR and other IS/IT), we suggest the following and reconstruct any hospital PACS case. It is our ambition to
three steps to be taken: extend the application of the proposed model (also
J Digit Imaging

longitudinal). The model can then be reassessed and evaluated Appendix: A Multistep Model Development Approach
for its robustness, and the established higher-order constructs
of our model can be validated through larger sample sizes. In developing our conceptual model, we applied a mul-
Comparing results across countries and groups might well tistep approach using path modeling to hierarchically
contribute to the generalizability of our findings. construct latent variables as the independent part (i.e.
Second, our obtained data included various demographic PACS alignment) of the conceptual model and latent variables
variables (e.g. type, size and region), but our empirical anal- as the dependent part (i.e. PACS performance) of the concep-
ysis did not consider in depth the possible differences among tual model.
group segments. Like in any type of modeling, we had to balance between
Using finite mixture (FIMIX)-PLS procedures, segmenta- recognizing the details of practice and complying the need for
tion can be applied to the empirical data. This approach allows overview and limitation.
model parameters to be estimated and observations’ affilia- With regard to the independent part of the conceptual
tions to be simultaneously segmented [27]. This has the ad- model, we define the following:
vantage compared with an a priori segmentation scheme in
that derived segments are homogeneous in terms of model 1. The second-order construct as the five organizational
(structural) relationships based on fully available information domains, each representing different maturity levels, the
for both manifest and latent variable scores. Initial results first-order exogenous constructs;
indicate that by segmenting data—using an extended ex post 2. The third-order construct, labeled as PACS alignment, as
analysis—higher levels of explained variance can be achieved related to the underlying second-order constructs (i.e. step
for various homogeneous sub-groups. These findings provide 1).
a platform for acquiring further differentiated PLS path With regard to the dependent part of the conceptual
modeling conclusions given segment-specific estimations. model, we define the following:
Third, the primary focus of our study was on PACS within 3. The second-order constructs (organizational construct and
hospitals. In that respect, it would be a logical step to extend clinical performance construct), as related to the block of
our concepts on alignment, maturity and complexity theory to the underlying first-order latent constructs, i.e. patient
medical IS/IT in general (e.g. including ePR, clinical decision service, end-user service, organizational efficiency, diag-
support and computerized provider order entry). nostic efficacy and communication efficacy. For the sake
A final remark is that in our study, we might have encoun- of simplicity, these constructs were left out;
tered common method variance (CMV, a subset of method 4. The third-order construct, labeled as PACS performance,
bias)3, which is a common phenomenon in survey research as related to the underlying second-order constructs (i.e.
and can cause problems (e.g. with construct validity). This can step 3).
specifically occur when respondents rate survey items at the
same point in time, and both exogenous and endogenous
constructs are self-perceived by the same respondents. There
still is little consensus about the extent of common method
References
biases and variances or the seriousness of these effects.
In summary, the current study validated a theorized PACS
performance framework. We argue that the adopted complex- 1. Huang HK: Some historical remarks on picture archiving and
communication systems. Comput Med Imaging Graph 27:93–
ity perspectives are crucial for explaining PACS performance 99, 2003
and systematically identifying improvement areas within hos- 2. Andriole KP, Khorasani R: Implementing a replacement PACS:
pital operations as well as being aligned with the situational issues to consider. J Am Coll Radiol 4(6):416–418, 2007
context of hospital strategies. In practice, the validated PACS 3. Van de Wetering R, et al: A balanced evaluation perspective: picture
archiving and communication system impacts on hospital workflow.
performance framework is a useful checklist to systematically J Digit Imaging 19(Suppl. 1):10–17, 2006
identify improvement areas for hospitals in the PACS domain. 4. Venkatraman N: The concept of fit in strategy research: towards
verbal and statistical correspondence. Acad Manage Rev 14(3):
423–444, 1989
Acknowledgments The authors wish to thank participants from all 5. Gibson CF, Nolan RL: Managing the four stages of EDP growth.
hospitals for their kind cooperation and so made this study possible. In Harv Bus Rev 52(1):76–88, 1974
particular, we wish to thank Dr. P.M. Algra of the radiology department of 6. Van de Wetering R, Batenburg RS: A PACS maturity model: a
the Alkmaar Medical Centre, The Netherlands, for his enthusiasm for our systematic meta-analytic review on maturation and evolvability of
research, and support and assistance in stimulating higher response rates. PACS in the hospital enterprise. Int J Med Inform 78(2):127–140,
2009
3
Principle component analysis (using SPSS v18) among all manifest 7. Edgeworth F: Mathematical physics: an essay on the applica-
variables of the model showed that multiple components/factors were tion of mathematics to the moral sciences. C. Kegan Paul &
present, making CMV unlikely. Co., London, 1881
 ROGIER ET AL.

8. Milgrom P, Roberts J: Complementarities and fit strategy, structure, 18. Jöreskog K: Structural analysis of covariance and correlation matri-
and organizational change in manufacturing. J Account Econ 19(2– ces. Psychometrika 43(4):443–477, 1978
3):179–208, 1995 19. Marcoulides GA, Saunders C: PLS: a silver bullet? MIS Quart 30(2):
9. Henderson JC, Venkatraman N: Strategic alignment: leveraging in- 3–9, 2006
formation technology for transforming organisations. IBM Syst J 20. Tenenhaus M, et al: PLS path modeling. Comput Stat Data An 48(1):
32(1):4–16, 1993 159–205, 2005
10. Chan Y, Reich B: IT alignment: an annotated bibliography. J Inform 21. Fornell C, Bookstein F: Two structural equation models: LISREL and
Technol 22(4):316–396, 2007 PLS applied to consumer exit-voice theory. J Marketing Res 19(4):
11. Scheper WJ: Business IT Alignment: solution for the productivity 440–452, 1982
paradox (in Dutch). Deloitte & Touche, The Netherlands, 2002 22. Henseler J, Fassott G: Testing moderating effects in PLS path models:
12. Scott Morton MS: The corporation of the 1990s: information tech- an illustration of available procedures. Handbook of Partial Least
nology and organizational transformation. Oxford Press, London, Squares. New York: Springer-Verlag, 2010, pp 713–735
1991 23. Faul F, et al: G*Power 3: a flexible statistical power analysis program
13. Benbya H, McKelvey B: Using coevolutionary and complexity the- for the social, behavioral, and biomedical sciences. Behav Res
ories to improve IS alignment: a multi-level approach. J Inform Methods 39(2):175–191, 2007
Technol 21(4):284–298, 2006 24. Cohen J: Statistical power analysis for the behavioral sciences. New
14. McKelvey B: Complexity theory in organization science: seizing the Jersey: Lawrence Erlbaum, 1988
promise or becoming a fad? Emergence 1(1):5–32, 1999 25. Jarvis C, MacKenzie S, Podsakoff P: A critical review of construct
15. Van de Wetering R, Batenburg R: Defining and formalizing: a syn- indicators and measurement model misspecification in marketing and
thesized review on the multifactorial nature of PACS performance. consumer research. J Consum Res 30(2):199–218, 2003
Int J Comput Assist Radiol Surg 5(Suppl. 1):170, 2010 26. Wetzels M, Odekerken-Schröder G, Van Oppen C: Using PLS path
16. Bagozzi R: Expectancy-value attitude models: an analysis of critical modeling for assessing hierarchical construct models: guidelines and
theoretical issues. Int J Res Mark 2(1):43–60, 1985 empirical illustration. MIS Quart 33(1):177–195, 2009
17. Hughes M, Price R, Marrs D: Linking theory construction and theory 27. Sarstedta M, Ringleb C: Treating unobserved heterogeneity in PLS
testing: models with multiple indicators of latent variables. Acad path modeling: a comparison of FIMIX-PLS with different data
Manage Rev 11(1):128–144, 1986 analysis strategies. J Appl Stat 37(8):1299–1318, 2010