Available online at www.sciencedirect.

com

ScienceDirect
Procedia Computer Science 37 (2014) 253 – 260

The 4th International Conference on Current and Future Trends of Information and
Communication Technologies in Healthcare (ICTH-2014)

Reducing Unnecessary Laboratory Testing Using Health

Informatics Applications: A Case Study on a Tertiary Care Hospital
Mohamed Khalifa, MD * and Parwaiz Khalid, PhD
King Faisal Specialist Hospital & Research Center, Jeddah 21499, Saudi Arabia

Abstract

BACKGROUND: Like many healthcare resources, laboratory testing has been over-utilized for years with a huge number of
unnecessary lab tests being done. The elimination of unnecessary laboratory testing is becoming more and more important in the
control and management of the rapid growth of healthcare costs. OBJECTIVES: To develop a quantitative tool to identify
unnecessary laboratory tests, based on quantitative over-utilization, and recommend a plan of control based on results and
findings. METHODS: The study used the data warehouse of a tertiary care hospital to extract relevant information about
laboratory tests ordered over a specific period of time then used statistical analysis to detect frequency of ordering lab tests to
highlight both tests and users who are adding loads to the laboratory testing process and are potential for improvement with
different methodologies and approaches. RESULTS: The study identified that more than 11% of ordered tests are repeated, over-
utilized and simply unnecessary and could be eliminated. 3 tests only; Complete Blood Count, Renal Profile and Blood Glucose
constitute 35% of all hospital inpatient lab tests. 10% of ordering physicians were responsible for the actual over-utilization of
the lab testing. RECOMMENDATIONS: The study recommended two types of approaches; a user approach and a system
approach, where user approach includes different types of orientation, education and training of physicians and other users on the
importance and ways of decreasing unnecessary lab test ordering, mainly through avoiding unnecessarily repeated tests, while
system approach includes the implementation of different computerized clinical decision support interventions that would help
during the order entry process to alert and remind users with the potential of ordering an unnecessarily repeated lab test.
© 2014 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
© 2014 The Authors. Published by Elsevier B.V.
(http://creativecommons.org/licenses/by-nc-nd/3.0/).
Peer-review under
Peer-review underresponsibility
responsibilityofofthe
theProgram
ProgramChairs
Chairsof of
ICTH-2014.
EUSPN-2014 and ICTH 2014.
Keywords: Health Informatics; Reducing Unnecessary Laboratory Testing; Laboratory Over-utilization; Hospitals

* Corresponding author. Tel.: +966564520916; fax: +966126677777.

E-mail address: [email protected]

1877-0509 © 2014 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/3.0/).
Peer-review under responsibility of the Program Chairs of EUSPN-2014 and ICTH 2014.
doi:10.1016/j.procs.2014.08.038
254 Mohamed Khalifa and Parwaiz Khalid / Procedia Computer Science 37 (2014) 253 – 260

1. Introduction

Like many other healthcare resources, laboratory testing has been over utilized for many years with a huge
number of unnecessary lab tests being done routinely in many healthcare settings. The elimination of unnecessary
laboratory testing is becoming more and more important in the control and management of the rapid growth of
healthcare costs [1]. Population worldwide are getting older with a growing need for more sophisticated healthcare
techniques, in addition, technology innovations are leading to new and expensive diagnostic and therapeutic methods
and options which will lead eventually to a greater rise in healthcare costs [2], this is why the burden of health care
expenditure on national budgets for most world countries has increased dramatically over the last ten years [3].

There are widely variable test ordering patterns at different sites for similar patient populations. Several studies
had observed that test ordering sometimes varies by the day of the week even though the patient population remains
constant and that there is also some variability in individual physician test ordering or to determine the number of
tests necessary for diagnosis and patient management. Further complicating this issue is the apparent lack of
agreement about what constitutes appropriate laboratory testing [4,5].

Physician ordering practices have been analyzed extensively in many studies, the inappropriate test ordering
found to be a primary reason for increased laboratory use [6]. Over ordering may be the result of healthcare
professionals’ inexperience or lack of knowledge about the appropriate use of tests [7], failure to check previous
results due to huge or messed-up patient files [8], test ordering routines that are difficult to change due to a non-user-
friendly electronic medical record system, or fear of errors of omission and litigation [9]. Moreover, patients actively
ask for tests and often attach greater value to test results than is justified [10].

World-wide attempts to cut unnecessary laboratory testing have not documented sustained results. Several
measures can help to reduce unnecessary laboratory testing, among these; studies confirmed that advising doctors
about rational use of clinical laboratory is effective but not sufficient [1]. Educational efforts directed at changing
physician practice have clearly demonstrated a 25% or smaller decrease in laboratory test ordering, although such
decreases are transient and time-limited [11]. Changes in requisition design have had a more durable effect but are
labor-intensive to design and require dedicated subspecialty expertise [12].

The challenges are actually found in all of the stages and domains of this improvement project, starting from
identifying what we do really mean by unnecessary laboratory testing and setting criteria to identify, evaluate and
highlight those tests, which have no direct positive impact on patient outcomes. We need to set standards for the
proper testing and the proper frequency of repeating tests, depending on the specialty of medicine and also on the
patient’s clinical condition and case severity [13]. Then we need to go through suggesting and planning different
approaches and methodologies to manage these unnecessary tests, mainly by using health informatics applications
and clinical decision support systems, and finally we need to implement such methodologies and measure their
short-term as well as long term results in the form of achievable reduction of such tests [14].

In this case study at King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia, about one million
laboratory tests were done in 2012 and a larger number, about 5 to 10% more, was expected to be ordered by
different hospital departments and services by the end of 2013. This rate was actually increasing over the last few
years by 5 – 10% each year, which evoked the necessity to study laboratory test ordering behaviors and patterns.
One of the critical issues in this increasing rate is the pattern of utilization of different tests; such as increased
frequency of ordering of tests, repeating some tests and test over-utilization. This is why the focus of the study was
mainly on the quantitative over-utilization; unnecessarily repeating tests. The study focused on using health
informatics applications in two phases; first discovering unnecessary laboratory tests based on quantitative analysis
of increased frequencies of test ordering, identifying quantitative over-utilization, and percentages of contribution of
ordering physicians, then suggesting a plan to reduce those tests according to the discovered patterns or potential
reasons behind repeating unnecessary tests.
 Mohamed Khalifa and Parwaiz Khalid / Procedia Computer Science 37 (2014) 253 – 260 255

2. Methodology

To collect the study data, we needed to generate a joint table from the data warehouse using multiple database
components to cover and enumerate all ordered and performed lab tests over a selected period of time, which was
the first six months of 2013 as a sample data at the time of conducting this study in July 2013. This table included
mainly basic data elements about lab tests such as lab test type, order date, order time, ordering department, ordering
physician, encounter type, patient medical record number and patient encounter number. In addition to many other
data elements, these main elements were prepared for the statistical analysis.

We needed to study the pattern of repeated tests to identify tests that were more frequently ordered and repeatedly
performed, for example, we needed to know how many CBC – Complete Blood Count – tests were ordered for the
same patient over a specific period of time, for example during the whole inpatient encounter. We needed to identify
departments and physicians who are ordering repeated tests for the same patient more frequently than other
departments or physicians to understand the factors that might contribute to the differences among departments and
physicians in test ordering behaviors. It is expected that some medical specialties, such as critical care for example,
could have a higher frequency in ordering certain tests, such as monitoring tests, since the condition of the patients in
these specialties are more severe than the others [15]. We also needed to identify test types that are more frequently
ordered by these departments or physicians.

We needed to set some comparison criteria or standards against which we can judge the level of over-utilization,
or otherwise under-utilization, of each test type, since some physicians might repeat certain tests many times during
the same inpatient encounter despite that none of these repeated tests, not even the first one, showed abnormality
[16]. Like many other studies, the decision was made to use our own data to set these standards for each test type
and for each department or medical specialty [17]. So an ordering index was developed and calculated for each test
type, medical specialty and physician, it was a ratio that could reflect the utilization level of each lab test by different
departments and physicians, it was used to compare departments, sections and physicians to each other. The ordering
index was calculated by dividing the total number of tests performed by the total number of patients who had their
encounters during the studied period of time.

We also needed to identify if this over-utilized test or repeated test order was done by the same physician for the
same patient or done by another physician for the same patient, since this will have an impact on the plan and
suggested actions to decrease the frequency of ordering, because some studies confirmed that timely presentation of
previous test result to physicians, can easily reduce the re-ordering of those tests [8]. Was this repeated order done
for the same patient during the same encounter or during a different subsequent encounter, another inpatient or
another outpatient encounter? Outpatient tests should not be ordered for inpatients. What is the general average of
repeating tests for each test that we have considering one patient? Most of these questions were answered during the
analysis of the data.

3. Results

Data were retrieved according to the research criteria, cleaned and validated for any redundancy or bad content.
Descriptive statistical analysis was performed on the collected data elements and results were generated in the form
of tables and reports. 537,177 laboratory tests were ordered and performed during the studied period of time, six
months, from January to June 2013. Inpatient encounters were responsible for ordering 52.8% of these thousands of
tests, while outpatient encounters were responsible for about 37.3% and emergency department was responsible for
about 8.8% of laboratory tests. Only 305 test types were ordered. Since the major portion of tests has been ordered
for inpatient encounters, and since inpatients have a higher potential for repeating lab tests over their long inpatient
days of stay, and for the purpose of simplifying the analysis, we decided to focus mainly on analyzing and
calculating inpatient lab tests ordering; outpatient and ER test ordering were postponed to a later study.
256 Mohamed Khalifa and Parwaiz Khalid / Procedia Computer Science 37 (2014) 253 – 260

Table 1. Laboratory tests order source.

Encounter Type Number of Tests Percentage Cum Percent
Inpatient 283,627 52.8% 52.8%
Outpatient 200,267 37.3% 90.1%
Emergency 47,358 8.8% 98.9%
Others (e.g. Home Healthcare, Radiology …etc.) 5,925 1.1% 100.0%
Total 537,177 100.0% 100.0%

1.1%

8.8%

Inpatient
Outpatient

52.8% Emergency
37.3%
Others

Fig. 1. Laboratory tests order source.

Three test types (CBC, Renal Profiles and Random Glucose Level) were responsible for about 35% of the total
lab workload for inpatient encounters, seven test types were responsible for 60%, twelve test types were responsible
for 70%, eighteen test types were responsible for 80% and 34 test types were responsible for 90% of the total lab
workload. This sets priority when planning to focus on the most important tests, to start with the vital few!

100% 100%
90% 90%
80% 80%
70% 70%
60% 60%
50%
40% 35%
35%
30% 25%
20%
10% 10% 10% 10%
10%
0%
1 to 3 4 to 7 8 to 12 13 to 18 19 to 34 35 to 305

Fig. 2. Cumulative inpatient test numbers and percentages.

 Mohamed Khalifa and Parwaiz Khalid / Procedia Computer Science 37 (2014) 253 – 260 257

The analysis showed that 3,694 patients had inpatient encounters, some more than one, during the six months of
the study, with 283,627 lab tests ordered and performed, some patients had more than a thousand tests of course
during their long inpatient encounters, so the average of number of tests ordered for each patient was 76.8. Only
20% of patients exceeded this limit, these are mostly patients in the critical care and surgical specialties. The total of
283,627 tests performed were ordered through 62,035 test types for all inpatients, with a general ordering index =
4.6 for each test type, which means that on average any test is being ordered 4.6 times for the patient during his/her
inpatient encounter, but not necessarily all patients had the same mix of lab texts ordered.

So, if a test ordering index is near to 4.6, it means that it is on a balanced utilization, but if the test type is ordered
in an index much higher than 4.6, it means that the test type is being over utilized compared to other test types. Of
course test types are not equally utilized in hospitals, some tests need to be ordered much frequently than others,
since their resulting and monitoring values are changing physiologically and pathologically more than other tests.
Some tests are needed to be done every 10 or 15 minutes in certain patient conditions, but at least this can give us an
idea about the tests that we should focus on to achieve better utilization.

A test that has a high frequency of ordering and a high ordering index is the optimal target for this project, we
should begin with these. E.g. a “Capillary Blood Gas” test had an ordering index of 14.6 (done 1,449 times for 99
different patients) is actually much less important than “CBC” which actually had an ordering index of 9.2 (because
it was ordered 31,918 times for 3,452 different patients). Any effort done on large number tests will achieve
significant results.

Table 2. A sample of calculations of test ordering index for different test types that are above the hospital average test ordering index.
Test Patient Count Test Count Percentage Ordering Index
Blood Gas, Capillary 99 1,449 0.5% 14.6
Renal Profile 3,256 33,375 11.8% 10.3
Glucose, Random 3,257 33,299 11.7% 10.2
Bone Profile 2,170 20,874 7.4% 9.6
CBC 3,452 31,918 11.3% 9.2
Blood Gas, Arterial 874 7,097 2.5% 8.1
CSA Level 33 241 0.1% 7.3
FK506 Level 128 934 0.3% 7.3
Total Plasma Exchange 14 100 0.0% 7.1
Albumin Level 1,655 11,231 4.0% 6.8
Alkaline Phosphatase 1,645 11,137 3.9% 6.8
WBC Differential 2,215 14,969 5.3% 6.8
Methotrexate Level 13 87 0.0% 6.7
Platelet Pheresis Unit 122 817 0.3% 6.7
PTT 2,873 18,670 6.6% 6.5
PT 2,872 17,954 6.3% 6.3
Blood Gas, Venous 519 3,184 1.1% 6.1
Hepatic Profile 2,279 13,331 4.7% 5.8
Total 62,035 283,627 100.0% 4.6

Tests that have high ordering index, above the hospital average, but still have a small percentage of contribution
to the total inpatient ordered tests should be excluded from being a good candidate for a CDS rule, e.g. “Total
Plasma Exchange” and “Methotrexate Level” both have a high ordering index but were ordered less than a 100 times
each; with less than 0.1% contribution. Tests that have a high ordering index, above the hospital average and, at the
same time, have a high percentage of contribution to the total inpatient ordered tests are very good candidates for
developing a specific CDS rule to remind or alert users about their high ordering frequency, e.g. “Renal Profile”,
“Glucose, Random”, “Bone Profile” and “CBC”. Some tests showed under-utilization, with a low ordering index,
this might need future analysis. To keep the focus on the simple quantitative analysis, the study did not include the
258 Mohamed Khalifa and Parwaiz Khalid / Procedia Computer Science 37 (2014) 253 – 260

qualitative dimension of considering test cost, which might have a significant positive impact on reducing tests that
are very expensive even if they are not ordered very frequently. This objective has been planned into a later study.

Using a type of Pareto analysis to see which department, specialty or physician should be approached first; results
showed that 10% of physicians significantly exceeded their department/specialty ordering index for tests, through
comparing the ordering index of each physician to the average and median ordering index of his/her colleagues in
the same medical specialty. Again, it might be by chance or due to certain preferences or experience differences that
certain level of patient condition acuity or severity are selected by or for certain treating physicians, so it is just a
rough estimate to say that a certain physician is ordering more laboratory tests for his patients than another physician
at the same department. A second phase of analysis can demonstrate which specific tests are repeated more by which
physicians; studying only the high index physicians for their detailed pattern or repeating tests. This might help to
explain why certain tests are ordered more often and/or why certain physicians are ordering more often.

Now we have the 5% most frequently ordered tests and the 10% most frequently ordering physicians, who are
actually increasing the hospital lab work by 11%, if we assumed that they were ordering lab tests the same rate like
their department colleagues, third of these physicians (3.3%) are responsible for 7% increase in the lab workload,
these are our target users. We can now work on ideas and solutions to reduce unnecessary lab tests.

Table 3. A sample table to compare ordering index of the top ordering physicians to their specialties, showing percentage of increased tests
and calculated over-utilization above the average for their specialties.
Physician Unnecessary
Section Index Physician Patients Tests Increase
Index Tests
Medical Oncology 18.9 Physician 1 110 3,251 29.6 56.6% 1,175
Adult Critical Care 16.2 Physician 1 72 2,463 34.2 111.1% 1,296
Physician 2 84 2,859 34 109.9% 1,497
Neonatology/Perinatology 12.6 Physician 1 62 1,422 22.9 81.7% 640
Internal Medicine 12.2 Physician 1 246 5,421 22 80.3% 2,415
Physician 2 195 4,133 21.2 73.8% 1,755
Physician 1 37 3,428 92.6 671.7% 2,984
Physician 2 29 2,325 80.2 568.3% 1,977
Physician 3 39 3,039 77.9 549.2% 2,571
Pediatric Hematology 12 Physician 4 33 2,490 75.5 529.2% 2,094
Physician 5 7 375 53.6 346.7% 291
Physician 6 28 648 23.1 92.5% 311
Physician 7 27 529 19.6 63.3% 205
Cardiac Surgery 11.7 Physician 1 102 2,625 25.7 119.7% 1,430
Nephrology 9.4 Physician 1 102 1,713 16.8 78.7% 755
Gynecology/Gynecologic 8.9 Physician 1 75 1,712 22.8 156.2% 1,044
General Pediatrics 8.5 Physician 1 161 2,667 16.6 95.3% 1,301
Adult Cardiology 8.3 Physician 1 219 2,948 13.5 62.7% 1,136
Reproductive 8.2 Physician 1 30 373 12.4 51.2% 126
Neurosurgery 7.2 Physician 1 67 864 12.9 79.2% 382
Physician 2 133 1,447 10.9 51.4% 491
All Hospital 10.8 All 21,801 235,440 10.8 11% 25,876

4. Recommendations

Laboratory tests are being ordered through the hospital information system and the computerized physician order
entry systems available for the users, this leads the planning into two approaches; one from the user side, the
ordering physicians, and the other from the system side, the hospital information system and the computerized order
entry system. Both approaches could be implemented simultaneously. Base line measurements are already provided
and a monthly measurement – for comparison and monitoring of the effects – should be made after the
implementation of the suggested approaches, to see if the solutions are effective and to what extent and rate.
 Mohamed Khalifa and Parwaiz Khalid / Procedia Computer Science 37 (2014) 253 – 260 259

4.1. User Approach

This approach depends mainly on modifying user’s behavior regarding ordering tests in general or ordering
specific tests through highlighting physicians who have the highest ordering index and the highest patient and test
numbers; those are the top priority users. In phase one, we should approach the top 3.3% of ordering physicians and
phase two should approach the top 10% of ordering physicians. Target physicians should be approached for
orientation and education about the importance of eliminating unnecessary laboratory testing and the impact of this
over-utilization on both the managerial and the clinical sides of the healthcare process, such as the implications of
unnecessary testing on the quality of care, which proved effective in some studies [18], or by directly educating
physicians about the costs of the unnecessary repeated tests that could have been avoided [19], physicians should
always keep in mind the value of the diagnostic tests, which means to assess whether a test provides healthcare
benefits that are worth its costs or harms [20]. User approach should also include using printed and/or electronic
materials sent to the users, department chairmen involvement through meetings departmental meetings, grand rounds
and individualized meetings, orientation and training sessions conducted for target users. A general hospital
orientation campaign could also be very helpful.

4.2. System Approach

This approach will depend mainly on two tasks; the first will be implementing specific computerized clinical
decision support interventions and system alerts, since the implementation of specific clinical decision support
interventions in the CPOE can help to reduce unnecessary lab testing through enhancing the appropriate use of
laboratory tests and other diagnostics studies [21], to alert the users when they are re-ordering a specific test again or
when they are ordering this test more than the average frequency exceeding their own specialty, department or
section average ratio or the specified pre-determined ratio. The second task will include suggestions to change
existing system components to help decreasing ordering group tests and frequent re-ordering of tests.

For the first task, a group of the medical informatics department was assigned to work on developing rules for the
alerts of repeated ordering of tests based either on outside benchmarks; identified international standards, best
practice or scientific evidence on literature or an inside benchmark; using average test ordering index for each
specialty, department and section through available hospital data on data warehouse. The system should show the
user the date and time of the last ordered and performed test of the same type when the user is ordering a new test.
Decision rules to define appropriate intervals at which repeat tests might be indicated for commonly ordered
laboratory tests have proved their success in safely reducing laboratory test over-utilization [22].

This CDS intervention implementation was planned to be phased according to the tests priorities and workloads
as mentioned above; where phase one should include the top 3 tests, 35% of the total workload, phase two should
include the top 12 tests, 70% of the total workload and phase three should include the top 34 tests, 90% of the total
workload. User orientation about the newly implemented CDS intervention should be planned to make them aware
of the functions and advantages of the new alerts and how to work with these alerts and use them properly.

For the second task, another group of the medical informatics department was assigned to work on conducting
some system changes, mainly in the form of modifying test ordering screens to replace grouped test panels
(Predefined Multi-Test Panels) with individualized test ordering check boxes and drop-down lists, which can have a
great impact on reducing the number of unnecessary tests ordered through the panel [23]. Another modification was
to use disease specific test ordering guidelines and ordering pathways, which can be effective in target areas or for
target diseases, many institutes embedded specific disease treatment clinical guidelines into ordering pathways
[24,25]. In case of ordering a repeated tests, or recently done test, a clinical justification, or rationale, text box is to
be added to the order to make it done, otherwise the system would not allow the user to repeat or re-order the test,
this helps the user to explain the reason behind re-ordering any test and will also help to minimize this type of
orders.
260 Mohamed Khalifa and Parwaiz Khalid / Procedia Computer Science 37 (2014) 253 – 260

References

1. Ohkubo, A. (1995). Reducing unnecessary laboratory use through education and regulation. Rinsho byori The Japanese journal of clinical
pathology, 43(7), 647-652.
2. Bailey, J., Jennings, A., & Parapia, L. (2005). Change of pathology request forms can reduce unwanted requests and tests. Journal of Clinical
Pathology, 58(8), 853-855. Copyright 2005 Journal of Clinical Pathology.
3. Vegting, I. L., Van Beneden, M., Kramer, M. H. H., Thijs, A., Kostense, P. J., & Nanayakkara, P. W. B. (2011). How to save costs by
reducing unnecessary testing: lean thinking in clinical practice. European Journal of Internal Medicine, 23(1), 70-5. European Federation of
Internal Medicine.
4. Flabouris, A., Bishop, G., Williams, L., & Cunningham, M. (2000). Routine blood test ordering for patients in intensive care. Anaesthesia
and intensive care, 28(5), 562-565.
5. Larsson, A., Palmer, M., Hultén, G., & Tryding, N. (2000). Large differences in laboratory utilization between hospitals in Sweden. Clinical
chemistry and laboratory medicine CCLM FESCC, 38(5), 383-389.
6. Kwok, J., & Jones, B. (2005). Unnecessary repeat requesting of tests: an audit in a government hospital immunology laboratory. Journal of
clinical pathology, 58(5), 457-462.
7. Wong, E. T. (1995). Improving laboratory testing: can we get physicians to focus on outcome? Clinical chemistry, 41(8 Pt 2), 1241-1247.
8. Tierney, W. M., McDonald, C. J., Martin, D. K., & Rogers, M. P. (1987). Computerized display of past test results. Effect on outpatient
testing. Annals of internal medicine.
9. Tabriz, M. S., Riederer, K., Baran, J., & Khatib, R. (2004). Repeating blood cultures during hospital stay: practice pattern at a teaching
hospital and a proposal for guidelines. Clinical microbiology and infection: the official publication of the European Society of Clinical
Microbiology and Infectious Diseases, 10(7), 624-627.
10. McDonald, I. G., Daly, J., Jelinek, V. M., Panetta, F., & Gutman, J. M. (1996). Opening Pandoraʼs box: the unpredictability of reassurance
by a normal test result. BMJ (Clinical research ed.), 313(7053), 329-332.
11. Durieux, P., Ravaud, P., Porcher, R., Fulla, Y., Manet, C.-S., & Chaussade, S. (2003). Long-term impact of a restrictive laboratory test
ordering form on tumor marker prescriptions. International journal of technology assessment in health care, 19(1), 106-113.
12. Emerson, J. F., & Emerson, S. S. (2001). The impact of requisition design on laboratory utilization. American Journal of Clinical Pathology,
116(6), 879-884.
13. May, T. A., Clancy, M., Critchfield, J., Ebeling, F., Enriquez, A., Gallagher, C., Genevro, J., et al. (2006). Reducing unnecessary inpatient
laboratory testing in a teaching hospital. American journal of clinical pathology, 126(2), 200-206.
14. Roshanov, P. S., You, J. J., Dhaliwal, J., Koff, D., Mackay, J. A., Weise-Kelly, L., Navarro, T., et al. (2011). Can computerized clinical
decision support systems improve practitioners’ diagnostic test ordering behavior? A decision-maker-researcher partnership systematic
review, 6, 88. BioMed Central.
15. Bugter-Maessen, A. M., Winkens, R. A., Grol, R. P., Knottnerus, J. A., Kester, A. D., Beusmans, G. H., & Pop, P. (1996). Factors predicting
differences among general practitioners in test ordering behavior and in the response to feedback on test requests. Family practice, 13(3),
254-258.
16. Weydert, J. A., Nobbs, N. D., Feld, R., & Kemp, J. D. (2005). A simple, focused, computerized query to detect overutilization of laboratory
tests. Archives of pathology & laboratory medicine, 129(9), 1141-1143.
17. Eisenberg, J. M., Williams, S. V., Garner, L., Viale, R., & Smits, H. (1977). Computer-based audit to detect and correct overutilization of
laboratory tests.Medical care, 15(11), 915-921.
18. Dowling, P. T., Alfonsi, G., Brown, M. I., & Culpepper, L. (1989). An education program to reduce unnecessary laboratory tests by
residents. Academic medicine : journal of the Association of American Medical Colleges, 64(7), 410-412.
19. Attali, M., Barel, Y., Somin, M., Beilinson, N., Shankman, M., Ackerman, A., & Malnick, S. D. H. (2006). A cost-effective method for
reducing the volume of laboratory tests in a university-associated teaching hospital. The Mount Sinai journal of medicine, New York, 73(5),
787-794.
20. Qaseem, A., Alguire, P., Dallas, P., Feinberg, L. E. L. E., Fitzgerald, F. T. F. T., Horwitch, C., Humphrey, L., et al., others. (2012).
Appropriate use of screening and diagnostic tests to foster high-value, cost-conscious care. Annals of internal medicine, 156(2), 147–149.
Am Coll Physicians.
21. Levick, D. L., Stern, G., Meyerhoefer, C. D., Levick, A., & Pucklavage, D. (2013). “Reducing unnecessary testing in a CPOE system through
implementation of a targeted CDS intervention”. BMC medical informatics and decision making, 13, 43.
22. Schubart, J. R., Fowler, C. E., Donowitz, G. R., & Connors, A. F. (2001). Algorithm-based decision rules to safely reduce laboratory test
ordering. Studies in health technology and informatics, 84(Pt 1), 523-527.
23. Pysher TJ, Bach PR, Lowichik A, et al. Chemistry test ordering patterns after elimination of predefined multitest chemistry panels in a
children's hospital. Pediatr Dev Pathol. 1999;2:446-453.
24. Chin, H. L., & Wallace, P. (1999). Embedding guidelines into direct physician order entry: simple methods, powerful results. Proceedings /
AMIA ... Annual Symposium. AMIA Symposium, 221-225.
25. Brown, S. R., & Brown, J. (2011). Why do physicians order unnecessary preoperative tests? A qualitative study. Family medicine, 43(5),
338-343.