Migrating Clinical Data in Various Formats to a Clinical Data Management System
Download as PPTX, PDF1 like2,418 views
This document summarizes the process of migrating clinical data from various legacy formats, including SAS datasets, Word/PDF listings, Excel files, and scans, into a clinical data management system (CDMS). A team was assembled with roles for building the studies, loading the data, quality control, and data entry. The overall process involved building the studies, parsing the source data into a loadable format, quality checks, and loading the data. Lessons learned focused on standardizing the data format versus loading "as is" and improving communication and consistency.
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Migrating Clinical Data in Various Formats to a Clinical Data Management System (20)
Ad
More from Perficient, Inc. (20)
Recently uploaded (20)
Migrating Clinical Data in Various Formats to a Clinical Data Management System
- 1. Migrating Clinical Data in Various Formats to a Clinical Data Management System Tammy Dutkin, Director of Clinical Data Management, Life Sciences, Perficient Michelle Engler, Senior Application Architect, Life Sciences, Perficient facebook.com/perficient linkedin.com/company/perficient twitter.com/Perficient_LS
- 2. About Perficient Perficient is a leading information technology consulting firm serving clients throughout North America and Europe. We help clients implement business-driven technology solutions that integrate business processes, improve worker productivity, increase customer loyalty and create a more agile enterprise to better respond to new business opportunities.
- 3. • Founded in 1997 • Public, NASDAQ: PRFT • 2013 revenue ~$373 million Perficient Profile • Major market locations throughout North America • Atlanta, Boston, Charlotte, Chicago, Cincinnati, Cleveland, Columbus, Dallas, Denver, Detroit, Fairfax, Houston, Indianapolis, Los Angeles, Minneapolis, New Orleans, New York City, Northern California, Philadelphia, Southern California, St. Louis, Toronto and Washington, D.C. • Global delivery centers in China, Europe and India • >2,100 colleagues • Dedicated solution practices • ~85% repeat business rate • Alliance partnerships with major technology vendors • Multiple vendor/industry technology and growth awards
- 4. Life Sciences Practices / Solutions Deep Clinical and Pharmacovigilance Applications Expertise Implementation Migration Integration Validation Consulting Upgrades Managed Services Application Development Private Cloud Hosting Application Support Sub-licensing Study Setup Services Clinical Trial Management Clinical Trial Planning and Budgeting Oracle ClearTrial CTMS Oracle Siebel CTMS / ASCEND Mobile CRA Clinical Data Management & Electronic Data Capture CDMS Oracle Clinical Electronic Data Capture Oracle Remote Data Capture Oracle InForm Medical Coding Oracle Thesaurus Management System Safety & Pharmacovigilance Adverse Event Reporting Oracle Argus Safety Suite Oracle AERS / EmpiricaTrace Axway Synchrony Gateway Signal Management Oracle Empirica Signal/Topics Medical Coding Oracle Thesaurus Management System Clinical Data Warehousing & Analytics Clinical Data Warehousing Oracle Life Sciences Data Hub Clinical Data Analytics Oracle Clinical Development Analytics JReview Data Review and Cleansing Oracle Data Management Workbench Clients
- 5. Welcome & Introduction Tammy Dutkin Director of Clinical Data Management & EDC Life Sciences, Perficient Michelle Engler Senior Solutions Architect Life Sciences, Perficient
- 6. Agenda • Data Migration from CRO to OC Business Case • Team • Overall Process • Quality Control • Parsing Data into BDL Format • Converting PDF/Word Listings to BDL Format • Lessons Learned • Contact Information and Questions
- 7. Data Migration from CRO to Oracle Clinical Business Case • Client had 24 legacy studies in various formats from their CROs and partners. • It was not possible to obtain SAS dataset files or further data transfers for all of these legacy studies. • Background information for many of the studies was limited. In many cases the protocol and clinical study report were all that were available. • The legacy studies were available in various formats: – SAS dataset files – Word (or Word converted to PDF) listings and Clinical Study Reports – Scans of SAS listings – Excel files • Client had Oracle Clinical and wanted all of their data stored within their production instance of Oracle Clinical for future cross analyzes and reporting.
- 8. Team Role Responsibilities and Experience OC Study Builders • Built the study objects in Oracle Clinical (OC) • Handled the annotations and mapping documents • Had 8+ years of experience building studies in OC OC Study Loaders • Used the mapping documents and the Excel files to load the data into OC • Had 8+ years of experience building studies in OC and loading data with the batch data loader Technical Experts • Handled the parsing of the Word/PDF listings into Excel Data Managers / Quality Control • Performed the QC of the study build, load, and final quality control Summary report (Note: often the same person(s) building the studies) Data Entry • Performed data entry on those listings that could not be parsed/loaded • Had 5-10 years of DM and entry experience Project Manager • Managed the timelines, quality, and process • Reviewed all documents prior to sending to client
- 9. Overall Process The process followed was dependent on the type of file available. • For studies where the SAS datasets exported into Microsoft Excel were available, the process included study setup in OC, creating maps between the Microsoft Excel file data extractions and the OC Study, pre-processing the Microsoft Excel file data using the existing Generic Data Parser program and finally, loading the data into OC. • For those studies where PDF or Word listings are available, the data was first parsed into Excel format and then followed the process of building, mapping, preprocessing and loading. • For the studies where only scans of the listings were available, the study was setup in OC and then double data entered into the database. Quality control was performed at each step
- 10. Quality Control • QC was built into every major step of the process – Builder Peer Review (checklist) • Study Design / Study Definition • Documentation (annotations, assumptions, mapping, DVG document) – Parsing Review (if applicable) – Loader Peer Review (checklist) • Discrepancy check • Verify Source data vs. loadable Excel File • Data Extract views • Data spot checking • Data entry browse checking – QC Summary Report • Response Counts and DCI Counts done via SQL • Spot Checks (5 records checked from the original source file to OC) • Any assumptions made during the build or the load/entry was documented in a Study Assumptions document
- 11. Listing Formats and Processing Word (or Word converted to PDF) Listings 1. Convert to TXT and split into 1 doc per page 2. Parse data into Excel format 3. Parse to BDL format 4. Load into Oracle Clinical Txt Listings 1. Split into 1 doc per page 2. Parse Data into Excel format 3. Parse to BDL format 4. Load into Oracle Clinical SAS Listings 1. Convert to Excel 2. Parse to BDL format 3. Load into Oracle Clinical Excel Listings 1. Parse to BDL format 2. Load into Oracle Clinical Scans of SAS Listings Perform 1st and 2nd pass data entry on the listings
- 12. Converting PDF/Word Listings to BDL Format • Data format comes in PDF or Word listings • Listings had to be split into 1 document per page • Listings had to be converted to TXT format • Simple Java Converter was written to perform this task Simple Java PDF->TXT Converter Simple Java RTF->TXT Converter
- 13. Converting PDF/Word Listings to BDL Format C:16 C:20 C:59 C:77 C:85 C:94 C:115 C:125 • Wrapping columns of data • Fields split by slash e.g. start date/stop date • Multiple records for a single patient without the patient # repeating • Key to parsing this type of listing is to identify patterns – What signifies the start or end of the data? – Where do I get the patient number? – How can I determine when a field is wrapping vs. a new record? – What column positions constitute a new response?
- 14. Converting PDF/Word Listings to BDL Format • Case #1: Additional decoding information e.g. 1 = Yes, 2= No, then the information was incorporated using the DVG Long Value • Case #2: Comment was for a particular field for all patients, then the field was added and loaded the footnote for every single record • Case #3: Comment was just for a specific patient, then the comment was hand entered for the patient in a comment field in the question group • Case #4: If the footnote was giving unit information (e.g. weight in lbs), then it was incorporated into the SAS Label within the OC Question Definition • The handling of the footnotes was put in an Assumptions doc that was recorded for each study
- 15. Converting PDF/Word Listings to BDL Format Configuration file used to communicate details for the listing: • Column cutoff positions • Heading titles • Patient number headings, location, and amount of fields to split • What the patient number consisted of • What signifies the start of the parsing • What signifies the end of the parsing • Other keys that the parser would need for interpreting the file: – Keep trail for – Max rows allowed per record – What type of delimiter separates the patient fields – Etc.
- 16. Converting PDF/Word Listings to BDL Format C:16 C:20 C:59 C:77 C:85 C:94 C:11 5 C:12 5 TXT Converter (custom utility created by Perficient) TXT Listing Configuration File Excel File for GDP Input
- 17. Parsing Data into BDL Format • Data must be in Excel file format • Create mapping file to define relationship between OC and data file • Create Lists file for mapping data values • Run the generic data parser to output the BDL file • Load BDL file into Oracle Clinical Lists Data Map Generic Data Parser BDL File Oracle Clinical
- 18. Parsing Data into BDL Format Generic Data Parser Mapping File
- 19. Parsing Data into BDL Format Generic Data Parser Mapping File Lists Map Data Generic Data Parser Batch Data Load File (fixed format ASCII Text File) Oracle Clinical
- 20. Lessons Learned • Confusion about whether to format the studies to a standard or load them “as is” – The decision was made to load the data “as is” and using any data standards the legacy study had available. – Although it would have taken much longer, converting the data into company/CDISC standards would have been beneficial. • Experienced communication and consistency Issues which were addressed with: – Weekly status meeting – Added checklists, templates and process flowcharts – Keeping running issue Log – Communication and process had to be consistent and formalized in order to ensure success • Flexibility required – In some cases, the data could not be effectively parsed/loaded; therefore, double data entry was the better option – New Data Challenges: Column wrapping was out of alignment with column headings – the parsing had to be able to be separated by column positions as opposed to column headings. Programmatic changes had to be made as they were needed. • Consistency Checks – 100% verification manual quality checks were performed at several points to ensure that the output of a parsing prior to loading were crucial to ensure success
- 21. Questions?
- 22. Thank You! For more information, please contact: [email protected] [email protected] [email protected] (Sales) +1 303 570 8464 (U.S. Sales) +44 (0) 1865 910200 (U.K. Sales) www.perficient.com www.facebook.com/perficient www.twitter.com/perficient_LS
Editor's Notes
- #3: For those of you who might not know, Perficient is a large IT consulting organization. Actually, we serve throughout North-America and Europe and really our strategy and goal is to help provide business solutions and technologies for companies to be able to be more efficient in what they do on the daily basis. One of our primary goals is to work with you not only to deliver solution, but also to really become a partner with you. We don’t want to implement something and walk away; we want to continue to be there to help provide solutions for you; to help you meet your business needs. Perficient was founded in 1997. Again, you’ll see that we have many offices throughout the United States. We also have some global, offshore resources in different countries, as well. Again, repeat business is one of our huge things that we like to talk about, because we want to build partnerships with organizations. So, little bit about Life Sciences. A lot of you probably know the name, “BioPharm Systems”, and have heard it for years. We were acquired by Perficient on April 1st of this year, and now we are the Life Science Practice within Perficient’s vertical business units (some of those other ones may be healthcare etc). So, our services of what we offer are still the same, across the Life Sciences arena, in all of the different areas; anywhere from implementation to hosting, to application support… Actually, now with Oracle becoming part of Perficient, that allows us to resell Oracle’s first solutions to our customers, as well - if they would like to work with us.
- #4: For those of you who might not know, Perficient is a large IT consulting organization. Actually, we serve throughout North-America and Europe and really our strategy and goal is to help provide business solutions and technologies for companies to be able to be more efficient in what they do on the daily basis. One of our primary goals is to work with you not only to deliver solution, but also to really become a partner with you. We don’t want to implement something and walk away; we want to continue to be there to help provide solutions for you; to help you meet your business needs. Perficient was founded in 1997. Again, you’ll see that we have many offices throughout the United States. We also have some global, offshore resources in different countries, as well. Again, repeat business is one of our huge things that we like to talk about, because we want to build partnerships with organizations. So, little bit about Life Sciences. A lot of you probably know the name, “BioPharm Systems”, and have heard it for years. We were acquired by Perficient on April 1st of this year, and now we are the Life Science Practice within Perficient’s vertical business units (some of those other ones may be healthcare etc). So, our services of what we offer are still the same, across the Life Sciences arena, in all of the different areas; anywhere from implementation to hosting, to application support… Actually, now with Oracle becoming part of Perficient, that allows us to resell Oracle’s first solutions to our customers, as well - if they would like to work with us.
- #5: For those of you who might not know, Perficient is a large IT consulting organization. Actually, we serve throughout North-America and Europe and really our strategy and goal is to help provide business solutions and technologies for companies to be able to be more efficient in what they do on the daily basis. One of our primary goals is to work with you not only to deliver solution, but also to really become a partner with you. We don’t want to implement something and walk away; we want to continue to be there to help provide solutions for you; to help you meet your business needs. Perficient was founded in 1997. Again, you’ll see that we have many offices throughout the United States. We also have some global, offshore resources in different countries, as well. Again, repeat business is one of our huge things that we like to talk about, because we want to build partnerships with organizations. So, little bit about Life Sciences. A lot of you probably know the name, “BioPharm Systems”, and have heard it for years. We were acquired by Perficient on April 1st of this year, and now we are the Life Science Practice within Perficient’s vertical business units (some of those other ones may be healthcare etc). So, our services of what we offer are still the same, across the Life Sciences arena, in all of the different areas; anywhere from implementation to hosting, to application support… Actually, now with Oracle becoming part of Perficient, that allows us to resell Oracle’s first solutions to our customers, as well - if they would like to work with us.
- #6: Let me start out by introducing myself. My name is Param Singh and I am the Director of the Clinical Trial Management Solutions practice within Perficient’s Life Science Business Unit. I have been working in the industry since 1999, and have almost exclusively been working with Siebel Clinical during that time. I have been with Perficient’s Life Science BU for over 6 years, and before that I was part if Accenture’s Pharma R&D practice and leading Siebel CTMS implementations there as well. Overall I have been a part of, over 30 implementations of Siebel CTMS now. These vary from implementations fro Pharma, CROs, Medical Device companies, and also range from anywhere from 30 user to global implementations of over 4500 users. And each type and size of organization has its own approach to selecting and implementing this solution and a lot of what we will discuss is direct feedback from my clients from over the years.
- #22: We’ll now open up the lines for any questions you may have. If you have a question, the best method is to use the chat feature to submit your question and we will address the questions as we receive them. If you would like to ask your question live, As a reminder, to unmute your line, press *7 on your phone, and to mute it again, press *6. It’s up to you if you would like to introduce yourself - as a reminder, this presentation is being recorded and will be posted on our website.