Using Qualtrics to Create Automated Online Trainings
2 likes543 views
The document discusses the use of the Qualtrics research suite to create automated online trainings as part of the Big XII Teaching and Learning Conference at Kansas State University. It highlights the suite's features such as multimedia integration, customization options, data analytics capabilities, and security measures, emphasizing its potential to enhance training efficiency and accessibility. Additionally, it notes the challenges related to system limits and the need for developer support for integration with existing information systems.
Using Qualtrics to Create Automated Online Trainings
- 1. USING QUALTRICS TO CREATE AUTOMATED ONLINE TRAININGS Digital Poster Session Big XII Teaching and Learning Conference (3rd Annual) Theme: Transformational Teaching and Learning Kansas State University June 2 – 3, 2016
- 2. OVERVIEW When thinking about “transformational teaching and learning,” training would not be the first thing to come to mind. The Qualtrics® research suite offers a number of design tools and features that enable the building of automated online trainings. There are the baseline features such as the ability to integrate multimedia, apply various question designs, enable accessibility features (like alt-texting), deliver a mobile experience, reach learners across distances, and provide basic security and data integrity features. 2
- 3. OVERVIEW (CONT.) Other features actually make this tool phenomenally powerful. One is the ability to richly customize learning sequences—by learner profile, by performance (behavior), by selection, or a mix of factors. There is a feature that enables the scoring of learner responses and the ability to set a threshold for passing. This tool has a rich data analytics capability (including a light item analysis), including online analytics and even cross-tabulation analysis. A Qualtrics® API enables the recording of online assessment scores and learner behaviors, in an automated way to faculty / staff / student information systems. Trainings are critical for effective workplace functioning and professional development. The same features in Qualtrics® that enable the effective building of automated trainings also enable the effective building of pre-learning modules or sequences for learners who need to refresh their skills for a new course. This digital slideshow introduces the use of Qualtrics® as a customizable training and pre-learning module tool. 3
- 4. 4
- 5. AUTOMATED…ONLINE…TRAININGS Automated: learner and the machine, non-human-led, computer-captured details of the learning, computerized record-keeping Online: deliverable to any location on Earth (and in space) with Web connectivity Training: workplace-based teaching of skills, standards, values, behaviors, and attitudes 5
- 6. GENERAL PARTS TO AN ONLINE TRAINING Title Overview of learning objectives (observable, measurable) Overview of topics covered Authorizing regulation or policy (if relevant) Estimated length of training (time commitment) Content (information): Text, images, stories, audio, and video (with source citations) Decision-making and walk-throughs of real- world scenarios Assignments (actionable, optional) Formative questions and answers Summative assessment (with results to the learner) Attestation Feedback from learners about the training and assessment (for continuing improvement) Contact information of the training provider 6
- 7. POST-TRAINING RESOURCES Downloadables and takeaways (checklists, tip sheets, posters) Memory enhancers like mnemonics, images, and stories Refresher contents (post-training): Online sites, scenarios, decision walk-throughs Encouragement to practice-practice-practice Support and direction to continue learning (through other reputable sources) 7
- 8. METADATA ABOUT THE ONLINE TRAINING Descriptive metadata (to help tie to other trainings and training sequences, to enhance re-usability in various contexts) 8
- 9. AFFORDANCES OF THE QUALTRICS RESEARCH SUITE FOR AUTOMATED ONLINE TRAININGS 9
- 10. WHY THE QUALTRICS RESEARCH SUITE? #1: Rich design structures for experiential learning sequences #2: Multimedia-rich builds (with accessibility accommodations) #3: Connections to information systems for data tracking #4: Branching logic #5: Customizations #6: Ability to build questions on prior dynamically captured data #7: Smart data collection 10
- 11. WHY THE QUALTRICS RESEARCH SUITE? (CONT.) #8: Smart in-tool and out-tool data analytics #9: Scoring and threshold features #10: Panel triggers #11: Email triggers #12: Fast development and deployment cycles #13: On-the-fly revision #14: Ease-of-collaboration through email verified accounts and shared libraries / folders 11
- 12. WHY THE QUALTRICS RESEARCH SUITE? (CONT.) #15: Templating through libraries #16: Security features for identity-supported controls #17: Data integrity protections 12
- 13. #1: RICH DESIGN STRUCTURES FOR EXPERIENTIAL LEARNING SEQUENCES Ability to build surveys / blocks / questions and to mix-and-match Diverse range of question types Complex sequencing based on various types of dependencies (geographical location, technologies used, answers to certain questions, performance on certain questions) Customization capabilities 13
- 14. 14
- 15. #2: MULTIMEDIA-RICH BUILDS (WITH ACCESSIBILITY ACCOMMODATIONS) Ability to integrate links, iframes (inline frames), video, imagery, interactive objects, and other elements Built-in accessibility tests of the sequences Alt-texting capability for imagery 15
- 16. #3: CONNECTIONS TO INFORMATION SYSTEMS FOR DATA TRACKING Computerized record-keeping and tracking over time (legal-grade verification) using a Qualtrics API Enablement for mass-scale delivery of trainings (such as compliance trainings, which occur regularly and require records) 16
- 17. #4: BRANCHING LOGIC Ability to create various training experiences for various trainees with different needs May be based on… performance (online behavior) time spent in a certain block of learning submitted (or pre-determined) learner profile professional role (and need to acquire certain skills and knowledge sets) geographical location technologies used to access the training (particularly for cases where the underlying technologies change the information needed for the training) other factors or mixes of factors 17
- 18. #4: BRANCHING LOGIC(CONT.) May branch at any point in time in a learning sequence between blocks and block sequences May branch between surveys If used, must test every path through the training to ensure accuracy Will result in different extracted performance data at the end of the training (if the particular branching paths are scored) 18
- 19. 19
- 20. #5: CUSTOMIZATIONS Piped text function {a} to deliver unique information to various parts of the training Customization of trainings to the individual’s name and the individual’s responses 20
- 21. #6: ABILITY TO BUILD QUESTIONS ON PRIOR DYNAMICALLY CAPTURED DATA Ability to build subsequent questions on prior dynamically captured data to elicit elaborations “Loop & merge” question types enable asking a closed-answer multiple selection question prior and then using the unique selected selections to elicit textual responses about those selections “Carry forward” responses enable the repeating of responses to a prior question (including open- ended ones) in a future question for more elaboration or a new elicitation 21
- 22. 22
- 23. #7: SMART DATA COLLECTION Ability to capture a wide range of quantitative and qualitative data Capability to capture text responses Enablement to capture digital file uploads (for digital imagery, text files, audio, video, and other basic digital file formats) May designated acceptable file extensions only 23
- 24. #8: SMART IN-TOOL AND OUT-TOOL DATA ANALYTICS About the Training Analysis of the training through the Reporting -> Survey Statistics function, including light item analysis and summary performance data About Learner Performance Automated data summarization visualizations (both in the dashboard and the auto-created reports function) Easy data download Cross-tabulation analysis for certain close-ended question types 24
- 25. #9: SCORING AND THRESHOLD FEATURES Ability to score question responses based on point values Capacity to set thresholds for certain learning sequences Those who do not apparently grasp certain aspects of the training may be sent for more extensive training on-the-fly Capability to set thresholds for pass/fail (binary) or thresholds to go on various learning sequences (at a number of thresholds) Can set thresholds to populate various panels that may lead to invites to additional other trainings (with human mediation) Can set thresholds to send learners to new learning branches or whole trainings in an automated way 25
- 26. 26
- 27. #10: PANEL TRIGGERS Enablement to populate panels based on various factors, such as performance, geographical region, and other factors (as trigger conditions) 27
- 28. #11: EMAIL TRIGGERS Ability to have automated emails sent when certain trigger conditions are met: Reaching pre-set quotas (of numbers of respondents to a certain survey) When certain answers are given Email recipients may be to certain members (to control for potential spam) 28
- 29. #12: FAST DEVELOPMENT AND DEPLOYMENT CYCLES Capability of developing automated online trainings in a fast way Ability to deploy quickly in an open way (public shared URL, including semantic- friendly ones) or in a closed invitation-only way Ability to pilot-test a training and to collect information about the training on Qualtrics® Light item analysis 29
- 30. #13: ON-THE-FLY REVISION Enablement to update or change a training on-the-fly Ability to preserve the then-version of a training (vs. an updated one) for a particular trainee at the time of the first log-in 30
- 31. #14: EASE-OF-COLLABORATION THROUGH EMAIL VERIFIED ACCOUNTS AND SHARED LIBRARIES / FOLDERS Ability to invite others (even those without Qualtrics) by email to collaborate on this cloud-based tool Ability to share user-owned survey libraries / folders of trainings (surveys), blocks, and questions 31
- 32. #15: TEMPLATING THROUGH LIBRARIES Collection of original templates through user-owned survey libraries on Qualtrics Access to Qualtrics’ survey templates 32
- 33. 33
- 34. #16: SECURITY FEATURES FOR IDENTITY- SUPPORTED CONTROLS Ability to hide open trainings from web browser indexers or spiders Ability to create unique one-use URLs for trainees invited by email Ability to analyze amount-of-time spent on training to detect potential ‘bot responses and potential dishonesty (using a crib sheet or cheat sheet to finish the training quickly) Ability to prevent access to trainings from the same IP address (what is called prevention against “ballot box stuffing” but may serve a different purpose in an automated online training context) 34
- 35. #17: DATA INTEGRITY PROTECTIONS Protection of data stateside, without data moving across undersea lines to overseas data centers 35
- 36. IN SUMMARY: Not about stand-alone capabilities but about tactical combinations in order to achieve training objectives 36
- 38. AND SOME SYSTEM LIMITS… Trainings are non-portable. Trainings are downloaded as proprietary .qsf files, and linked data may be downloaded as .csv, but they require Qualtrics to reconstitute online. Currently, it is not possible to archive trainings with data in a library (only the trainings themselves can be archived, without the related data). All complex sequences have to be thoroughly tested for clear behaviors. (The branching logic visualizations could benefit from richer details, such as a full decision- tree layout vs. just a linear listing.) There could be more effective in-system analytics that focus on learner behaviors and inferences about their learning. 38
- 39. REQUIREMENT FOR DEVELOPER SUPPORT Connecting to information systems (databases) will require developer work to use the Qualtrics API for one-directional automated data sharing. This feature usually works with closed panel invites to trainings (with learner identifiers emplaced). Another option is to have an open URL training and the collection of identifiers during the training (but this may introduce some noise and some room for mis-identification of the self). 39
- 40. AND? SO? 40
- 41. THINK AFFORDANCES AND EFFICIENCIES Think about tech tools based on capabilities, not the initial design intentions of the developers or the depictions of the marketers. Integrated solutions, which address a broad range of interests (legal, record-keeping, training designer, learner), benefit the entire process. There are costs to investing in any socio-technical platform, and it helps to consider various scenarios and use cases. 41
- 42. CONCLUSION AND CONTACT Dr. Shalin Hai-Jew iTAC, Kansas State University 212 Hale / Farrell Library 785-532-5262 [email protected] Another Resource: Using Qualtrics for Online Trainings The presenter has no formal tie to Qualtrics, Inc. Thanks to Dr. Jana R. Fallin and her team at the K-State Teaching & Learning Center for accepting this digital poster session for the Big XII Teaching and Learning Conference (2016)! Thanks to Dr. Rebecca Gould for supporting this work! 42