VII Jornadas eMadrid "Education in exponential times". "Supporting higher education in integrating learning analytics". Dragan Gasevic. U Edinburgh, UK. 05/07/2017.
Download as PPTX, PDF2 likes1,109 views
The document discusses the challenges and necessary steps for higher education institutions to adopt learning analytics effectively. It emphasizes the importance of stakeholder engagement, policy development, and the establishment of a supportive framework to ensure successful integration and use of learning analytics. Key points include the need for data literacy among stakeholders and the critical role of leadership in fostering innovation and addressing ethical considerations.
VII Jornadas eMadrid "Education in exponential times". "Supporting higher education in integrating learning analytics". Dragan Gasevic. U Edinburgh, UK. 05/07/2017.
- 1. Supporting higher education in integrating learning analytics Dragan Gašević @dgasevic July 5, 2017 LASI Spain Madrid, Spain http://sheilaproject.eu/
- 2. Current state Understanding & supporting learning Moving away from deficit models
- 3. Learning analytics is about learning Gašević, D., Dawson, S., Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.
- 4. Field of research and practice Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the Learning Analytics Puzzle: A Consolidated Model of a Field of Research and Practice. Learning: Research and Practice, 3(2), 63-78. doi:10.1080/23735082.2017.1286142
- 5. Our institution is in early days of adoption
- 7. Current state – Oz and Europe http://sheilaproject.eu/http://he-analytics.com
- 8. Adoption challenge Leadership for strategic implementation & monitoring Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).
- 9. Adoption challenge Equal engagement with different stakeholders Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).
- 10. Adoption challenge Training to cultivate data literacy among primary stakeholders Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).
- 11. Adoption challenge Policies for learning analytics practice Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).
- 12. What’s necessary to move forward?
- 13. DIRECTIONS
- 14. Data – Model – Transformation Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and Distance Education. http://bit.ly/icde_la_16
- 17. Inclusive adoption process Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
- 18. SHEILA policy making framework Action – Challenges – Policy – Instruments http://sheilaproject.eu/
- 19. SHEILA policy making framework Action – Challenges – Policy – Instruments http://sheilaproject.eu/
- 20. SHEILA policy making framework Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
- 21. Step 1 – Map political context Internal and external drivers for learning analytics adoption
- 23. Step 1 – Map political context One size fits all does not work in learning analytics
- 24. Step 1 – Map political context Opportunities to build learning analytics on existing projects/practice
- 25. SHEILA policy making framework Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
- 26. Step 2 – Identify key stakeholders Primary users of learning analytics
- 27. Step 2 – Identify key stakeholders The project sponsor on the senior management team
- 28. Step 2 – Identify key stakeholders Other critical stakeholders to consider Internal – professional and academic teams External – service providers/vendors and collaborators Champions of learning analytics (bottom up)
- 29. Students’ perspective Students expect the use of their data provided ethics & privacy consideration
- 30. Students’ perspective But, they are not sure if teaching staff will know to use learning analytics
- 31. Teaching staff’s perspective Concerned about their workload
- 32. Contradictory views Students and teaching staff don’t share the same perspectives
- 33. Experts’ perspective Privacy and ethics are most important but easy to implement
- 34. SHEILA policy making framework Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
- 35. Step 3 – Identify desired behavior changes Identify areas where decisions will be informed by learning analytics
- 36. Step 3 – Identify desired behavior changes Define responsibilities and implications for primary users
- 37. Step 3 – Identify desired behavior changes Identification of possible inadvertent consequences
- 38. SHEILA policy making framework Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
- 39. Step 4 – Develop engagement strategy Alignment of learning analytics with the wider institutional strategies
- 40. Step 4 – Develop engagement strategy Secure funding, establish a working group, and raise awareness
- 41. Step 4 – Develop engagement strategy Select data that will be fed back to users
- 42. Step 4 – Develop engagement strategy How interventions will be triggered and who is responsible?
- 43. SHEILA policy making framework Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
- 44. Step 5 – Analyze internal capacity Data storage, disposal, and security evaluation
- 45. Step 5 – Analyze internal capacity Human, financial, legal, and infrastructural capacity
- 46. Step 5 – Analyze internal capacity Evaluate institutional culture Trust in data Decision-making based on data Openness to changes and innovation
- 47. SHEILA policy making framework Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
- 48. Step 6 – Establish monitoring & learning frameworks Establish qualitative and quantitative indicators of success Stage the process to recognize institutional development
- 49. Step 6 – Establish monitoring & learning frameworks Seek feedback from primary users through various channels
- 50. Step 6 – Establish monitoring & learning frameworks Recognizing and addressing limitations observed
- 51. Systemic Adoption Model Colvin, C., et al. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney: Australian Office for Learning and Teaching.
- 52. Solution-focused Model Colvin, C., et al. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney: Australian Office for Learning and Teaching.
- 53. Process-focused Model Colvin, C., et al. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney: Australian Office for Learning and Teaching.
- 54. How is innovation recognized, supported, and promoted?
- 55. FINAL REMARKS
- 56. Learning analytics principles Data incompleteness, bias perpetuation, avoidance of deficit models, facilitation of training, not used for performance assessment The University of Edinburgh (2017). Learning Analytics Policy, http://www.ed.ac.uk/academic-services/projects/learning-analytics-policy
- 57. Learning analytics purposes Quality, equity, personalized feedback, coping with scale, student experience, skills, and efficiency The University of Edinburgh (2017). Learning Analytics Policy, http://www.ed.ac.uk/academic-services/projects/learning-analytics-policy
- 58. Critical role of leadership for adoption of learning analytics
- 59. Skill development must not be underestimated
- 60. Promoting and supporting innovation
- 61. Development of institutional policy as a critical enabler
- 62. Supporting higher education in integrating learning analytics Dragan Gašević @dgasevic June 20, 2017 Sydney, NSW, Australia
Editor's Notes
- #22: Methodology Institutions feel pressure to adopt learning analytics without having identified the needs first. Wrongly assume that learning analytics can provide all answers without having identified a question first – data driven. Learning analytics does not generate new knowledge.
- #24: Infrastructure Existing solutions in the market focus on addressing retention problems. There is no one-size-fits-all model, even within one institution (different disciplines and learning modes).
- #25: Management Learning analytics needs to compete with other institutional priorities.
- #28: Management Define ownership and responsibilities among professional groups within the university
- #29: Privacy Sharing data with third parties requires a careful check of security issues and breaches of privacy.
- #38: Management Students may be prone to choose subjects where they are likely to perform well. Users may game a system. People mistrust the result of an analysis if the process is not transparent or if the analytical model is too complicated to understand. Those who need support may not necessarily seek information from learning analytics.
- #41: .
- #42: Management Learning analytics may induce fear and discomfort about surveillance. Universities overload students with too many e-mails. Strict data protection laws could hamper the adoption of learning analytics
- #45: Infrastructure Some useful data remains inaccessible. Data is held in silos. Setting up a learning analytics environment is costly.
- #46: Capabilities The maturity of data literacy varies among stakeholders. The lack of critical self-reflection skills reduces the chance to benefit from learning analytics. Gaps exist in the understanding/ interpretation of data protection regulations between legal officers and researchers or practitioners. Digital capabilities affect the desire to opt into a learning analytics service.
- #47: Culture Institution-wide buy-in is hard to reach. Instructors are more interested in establishing a research profile than enhancing teaching and learning. Senior managers are more interested in financial benefits to the institution. There is unequal engagement/ interest in learning analytics among primary users (e.g., gender, age, disciplines). There is no common understanding of learning analytics among stakeholders at different levels. Concerns of data protection impede buy-in. Management 2018 GDPR requires changes in existing practice and system (e.g., coping with individual opt-outs). Central steering groups and individual project groups do not coordinate. The Difficulty of engaging students with institutional policies in an informed way.
- #50: Culture Low participation of primary stakeholders in top-down consultations (e.g., survey and meetings). Management Manage expectations (e.g., deliverables and impact).
- #51: Methodology It could be hard to isolate learning analytics from parallel projects that support the same goals (e.g., enhance learning and teaching). Fail to recognise and address limitations of data and analytics models (e.g., uncapturable factors of learning, ineffective metrics, existing bias, inaccuracy of predictions). Overly depend on data that is conveniently available to justify a learning phenomenon. Fail to contextualise data. Wrongly assume causal relationship between certain learning outcomes and interventions. Interventions introduced to one course may have negative impact on student engagement in another course.
- #52: These findings informed the development of a model. As per slide. In the following both the solutions focused and process focused models are displayed.
- #53: Solutions focused. – The goal is to quickly deploy analytics hence the initial emphasis in on implementation. For example deploying BlackBoard analytics. At this stage there is limited attention on dimensions such as innovation (interested staff) and developing staff capacity.
- #54: In contrast the process focused model has all dimensions however, these are not fully developed. The diagrqam here is generous as the dominant focus has been on innovation through interested staff. There has been very little uptake to for broader implementation. This can be due to the perceived lack of an articulated institutional problem or pressure for LA to address. The difficulty remains in how to transition innovations to mainstream