Using Data to Inform Decisions in Agile Software Development
0 likes205 views
Presentation of the paper "Towards using Data to Inform Decisions in Agile Software Development: Views of Available Data" held on July 28th 2019 at the 14th International Conference on Software Technologies (ICSOFT'19) in Prague. Paper authors: Christoph Matthies, Guenter Hesse. Hasso Plattner Institute, University of Potsdam, Germany
![Motivation
2
for using data in software engineering
Software engineering must shed the folkloric advice and
anecdotal evidence [...] and replace them with
convincing results, established with [...] empirical methods
– Bertrand Meyer [Meyer, 2013]
“
”[Meyer, 2013] B. Meyer, H. Gall, M. Harman, and G. Succi, “Empirical Answers to Fundamental Software
Engineering Problems (Panel),” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software
Engineering, ser. ESEC/FSE 2013. New York, USA: ACM, 2013, pp. 14–18.
Picture: https://commons.wikimedia.org/wiki/File:Bertrand_Meyer_recent.jpg](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-2-320.jpg)
![Importance of Decision-Making
3
in software development projects
■ Software project success hinges on many factors
□ Budget, scope, team organization, technology, architecture
□ Customer engagement, requirements analysis
[Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making
in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254.
[Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure
Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-3-320.jpg)
![Importance of Decision-Making
4
in software development projects
■ Software project success hinges on many factors
□ Budget, scope, team organization, technology, architecture
□ Customer engagement, requirements analysis
■ Correct project decisions critical for success [Drury, 2012]
□ Overruns of budget and schedule
□ Project cancellation [Taherdoost, 2018]
[Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making
in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254.
[Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure
Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-4-320.jpg)
![Motivation
5
for using data in Agile software development processes
Scrum is founded on empirical process control theory [...]
knowledge comes from experience and making decisions
based on what is known. [...]
– The Scrum Guide [Schwaber, 2017]
[Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” 2017,
[online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
Picture: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland
“
”](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-5-320.jpg)
![Decision-Making in Agile
6
Evidence-based decision-making in teams
■ Agile teams should make use of available data to enable
“evidence-based decision making” [Fitzgerald, 2014]
■ However, few details available on concrete implementations
[Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In
Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York,
New York, USA. ACM Press.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-6-320.jpg)
![Decision-Making in Agile
7
Evidence-based decision-making in teams
■ Agile teams should make use of available data to enable
“evidence-based decision making” [Fitzgerald, 2014]
■ However, few details available on concrete implementations
Main Research Questions
■ What data is available and relevant to Agile teams?
■ What methods are transferable from other domains?
[Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In
Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York,
New York, USA. ACM Press.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-7-320.jpg)
![Related Work [Svensson, 2019]
8
Unfulfilled Potential of DDDM in Agile
[Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”,
20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-8-320.jpg)
![Potential of DDDM
9
Lots of potential, little usage [Svensson, 2019]
■ Survey of software practitioners
■ How is data used in the company for making decisions?
[Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”,
20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-9-320.jpg)
![Data-Driven Decision-Making
10
In business contexts
■ DDDM contrasted with “gut feeling” [Provost, 2013]
□ Standard in business in the absence of analysis technologies
[Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data
and Data-Driven Decision Making. Big Data, 1(1):51–59.
[Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers:
How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal.
[Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making.
Harvard Business Review Digital Articles, pages 2–5](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-10-320.jpg)
![Data-Driven Decision-Making
11
In business contexts
■ DDDM contrasted with “gut feeling” [Provost, 2013]
□ Standard in business in the absence of analysis technologies
■ Technology use in business has fueled DDDM adoption
□ Increased company productivity [Brynjolfsson, 2011]
□ Transparency & real-time information intensifies tensions,
management must adapt [Schrage, 2016]
[Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data
and Data-Driven Decision Making. Big Data, 1(1):51–59.
[Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers:
How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal.
[Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making.
Harvard Business Review Digital Articles, pages 2–5](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-11-320.jpg)
![Agile Software Development
12
Development Processes and their connection to data
■ Plan-driven, rational decision-making [Moe, 2012]
□ Sets of rules that govern behavior
□ Assumption: outcome probabilities are known [Zannier, 2007]
[Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making:
A multiple case study of agile software development. Information and Software Technology, 54(8):853–865.
[Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors
and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and
Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-12-320.jpg)
![Agile Software Development
13
Development Processes and their connection to data
■ Plan-driven, rational decision-making [Moe, 2012]
□ Sets of rules that govern behavior
□ Assumption: outcome probabilities are known [Zannier, 2007]
■ Data enables teams to implement Agile feedback cycles
■ Agile, naturalistic decision-making
□ Situational behavior, context-dependent rules
[Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making:
A multiple case study of agile software development. Information and Software Technology, 54(8):853–865.
[Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors
and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and
Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-13-320.jpg)
![Data Use in Business
15
A wealth of data to use for decisions
■ Businesses gather detailed statistics
□ Users, suppliers, partners & competitors
□ ERP systems for collection, BI for analysis [Chen, 2012]
[Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics:
From big data to big impact. MIS quarterly, 36(4).
[Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven
Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme
Programming, pages 69–85. Springer International Publishing.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-15-320.jpg)
![Data Use in Business
16
A wealth of data to use for decisions
■ Businesses gather detailed statistics
□ Users, suppliers, partners & competitors
□ ERP systems for collection, BI for analysis [Chen, 2012]
■ Continuing digitization allows additional data collection
□ “Reality mining”: cars, smart devices, etc. [Svensson, 2019]
□ Clickstream data, UI interactions, high time-resolution
[Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics:
From big data to big impact. MIS quarterly, 36(4).
[Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven
Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme
Programming, pages 69–85. Springer International Publishing.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-16-320.jpg)
![Data in Software Development
17
Much more than code
■ Software development process itself: source of valuable data
□ Intrinsically relevant to making decisions
□ Not only code, but a multitude of supporting artifacts
[de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In
Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA.
ACM, ACM Press.
[Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous
integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in
Practice Track, ICSE-SEIP 2017, pages 113–122.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-17-320.jpg)
![Data in Software Development
18
Much more than code
■ Software development process itself: source of valuable data
□ Intrinsically relevant to making decisions
□ Not only code, but a multitude of supporting artifacts
■ Processes are “inscribed” into software artifacts [de Souza, 2005]
□ E.g. VCS: who, when, what, why, status
□ Also includes evidence of failures [Ziftci & Reardon, 2017]
[de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In
Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA.
ACM, ACM Press.
[Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous
integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in
Practice Track, ICSE-SEIP 2017, pages 113–122.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-18-320.jpg)
![Software Project Data
19
[Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the
13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016.
“a gold-mine of actionable information” [Guo et al., 2016]](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-19-320.jpg)
![Software Project Data
20
“a gold-mine of actionable information” [Guo et al., 2016]
Primary purpose: communication
[Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the
13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-20-320.jpg)
![Software Project Data
21
Use this data for reflection
[Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the
13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016.
“a gold-mine of actionable information” [Guo et al., 2016]](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-21-320.jpg)
![Scrum Process
22
Where Agile decision-making happens [Schwaber & Sutherland, 2017]
1
2
3
4
5
6
7
Sprint
Planning
Sprint
BacklogBacklog
Sprint
Daily
Stakeholder
Input
Product
4 iterations
Sprint
Retrospective Finished
Increment
Sprint Review
[Schwaber & Sutherland, 2017] Schwaber, K., & Sutherland, J, “The Scrum Guide - The Definitive Guide to Scrum: The Rules of the Game”, 2017,
[online] Available: http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-22-320.jpg)
![Sprint Planning
24
Building a backlog of work for the next Sprint
■ Selection and prioritization of features for next iteration
□ Based on stakeholders’ experiences & opinions [Svensson, 2019]
□ Lack explanation and links to evidence](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-24-320.jpg)
![Sprint Planning
25
Building a backlog of work for the next Sprint
■ Selection and prioritization of features for next iteration
□ Based on stakeholders’ experiences & opinions [Svensson, 2019]
□ Lack explanation and links to evidence
■ Enable short feedback cycles between devs and real customers
□ Metrics from deployed software, real use cases
□ Observe actual (mis)usage, engineering relevant metrics
□ Base changes on results](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-25-320.jpg)
![Related Research Fields
34
Mining Software Repositories
■ Draw from MSR techniques [Prana, 2018]
■ MSR: Focus on large amounts of repositories
□ “most widely used open source license?” [Dyer, 2013]
■ Little research: Few repositories,
intricate knowledge of creators / users
[Prana, 2018] Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. “Categorizing the Content of GitHub
README Files”. Empirical Software Engineering. 2018. https://doi.org/10.1007/s10664-018-9660-3
[Dyer, 2013] Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. “Boa: A language and infrastructure for
analyzing ultra-large-scale software repositories”. In Proceedings - International Conference on Software
Engineering. pp. 422–431. 2013. IEEE.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-34-320.jpg)
![Conclusion
35
■ Agile requires teams to make decisions based on
best information available [Schuh, 2004]
■ Definition of “best information” ongoing research challenge
[Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development:
A systematic review. Information and Software Technology, 50(9-10):833–859.
[Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc.,
Rockland, MA, USA, 1st edition.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-35-320.jpg)
![Conclusion
36
■ Agile requires teams to make decisions based on
best information available [Schuh, 2004]
■ Definition of “best information” ongoing research challenge
■ Individuals can ignore analyses or own preferences due to rules,
traditions or influence of others [Dybå and Dingsøyr, 2008]
[Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development:
A systematic review. Information and Software Technology, 50(9-10):833–859.
[Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc.,
Rockland, MA, USA, 1st edition.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-36-320.jpg)
![Conclusion
37
■ Agile requires teams to make decisions based on
best information available [Schuh, 2004]
■ Definition of “best information” ongoing research challenge
■ Individuals can ignore analyses or own preferences due to rules,
traditions or influence of others [Dybå and Dingsøyr, 2008]
Data analysis and interpretation by Agile teams (supported by
context changes) can enable better informed decision
[Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development:
A systematic review. Information and Software Technology, 50(9-10):833–859.
[Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc.,
Rockland, MA, USA, 1st edition.](https://image.slidesharecdn.com/icsoft19-190728145747/85/Using-Data-to-Inform-Decisions-in-Agile-Software-Development-37-320.jpg)
Ad
More Related Content
Similar to Using Data to Inform Decisions in Agile Software Development (20)
More from Christoph Matthies (17)
Ad
Recently uploaded (20)
![PRTG Network Monitor Crack Latest Version & Serial Key 2025 [100% Working]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250504182022-6534c7c0-thumbnail.jpg?width=560&fit=bounds)
Ad
Using Data to Inform Decisions in Agile Software Development
- 1. Hasso Plattner Institute University of Potsdam, Germany [email protected] @chrisma0 Using Data to Inform Decisions in Agile Software Development Christoph Matthies, Guenter Hesse Prague, July ’19
- 2. Motivation 2 for using data in software engineering Software engineering must shed the folkloric advice and anecdotal evidence [...] and replace them with convincing results, established with [...] empirical methods – Bertrand Meyer [Meyer, 2013] “ ”[Meyer, 2013] B. Meyer, H. Gall, M. Harman, and G. Succi, “Empirical Answers to Fundamental Software Engineering Problems (Panel),” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ser. ESEC/FSE 2013. New York, USA: ACM, 2013, pp. 14–18. Picture: https://commons.wikimedia.org/wiki/File:Bertrand_Meyer_recent.jpg
- 3. Importance of Decision-Making 3 in software development projects ■ Software project success hinges on many factors □ Budget, scope, team organization, technology, architecture □ Customer engagement, requirements analysis [Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254. [Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering
- 4. Importance of Decision-Making 4 in software development projects ■ Software project success hinges on many factors □ Budget, scope, team organization, technology, architecture □ Customer engagement, requirements analysis ■ Correct project decisions critical for success [Drury, 2012] □ Overruns of budget and schedule □ Project cancellation [Taherdoost, 2018] [Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254. [Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering
- 5. Motivation 5 for using data in Agile software development processes Scrum is founded on empirical process control theory [...] knowledge comes from experience and making decisions based on what is known. [...] – The Scrum Guide [Schwaber, 2017] [Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” 2017, [online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf Picture: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland “ ”
- 6. Decision-Making in Agile 6 Evidence-based decision-making in teams ■ Agile teams should make use of available data to enable “evidence-based decision making” [Fitzgerald, 2014] ■ However, few details available on concrete implementations [Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York, New York, USA. ACM Press.
- 7. Decision-Making in Agile 7 Evidence-based decision-making in teams ■ Agile teams should make use of available data to enable “evidence-based decision making” [Fitzgerald, 2014] ■ However, few details available on concrete implementations Main Research Questions ■ What data is available and relevant to Agile teams? ■ What methods are transferable from other domains? [Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York, New York, USA. ACM Press.
- 8. Related Work [Svensson, 2019] 8 Unfulfilled Potential of DDDM in Agile [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
- 9. Potential of DDDM 9 Lots of potential, little usage [Svensson, 2019] ■ Survey of software practitioners ■ How is data used in the company for making decisions? [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
- 10. Data-Driven Decision-Making 10 In business contexts ■ DDDM contrasted with “gut feeling” [Provost, 2013] □ Standard in business in the absence of analysis technologies [Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data and Data-Driven Decision Making. Big Data, 1(1):51–59. [Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal. [Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making. Harvard Business Review Digital Articles, pages 2–5
- 11. Data-Driven Decision-Making 11 In business contexts ■ DDDM contrasted with “gut feeling” [Provost, 2013] □ Standard in business in the absence of analysis technologies ■ Technology use in business has fueled DDDM adoption □ Increased company productivity [Brynjolfsson, 2011] □ Transparency & real-time information intensifies tensions, management must adapt [Schrage, 2016] [Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data and Data-Driven Decision Making. Big Data, 1(1):51–59. [Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal. [Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making. Harvard Business Review Digital Articles, pages 2–5
- 12. Agile Software Development 12 Development Processes and their connection to data ■ Plan-driven, rational decision-making [Moe, 2012] □ Sets of rules that govern behavior □ Assumption: outcome probabilities are known [Zannier, 2007] [Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making: A multiple case study of agile software development. Information and Software Technology, 54(8):853–865. [Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.
- 13. Agile Software Development 13 Development Processes and their connection to data ■ Plan-driven, rational decision-making [Moe, 2012] □ Sets of rules that govern behavior □ Assumption: outcome probabilities are known [Zannier, 2007] ■ Data enables teams to implement Agile feedback cycles ■ Agile, naturalistic decision-making □ Situational behavior, context-dependent rules [Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making: A multiple case study of agile software development. Information and Software Technology, 54(8):853–865. [Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.
- 14. Levels of Decisions 14 The impact of Agile “self-organizing” teams ■ Strategic plans define long-term goals ■ Tactical plans involve managing projects to achieve goals ■ Operational plans define milestones, and the actual work
- 15. Data Use in Business 15 A wealth of data to use for decisions ■ Businesses gather detailed statistics □ Users, suppliers, partners & competitors □ ERP systems for collection, BI for analysis [Chen, 2012] [Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS quarterly, 36(4). [Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme Programming, pages 69–85. Springer International Publishing.
- 16. Data Use in Business 16 A wealth of data to use for decisions ■ Businesses gather detailed statistics □ Users, suppliers, partners & competitors □ ERP systems for collection, BI for analysis [Chen, 2012] ■ Continuing digitization allows additional data collection □ “Reality mining”: cars, smart devices, etc. [Svensson, 2019] □ Clickstream data, UI interactions, high time-resolution [Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS quarterly, 36(4). [Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme Programming, pages 69–85. Springer International Publishing.
- 17. Data in Software Development 17 Much more than code ■ Software development process itself: source of valuable data □ Intrinsically relevant to making decisions □ Not only code, but a multitude of supporting artifacts [de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA. ACM, ACM Press. [Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, pages 113–122.
- 18. Data in Software Development 18 Much more than code ■ Software development process itself: source of valuable data □ Intrinsically relevant to making decisions □ Not only code, but a multitude of supporting artifacts ■ Processes are “inscribed” into software artifacts [de Souza, 2005] □ E.g. VCS: who, when, what, why, status □ Also includes evidence of failures [Ziftci & Reardon, 2017] [de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA. ACM, ACM Press. [Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, pages 113–122.
- 19. Software Project Data 19 [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016. “a gold-mine of actionable information” [Guo et al., 2016]
- 20. Software Project Data 20 “a gold-mine of actionable information” [Guo et al., 2016] Primary purpose: communication [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016.
- 21. Software Project Data 21 Use this data for reflection [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016. “a gold-mine of actionable information” [Guo et al., 2016]
- 22. Scrum Process 22 Where Agile decision-making happens [Schwaber & Sutherland, 2017] 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review [Schwaber & Sutherland, 2017] Schwaber, K., & Sutherland, J, “The Scrum Guide - The Definitive Guide to Scrum: The Rules of the Game”, 2017, [online] Available: http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
- 23. Sprint Planning 23 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review Building a backlog of work for the next Sprint
- 24. Sprint Planning 24 Building a backlog of work for the next Sprint ■ Selection and prioritization of features for next iteration □ Based on stakeholders’ experiences & opinions [Svensson, 2019] □ Lack explanation and links to evidence
- 25. Sprint Planning 25 Building a backlog of work for the next Sprint ■ Selection and prioritization of features for next iteration □ Based on stakeholders’ experiences & opinions [Svensson, 2019] □ Lack explanation and links to evidence ■ Enable short feedback cycles between devs and real customers □ Metrics from deployed software, real use cases □ Observe actual (mis)usage, engineering relevant metrics □ Base changes on results
- 26. Daily Scrum 26 Quick status update 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review
- 27. Daily Scrum 27 Quick status update ■ Short daily status meeting □ “What did I do?”, “What will I do?”, “Blockers?” □ Strictly time-boxed
- 28. Daily Scrum 28 Quick status update ■ Short daily status meeting □ “What did I do?”, “What will I do?”, “Blockers?” □ Strictly time-boxed ■ Development artifacts hold information on what was done □ Concrete items that were completed □ Rely less on personal recollection □ Optimize for speed
- 29. Sprint Retrospective 29 Reflecting on past work and improving 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review
- 30. Retrospectives 30 Reflecting on past work and improving What should be improved? Retrospective Meeting
- 31. 31 Did we improve what we planned? Retrospective Meeting Retrospectives Reflecting on past work and improving
- 32. 32 Did we improve what we planned? Retrospective Meeting Project DataEvidence of last iteration’s work code docs stats Retrospectives Reflecting on past work and improving
- 33. 33 Did we improve what we planned? Retrospective Meeting Project DataEvidence of last iteration’s work code docs stats Retrospectives Reflecting on past work and improving
- 34. Related Research Fields 34 Mining Software Repositories ■ Draw from MSR techniques [Prana, 2018] ■ MSR: Focus on large amounts of repositories □ “most widely used open source license?” [Dyer, 2013] ■ Little research: Few repositories, intricate knowledge of creators / users [Prana, 2018] Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. “Categorizing the Content of GitHub README Files”. Empirical Software Engineering. 2018. https://doi.org/10.1007/s10664-018-9660-3 [Dyer, 2013] Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. “Boa: A language and infrastructure for analyzing ultra-large-scale software repositories”. In Proceedings - International Conference on Software Engineering. pp. 422–431. 2013. IEEE.
- 35. Conclusion 35 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
- 36. Conclusion 36 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge ■ Individuals can ignore analyses or own preferences due to rules, traditions or influence of others [Dybå and Dingsøyr, 2008] [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
- 37. Conclusion 37 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge ■ Individuals can ignore analyses or own preferences due to rules, traditions or influence of others [Dybå and Dingsøyr, 2008] Data analysis and interpretation by Agile teams (supported by context changes) can enable better informed decision [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
- 38. Summary 38
- 39. Image Credits 39 In order of appearance ■ Data directed Decision Making by H Alberto Gongora from the Noun Project (CC BY 3.0 US) ■ Project by useiconic.com from the Noun Project (CC BY 3.0 US) ■ Question by unlimicon from the Noun Project (CC BY 3.0 US) ■ Business by David Glöckler from the Noun Project (CC BY 3.0 US) ■ Agile Development by Bê Sabino from the Noun Project (CC BY 3.0 US) ■ Digitization by priyanka from the Noun Project (CC BY 3.0 US) ■ Developer by becris from the Noun Project (CC BY 3.0 US) ■ Plan by Gregor Cresnar from the Noun Project (CC BY 3.0 US) ■ Daily Scrum by Swen-Peter Ekkebus from the Noun Project (CC BY 3.0 US) ■ Documents by LUFTFI GANI AL ACHMAD from the Noun Project (CC BY 3.0 US) ■ Idea Generation by priyanka from the Noun Project (CC BY 3.0 US)