SlideShare a Scribd company logo
Expert Mining for Evaluating Risk Indicators ScenariosOscar Franco-Bedoya, Dolors Costal, Soraya Hidalgo, Ron Ben-JacobMonday 21st July 2014
2 
Background 
Workshop procedure 
Generalization of the approach 
Applications and lessons learned 
Related work 
Conclusions and further work 
Outline
3 
Background
4 
Background
5 
Background 
Managing risk in 
open source adoption
6 
Identification 
Mitigation 
methods 
Management 
Advanced tools 
Provides 
Platform 
Methods 
OSS 
adoption 
projects 
To support 
a 
In 
Uses 
Ecosystem 
modeling 
Statistical 
tools 
Risk 
Management 
i.e. 
i.e. 
i.e. 
Bayesian 
Networks 
Social 
network 
analysis 
Expert 
scenarios 
assessment 
e.g. 
e.g. 
e.g. 
Background 
Risks 
Systematic 
protocol 
uses
7 
Background 
Project Site 
Code Version 
Repository 
Bug Tracker Mailing List IRC 
Ecosystem hubs 
Project indexes 
Social Media 
Twitter Facebook 
I 
Raw 
Data 
SNA 
Measures 
Risk Indicators: 
• Project 
• Community 
• Contextual 
II 
Indicators 
Scenario-based 
Assessment Domain 
Expert 
Business Analysis 
• Business goals 
III 
Business 
Goals 
3-Layered 
RISCOSS approach 
Number of 
downloads 
Number of 
event 
references 
Centrality 
Number of 
open bugs
8 
Outline 
Background 
Workshop procedure 
Generalization of the approach 
Applications and lessons learned 
Related work 
Conclusions and further work 
Outline
9 
The RISCOSS tactical workshop is designed to permit experts to assess risk indicators 
Workshop procedure
10 
Tacticalworkshop 
protocol 
Workshop procedure 
Pre-TasksPart II: Community dinamicsPart I : Community dataTacticalWorkshop OrganizerDomainExpert6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WSTabsTimelinessActivenessCommunity10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicatorPDF File:KPA RISCOSS Analytics [more risk drivers] [not more risk drivers] RISCOSS Analytics Team1.Determine drivers and risk indicators2.Construct Bayesian networks3.Define scenarios4.Identify and contact experts5.Workshops planning and preparation16.Send the scenarios judgementPopulate BN nodes
11 
Pre-Tasks 
Tactical 
Workshop 
Organizer 
RISCOSS 
Analytics 
Team 
1.Determine 
drivers and 
risk indicators 
2.Construct 
Bayesian 
networks 
3.Define 
scenarios 
4.Identify and 
contact 
experts 
5.Workshops 
planning and 
preparation 
Correspond to 
tasks that must 
be done before 
conducting the 
tactical workshop 
Workshop procedure
12 
Pre-Tasks 
Tactical 
Workshop 
Organizer 
RISCOSS 
Analytics 
Team 
1.Determine 
drivers and 
risk indicators 
2.Construct 
Bayesian 
networks 
3.Define 
scenarios 
4.Identify and 
contact 
experts 
5.Workshops 
planning and 
preparation 
Workshop procedure 
Risk Driver 
Forum posts per day 
Forum messages per thread 
Mail per day 
Overall community size 
Number of developers involved 
Number of testers (individuals 
providing feedback) 
Number of companies using the 
software 
Companies supporting the project 
(adding to code) 
Risk Indicator Activeness
13 
Pre-Tasks 
Tactical 
Workshop 
Organizer 
RISCOSS 
Analytics 
Team 
1.Determine 
drivers and 
risk indicators 
2.Construct 
Bayesian 
networks 
3.Define 
scenarios 
4.Identify and 
contact 
experts 
5.Workshops 
planning and 
preparation 
Workshop procedure
14 
Pre-Tasks 
Tactical 
Workshop 
Organizer 
RISCOSS 
Analytics 
Team 
1.Determine 
drivers and 
risk indicators 
2.Construct 
Bayesian 
networks 
3.Define 
scenarios 
4.Identify and 
contact 
experts 
5.Workshops 
planning and 
preparation 
Workshop procedure
15 
Pre-Tasks 
Tactical 
Workshop 
Organizer 
RISCOSS 
Analytics 
Team 
1.Determine 
drivers and 
risk indicators 
2.Construct 
Bayesian 
networks 
3.Define 
scenarios 
4.Identify and 
contact 
experts 
5.Workshops 
planning and 
preparation 
Workshop procedure
16 
The tactical 
workshops begin 
with an exposition 
about the main 
topics that will be 
covered 
Workshop procedure 
6.Make 
presentation 
of RISCOSS 
project 
summary. 
7.Explain the 
RISCOSS 
analytics 
platform. 
8.Explain the 
tactical 
workshop 
Part I and 
Part II 
PDF File:KPA 
RISCOSS 
Analytics 
Tactical 
Workshop 
Organizer
17 
Workshop procedure 
9.Explain the 
risk driver 
selection WS 
Excel File 
Excel File: risk driver 
selection WS 
Tabs 
Timeliness 
Activeness 
Community 
Tactical 
Workshop 
Organizer
18 
Workshop procedure 
Part I : 
Community data 
Domain 
Expert 
10.Study the 
use case 
scenario 
11.Assess the 
use case 
scenario 
overall state 
12.Determine 
the level of 
the risk 
indicator
19 
Workshop procedure 
Part II: 
Community 
dinamics 
13.Study the 
use case 
scenario 
14.Assess the 
use case 
scenario 
overall state 
15.Determine 
the level of 
the 
community 
risk indicator 
16.Send the 
scenarios 
judgement 
Domain 
Expert
20 
Outline 
Outline 
Background 
Workshop procedure 
Generalization of the approach 
Applications and lessons learned 
Related work 
Conclusions and further work
21 
Generalization of the approach
22 
Outline 
Outline 
Background 
Workshop procedure 
Generalization of the approach 
Applications and lessons learned 
Related work 
Conclusions and further work
23 
Applications and lessons learned 
Applications and lessons learned 
We have conducted 10 technical workshops 
In private and official organizations and academic institutions. 
The experts were from different countries. 
France, Israel, Italy, Spain, and Netherlands 
There are some inconsistencies in the scenarios 
The scenarios were designed using random number generators 
While the domain experts are conducting the tactical workshops, 
The degree of "calibration" of their judgement improves
24 
Outline 
Outline 
Background 
Workshop procedure 
Generalization of the approach 
Applications and lessons learned 
Related work 
Conclusions and further work
25 
Related Work 
Related Work 
Delphi method 
(QUELCE) 
QuantifyingUncertaintyinEarlyCostEstimation[2] 
Reliableconsensusofopinionofagroupofexperts[1] 
[1]N.DalkeyandO.Helmer,“AnexperimentalapplicationoftheDelphimethodtotheuseofexperts,”. 
[2]R.W.Ferguson,D.Goldenson,J.M.McCurley,R.W.Stoddard,andD.Zubrow,“QuantifyingUncertaintyinEarlyLifecycleCostEstimation(QUELCE),”
26 
Outline 
Outline 
Background 
Workshop procedure 
Generalization of the approach 
Applications and lessons learned 
Related work 
Conclusions and further work
27 
Domain 
Expert 
Conclusions and further work 
Conclusions and further work 
Pre-Tasks 
Part II: 
Community 
dinamics 
Part I : 
Community data 
Tactical 
Workshop 
Organizer 
Domain 
Expert 
6.Make 
presentation 
of RISCOSS 
project 
summary. 
7.Explain the 
RISCOSS 
analytics 
platform. 
8.Explain the 
tactical 
workshop 
Part I and 
Part II 
9.Explain the 
risk driver 
selection WS 
Excel File 
Excel File: risk driver 
selection WS 
Tabs 
Timeliness 
Activeness 
Community 
10.Study the 
use case 
scenario 
11.Assess the 
use case 
scenario 
overall state 
12.Determine 
the level of 
the risk 
indicator 
13.Study the 
use case 
scenario 
14.Assess the 
use case 
scenario 
overall state 
15.Determine 
the level of 
the 
community 
risk indicator 
PDF File:KPA 
RISCOSS 
Analytics 
[more 
risk drivers] 
[not more 
risk drivers] 
RISCOSS 
Analytics 
Team 
1.Determine 
drivers and 
risk indicators 
2.Construct 
Bayesian 
networks 
3.Define 
scenarios 
4.Identify and 
contact 
experts 
5.Workshops 
planning and 
preparation 
16.Send the 
scenarios 
judgement 
Populate BN 
nodes 
Domain 
Expert SNA 
Project Site 
Code Version 
Repository 
Bug Tracker Mailing List IRC 
Step-by-step 
protocol 
Data used to 
construct 
Bayesian 
networks 
Future work 
Combines opinion 
of domain experts 
with OSS raw data 
Empirical 
application 
& 
lessons
28 
Outline 
Outline 
Background 
Workshop procedure 
Generalization of the approach 
Applications and lessons learned 
Related work 
Conclusions and further work
29 
SEeD4FI
Thanks for your attention 
Comments and Questions
31 
SEeD4FI
Ad

Recommended

Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Tom Mens
 
On the Relation between Outdated Docker Containers, Severity Vulnerabilities,...
On the Relation between Outdated Docker Containers, Severity Vulnerabilities,...
Tom Mens
 
Binary Analysis - Luxembourg
Binary Analysis - Luxembourg
Abhik Roychoudhury
 
UCC-2016, 6-9 May December, Shanghai, China
UCC-2016, 6-9 May December, Shanghai, China
Charith Perera
 
Cesi2014
Cesi2014
GESSI UPC
 
Jcis 2015-Towards Assessing Open Source Communities' Health using SOC Concepts
Jcis 2015-Towards Assessing Open Source Communities' Health using SOC Concepts
GESSI UPC
 
Electronica informatoca
Electronica informatoca
Aidessita Gali
 
Ossap final
Ossap final
GESSI UPC
 
Agile and the nature of decision making
Agile and the nature of decision making
Dennis Stevens
 
Agile and the nature of decision making
Agile and the nature of decision making
drewz lin
 
RISCOSS platform: evaluation results
RISCOSS platform: evaluation results
Silvia Valentini
 
Angelo Susi' s presentation at PMI Academic Workshop 2016
Angelo Susi' s presentation at PMI Academic Workshop 2016
Silvia Valentini
 
RISCOSS platform: evaluation results
RISCOSS platform: evaluation results
Silvia Valentini
 
crisc_wk_3.pptx
crisc_wk_3.pptx
dotco
 
Information systems risk assessment frame workisraf 130215042410-phpapp01
Information systems risk assessment frame workisraf 130215042410-phpapp01
S Periyakaruppan CISM,ISO31000,C-EH,ITILF
 
Riscoss models u-qasar meeting
Riscoss models u-qasar meeting
Xavier Franch
 
Hands on IT risk assessment
Hands on IT risk assessment
George Delikouras
 
Risk profiling
Risk profiling
Jaume Jornet Rivas
 
Data Driven Risk Management
Data Driven Risk Management
Resolver Inc.
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and Development
IJERD Editor
 
Introduction to FAIR - Factor Analysis of Information Risk
Introduction to FAIR - Factor Analysis of Information Risk
Osama Salah
 
Measuring DDoS Risk using FAIR (Factor Analysis of Information Risk
Measuring DDoS Risk using FAIR (Factor Analysis of Information Risk
Tony Martin-Vegue
 
Dynamic vs. Traditional Probabilistic Risk Assessment Methodologies - by Huai...
Dynamic vs. Traditional Probabilistic Risk Assessment Methodologies - by Huai...
ASQ Reliability Division
 
Risk Assessment
Risk Assessment
Glen Alleman
 
Assessment Of Risk Mitigation
Assessment Of Risk Mitigation
Eneni Oduwole
 
Optimization of different objective function in risk assessment system
Optimization of different objective function in risk assessment system
Alexander Decker
 
NIST 800 30 revision Sep 2012
NIST 800 30 revision Sep 2012
S Periyakaruppan CISM,ISO31000,C-EH,ITILF
 
Ratneshwar jha risk analysis and mitigation precedence modeling
Ratneshwar jha risk analysis and mitigation precedence modeling
PMInstituteIndia
 
iStarJSON: A Lightweight Data-Format for i* Models
iStarJSON: A Lightweight Data-Format for i* Models
GESSI UPC
 
Towards iStarML 2.0: Closing Gaps from Evolved Requirements
Towards iStarML 2.0: Closing Gaps from Evolved Requirements
GESSI UPC
 

More Related Content

Similar to Expert mining compsac-2014 (20)

Agile and the nature of decision making
Agile and the nature of decision making
Dennis Stevens
 
Agile and the nature of decision making
Agile and the nature of decision making
drewz lin
 
RISCOSS platform: evaluation results
RISCOSS platform: evaluation results
Silvia Valentini
 
Angelo Susi' s presentation at PMI Academic Workshop 2016
Angelo Susi' s presentation at PMI Academic Workshop 2016
Silvia Valentini
 
RISCOSS platform: evaluation results
RISCOSS platform: evaluation results
Silvia Valentini
 
crisc_wk_3.pptx
crisc_wk_3.pptx
dotco
 
Information systems risk assessment frame workisraf 130215042410-phpapp01
Information systems risk assessment frame workisraf 130215042410-phpapp01
S Periyakaruppan CISM,ISO31000,C-EH,ITILF
 
Riscoss models u-qasar meeting
Riscoss models u-qasar meeting
Xavier Franch
 
Hands on IT risk assessment
Hands on IT risk assessment
George Delikouras
 
Risk profiling
Risk profiling
Jaume Jornet Rivas
 
Data Driven Risk Management
Data Driven Risk Management
Resolver Inc.
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and Development
IJERD Editor
 
Introduction to FAIR - Factor Analysis of Information Risk
Introduction to FAIR - Factor Analysis of Information Risk
Osama Salah
 
Measuring DDoS Risk using FAIR (Factor Analysis of Information Risk
Measuring DDoS Risk using FAIR (Factor Analysis of Information Risk
Tony Martin-Vegue
 
Dynamic vs. Traditional Probabilistic Risk Assessment Methodologies - by Huai...
Dynamic vs. Traditional Probabilistic Risk Assessment Methodologies - by Huai...
ASQ Reliability Division
 
Risk Assessment
Risk Assessment
Glen Alleman
 
Assessment Of Risk Mitigation
Assessment Of Risk Mitigation
Eneni Oduwole
 
Optimization of different objective function in risk assessment system
Optimization of different objective function in risk assessment system
Alexander Decker
 
NIST 800 30 revision Sep 2012
NIST 800 30 revision Sep 2012
S Periyakaruppan CISM,ISO31000,C-EH,ITILF
 
Ratneshwar jha risk analysis and mitigation precedence modeling
Ratneshwar jha risk analysis and mitigation precedence modeling
PMInstituteIndia
 
Agile and the nature of decision making
Agile and the nature of decision making
Dennis Stevens
 
Agile and the nature of decision making
Agile and the nature of decision making
drewz lin
 
RISCOSS platform: evaluation results
RISCOSS platform: evaluation results
Silvia Valentini
 
Angelo Susi' s presentation at PMI Academic Workshop 2016
Angelo Susi' s presentation at PMI Academic Workshop 2016
Silvia Valentini
 
RISCOSS platform: evaluation results
RISCOSS platform: evaluation results
Silvia Valentini
 
crisc_wk_3.pptx
crisc_wk_3.pptx
dotco
 
Riscoss models u-qasar meeting
Riscoss models u-qasar meeting
Xavier Franch
 
Data Driven Risk Management
Data Driven Risk Management
Resolver Inc.
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and Development
IJERD Editor
 
Introduction to FAIR - Factor Analysis of Information Risk
Introduction to FAIR - Factor Analysis of Information Risk
Osama Salah
 
Measuring DDoS Risk using FAIR (Factor Analysis of Information Risk
Measuring DDoS Risk using FAIR (Factor Analysis of Information Risk
Tony Martin-Vegue
 
Dynamic vs. Traditional Probabilistic Risk Assessment Methodologies - by Huai...
Dynamic vs. Traditional Probabilistic Risk Assessment Methodologies - by Huai...
ASQ Reliability Division
 
Assessment Of Risk Mitigation
Assessment Of Risk Mitigation
Eneni Oduwole
 
Optimization of different objective function in risk assessment system
Optimization of different objective function in risk assessment system
Alexander Decker
 
Ratneshwar jha risk analysis and mitigation precedence modeling
Ratneshwar jha risk analysis and mitigation precedence modeling
PMInstituteIndia
 

More from GESSI UPC (20)

iStarJSON: A Lightweight Data-Format for i* Models
iStarJSON: A Lightweight Data-Format for i* Models
GESSI UPC
 
Towards iStarML 2.0: Closing Gaps from Evolved Requirements
Towards iStarML 2.0: Closing Gaps from Evolved Requirements
GESSI UPC
 
Monitoring the service-based system lifecycle with SALMon
Monitoring the service-based system lifecycle with SALMon
GESSI UPC
 
Aligning Business Goals and Risks in OSS Adoption
Aligning Business Goals and Risks in OSS Adoption
GESSI UPC
 
RISCOSS: Gestión del riesgo en proyectos open source (Open Expo Day2015)
RISCOSS: Gestión del riesgo en proyectos open source (Open Expo Day2015)
GESSI UPC
 
Open expo2015 riscoss
Open expo2015 riscoss
GESSI UPC
 
Oss2015
Oss2015
GESSI UPC
 
Mobility4 all
Mobility4 all
GESSI UPC
 
Er14
Er14
GESSI UPC
 
QuESo: a Quality Model for Open Source Software Ecosystems
QuESo: a Quality Model for Open Source Software Ecosystems
GESSI UPC
 
MoDRE 2014 @ RE keynote -- NFR-Aware MDD Processes
MoDRE 2014 @ RE keynote -- NFR-Aware MDD Processes
GESSI UPC
 
Quantifying the Impact of OSS Adoption Risks with the help of i* Models
Quantifying the Impact of OSS Adoption Risks with the help of i* Models
GESSI UPC
 
Applying Business Strategy Models in Organizations
Applying Business Strategy Models in Organizations
GESSI UPC
 
Slides refsq'14 ds v1
Slides refsq'14 ds v1
GESSI UPC
 
A Context Ontology for Service Provisioning and Consumption
A Context Ontology for Service Provisioning and Consumption
GESSI UPC
 
Practical Experiences in Designing and Conducting Empirical Studies in Indust...
Practical Experiences in Designing and Conducting Empirical Studies in Indust...
GESSI UPC
 
Assessing Open Source Communities' using Service Oritented Computing concepts
Assessing Open Source Communities' using Service Oritented Computing concepts
GESSI UPC
 
DB searches vs. snowballing
DB searches vs. snowballing
GESSI UPC
 
AK+MDD+NFRs
AK+MDD+NFRs
GESSI UPC
 
Arteon: Architectural and Technology Ontology
Arteon: Architectural and Technology Ontology
GESSI UPC
 
iStarJSON: A Lightweight Data-Format for i* Models
iStarJSON: A Lightweight Data-Format for i* Models
GESSI UPC
 
Towards iStarML 2.0: Closing Gaps from Evolved Requirements
Towards iStarML 2.0: Closing Gaps from Evolved Requirements
GESSI UPC
 
Monitoring the service-based system lifecycle with SALMon
Monitoring the service-based system lifecycle with SALMon
GESSI UPC
 
Aligning Business Goals and Risks in OSS Adoption
Aligning Business Goals and Risks in OSS Adoption
GESSI UPC
 
RISCOSS: Gestión del riesgo en proyectos open source (Open Expo Day2015)
RISCOSS: Gestión del riesgo en proyectos open source (Open Expo Day2015)
GESSI UPC
 
Open expo2015 riscoss
Open expo2015 riscoss
GESSI UPC
 
Mobility4 all
Mobility4 all
GESSI UPC
 
QuESo: a Quality Model for Open Source Software Ecosystems
QuESo: a Quality Model for Open Source Software Ecosystems
GESSI UPC
 
MoDRE 2014 @ RE keynote -- NFR-Aware MDD Processes
MoDRE 2014 @ RE keynote -- NFR-Aware MDD Processes
GESSI UPC
 
Quantifying the Impact of OSS Adoption Risks with the help of i* Models
Quantifying the Impact of OSS Adoption Risks with the help of i* Models
GESSI UPC
 
Applying Business Strategy Models in Organizations
Applying Business Strategy Models in Organizations
GESSI UPC
 
Slides refsq'14 ds v1
Slides refsq'14 ds v1
GESSI UPC
 
A Context Ontology for Service Provisioning and Consumption
A Context Ontology for Service Provisioning and Consumption
GESSI UPC
 
Practical Experiences in Designing and Conducting Empirical Studies in Indust...
Practical Experiences in Designing and Conducting Empirical Studies in Indust...
GESSI UPC
 
Assessing Open Source Communities' using Service Oritented Computing concepts
Assessing Open Source Communities' using Service Oritented Computing concepts
GESSI UPC
 
DB searches vs. snowballing
DB searches vs. snowballing
GESSI UPC
 
Arteon: Architectural and Technology Ontology
Arteon: Architectural and Technology Ontology
GESSI UPC
 
Ad

Recently uploaded (20)

Structural Wonderers_new and ancient.pptx
Structural Wonderers_new and ancient.pptx
nikopapa113
 
machine learning is a advance technology
machine learning is a advance technology
ynancy893
 
Proposal for folders structure division in projects.pdf
Proposal for folders structure division in projects.pdf
Mohamed Ahmed
 
Industry 4.o the fourth revolutionWeek-2.pptx
Industry 4.o the fourth revolutionWeek-2.pptx
KNaveenKumarECE
 
retina_biometrics ruet rajshahi bangdesh.pptx
retina_biometrics ruet rajshahi bangdesh.pptx
MdRakibulIslam697135
 
Rapid Prototyping for XR: Lecture 1 Introduction to Prototyping
Rapid Prototyping for XR: Lecture 1 Introduction to Prototyping
Mark Billinghurst
 
Complete University of Calculus :: 2nd edition
Complete University of Calculus :: 2nd edition
Shabista Imam
 
Structured Programming with C++ :: Kjell Backman
Structured Programming with C++ :: Kjell Backman
Shabista Imam
 
Abraham Silberschatz-Operating System Concepts (9th,2012.12).pdf
Abraham Silberschatz-Operating System Concepts (9th,2012.12).pdf
Shabista Imam
 
Rapid Prototyping for XR: Lecture 5 - Cross Platform Development
Rapid Prototyping for XR: Lecture 5 - Cross Platform Development
Mark Billinghurst
 
Rapid Prototyping for XR: Lecture 4 - High Level Prototyping.
Rapid Prototyping for XR: Lecture 4 - High Level Prototyping.
Mark Billinghurst
 
Complete guidance book of Asp.Net Web API
Complete guidance book of Asp.Net Web API
Shabista Imam
 
How to Un-Obsolete Your Legacy Keypad Design
How to Un-Obsolete Your Legacy Keypad Design
Epec Engineered Technologies
 
Machine Learning - Classification Algorithms
Machine Learning - Classification Algorithms
resming1
 
(Continuous Integration and Continuous Deployment/Delivery) is a fundamental ...
(Continuous Integration and Continuous Deployment/Delivery) is a fundamental ...
ketan09101
 
System design handwritten notes guidance
System design handwritten notes guidance
Shabista Imam
 
Generative AI & Scientific Research : Catalyst for Innovation, Ethics & Impact
Generative AI & Scientific Research : Catalyst for Innovation, Ethics & Impact
AlqualsaDIResearchGr
 
International Journal of Advanced Information Technology (IJAIT)
International Journal of Advanced Information Technology (IJAIT)
ijait
 
60 Years and Beyond eBook 1234567891.pdf
60 Years and Beyond eBook 1234567891.pdf
waseemalazzeh
 
Call For Papers - 17th International Conference on Wireless & Mobile Networks...
Call For Papers - 17th International Conference on Wireless & Mobile Networks...
hosseinihamid192023
 
Structural Wonderers_new and ancient.pptx
Structural Wonderers_new and ancient.pptx
nikopapa113
 
machine learning is a advance technology
machine learning is a advance technology
ynancy893
 
Proposal for folders structure division in projects.pdf
Proposal for folders structure division in projects.pdf
Mohamed Ahmed
 
Industry 4.o the fourth revolutionWeek-2.pptx
Industry 4.o the fourth revolutionWeek-2.pptx
KNaveenKumarECE
 
retina_biometrics ruet rajshahi bangdesh.pptx
retina_biometrics ruet rajshahi bangdesh.pptx
MdRakibulIslam697135
 
Rapid Prototyping for XR: Lecture 1 Introduction to Prototyping
Rapid Prototyping for XR: Lecture 1 Introduction to Prototyping
Mark Billinghurst
 
Complete University of Calculus :: 2nd edition
Complete University of Calculus :: 2nd edition
Shabista Imam
 
Structured Programming with C++ :: Kjell Backman
Structured Programming with C++ :: Kjell Backman
Shabista Imam
 
Abraham Silberschatz-Operating System Concepts (9th,2012.12).pdf
Abraham Silberschatz-Operating System Concepts (9th,2012.12).pdf
Shabista Imam
 
Rapid Prototyping for XR: Lecture 5 - Cross Platform Development
Rapid Prototyping for XR: Lecture 5 - Cross Platform Development
Mark Billinghurst
 
Rapid Prototyping for XR: Lecture 4 - High Level Prototyping.
Rapid Prototyping for XR: Lecture 4 - High Level Prototyping.
Mark Billinghurst
 
Complete guidance book of Asp.Net Web API
Complete guidance book of Asp.Net Web API
Shabista Imam
 
Machine Learning - Classification Algorithms
Machine Learning - Classification Algorithms
resming1
 
(Continuous Integration and Continuous Deployment/Delivery) is a fundamental ...
(Continuous Integration and Continuous Deployment/Delivery) is a fundamental ...
ketan09101
 
System design handwritten notes guidance
System design handwritten notes guidance
Shabista Imam
 
Generative AI & Scientific Research : Catalyst for Innovation, Ethics & Impact
Generative AI & Scientific Research : Catalyst for Innovation, Ethics & Impact
AlqualsaDIResearchGr
 
International Journal of Advanced Information Technology (IJAIT)
International Journal of Advanced Information Technology (IJAIT)
ijait
 
60 Years and Beyond eBook 1234567891.pdf
60 Years and Beyond eBook 1234567891.pdf
waseemalazzeh
 
Call For Papers - 17th International Conference on Wireless & Mobile Networks...
Call For Papers - 17th International Conference on Wireless & Mobile Networks...
hosseinihamid192023
 
Ad

Expert mining compsac-2014

  • 1. Expert Mining for Evaluating Risk Indicators ScenariosOscar Franco-Bedoya, Dolors Costal, Soraya Hidalgo, Ron Ben-JacobMonday 21st July 2014
  • 2. 2 Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work Outline
  • 5. 5 Background Managing risk in open source adoption
  • 6. 6 Identification Mitigation methods Management Advanced tools Provides Platform Methods OSS adoption projects To support a In Uses Ecosystem modeling Statistical tools Risk Management i.e. i.e. i.e. Bayesian Networks Social network analysis Expert scenarios assessment e.g. e.g. e.g. Background Risks Systematic protocol uses
  • 7. 7 Background Project Site Code Version Repository Bug Tracker Mailing List IRC Ecosystem hubs Project indexes Social Media Twitter Facebook I Raw Data SNA Measures Risk Indicators: • Project • Community • Contextual II Indicators Scenario-based Assessment Domain Expert Business Analysis • Business goals III Business Goals 3-Layered RISCOSS approach Number of downloads Number of event references Centrality Number of open bugs
  • 8. 8 Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work Outline
  • 9. 9 The RISCOSS tactical workshop is designed to permit experts to assess risk indicators Workshop procedure
  • 10. 10 Tacticalworkshop protocol Workshop procedure Pre-TasksPart II: Community dinamicsPart I : Community dataTacticalWorkshop OrganizerDomainExpert6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WSTabsTimelinessActivenessCommunity10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicatorPDF File:KPA RISCOSS Analytics [more risk drivers] [not more risk drivers] RISCOSS Analytics Team1.Determine drivers and risk indicators2.Construct Bayesian networks3.Define scenarios4.Identify and contact experts5.Workshops planning and preparation16.Send the scenarios judgementPopulate BN nodes
  • 11. 11 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Correspond to tasks that must be done before conducting the tactical workshop Workshop procedure
  • 12. 12 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure Risk Driver Forum posts per day Forum messages per thread Mail per day Overall community size Number of developers involved Number of testers (individuals providing feedback) Number of companies using the software Companies supporting the project (adding to code) Risk Indicator Activeness
  • 13. 13 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure
  • 14. 14 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure
  • 15. 15 Pre-Tasks Tactical Workshop Organizer RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation Workshop procedure
  • 16. 16 The tactical workshops begin with an exposition about the main topics that will be covered Workshop procedure 6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II PDF File:KPA RISCOSS Analytics Tactical Workshop Organizer
  • 17. 17 Workshop procedure 9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WS Tabs Timeliness Activeness Community Tactical Workshop Organizer
  • 18. 18 Workshop procedure Part I : Community data Domain Expert 10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator
  • 19. 19 Workshop procedure Part II: Community dinamics 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicator 16.Send the scenarios judgement Domain Expert
  • 20. 20 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
  • 21. 21 Generalization of the approach
  • 22. 22 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
  • 23. 23 Applications and lessons learned Applications and lessons learned We have conducted 10 technical workshops In private and official organizations and academic institutions. The experts were from different countries. France, Israel, Italy, Spain, and Netherlands There are some inconsistencies in the scenarios The scenarios were designed using random number generators While the domain experts are conducting the tactical workshops, The degree of "calibration" of their judgement improves
  • 24. 24 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
  • 25. 25 Related Work Related Work Delphi method (QUELCE) QuantifyingUncertaintyinEarlyCostEstimation[2] Reliableconsensusofopinionofagroupofexperts[1] [1]N.DalkeyandO.Helmer,“AnexperimentalapplicationoftheDelphimethodtotheuseofexperts,”. [2]R.W.Ferguson,D.Goldenson,J.M.McCurley,R.W.Stoddard,andD.Zubrow,“QuantifyingUncertaintyinEarlyLifecycleCostEstimation(QUELCE),”
  • 26. 26 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
  • 27. 27 Domain Expert Conclusions and further work Conclusions and further work Pre-Tasks Part II: Community dinamics Part I : Community data Tactical Workshop Organizer Domain Expert 6.Make presentation of RISCOSS project summary. 7.Explain the RISCOSS analytics platform. 8.Explain the tactical workshop Part I and Part II 9.Explain the risk driver selection WS Excel File Excel File: risk driver selection WS Tabs Timeliness Activeness Community 10.Study the use case scenario 11.Assess the use case scenario overall state 12.Determine the level of the risk indicator 13.Study the use case scenario 14.Assess the use case scenario overall state 15.Determine the level of the community risk indicator PDF File:KPA RISCOSS Analytics [more risk drivers] [not more risk drivers] RISCOSS Analytics Team 1.Determine drivers and risk indicators 2.Construct Bayesian networks 3.Define scenarios 4.Identify and contact experts 5.Workshops planning and preparation 16.Send the scenarios judgement Populate BN nodes Domain Expert SNA Project Site Code Version Repository Bug Tracker Mailing List IRC Step-by-step protocol Data used to construct Bayesian networks Future work Combines opinion of domain experts with OSS raw data Empirical application & lessons
  • 28. 28 Outline Outline Background Workshop procedure Generalization of the approach Applications and lessons learned Related work Conclusions and further work
  • 30. Thanks for your attention Comments and Questions