Finding Help with Programming Errors: An Exploratory Study of Novice Software Engineers’ Focus in Stack Overflow Posts
Download as PPTX, PDF0 likes62 views
The study explores how novice software engineers navigate and extract information from Stack Overflow posts, highlighting significant challenges like information overload and difficulty in determining the relevance of embedded code segments. It finds that novices focus on only 27% of the code and 15-21% of the text within posts, leading to inefficiencies in identifying solutions. Recommendations include improving the Q&A forum interface and providing better tools for extracting relevant information.
![Finding Help with Programming Errors: An Exploratory Study
of Novice Software Engineers’ Focus in Stack Overflow Posts
Preetha Chatterjee Minji Kong Lori Pollock
Journal of Systems and Software (JSS), Volume 159, Jan 2020
Asst. Prof.
[Fall ’21]
Ph.D.](https://image.slidesharecdn.com/sofocus-210408152749/85/Finding-Help-with-Programming-Errors-An-Exploratory-Study-of-Novice-Software-Engineers-Focus-in-Stack-Overflow-Posts-1-320.jpg)
![Stack Overflow Usage: Statistics and Challenges
40% participants are novice programmers (< 5 yrs experience)
who struggle to find relevant information from a Stack Overflow post!!
Information Overload:
The top 5 Stack Overflow
questions for a sample
query “Differences
Hashmap Hashtable Java”,
consist of 51 answers
which have 6,771 words in
total [Xu et. al. 2017]
https://insights.stackoverflow.com/survey/2020
We aim to understand:
How novice software engineers direct their efforts and what kinds of
information they focus on within a Stack Overflow post](https://image.slidesharecdn.com/sofocus-210408152749/85/Finding-Help-with-Programming-Errors-An-Exploratory-Study-of-Novice-Software-Engineers-Focus-in-Stack-Overflow-Posts-2-320.jpg)
Finding Help with Programming Errors: An Exploratory Study of Novice Software Engineers’ Focus in Stack Overflow Posts
- 1. Finding Help with Programming Errors: An Exploratory Study of Novice Software Engineers’ Focus in Stack Overflow Posts Preetha Chatterjee Minji Kong Lori Pollock Journal of Systems and Software (JSS), Volume 159, Jan 2020 Asst. Prof. [Fall ’21] Ph.D.
- 2. Stack Overflow Usage: Statistics and Challenges 40% participants are novice programmers (< 5 yrs experience) who struggle to find relevant information from a Stack Overflow post!! Information Overload: The top 5 Stack Overflow questions for a sample query “Differences Hashmap Hashtable Java”, consist of 51 answers which have 6,771 words in total [Xu et. al. 2017] https://insights.stackoverflow.com/survey/2020 We aim to understand: How novice software engineers direct their efforts and what kinds of information they focus on within a Stack Overflow post
- 3. Study Design RQ1. Challenges - What slows a novice software engineer down in identifying the solution most appropriate for their problem? RQ2. Focus - Which parts of a post novice software engineers focus on to reduce their time in locating information?
- 4. RQ1 Findings: Developer Challenges Survey Question 69% participants indicated “Too much text containing unnecessary details” 51% participants indicated “Determining embedded code segment relevance” 39% indicated “Too much code containing unnecessary details”
- 5. Question Title: Polymorphism in Java error: cannot find Symbol I've just started learning object oriented programming from the book head first java. It said that polymorphism enables me to create an array of the superclass type and then have all the subclasses as the array elements. But when I tried writing code using the same principles it ran into error saying error: cannot find symbol I made the classes the superclass was animal and the dog class extended the animal class having a fetch method of its own, but when I referenced the dog variable as animal it did not work here is the code … The error tester.java:4: error: cannot find symbol doggie.fetch(); symbol: method fetch() location: variable doggie of type animal When using polymorphism, if you create an instance of the subclass and store its reference in a variable of superclass type, you can only call those methods on the newly created instance which are present in the super class. In your code, you created an instance of dog class and stored its reference in doggie which is of type animal (super class of dog), In such case, you can't call any method on dog class instance that isn't available in animal class. fetch method is not defined in the animal class hence you get the error. Either define the fetch method in the animal class OR Change animal doggie = new dog(); to dog doggie = new dog(); Developer’s Goal Developer’s Pertinent Action System Symptom Accepted Answer Symptom Cause Posted Solution Solution Justification RQ2 Findings: Developer Focus
- 6. • Too much text including unnecessary details (69%) • Determining embedded code segment relevance (51%) • Too much code including unnecessary details (39%) Developer’s Goal Developer’s Pertinent Action System Symptom Symptom Cause Posted Solution Solution Justification Novice software engineers pay attention to only 27% code and 15-21% of text in a SO post! @PreethaChatterj [email protected] sites.udel.edu/preethac/ Significance: improve the Q&A forum interface guide tools for mining forums improve granularity of traceability mappings involving forum posts Paper link: https://doi.org/10.1016/j.jss.2019.110454 Summary and Overview RQ1. Challenges - What slows a novice software engineer down in identifying the solution most appropriate for their problem? RQ2. Focus - Which parts of a post novice software engineers focus on to reduce their time in locating information?
Editor's Notes
- #3: The recent Stack Overflow survey shows that 40% participants are novice programmers i.e. people with less than 5 years of coding experience. SO often contain long threads of discussions. A previous study suggests that reading answers of a popular SO post could take about 30 minutes!! Because of this information overload, Finding help from SO is especially difficult for novice software engineers. We conducted an exploratory study to understand how novice software engineers direct their efforts and what kinds of information they focus on within a post.
- #4: Specifically, our exploratory case study answers 2 research questions: To answer RQ1, we conducted a survey to gather novice software engineers’ perceptions of what slows them down in their Stack Overflow usage. For (RQ2) the software engineers first annotated 400 Stack Overflow posts to indicate what they would recommend to highlight to focus attention. Then, we analyzed their annotations, to gain more insight into their recommendations.
- #5: Findings from RQ1 suggest that, majority of participants indicated the obstacles of “Too much text containing unnecessary details”, followed by “Determining embedded code segment relevance”, and “Too much code containing unnecessary details”.
- #6: The results from RQ2 discern the kinds of information prominent in the developers’ focus. In the question we find the: Developers’ Goal: or their intended outcome. Developer’s Pertinent Action: that led to the error. Observed system response to the developer’s action. In the answer we find the: cause of the error. Suggested approach to fix the error. Reason why the suggested Solution works.
- #7: The results from this study can be leveraged to guide tools for mining forums, and improve granularity of traceability mappings involving forum posts.