How and Why Developers Migrate Python Tests (SANER 2022)
0 likes28 views
Nowadays, Python developers can rely on two major testing frameworks: unittest and pytest. Due to the benefits of pytest (e.g., fixture reuse), several relevant projects in the Python ecosystem have migrated from unittest to pytest. Despite being performed by the Python community, we are not yet aware of how systems are migrated from unittest to pytest nor the major reasons behind the migration. In this paper, we provide the first empirical study to assess testing framework migration. We analyze how and why developers migrate from unittest to pytest. We mine 100 popular Python systems and assess their migration status. We find that 34% of the systems rely on both testing frameworks and that Python projects are moving to pytest. While some systems have fully migrated, others are still migrating after a long period, suggesting that the migration is not always straightforward. Overall, the migrated test code is smaller than the original one. Furthermore, developers migrate to pytest due to several reasons, such as the easier syntax, interoperability, easier maintenance, and fixture flexibility/reuse, however, the implicit mechanics of pytest is a concern. We conclude by discussing practical implications for practitioners and researchers.
![Easier syntax: “[...] the pytest syntax is nicer, and allows us to
take advantage of things like fixtures”
Interoperability: "we can run the legacy unittests"
Easier maintenance: "It would be nice to migrate the test over
to pytest-django to get rid of most of the boilerplate code"
Fixture flexibility/reuse: "There is a powerful fixtures system
to support cleaner setup/teardown code, which supports
per-test, per-class, per-module and global fixtures"
36
RQ4: Reasons](https://image.slidesharecdn.com/howandwhydevelopersmigratepythontestssaner2022-240725125338-18376614/85/How-and-Why-Developers-Migrate-Python-Tests-SANER-2022-36-320.jpg)
![Implicit mechanics: “There's a lot of 'magic' involved in the
internals, which can be confusing”
"Be careful with the 'magic': in particular, fixtures can
sometimes be overused in ways that make test code hard to
follow because too much is happening behind the scenes"
New tool to learn, Multiple test styles and Migration duration:
"Another tool to learn for contributors [...] Either we end up
with multiple styles of test or there's a lot of work in rewriting
existing tests"
37
RQ4: Reasons](https://image.slidesharecdn.com/howandwhydevelopersmigratepythontestssaner2022-240725125338-18376614/85/How-and-Why-Developers-Migrate-Python-Tests-SANER-2022-37-320.jpg)
Ad
More Related Content
Similar to How and Why Developers Migrate Python Tests (SANER 2022) (20)
More from Andre Hora (14)
Ad
Recently uploaded (20)
![PRTG Network Monitor Crack Latest Version & Serial Key 2025 [100% Working]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250504182022-6534c7c0-thumbnail.jpg?width=560&fit=bounds)
![Get & Download Wondershare Filmora Crack Latest [2025]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250429170801-59e1b240-thumbnail.jpg?width=560&fit=bounds)
Ad
How and Why Developers Migrate Python Tests (SANER 2022)
- 1. How and Why Developers Migrate Python Tests Lívia Barbosa, Andre Hora 1 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER, 2022)
- 2. Motivation Why Python? Why test migration? 2
- 3. 3 TIOBE Index: https:/ /www.tiobe.com/tiobe-index Github Search Results One of the most popular programming languages
- 4. Testing Frameworks 4 Unittest - JUnit based - Included in the Python Standard Library
- 5. Testing Frameworks 5 Unittest - JUnit based - Included in the Python Standard Library Pytest - Third-party testing framework - Lighter-weight syntax
- 8. Class and Assert Migration (TermGraph, d56652) 8 unittest pytest
- 9. Fixture Migration (pyvim, 7e1c7b) 9 unittest pytest
- 10. 10 Problem Despite being largely performed by the Python community, we are not yet aware of: - How software projects are migrated from unittest to pytest - The reasons behind the migrations
- 11. Research Questions 0. (extension) To what extent are unittest and pytest adopted in the Python ecosystem over time? 1. (frequency) How frequent is code migrated from unittest to pytest? 2. (duration) How long does it take to migrate from unittest to pytest? 3. (transformations) What code is migrated from unittest to pytest? 4. (reasons) Why is code migrated from unittest to pytest? 11
- 12. Study design 12
- 13. Study Design 1. Case Studies: Top-100 real-world systems 2. Detecting Testing Frameworks over time 3. Assessing migration 4. Pull Requests and Issues 13
- 14. Top-100 most popular real-world systems 14 1 - Case Studies
- 15. 2 - Detecting Testing Frameworks Assess present and past versions of the system 15
- 16. 3 - Assessing migration ◉ Migration Commits ○ PyDriller ○ Verify removed and added lines per commit ◉ Migration Stage 16
- 17. 17 3 - Assessing migration
- 18. Code Transformations 18 Type of Migration Unittest API Pytest API Assert self.assert* assert Fixture setUp, tearDown & its variations @pytest.fixture, @pytest.mark.usefixtures Import import unittest import pytest Skip @unittest.skip* @pytest.mark.skip, @pytest.mark.skipif Expected failure @unittest.expectedFailure @pytest.mark.xfail
- 19. 3 - Assessing migration 19
- 20. ◉ Query Github Search API for "unittest to pytest" ◉ Filtered out false positives, resulting in 61 ◉ Thematic analysis to classify them 20 4 - Pull Request and Issues
- 21. Results RQ0: Extension, RQ1: frequency, RQ2: duration, RQ3: transformations, RQ4 reasons 21
- 22. RQ0: Extension To what extent are unittest and pytest adopted in the Python ecosystem over time? 22
- 23. RQ0: Extension To what extent are unittest and pytest adopted in the Python ecosystem over time? 23
- 24. RQ0: Extension To what extent are unittest and pytest adopted in the Python ecosystem over time? 24
- 25. RQ0: Extension To what extent are unittest and pytest adopted in the Python ecosystem over time? 25
- 26. RQ0: Extension To what extent are unittest and pytest adopted in the Python ecosystem over time? Most systems (77 out of 100) rely on unittest (20%), pytest (23%), or both (34%). Moreover, projects with unittest are moving to pytest: 66% (39 out of 59) of the systems initially with unittest now rely on pytest. 26
- 27. RQ1: Frequency How frequent is code migrated from unittest to pytest? 27
- 28. RQ1: Frequency How frequent is code migrated from unittest to pytest? From the 39 systems that started with unittest and adopted pytest over time, 28% (11) have fully migrated to pytest, 41% (16) are still migrating, and 31% (12) did not migrate. 28
- 29. RQ2: Duration How long does it take to migrate from unittest to pytest? 29
- 30. RQ2: Duration How long does it take to migrate from unittest to pytest? When the migration started? 30
- 31. RQ2: Duration How long does it take to migrate from unittest to pytest? When the migration started? The migration may be fast (up to one week) or take a long period to be concluded, from months to years. Most migrated systems (8 out of 11) tend to concentrate the migration commits, while only 3 perform the migration more sparsely. Systems start the migration in distinct development stages, from early to late ones. 31
- 32. RQ3: Transformations What code is migrated from unittest to pytest? 32
- 33. RQ3: Transformations What code is migrated from unittest to pytest? 33
- 34. RQ3: Transformations What code is migrated from unittest to pytest? The majority (90%) of the migration commits include assert migrations. Developers also tend to migrate fixtures (18%) and imports (13%). Overall, the migrated test code is 34% smaller than the original one, meaning fewer test code to be maintained. 34
- 35. RQ4: Reasons Why is code migrated from unittest to pytest? 35
- 36. Easier syntax: “[...] the pytest syntax is nicer, and allows us to take advantage of things like fixtures” Interoperability: "we can run the legacy unittests" Easier maintenance: "It would be nice to migrate the test over to pytest-django to get rid of most of the boilerplate code" Fixture flexibility/reuse: "There is a powerful fixtures system to support cleaner setup/teardown code, which supports per-test, per-class, per-module and global fixtures" 36 RQ4: Reasons
- 37. Implicit mechanics: “There's a lot of 'magic' involved in the internals, which can be confusing” "Be careful with the 'magic': in particular, fixtures can sometimes be overused in ways that make test code hard to follow because too much is happening behind the scenes" New tool to learn, Multiple test styles and Migration duration: "Another tool to learn for contributors [...] Either we end up with multiple styles of test or there's a lot of work in rewriting existing tests" 37 RQ4: Reasons
- 38. Why is code migrated from unittest to pytest? Developers migrate from unittest to pytest mostly due to the easier syntax, interoperability, easier maintenance, and fixture flexibility/reuse. Disadvantages are less discussed, but the implicit mechanics of pytest is the main concern. 38 RQ4: Reasons
- 40. ◉ Migration practices, advantages, and disadvantages. ◉ Keep track of the migration. ◉ Improve migration guidelines. 40 For practitioners
- 41. For Researchers ◉ Why the migration is not concluded. ◉ Tools and techniques to document and automate the migration. 41
- 42. Threats to Validity ◉ Framework usage and migration commits ◉ Mocking libraries ◉ Manual classification of issues/PRs ◉ Generalization of the results 42
- 43. Review To what extent are they adopted? 20% unittest, 23% pytest, 34% on both. 66% (39 out of 59) unittest projects are moving to pytest. How frequent is the migration? From the 39 systems initially with unittest, 28% fully migrated, 41% is still migrating How long does it take? May be fast or slow, most systems concentrate migration commits, and they start the migration in different development stages. 43 Research Questions and Results
- 44. Review 44 Research Questions and Results What code is migrated? 90% of the migration commits include assert migrations. Overall, the migrated test code is 34% smaller Why does it happen? Easier syntax, interoperability, easier maintenance, and fixture flexibility/reuse. Implicit mechanics is a concern.
- 45. ◉ The first empirical assess the migration of testing frameworks in Python systems ◉ How code is migrated and the reasons for the migration ◉ Propose practical implications for practitioners and researchers 45 Conclusion