Skip to content

test_concurrent_futures does not exercise each mp_context method reliably due to code smell #91607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gpshead opened this issue Apr 16, 2022 · 0 comments · Fixed by #91600, #91612 or #91617
Closed

test_concurrent_futures does not exercise each mp_context method reliably due to code smell #91607

gpshead opened this issue Apr 16, 2022 · 0 comments · Fixed by #91600, #91612 or #91617
Assignees
@gpshead
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@gpshead
Copy link
Member

gpshead commented Apr 16, 2022 •
edited
Loading

While working on PRs to fix #90622 in concurrent.futures I've come across at mistakes intest_concurrent_futures.py that mean it wasn't testing the multiprocessing start methods desired of the tests in many situations. PR #91600.
@gpshead gpshead added type-bug An unexpected behavior, bug, or error tests Tests in the Lib/test dir stdlib Python modules in the Lib dir 3.11 only security fixes 3.10 only security fixes 3.9 only security fixes labels Apr 16, 2022
@gpshead gpshead self-assigned this Apr 16, 2022
gpshead added a commit that referenced this issue Apr 16, 2022
@gpshead @blurb-it
gh-91607: Fix several test_concurrent_futures tests to actually test …
7fa3a5a 
…what they claim (#91600)

* Fix test_concurrent_futures to actually test what it says.

Many ProcessPoolExecutor based tests were ignoring the mp_context
and using the default instead.  This meant we lacked proper test
coverage of all of them.

Also removes the old _prime_executor() worker delay seeding code
as it appears to have no point and causes 20-30 seconds extra
latency on this already long test.  It also interfered with some
of the refactoring to fix the above to not needlessly create their
own executor when setUp has already created an appropriate one.

* Don't import the name from multiprocessing directly to avoid confusion.

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
gpshead added a commit to gpshead/cpython that referenced this issue Apr 16, 2022
@gpshead
[3.10] pythongh-91607: Fix several test_concurrent_futures tests to a…
631fa3c 
…ctually test what they claim (pythonGH-91600)

* Fix test_concurrent_futures to actually test what it says.

Many ProcessPoolExecutor based tests were ignoring the mp_context
and using the default instead.  This meant we lacked proper test
coverage of all of them.

Also removes the old _prime_executor() worker delay seeding code
as it appears to have no point and causes 20-30 seconds extra
latency on this already long test.  It also interfered with some
of the refactoring to fix the above to not needlessly create their
own executor when setUp has already created an appropriate one.

* Don't import the name from multiprocessing directly to avoid confusion.

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>.
(cherry picked from commit 7fa3a5a)

Co-authored-by: Gregory P. Smith <[email protected]>
gpshead added a commit that referenced this issue Apr 16, 2022
@gpshead
[3.10] gh-91607: Fix several test_concurrent_futures tests to actuall…
9a45893 
…y test what they claim (GH-91600) (#91612)

* Fix test_concurrent_futures to actually test what it says.

Many ProcessPoolExecutor based tests were ignoring the mp_context
and using the default instead.  This meant we lacked proper test
coverage of all of them.

Also removes the old _prime_executor() worker delay seeding code
as it appears to have no point and causes 20-30 seconds extra
latency on this already long test.  It also interfered with some
of the refactoring to fix the above to not needlessly create their
own executor when setUp has already created an appropriate one.

* Don't import the name from multiprocessing directly to avoid confusion.

(cherry picked from commit 7fa3a5a)

Co-authored-by: Gregory P. Smith <[email protected]>
gpshead added a commit to gpshead/cpython that referenced this issue Apr 16, 2022
@gpshead
[3.9] [3.10] pythongh-91607: Fix several test_concurrent_futures test…
d701602 
…s to actually test what they claim (pythonGH-91600) (pythonGH-91612)

* Fix test_concurrent_futures to actually test what it says.

Many ProcessPoolExecutor based tests were ignoring the mp_context
and using the default instead.  This meant we lacked proper test
coverage of all of them.

Also removes the old _prime_executor() worker delay seeding code
as it appears to have no point and causes 20-30 seconds extra
latency on this already long test.  It also interfered with some
of the refactoring to fix the above to not needlessly create their
own executor when setUp has already created an appropriate one.

* Don't import the name from multiprocessing directly to avoid confusion.

(cherry picked from commit 7fa3a5a)

Co-authored-by: Gregory P. Smith <[email protected]>.
(cherry picked from commit 9a45893)

Co-authored-by: Gregory P. Smith <[email protected]>
gpshead added a commit that referenced this issue Apr 17, 2022
@gpshead
[3.9] [3.10] gh-91607: Fix several test_concurrent_futures tests to a…
2a43afd 
…ctually test what they claim (GH-91600) (GH-91612) (#91617)

* Fix test_concurrent_futures to actually test what it says.

Many ProcessPoolExecutor based tests were ignoring the mp_context
and using the default instead.  This meant we lacked proper test
coverage of all of them.

Also removes the old _prime_executor() worker delay seeding code
as it appears to have no point and causes 20-30 seconds extra
latency on this already long test.  It also interfered with some
of the refactoring to fix the above to not needlessly create their
own executor when setUp has already created an appropriate one.

* Don't import the name from multiprocessing directly to avoid confusion.

(cherry picked from commit 7fa3a5a)
(cherry picked from commit 9a45893)
@gpshead gpshead closed this as completed Apr 17, 2022
This was linked to pull requests Apr 17, 2022
gh-91607: Fix several test_concurrent_futures tests to actually test what they claim #91600
Merged
[3.10] gh-91607: Fix several test_concurrent_futures tests to test what they claim #91612
Merged
[3.9] [3.10] gh-91607: Fix several test_concurrent_futures tests to actually test what they claim (GH-91600) (GH-91612) #91617
Merged
hello-adam pushed a commit to hello-adam/cpython that referenced this issue Jun 2, 2022
@gpshead @hello-adam
[3.9] [3.10] pythongh-91607: Fix several test_concurrent_futures test…
bb9db97 
…s to actually test what they claim (pythonGH-91600) (pythonGH-91612) (python#91617)

* Fix test_concurrent_futures to actually test what it says.

Many ProcessPoolExecutor based tests were ignoring the mp_context
and using the default instead.  This meant we lacked proper test
coverage of all of them.

Also removes the old _prime_executor() worker delay seeding code
as it appears to have no point and causes 20-30 seconds extra
latency on this already long test.  It also interfered with some
of the refactoring to fix the above to not needlessly create their
own executor when setUp has already created an appropriate one.

* Don't import the name from multiprocessing directly to avoid confusion.

(cherry picked from commit 7fa3a5a)
(cherry picked from commit 9a45893)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gpshead gpshead

Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
1 participant
@gpshead