BUG: Fix input bytes conversion in Py3 to return str #4783
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
bz2 in python 3.2 doesn't support a |
This was referenced Sep 9, 2013
|
@y-p I'm getting closer on this, but decided to give up and just read the entire file in with 3.2, then decode and rewrap with StringIO -- otherwise we couldn't support compressed files (easily) in that version... |
|
Yep, seems like a reasonable minimal workaround to me. |
|
@y-p look okay to you? going to merge if so... |
|
Yep, go ahead. |
Fixed bug with reading compressed files in as `bytes` rather than `str` in Python 3. `gzip` and `bz2` both now return `bytes` rather than `str` in Python 3, so just needed to wrap them in an `io.TextIOWrapper` to make everything work. Only wrap BytesIO and compressed streams Read entire compressed file in 3.2 to get around inconsistencies with TextIOWrapper
jtratner
added a commit
that referenced
this pull request
Sep 14, 2013
…-py3 BUG: Fix input bytes conversion in Py3 to return str
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #3963, #4785
Fixed bug with reading compressed files in as
bytes(gzipandbz2both now returnbytesratherthan
strin Python 3) rather thanstrin Python 3, as well as the lack of conversion ofBytesIO. Now,_get_handleand_wrap_compressedboth wrap in anio.TextIOWrapper, so that the parsers work internally only withstrin Python 3. In Python 3.2, has to read the entire file in first (becausegzipandbz2files both lack aread1()method in 3.2)Also adds support for passing fileobjects with compression == 'bz2'.