read_fwf/table on py3 has trouble with BytesIO

[ghost]

>>> import pandas as pd
>>> from io import BytesIO
>>> pd.read_fwf(BytesIO("שלום".encode('utf8')),widths=[2])
>>>pandas/io/parsers.py", line 1944, in <listcomp>
>>>    for (fromm, to) in self.colspecs]
>>>TypeError: Type str doesn't support the buffer API

By another path:

>>> from io import BytesIO
>>> pd.read_table(BytesIO("שלום::1234\n".encode('cp1255')),sep="::", engine='python', encoding='cp1255')
  File "/usr/local/lib/python3.3/dist-packages/pandas-0.12.0_357_g218f334-py3.3-linux-x86_64.egg/pandas/io/parsers.py", line 1324, in _read
    yield pat.split(line.decode('utf-8').strip())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf9 in position 0: invalid start byte

is broken. Note that len(sep)>1 activates the python engine anyway right now.

related #4784

Edit: fixed incorrect encoding and updated error
Edit: Updated examples

[jtratner]

I don't think your example is due to pandas...I get the same error as you have above just by running the following 2 lines:

from io import BytesIO
BytesIO("שלום".encode('cp1252')).read()

[ghost]

You're right, that's the wrong codepage (Should be cp1255) and the example doesn't trigger that code path anyway. fixed.

[jtratner]

first error appears to be resolved by #4784 (since it's just that bytes aren't being decoded). Second one still raises after #4784.

[ghost]

Glad to see you've claimed this bug @jtratner.

Just noting here that #4784 is only a half-measure fix and so I've reopened the original #3963,
adding an explanation of the problem.

[jtratner]

@y-p How would you set up the test case in this issue for Python 2? I'm not sure whether I'm encountering a bug with this in PY2 or just translating this test incorrectly.

Doesn't need to work in Python 3...I can definitely figure out how to make a cross-compatible test case if I see the Python 2 version. It would be helpful if it didn't use the unicode literals future import.

[ghost]

Not sure I follow. It's a bug that manifests only on py3, so reasonably can be tested only on py3. Not so?

[jtratner]

Closed by #4783