pandas.read_hdf leaves file open when requested data key does not exist

[RomanKoziolek]

Code Sample, a copy-pastable example if possible

import pandas

data = pandas.read_hdf(<h5_file_name>, mode='r', key=key)

Problem description

When trying to read HDF5 file and the requested key object does not exist in the file, pandas pytables raises KeyError('No object named {key} in the file'.) and does not close the opened file. I would expect that the HDF5 file is closed after trying to read the key object, whether it was successful or not.
Trying to write to the file which is opened in read mode fails with exception.
The problematic code is in pandas.io.pytables.py file, method select, line 712.

Workaround

Check if the key exists before trying to read it from file.

import h5py
key_exists = True
with h5py.File(<h5_file_name>, mode='r') as file:
    group = file.get(key)
    if not group:
       key_exists = False

if key_exists:
    ...

Should you need more details, please let me know.

[WillAyd]

What version of pandas are you using? Have you tried on master? IIRC was fixed recently

…

Sent from my iPhone

On Jun 19, 2019, at 10:14 AM, Roman Koziolek ***@***.***> wrote: Code Sample, a copy-pastable example if possible import pandas data = pandas.read_hdf(<h5_file_name>, mode='r', key=key) Problem description When trying to read HDF5 file and the requested key object does not exist in the file, pandas pytables raises KeyError('No object named {key} in the file'.) and does not close the opened file. I would expect that the HDF5 file is closed after trying to read the key object, whether it was successful or not. Trying to write to the file which is opened in read mode fails with exception. The problematic code is in pandas.io.pytables.py file, method select, line 712. Workaround Check if the key exists before trying to read it from file. import h5py key_exists = True with h5py.File(<h5_file_name>, mode='r') as file: group = file.get(key) if not group: key_exists = False if key_exists: ... Should you need more details, please let me know. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[RomanKoziolek]

I'm using 0.24.1 but I see the same code is on master. The select method just raises exception, does not close the file.

[WillAyd]

Do you mind trying on master?

…

Sent from my iPhone

On Jun 19, 2019, at 10:41 AM, Roman Koziolek ***@***.***> wrote: I'm using 0.24.1 but I see the same code is on master. The select method just raises exception, does not close the file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[RomanKoziolek]

I'll verify that on Friday

[rbenes]

I fixed very similar issue (maybe the same) few weeks ago #25766 (it will be in next release)

[RomanKoziolek]

@rbenes your fix solves my issue.
@WillAyd, @rbenes thank you guys!