Skip to content

BUG: nanops._has_infs doesn't work with many dtypes #7357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
toddrjen opened this issue Jun 5, 2014 · 4 comments
Closed

BUG: nanops._has_infs doesn't work with many dtypes #7357

toddrjen opened this issue Jun 5, 2014 · 4 comments
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Internals Related to non-user accessible pandas implementation Numeric Operations Arithmetic, Comparison, and Logical operations
Milestone

Comments

@toddrjen
Copy link
Contributor

toddrjen commented Jun 5, 2014

nanops._has_infs, which is used in determining whether to employ bottleneck optimizations, fails on dtypes which should otherwise be supported, including 'f2', complex, and multidimensional 'f4' and 'f8'.

>>> from pandas.core import nanops
>>> import numpy as np
>>> 
>>> val0 = np.ones(10)*np.inf
>>> val1 = val0.astype('f2')
>>> val2 = val0+val0*1j
>>> val3 = np.tile(val0, (1, 10))
>>> 
>>> nanops._has_infs(val0)
True
>>> nanops._has_infs(val1)
False
>>> nanops._has_infs(val2)
False
>>> nanops._has_infs(val3)
ValueError: Buffer has wrong number of dimensions (expected 1, got 2)
@jreback jreback added this to the 0.15.0 milestone Jun 5, 2014
@jreback
Copy link
Contributor

jreback commented Jun 5, 2014

FYI, dtype f2 is not ALWAYS available on all platforms/numpies (nor are some complex numbers)

you can test if the dtype.name is in _TYPE_MAP (if its numeric), see here: https://github.com/pydata/pandas/blob/master/pandas/src/inference.pyx

@jreback
Copy link
Contributor

jreback commented Jun 5, 2014

IIRC bottleneck does NOT support f2 (not sure though)

@toddrjen
Copy link
Contributor Author

toddrjen commented Jun 5, 2014

The current implementation has 2 major problems:

  1. It assumes the array is 1D
  2. It assume any array that is not an f4 or f8 dtype cannot have infinite values

These problems are both easily fixed without much additional performance overhead.

@jreback jreback modified the milestones: 0.14.1, 0.15.0 Jun 5, 2014
jreback added a commit that referenced this issue Jun 13, 2014
Fix bug where ``nanops._has_infs`` doesn't work with many dtypes (issue #7357)
@jreback
Copy link
Contributor

jreback commented Jun 13, 2014

closed by #7448

@jreback jreback closed this as completed Jun 13, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Internals Related to non-user accessible pandas implementation Numeric Operations Arithmetic, Comparison, and Logical operations
Projects
None yet
Development

No branches or pull requests

2 participants