release:1.18.0-notes

NumPy 1.18.0 Release Notes

In addition to the usual bug fixes, this NumPy release cleans up and documents the new random C-API, expires a large number of old deprecations, and improves the appearance of the documentation. The Python versions supported are 3.5-3.8. This is the last NumPy release series that will support Python 3.5.

Downstream developers should use Cython >= 0.29.14 for Python 3.8 support and OpenBLAS >= 3.7 to avoid problems on the Skylake architecture.

Highlights

• The C-API for numpy.random has been defined and documented.

• Basic infrastructure for linking with 64 bit BLAS and LAPACK libraries.

• Many documentation improvements.

New functions

Multivariate hypergeometric distribution added to numpy.random

The method multivariate_hypergeometric has been added to the class numpy.random.Generator. This method generates random variates from the multivariate hypergeometric probability distribution. (gh-13794)

Deprecations

np.fromfile and np.fromstring will error on bad data

In future numpy releases, the functions np.fromfile and np.fromstring will throw an error when parsing bad data. This will now give a DeprecationWarning where previously partial or even invalid data was silently returned. This deprecation also affects the C defined functions PyArray_FromString and PyArray_FromFile (gh-13605)

Deprecate non-scalar arrays as fill values in ma.fill_value

Setting a MaskedArray.fill_value to a non-scalar array is deprecated since the logic to broadcast the fill value to the array is fragile, especially when slicing. (gh-13698)

Deprecate PyArray_As1D, PyArray_As2D

PyArray_As1D, PyArray_As2D are deprecated, use PyArray_AsCArray instead (gh-14036)

Deprecate np.alen

np.alen was deprecated. Use len instead. (gh-14181)

Deprecate the financial functions

In accordance with NEP-32, the financial functions fv ipmt, irr, mirr, nper, npv, pmt, ppmt, pv and rate are deprecated, and will be removed from NumPy 1.20.The replacement for these functions is the Python package numpy-financial. (gh-14720)

The axis argument to numpy.ma.mask_cols and numpy.ma.mask_row is deprecated

This argument was always ignored. (gh-14996)

Expired deprecations

• PyArray_As1D and PyArray_As2D have been removed in favor of PyArray_AsCArray (gh-14036)

• np.rank has been removed. This was deprecated in NumPy 1.10 and has been replaced by np.ndim. (gh-14039)

• The deprecation of expand_dims out-of-range axes in 1.13.0 has expired. (gh-14051)

• PyArray_FromDimsAndDataAndDescr and PyArray_FromDims have been removed (they will always raise an error). Use PyArray_NewFromDescr and PyArray_SimpleNew instead. (gh-14100)

• numeric.loads, numeric.load, np.ma.dump, np.ma.dumps, np.ma.load, np.ma.loads are removed, use pickle methods instead (gh-14256)

• arrayprint.FloatFormat, arrayprint.LongFloatFormat has been removed, use FloatingFormat instead

• arrayprint.ComplexFormat, arrayprint.LongComplexFormat has been removed, use ComplexFloatingFormat instead

• arrayprint.StructureFormat has been removed, use StructureVoidFormat instead (gh-14259)

• np.testing.rand has been removed. This was deprecated in NumPy 1.11 and has been replaced by np.random.rand. (gh-14325)

• Class SafeEval in numpy/lib/utils.py has been removed. This was deprecated in NumPy 1.10. Use np.safe_eval instead. (gh-14335)

• Remove deprecated support for boolean and empty condition lists in np.select (gh-14583)

• Array order only accepts 'C', 'F', 'A', and 'K'. More permissive options were deprecated in NumPy 1.11. (gh-14596)

• np.linspace parameter num must be an integer. Deprecated in NumPy 1.12. (gh-14620)

• UFuncs with multiple outputs must use a tuple for the out kwarg. This finishes a deprecation started in NumPy 1.10. (gh-14682)

The files numpy/testing/decorators.py, numpy/testing/noseclasses.py and numpy/testing/nosetester.py have been removed. They were never meant to be public (all relevant objects are present in the numpy.testing namespace), and importing them has given a deprecation warning since NumPy 1.15.0 (gh-14567)

Compatibility notes

numpy.lib.recfunctions.drop_fields can no longer return None

If drop_fields is used to drop all fields, previously the array would be completely discarded and None returned. Now it returns an array of the same shape as the input, but with no fields. The old behavior can be retained with

dropped_arr = drop_fields(arr, ['a', 'b'])
if dropped_arr.dtype.names == ():
    dropped_arr = None

converting the empty recarray to None (gh-14510)

numpy.argmin/argmax/min/max returns NaT if it exists in array

numpy.argmin, numpy.argmax, numpy.min, and numpy.max will return NaT if it exists in the array. (gh-14717)

np.can_cast(np.uint64, np.timedelta64, casting='safe') is now False

Previously this was True - however, this was inconsistent with uint64 not being safely castable to int64, and resulting in strange type resolution.

If this impacts your code, cast uint64 to int64 first. (gh-14718)

Changed random variate stream from numpy.random.Generator.integers

There was a bug in numpy.random.Generator.integers that caused biased sampling of 8 and 16 bit integer types. Fixing that bug has changed the output stream from what it was in previous releases. (gh-14777)

Add more ufunc loops for datetime64, timedelta64

np.datetime('NaT') should behave more like float('Nan'). Add needed infrastructure so np.isinf(a) and np.isnan(a) will run on datetime64 and timedelta64 dtypes. Also added specific loops for numpy.fmin and numpy.fmax that mask NaT. This may require adjustment to user- facing code. Specifically, code that either disallowed the calls to numpy.isinf or numpy.isnan or checked that they raised an exception will require adaptation, and code that mistakenly called numpy.fmax and numpy.fmin instead of numpy.maximum or numpy.minimum respectively will require adjustment. This also affects numpy.nanmax and numpy.nanmin. (gh-14841)

Moved modules in numpy.random

As part of the API cleanup, the submodules in numpy.random bit_generator, philox, pcg64, sfc64, common, generator, and bounded_integers were moved to _bit_generator, _philox, _pcg64, _sfc64, _common, _generator, and _bounded_integers respectively to indicate that they are not part of the public interface. (gh-14608)

C API changes

PyDataType_ISUNSIZED(descr) now returns False for structured datatypes

Previously this returned True for any datatype of itemsize 0, but now this returns false for the non-flexible datatype with itemsize 0, np.dtype([]). (gh-14393)

New Features

Add our own *.pxd cython import file

Added a numpy/__init__.pxd file. It will be used for cimport numpy (gh-12284)

A tuple of axes can now be input to expand_dims

The numpy.expand_dims axis keyword can now accept a tuple of axes. Previously, axis was required to be an integer. (gh-14051)

Support for 64-bit OpenBLAS

Added support for 64-bit (ILP64) OpenBLAS. See site.cfg.example for details. (gh-15012)

Add --f2cmap option to F2PY

Allow specifying a file to load Fortran-to-C type map customizations from. (gh-15113)

Improvements

Different C numeric types of the same size have unique names

On any given platform, two of np.intc, np.int_, and np.longlong would previously appear indistinguishable through their repr, despite their corresponding dtype having different properties. A similar problem existed for the unsigned counterparts to these types, and on some platforms for np.double and np.longdouble

These types now always print with a unique __name__. (gh-10151)

argwhere now produces a consistent result on 0d arrays

On N-d arrays, numpy.argwhere now always produces an array of shape (n_non_zero, arr.ndim), even when arr.ndim == 0. Previously, the last axis would have a dimension of 1 in this case. (gh-13610)

Add axis argument for random.permutation and random.shuffle

Previously the random.permutation and random.shuffle functions can only shuffle an array along the first axis; they now have a new argument axis which allows shuffle along a specified axis. (gh-13829)

method keyword argument for np.random.multivariate_normal

A method keyword argument is now available for np.random.multivariate_normal with possible values {'svd', 'eigh', 'cholesky'}. To use it, write np.random.multivariate_normal(..., method=<method>). (gh-14197)

Add complex number support for numpy.fromstring

Now numpy.fromstring can read complex numbers. (gh-14227)

numpy.unique has consistent axes order when axis is not None

Using moveaxis instead of swapaxes in numpy.unique, so that the ordering of axes except the axis in arguments will not be broken. (gh-14255)

numpy.matmul with boolean output now converts to boolean values

Calling numpy.matmul where the output is a boolean array would fill the array with uint8 equivalents of the result, rather than 0/1. Now it forces the output to 0 or 1 (NPY_TRUE or NPY_FALSE). (gh-14464)

numpy.random.randint produced incorrect value when the range was 2**32

The implementation introduced in 1.17.0 had an incorrect check when determining whether to use the 32-bit path or the full 64-bit path that incorrectly redirected random integer generation with a high - low range of 2**32 to the 64-bit generator. (gh-14501)

Add complex number support for numpy.fromfile

Now numpy.fromfile can read complex numbers. (gh-14730)

std=c99 added if compiler is named gcc

GCC before version 5 requires the -std=c99 command line argument. Newer compilers automatically turn on C99 mode. The compiler setup code will automatically add the code if the compiler name has gcc in it. (gh-14771)

Changes

NaT now sorts to the end of arrays

NaT is now effectively treated as the largest integer for sorting purposes, so that it sorts to the end of arrays. This change is for consistency with NaN sorting behavior. (gh-12658) (gh-15068)

Incorrect threshold in np.set_printoptions raises TypeError or ValueError

Previously an incorrect threshold raised ValueError; it now raises TypeError for non-numeric types and ValueError for nan values. (gh-13899)

Warn when saving a dtype with metadata

A UserWarning will be emitted when saving an array via numpy.save with metadata. Saving such an array may not preserve metadata, and if metadata is preserved, loading it will cause a ValueError. This shortcoming in save and load will be addressed in a future release. (gh-14142)

numpy.distutils append behavior changed for LDFLAGS and similar

numpy.distutils has always overridden rather than appended to LDFLAGS and other similar such environment variables for compiling Fortran extensions. Now the default behavior has changed to appending - which is the expected behavior in most situations. To preserve the old (overwriting) behavior, set the NPY_DISTUTILS_APPEND_FLAGS environment variable to 0. This applies to: LDFLAGS, F77FLAGS, F90FLAGS, FREEFLAGS, FOPT, FDEBUG, and FFLAGS. NumPy 1.16 and 1.17 gave build warnings in situations where this change in behavior would have affected the compile flags used. (gh-14248)

Remove numpy.random.entropy without a deprecation

numpy.random.entropy was added to the numpy.random namespace in 1.17.0. It was meant to be a private c-extension module, but was exposed as public. It has been replaced by numpy.random.SeedSequence so the module was completely removed. (gh-14498)

Add options to quiet build configuration and build with -Werror

Added two new configuration options. During the build_src subcommand, as part of configuring NumPy, the files _numpyconfig.h and config.h are created by probing support for various runtime functions and routines. Previously, the very verbose compiler output during this stage clouded more important information. By default the output is silenced. Running runtests.py --debug-info will add --verbose-cfg to the build_src subcommand,which will restore the previous behaviour.

Adding CFLAGS=-Werror to turn warnings into errors would trigger errors during the configuration. Now runtests.py --warn-error will add --warn-error to the build subcommand, which will percolate to the build_ext and build_lib subcommands. This will add the compiler flag to those stages and turn compiler warnings into errors while actually building NumPy itself, avoiding the build_src subcommand compiler calls.

(gh-14527) (gh-14518)