{ "__type": "IngestedDoc", "__tag": 4010, "_content": {}, "_ordered_sections": [], "item_file": null, "item_line": null, "item_type": null, "aliases": [], "example_section_data": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "see_also": [], "signature": null, "references": null, "qa": "dev:roadmap-detailed", "arbitrary": [ { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Most of this roadmap is intended to provide a high-level view on what is most needed per SciPy submodule in terms of new functionality, bug fixes, etc. Besides important \"business as usual\" changes, it contains ideas for major new features - those are marked as such, and are expected to take significant dedicated effort. Things not mentioned in this roadmap are not necessarily unimportant or out of scope, however we (the SciPy developers) want to provide to our users and contributors a clear picture of where SciPy is going and where help is needed most." } ] }, { "__type": "Admonition", "__tag": 4056, "kind": "note", "base_type": "note", "children": [ { "__type": "AdmonitionTitle", "__tag": 4055, "children": [ { "__type": "Text", "__tag": 4046, "value": "note This is the detailed roadmap. A very high-level overview with only" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "the most important ideas is " }, { "__type": "CrossRef", "__tag": 4002, "value": "scipy-roadmap", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "dev:roadmap" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Detailed SciPy Roadmap" } ], "level": 0, "target": "scipy-roadmap-detailed" }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This roadmap will be evolving together with SciPy. Updates can be submitted as pull requests. For large or disruptive changes you may want to discuss those first on the scipy-dev forum." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "General" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "In general, we want to evolve the API to remove known warts as much as possible, " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "however as much as possible without breaking backwards compatibility" } ] }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "API changes" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Test coverage of code added in the last few years is quite good, and we aim for a high coverage for all new code that is added. However, there is still a significant amount of old code for which coverage is poor. Bringing that up to the current standard is probably not realistic, but we should plug the biggest holes." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Besides coverage there is also the issue of correctness - older code may have a few tests that provide decent statement coverage, but that doesn't necessarily say much about whether the code does what it says on the box. Therefore code review of some parts of the code (" }, { "__type": "InlineCode", "__tag": 4051, "value": "stats" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "signal" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndimage" }, { "__type": "Text", "__tag": 4046, "value": " in particular) is necessary." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Test coverage" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The documentation is in good shape. Expanding of current docstrings - adding examples, references, and better explanations - should continue. Most modules also have a tutorial in the reference guide that is a good introduction, however there are a few missing or incomplete tutorials - this should be fixed." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Documentation" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The " }, { "__type": "InlineCode", "__tag": 4051, "value": "asv" }, { "__type": "Text", "__tag": 4046, "value": "-based benchmark system is in reasonable shape. It is quite easy to add new benchmarks, however running the benchmarks is not very intuitive. Making this easier is a priority." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Benchmarks" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Cython's old syntax for using NumPy arrays should be removed and replaced with Cython memoryviews." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Binary sizes of extensions built from Cython code are large, and compile times are long. We should aim to combine extension modules where possible (e.g., " }, { "__type": "InlineCode", "__tag": 4051, "value": "stats._boost" }, { "__type": "Text", "__tag": 4046, "value": " contains many extension modules now), and limit the use of Cython to places where it's the best choice. Note that conversion of Cython to C++ is ongoing in " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.special" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Use of Cython" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Pythran is still an optional build dependency, and can be disabled with " }, { "__type": "InlineCode", "__tag": 4051, "value": "-Duse-pythran=false" }, { "__type": "Text", "__tag": 4046, "value": ". The aim is to make it a hard dependency - for that to happen it must be clear that the maintenance burden is low enough." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Use of Pythran" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "SciPy owes a lot of its success to relying on wrapping well established Fortran libraries (QUADPACK, FITPACK, ODRPACK, ODEPACK etc). The Fortran 77 that these libraries are written in is quite hard to maintain, and the use of Fortran is problematic for many reasons; e.g., it makes our wheel builds much harder to maintain, it has repeatedly been problematic for supporting new platforms like macOS arm64 and Windows on Arm, and it is highly problematic for Pyodide's SciPy support. Our goal is to remove all Fortran code from SciPy by replacing the functionality with code written in other languages. Progress towards this goal is tracked in " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-18566" } ], "url": "https://github.com/scipy/scipy/issues/18566", "title": "" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Use of Fortran libraries" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Continuous integration currently covers 32/64-bit Windows, macOS on x86-64/arm, 32/64-bit Linux on x86, and Linux on aarch64 - as well as a range of versions of our dependencies and building release quality wheels. Reliability of CI has not been good recently (H1 2023), due to the large amount of configurations to support and some CI jobs needing an overhaul. We aim to reduce build times by removing the remaining distutils-based jobs when we drop that build system and make the set of configurations in CI jobs more orthogonal." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Continuous integration" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "SciPy binaries are quite large (e.g. an unzipped manylinux wheel for 1.7.3 is 39 MB on PyPI and 122 MB after installation), and this can be problematic - for example for use in AWS Lambda, which has a 250 MB size limit. We aim to keep binary size as low as possible; when adding new compiled extensions, this needs checking. Stripping of debug symbols in " }, { "__type": "InlineCode", "__tag": 4051, "value": "multibuild" }, { "__type": "Text", "__tag": 4046, "value": " can perhaps be improved (see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "this issue" } ], "url": "https://github.com/multi-build/multibuild/issues/162", "title": "" }, { "__type": "Text", "__tag": 4046, "value": "). An effort should be made to slim down where possible, and not add new large files. In the future, things that are being considered (very tentatively) and may help are separating out the bundled " }, { "__type": "InlineCode", "__tag": 4051, "value": "libopenblas" }, { "__type": "Text", "__tag": 4046, "value": " and removing support for " }, { "__type": "InlineCode", "__tag": 4051, "value": "long double" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "Size of binaries" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [], "title": [ { "__type": "Text", "__tag": 4046, "value": "Modules" } ], "level": 1, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "dendrogram" }, { "__type": "Text", "__tag": 4046, "value": " needs a rewrite, it has a number of hard to fix open issues and feature requests." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "cluster" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This module needs only periodic updates to the numerical values." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "constants" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This module was added with the understanding that its scope would be limited. The goal is to provide basic differentiation of black-box functions, not replicate all features of existing differentiation tools. With that in mind, items for future work include:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Improve support for callables that accept additional arguments (e.g. add " }, { "__type": "InlineCode", "__tag": 4051, "value": "*args" }, { "__type": "Text", "__tag": 4046, "value": " to " }, { "__type": "InlineCode", "__tag": 4051, "value": "jacobian" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "hessian" }, { "__type": "Text", "__tag": 4046, "value": "). Note that this is not trivial due to the way elements of the arrays are eliminated when their corresponding calculations have converged." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Improve implementation of " }, { "__type": "CrossRef", "__tag": 4002, "value": "scipy.differentiate.hessian", "reference": { "__type": "RefInfo", "__tag": 4000, "module": "scipy", "version": "*", "kind": "api", "path": "scipy.differentiate._differentiate:hessian" }, "kind": "module" }, { "__type": "Text", "__tag": 4046, "value": ": rather than chaining first-order differentiation, use a second-order differentiation stencil." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Consider the addition of an option to use relative step sizes." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Consider generalizing the approach to use \"polynomial extrapolation\"; i.e., rather than estimating derivatives of a given order from the minimal number of function evaluations, use a least-squares approach to improve robustness to numerical noise." } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "differentiate" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This module is in good shape." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "fft" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Complete the feature set of the new " }, { "__type": "InlineCode", "__tag": 4051, "value": "cubature" }, { "__type": "Text", "__tag": 4046, "value": " function, and add an interface tailored to one-dimensional integrals." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Migrate functions for generating quadrature rule points and weights from " }, { "__type": "InlineRole", "__tag": 4003, "value": "special", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", improve their reliability, and add support for other important rules." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Complete the feature set of " }, { "__type": "InlineCode", "__tag": 4051, "value": "solve_ivp" }, { "__type": "Text", "__tag": 4046, "value": ", including all functionality of the " }, { "__type": "InlineCode", "__tag": 4051, "value": "odeint" }, { "__type": "Text", "__tag": 4046, "value": " interface." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Gracefully sunset superseded functions and classes." } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "integrate" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "FITPACK replacements" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "We need to finish providing functional equivalents of the FITPACK Fortran library. The aim is to reimplement the algorithmic content of the FITPACK monolith in a modular fashion, to allow better user control and extensibility. To this end, we need" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "1D splines: add the periodic spline fitting to " }, { "__type": "InlineRole", "__tag": 4003, "value": "make_splrep", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " function." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "2D splines: provide functional replacements of the " }, { "__type": "InlineRole", "__tag": 4003, "value": "BivariateSpline", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " family of classes for scattered and gridded data." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Once we have a feature-complete set of replacements, we can gracefully sunset our use of FITPACK." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "Ideas for new features" } ] } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "add the ability to construct smoothing splines with variable number of knots to " }, { "__type": "InlineRole", "__tag": 4003, "value": "make_smoothing_spline", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ". Currently, " }, { "__type": "InlineRole", "__tag": 4003, "value": "make_smoothing_spline", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " always uses all data points for the knot vector." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "experiment with user-selectable smoothing criteria: add the P-splines penalty form to complement the 2nd derivative penalty of " }, { "__type": "InlineRole", "__tag": 4003, "value": "make_smoothing_spline", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " and jumps of the 3rd derivative of " }, { "__type": "InlineRole", "__tag": 4003, "value": "make_splrep", "domain": null, "role": null, "inventory": null } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "investigate ways of vectorizing the spline construction for batches of the independent variable. This class of features has been requested by " }, { "__type": "CrossRef", "__tag": 4002, "value": "scipy.stats", "reference": { "__type": "RefInfo", "__tag": 4000, "module": "scipy", "version": "*", "kind": "api", "path": "scipy.stats" }, "kind": "module" }, { "__type": "Text", "__tag": 4046, "value": " developers" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "allow specifying the boundary conditions for spline fitting." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "investigate constrained spline fitting problems which the FITPACK library was providing in Fortran, but were never exposed to the python interfaces" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "NURBS support can be added if there is sufficient user interest" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "Scalability and performance" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "We need to check performance on large data sets and improve where lacking. This is relevant for both regular grid N-D interpolators and QHull-based scattered N-D interpolators. For the latter ones, we additionally need to investigate if we can improve on the 32-bit integer limitations of the QHull library." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "interpolate" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "wavfile:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "PCM float will be supported, for anything else use " }, { "__type": "InlineCode", "__tag": 4051, "value": "audiolab" }, { "__type": "Text", "__tag": 4046, "value": " or other specialized libraries." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Raise errors instead of warnings if data not understood." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Other sub-modules (matlab, netcdf, idl, harwell-boeing, arff, matrix market) are in good shape." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "io" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "scipy.linalg" }, { "__type": "Text", "__tag": 4046, "value": " is in good shape." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Needed:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Reduce duplication of functions with " }, { "__type": "InlineCode", "__tag": 4051, "value": "numpy.linalg" }, { "__type": "Text", "__tag": 4046, "value": ", make APIs consistent." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "get_lapack_funcs" }, { "__type": "Text", "__tag": 4046, "value": " should always use " }, { "__type": "InlineCode", "__tag": 4051, "value": "flapack" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Wrap more LAPACK functions" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "One too many funcs for LU decomposition, remove one" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Ideas for new features:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Add type-generic wrappers in the Cython BLAS and LAPACK" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Make many of the linear algebra routines into gufuncs" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Complete support for batched operations (see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-21466" } ], "url": "https://github.com/scipy/scipy/issues/21466", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ")" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "BLAS and LAPACK" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The Python and Cython interfaces to BLAS and LAPACK in " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.linalg" }, { "__type": "Text", "__tag": 4046, "value": " are one of the most important things that SciPy provides. In general " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.linalg" }, { "__type": "Text", "__tag": 4046, "value": " is in good shape, however we can make a number of improvements:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Add support for ILP64 (64-bit) BLAS and LAPACK (see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-21889" } ], "url": "https://github.com/scipy/scipy/issues/21889", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ")" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Unify the two sets of low-level BLAS/LAPACK wrappers, probably dropping the " }, { "__type": "InlineCode", "__tag": 4051, "value": "f2py" }, { "__type": "Text", "__tag": 4046, "value": "-based ones (see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-20682" } ], "url": "https://github.com/scipy/scipy/issues/20682", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ")" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Improve and document the various ways we link to BLAS and LAPACK from C and C++ code internally in SciPy (see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-20002" } ], "url": "https://github.com/scipy/scipy/issues/20002", "title": "" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-21130" } ], "url": "https://github.com/scipy/scipy/pull/21130", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ")" } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "linalg" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "All features have been removed from " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.misc" }, { "__type": "Text", "__tag": 4046, "value": ", and the namespace itself will eventually be removed." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "misc" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Underlying " }, { "__type": "InlineCode", "__tag": 4051, "value": "ndimage" }, { "__type": "Text", "__tag": 4046, "value": " is a powerful interpolation engine. Users come with an expectation of one of two models: a pixel model with " }, { "__type": "InlineCode", "__tag": 4051, "value": "(1, 1)" }, { "__type": "Text", "__tag": 4046, "value": " elements having centers " }, { "__type": "InlineCode", "__tag": 4051, "value": "(0.5, 0.5)" }, { "__type": "Text", "__tag": 4046, "value": ", or a data point model, where values are defined at points on a grid. Over time, we've become convinced that the data point model is better defined and easier to implement, but this should be clearly communicated in the documentation." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "More importantly, still, SciPy implements one " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "variant" } ] }, { "__type": "Text", "__tag": 4046, "value": " of this data point model, where datapoints at any two extremes of an axis share a spatial location under " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "periodic wrapping" } ] }, { "__type": "Text", "__tag": 4046, "value": " mode. E.g., in a 1D array, you would have " }, { "__type": "InlineCode", "__tag": 4051, "value": "x[0]" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "x[-1]" }, { "__type": "Text", "__tag": 4046, "value": " co-located. A very common use-case, however, is for signals to be periodic, with equal spacing between the first and last element along an axis (instead of zero spacing). Wrapping modes for this use-case were added in " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-8537" } ], "url": "https://github.com/scipy/scipy/pull/8537", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ", next the interpolation routines should be updated to use those modes. This should address several issues, including gh-1323, gh-1903, gh-2045 and gh-2640." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The morphology interface needs to be standardized:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "binary dilation/erosion/opening/closing take a \"structure\" argument, whereas their grey equivalent take size (has to be a tuple, not a scalar), footprint, or structure." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "a scalar should be acceptable for size, equivalent to providing that same value for each axis." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "for binary dilation/erosion/opening/closing, the structuring element is optional, whereas it's mandatory for grey. Grey morphology operations should get the same default." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "other filters should also take that default value where possible." } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "ndimage" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This module is in reasonable shape, although it could use a bit more maintenance. No major plans or wishes here." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "odr" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "We aim to continuously improve the set of optimizers provided by this module. For large scale problems, the state of the art continues to advance and we aim to keep up by leveraging implementations from domain-specific libraries like HiGHS and PRIMA. Other areas for future work include the following." } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Improve the interfaces of existing optimizers (e.g. " }, { "__type": "InlineCode", "__tag": 4051, "value": "shgo" }, { "__type": "Text", "__tag": 4046, "value": ")." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Improve usability of the benchmark system, and add features for comparing results more easily (e.g. summary plots)." } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "optimize" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "Convolution and correlation" } ] }, { "__type": "Text", "__tag": 4046, "value": ": (Relevant functions are convolve, correlate, fftconvolve, convolve2d, correlate2d, and sepfir2d.) Eliminate the overlap with " }, { "__type": "InlineRole", "__tag": 4003, "value": "ndimage", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " (and elsewhere). From " }, { "__type": "InlineCode", "__tag": 4051, "value": "numpy" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.signal" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.ndimage" }, { "__type": "Text", "__tag": 4046, "value": " (and anywhere else we find them), pick the \"best of class\" for 1-D, 2-D and n-d convolution and correlation, put the implementation somewhere, and use that consistently throughout SciPy." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "B-splines" } ] }, { "__type": "Text", "__tag": 4046, "value": ": (Relevant functions are gauss_spline, cspline1d, qspline1d, cspline2d, qspline2d, cspline1d_eval, and spline_filter.) Move the good stuff to " }, { "__type": "InlineRole", "__tag": 4003, "value": "interpolate", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " (with appropriate API changes to match how things are done in " }, { "__type": "InlineRole", "__tag": 4003, "value": "interpolate", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": "), and eliminate any duplication." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "Filter design" } ] }, { "__type": "Text", "__tag": 4046, "value": ": merge " }, { "__type": "InlineRole", "__tag": 4003, "value": "firwin", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineRole", "__tag": 4003, "value": "firwin2", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " so " }, { "__type": "InlineRole", "__tag": 4003, "value": "firwin2", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " can be removed." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "Continuous-Time Linear Systems" } ] }, { "__type": "Text", "__tag": 4046, "value": ": Further improve the performance of " }, { "__type": "InlineCode", "__tag": 4051, "value": "ltisys" }, { "__type": "Text", "__tag": 4046, "value": " (fewer internal transformations between different representations). Fill gaps in lti system conversion functions." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "Second Order Sections" } ] }, { "__type": "Text", "__tag": 4046, "value": ": Make SOS filtering equally capable as existing methods. This includes ltisys objects, an " }, { "__type": "InlineRole", "__tag": 4003, "value": "lfiltic", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " equivalent, and numerically stable conversions to and from other filter representations. SOS filters could be considered as the default filtering method for ltisys objects, for their numerical stability." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "signal" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The sparse matrix formats are mostly feature-complete, however the main issue is that they act like " }, { "__type": "InlineCode", "__tag": 4051, "value": "numpy.matrix" }, { "__type": "Text", "__tag": 4046, "value": " (which will be deprecated in NumPy at some point)." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "What we want is sparse arrays that act like " }, { "__type": "InlineCode", "__tag": 4051, "value": "numpy.ndarray" }, { "__type": "Text", "__tag": 4046, "value": ". Initial support for a new set of classes (" }, { "__type": "InlineCode", "__tag": 4051, "value": "csr_array" }, { "__type": "Text", "__tag": 4046, "value": " et al.) was added in SciPy " }, { "__type": "InlineCode", "__tag": 4051, "value": "1.8.0" }, { "__type": "Text", "__tag": 4046, "value": " and stabilized in " }, { "__type": "InlineCode", "__tag": 4051, "value": "1.12.0" }, { "__type": "Text", "__tag": 4046, "value": " with construction functions for arrays, " }, { "__type": "InlineCode", "__tag": 4051, "value": "1.14.0" }, { "__type": "Text", "__tag": 4046, "value": " with 1D array support and " }, { "__type": "InlineCode", "__tag": 4051, "value": "1.15.0" }, { "__type": "Text", "__tag": 4046, "value": " with 1D indexing. The sparse array codebase now supports all sparse matrix features and in addition supports 1D arrays and the first steps toward nD arrays. There is a transition guide to help users and libraries convert their code to sparse arrays." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Next steps toward sparse array conversion:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "DefList", "__tag": 4033, "children": [ { "__type": "DefListItem", "__tag": 4037, "dt": { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Extend sparse array API to nD arrays." } ] }, "dd": [ { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Support for COO, CSR and DOK formats." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Some COO features exist in 1.15." } ] } ] } ] } ] } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Introduce support for broadcasting in operations where sparse formats can effectively do that." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Help other libraries convert to sparse arrays from sparse matrices. NetworkX, dipy, scikit-image, pyamg, cvxpy and scikit-learn are in progress or have completed conversion to sparse arrays." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Add deprecation warnings for sparse matrix." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Work with NumPy on deprecation/removal of " }, { "__type": "InlineCode", "__tag": 4051, "value": "numpy.matrix" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Deprecate and then remove sparse matrix in favor of sparse array." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "DefList", "__tag": 4033, "children": [ { "__type": "DefListItem", "__tag": 4037, "dt": { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Start API shift of construction function names (" }, { "__type": "InlineCode", "__tag": 4051, "value": "diags" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "block" }, { "__type": "Text", "__tag": 4046, "value": ", etc.)" } ] }, "dd": [ { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Note: as a whole, the construction functions undergo two name shifts. Once to move from matrix creation to new functions for array creation (i.e. " }, { "__type": "InlineCode", "__tag": 4051, "value": "eye" }, { "__type": "Text", "__tag": 4046, "value": " -> " }, { "__type": "InlineCode", "__tag": 4051, "value": "eye_array" }, { "__type": "Text", "__tag": 4046, "value": "). Then a second move to change names to match the " }, { "__type": "InlineCode", "__tag": 4051, "value": "array_api" }, { "__type": "Text", "__tag": 4046, "value": " name (i.e. " }, { "__type": "InlineCode", "__tag": 4051, "value": "eye_array" }, { "__type": "Text", "__tag": 4046, "value": " to " }, { "__type": "InlineCode", "__tag": 4051, "value": "eye" }, { "__type": "Text", "__tag": 4046, "value": ") after sparse matrices are removed. We will keep the " }, { "__type": "InlineCode", "__tag": 4051, "value": "*_array" }, { "__type": "Text", "__tag": 4046, "value": " versions for a long time as (maybe hidden) aliases." } ] } ] } ] } ] } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Add construction function names matching " }, { "__type": "InlineCode", "__tag": 4051, "value": "array_api" }, { "__type": "Text", "__tag": 4046, "value": " names." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Deprecate the transition construction function names." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "An alternative (more ambitious, and unclear if it will materialize) plan is being worked on in https://github.com/pydata/sparse. To support that effort we aim to support PyData Sparse in all functions that accept sparse arrays. Support for " }, { "__type": "InlineCode", "__tag": 4051, "value": "pydata/sparse" }, { "__type": "Text", "__tag": 4046, "value": " in " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.sparse.linalg" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.sparse.csgraph" }, { "__type": "Text", "__tag": 4046, "value": " is mostly complete." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Regarding the different sparse matrix formats: there are a lot of them. These should be kept, but improvements/optimizations should go into CSR/CSC, which are the preferred formats. LIL may be the exception, it's inherently inefficient. It could be dropped if DOK is extended to support all the operations LIL currently provides." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "sparse" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This module is in good shape." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "sparse.csgraph" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "There are a significant number of open issues for " }, { "__type": "InlineCode", "__tag": 4051, "value": "_arpack" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "lobpcg" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "_isolve" }, { "__type": "Text", "__tag": 4046, "value": ":" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "callback keyword is inconsistent" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "tol keyword is broken, should be relative tol" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Fortran code not re-entrant (but we don't solve, maybe reuse from PyKrilov)" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "_dsolve" }, { "__type": "Text", "__tag": 4046, "value": ":" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "add license-compatible sparse Cholesky or incomplete Cholesky" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "add license-compatible sparse QR" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "improve interface to SuiteSparse UMFPACK" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "add interfaces to SuiteSparse CHOLMOD and SPQR" } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "sparse.linalg" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "QHull wrappers are in good shape, as is " }, { "__type": "InlineCode", "__tag": 4051, "value": "KDTree" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A rewrite of " }, { "__type": "InlineCode", "__tag": 4051, "value": "spatial.distance" }, { "__type": "Text", "__tag": 4046, "value": " metrics in C++ is in progress - this should improve performance, make behaviour (e.g., for various non-float64 input dtypes) more consistent, and fix a few remaining issues with definitions of the math implement by a few of the metrics." } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "spatial" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Though there are still a lot of functions that need improvements in precision, probably the only show-stoppers are hypergeometric functions, parabolic cylinder functions, and spheroidal wave functions. Three possible ways to handle this:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Get good double-precision implementations. This is doable for parabolic cylinder functions (in progress). I think it's possible for hypergeometric functions, though maybe not in time. For spheroidal wavefunctions this is not possible with current theory." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Port Boost's arbitrary precision library and use it under the hood to get double precision accuracy. This might be necessary as a stopgap measure for hypergeometric functions; the idea of using arbitrary precision has been suggested before by @nmayorov and in " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-5349" } ], "url": "https://github.com/scipy/scipy/issues/5349", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ". Likely necessary for spheroidal wave functions, this could be reused: https://github.com/radelman/scattering." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Add clear warnings to the documentation about the limits of the existing implementations." } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "special" } ], "level": 2, "target": null }, { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.stats" }, { "__type": "Text", "__tag": 4046, "value": " subpackage aims to provide fundamental statistical methods as might be covered in standard statistics texts such as Johnson's \"Miller & Freund's Probability and Statistics for Engineers\", Sokal & Rohlf's \"Biometry\", or Zar's \"Biostatistical Analysis\". It does not seek to duplicate the advanced functionality of downstream packages (e.g. StatsModels, LinearModels, PyMC3); instead, it can provide a solid foundation on which they can build. (Note that these are rough guidelines, not strict rules. \"Advanced\" is an ill-defined and subjective term, and \"advanced\" methods may also be included in SciPy, especially if no other widely used and well-supported package covers the topic. Also note that " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "some" } ] }, { "__type": "Text", "__tag": 4046, "value": " duplication with downstream projects is inevitable and not necessarily a bad thing.)" } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The following improvements will help SciPy better serve this role." } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Improve statistical tests: include methods for generating confidence intervals, and implement exact and randomized p-value calculations - considering the possibility of ties - where computationally feasible." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Extend the new univariate distribution infrastructure, adding support for discrete distributions and circular continuous distributions. Add a selection of the most widely used statistical distributions under the new infrastructure, performing rigorous accuracy testing and documenting the results. Enable users to create custom distributions that leverage the new infrastructure." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Improve the multivariate distribution infrastructure to ensure a consistent API, thorough testing, and complete documentation." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Continue to make the APIs of functions more consistent, with standard support for batched calculations, NaN policies, and dtype preservation." } ] } ] } ] } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "stats" } ], "level": 2, "target": null } ], "local_refs": [] }