Toolchain Roadmap

The use of the SciPy library requires (or optionally depends upon) several other libraries in order to operate, the main dependencies being Python and NumPy. It requires a larger collection of libraries and tools in order to build the library or to build the documentation.

Of course, the tooling and libraries are themselves not static. This document aims to provide a guide as to how SciPy's use of these dynamic dependencies will proceed over time.

SciPy aims to be compatible with a number of releases of its dependent libraries and tools. Forcing the user base to upgrade other components for every release would greatly diminish the value of SciPy. However, maintaining backwards compatibility with very old tooling/libraries imposes limitations on which newer functionalities and capabilities can be incorporated. SciPy takes a somewhat conservative approach, maintaining compatibility with several major releases of Python and NumPy on the major platforms. (That may in and of itself impose further restrictions. See the C Compilers section for an example.)

• First and foremost, SciPy is a Python project, hence it requires a Python environment.

• BLAS and LAPACK numerical libraries need to be installed.

• Compilers for C, C++, Fortran code are needed, as well as for Cython & Pythran (the latter is opt-out currently)

• The Python environment needs the numpy package to be installed.

• Testing requires the pytest and hypothesis Python packages.

• Building the documentation requires the matplotlib, Sphinx and MyST-NB packages along with PyData theme.

The tooling used to build CPython has some implications for the tooling used in building SciPy. It also has implications for the examples used in the documentation (e.g., docstrings for functions), as these examples can only use functionality present in all supported configurations.

Building SciPy

Python Versions

SciPy is compatible with several versions of Python. When dropping support for older Python versions, SciPy takes guidance from [NEP29]. Generally, support for the oldest Python version is dropped 42 months after the original release. Following the acceptance of PEP 602, this mostly happens in April, and gets picked up by the mid-year release of SciPy.

NumPy

SciPy depends on NumPy but releases of SciPy are not tied to releases of NumPy. SciPy attempts to be compatible with at least the 4 previous releases of NumPy. In particular, SciPy cannot rely on features of just the latest NumPy, but needs to be written using what is common in all of those 4 NumPy releases.

In specific cases, such as a particular architecture, these requirements could vary. Please check the release notes and the meta-package oldest-supported-numpy for more info [OSN].

Compilers

Building SciPy requires compilers for C, C++, Fortran, as well as the python transpilers Cython and Pythran (the latter is an opt-out dependency starting from version 1.7.0).

To maintain compatibility with a large number of platforms & setups, especially where using the official wheels (or other distribution channels like Anaconda or conda-forge) is not possible, SciPy tries to keep compatibility with older compilers, on platforms that have not yet reached their official end-of-life.

As explained in more detail below, the current minimal compiler versions are:

==========  ===========================  ===============================  ============================
 Compiler    Default Platform (tested)    Secondary Platform (untested)    Minimal Version
==========  ===========================  ===============================  ============================
 GCC         Linux                        AIX, Alpine Linux, OSX           GCC 9.x
 LLVM        OSX                          Linux, FreeBSD, Windows          LLVM 12.x
 MSVC        Windows                      -                                Visual Studio 2019 (vc142)
==========  ===========================  ===============================  ============================

Note that there is currently no dedicated CI job to test the minimum supported LLVM/Clang version. Older versions than used in SciPy CI should work, as long as they support core (non-stdlib) C++17. Please file an issue if you encounter a problem during compilation.

Official Builds

Currently, SciPy wheels are being built as follows:

=========================   ==============================   ====================================   =============================
 Platform                    `CI`_ `Base`_ `Images`_          Compilers                              Comment
=========================   ==============================   ====================================   =============================
 Linux x86                   ``ubuntu-22.04``                 GCC 10.2.1                             ``cibuildwheel``
 Linux arm                   ``docker-builder-arm64``         GCC 11.3.0                             ``cibuildwheel``
 OSX x86_64 (OpenBLAS)       ``macos-12``                     Apple clang 13.1.6/gfortran 11.3.0     ``cibuildwheel``
 OSX x86_64 (Accelerate)     ``macos-13``                     Apple clang 15.0.0/gfortran 13.2.0     ``cibuildwheel``
 OSX arm64 (OpenBLAS)        ``macos-14``                     Apple clang 15.0.0/gfortran 12.1.0     ``cibuildwheel``
 OSX arm64 (Accelerate)      ``macos-14``                     Apple clang 15.0.0/gfortran 13.2.0     ``cibuildwheel``
 Windows                     ``windows-2019``                 GCC 10.3.0 (`rtools`_)                 ``cibuildwheel``
=========================   ==============================   ====================================   =============================

Note that the OSX wheels additionally vendor gfortran 11.3.0 for x86_64, and gfortran 12.1.0 for arm64. See tools/wheels/cibw_before_build_macos.sh.

C Compilers

SciPy is compatible with most modern C compilers (in particular clang). Nowadays, there is reasonable support for recent C language standards across all relevant compilers, though this is very different from how things used to be. The following paragraphs primarily discuss the evolution of these constraints; readers who do not care about historical context can skip ahead to the table at the end.

==============  =================  =================  =================
SciPy version    CPython support    MS Visual C++      Toolset version
==============  =================  =================  =================
 Until 1.2       2.7 & 3.4+         VS 2008 (9.0)      vc90
 1.3, 1.4        3.5+               VS 2010 (10.0)     vc100
 1.5             3.6+               VS 2015 (14.0)     vc140
 1.6, 1.7        3.7+               VS 2017 (14.1)     vc141
 1.8             3.8+               VS 2017 (14.1)     vc141
 1.9             3.8+               VS 2019 (14.20)    vc142
==============  =================  =================  =================

In terms of C language standards, it's relevant to note that C11 has optional features (e.g. atomics, threading), some of which (VLAs & complex types) were mandatory in the C99 standard. C17 (occasionally called C18) can be considered a bug fix for C11, so generally, C11 may be skipped entirely.

SciPy has been restricted in the use of more advanced language features by the available compiler support, and Microsoft in particular has taken very long to achieve conformance to C99/C11/C17, however starting from Visual Studio 16.8, C11/C17 is supported (though without the C11 optional features). C99 <complex.h> support would be particularly interesting for SciPy. However, it's still possible to use complex types on windows, provided that windows-specific types are used.

Therefore, using C features beyond C90 was only possible insofar as there was support on Windows; however, as of as of the end of 2021, a sufficiently recent compiler is used. This is because GCC & LLVM support all relevant C11 features with the oldest currently used versions, and C17 is just a bugfix for C11, as mentioned above. In short:

================  =======================================================================
 Date              C Standard
================  =======================================================================
 <= 2018           C90
 2019              C90 for old code, may consider C99 for new
 2020              C99 (no ``<complex.h>``, ``<stdatomic.h>``, ``<threads.h>`` & VLAs)
 2021              C17 (no ``<complex.h>``, ``<stdatomic.h>``, ``<threads.h>`` & VLAs)
 ?                 C23, ``<complex.h>``, ``<stdatomic.h>``, ...
================  =======================================================================

C++ Language Standards

C++ language standards for SciPy are generally guidelines rather than official decisions. This is particularly true of attempting to predict adoption timelines for newer standards.

================  =======================================================================
 Date              C++ Standard
================  =======================================================================
 <= 2019           C++03
 2020              C++11
 2021              C++14
 2022              C++17 (core language + universally available stdlib features)
 ?                 C++17 (with full stdlib), C++20, C++23, C++26
================  =======================================================================

This leaves all the main platforms and their compilers with comparatively recent versions. However, SciPy has historically also endeavored to support less common platforms as well - if not with binary artefacts (i.e. wheels), then at least by remaining compilable from source - which includes for example AIX, Alpine Linux and FreeBSD.

All the currently lowest-supported compiler versions (GCC 9, LLVM 14, VS2019 with vc142) have full support for the C++17 core language, which can therefore be used unconditionally. However, as of mid-2024, support for the entirety of the C++17 standard library has not yet been completed across all compilers [CPP], particularly LLVM. It is therefore necessary to check if a given stdlib-feature is supported by all compilers before it can be used in SciPy.

C++20 support is stabilizing very slowly, even aside from modules, coroutines and several not-yet-universally-supported stdlib features. Given how big of a release the C++20 standard was, it is expected that it will take a while yet before we can start considering moving our baseline. Compiler support for C++23 and C++26 is still under heavy development [CPP].

Fortran Compilers

Generally, any well-maintained compiler is likely suitable and can be used to build SciPy. That said, we do not test with old gfortran versions, which is why we are matching the lower bound with the one for GCC above.

============= =====================================
 Tool          Version
============= =====================================
gfortran       >= 9.x
ifort/ifx      A recent version (not tested in CI)
flang (LLVM)   >= 17.x
============= =====================================

Cython & Pythran

SciPy always requires a recent Cython compiler. Since 1.7, Pythran is a build dependency (currently with the possibility to opt out).

OpenMP support

For various reasons, SciPy cannot be distributed with built-in OpenMP support. When using the optional Pythran support, OpenMP-enabled parallel code can be generated when building from source.

Other Libraries

Any library conforming to the BLAS/LAPACK interface may be used. OpenBLAS, ATLAS, MKL, BLIS, and reference Netlib libraries are known to work.

=============== =====================================================
 Library           Minimum version
=============== =====================================================
LAPACK           3.7.1
BLAS             A recent version of OpenBLAS, MKL or ATLAS.
                 The Accelerate BLAS library is no longer supported.
=============== =====================================================

There are some additional optional dependencies.

=============== ======== ==========================================
 Library        Version   URL
=============== ======== ==========================================
mpmath          Recent    http://mpmath.org/
scikit-umfpack  Recent    https://pypi.org/project/scikit-umfpack/
pooch           Recent    https://pypi.org/project/pooch/
=============== ======== ==========================================

Moreover, SciPy supports interaction with other libraries. The test suite has additional compatibility tests that are run when these are installed:

=========================  ========  ====================================
 Tool                      Version    URL
=========================  ========  ====================================
pydata/sparse              Recent     https://github.com/pydata/sparse/
=========================  ========  ====================================

Testing and Benchmarking

Testing and benchmarking require recent versions of:

=========================  ========  ====================================
 Tool                      Version    URL
=========================  ========  ====================================
pytest                     Recent     https://docs.pytest.org/en/latest/
Hypothesis                 Recent     https://hypothesis.readthedocs.io/
asv (airspeed velocity)    Recent     https://asv.readthedocs.io/
=========================  ========  ====================================

Building the Documentation

============================  =================================================
 Tool                         Version
============================  =================================================
Sphinx                        Whatever recent versions work. >= 5.0.
PyData Sphinx theme           Whatever recent versions work. >= 0.15.2.
Sphinx-Design                 Whatever recent versions work. >= 0.4.0.
numpydoc                      Whatever recent versions work. >= 1.5.0.
matplotlib                    Generally suggest >= 3.5.
MyST-NB                       Whatever recent versions work. >= 0.17.1
jupyterlite-sphinx            Whatever recent versions work. >= 0.17.1
jupyterlite-pyodide-kernel    Whatever recent versions work. >= 0.1.0
============================  =================================================

Packaging

A Recent version of:

=============  ========  =============================================
 Tool          Version    URL
=============  ========  =============================================
setuptools     Recent     https://pypi.org/project/setuptools/
wheel          Recent     https://pythonwheels.com
multibuild     Recent     https://github.com/matthew-brett/multibuild
=============  ========  =============================================

making-a-release and distributing-a-release contain information on making and distributing a SciPy release.

References