{ "__type": "IngestedDoc", "__tag": 4010, "_content": {}, "_ordered_sections": [], "item_file": null, "item_line": null, "item_type": null, "aliases": [], "example_section_data": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "see_also": [], "signature": null, "references": null, "qa": "dev:core-dev:index", "arbitrary": [ { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "SciPy has a formal governance model, documented in " }, { "__type": "CrossRef", "__tag": 4002, "value": "governance", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "dev:governance" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": ". The section below documents in an informal way what happens in practice for decision making about code and commit rights. The formal governance model is leading, the below is only provided for context." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Any significant decisions on adding (or not adding) new features, breaking backwards compatibility or making other significant changes to the codebase should be made on the scipy-dev forum after a discussion (preferably with full consensus)." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Any non-trivial change (where trivial means a typo, or a one-liner maintenance commit) has to go in through a pull request (PR). It has to be reviewed by another developer. In case review doesn't happen quickly enough and it is important that the PR is merged quickly, the submitter of the PR should send a message to the forum saying they intend to merge that PR without review at time X for reason Y unless someone reviews it before then." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Changes and new additions should be tested. Untested code is broken code." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Who gets commit rights is decided by the SciPy Steering Council; changes in commit rights will then be announced on the scipy-dev forum." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The general decision rule to accept a proposed new feature has so far been conditional on:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The method is applicable in many fields and \"generally agreed\" to be useful," } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "It fits the topic of the submodule, and does not require extensive support frameworks to operate," } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The implementation looks sound and unlikely to need much tweaking in the future (e.g., limited expected maintenance burden)," } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Someone wants to contribute it, and" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Someone wants to review it." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The last criterion is often a sticking point for proposed features. Code cannot be merged until it has been thoroughly reviewed, and there is always a backlog of maintenance tasks that compete for reviewers' time. Ideally, contributors should line up a reviewer with suitable domain expertise before beginning work." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Although it's difficult to give hard rules on what \"generally useful and generally agreed to work\" means, it may help to weigh the following against each other:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Is the method used/useful in different domains in practice? How much domain-specific background knowledge is needed to use it properly?" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Consider the code already in the module. Is what you are adding an omission? Does it solve a problem that you'd expect the module be able to solve? Does it supplement an existing feature in a significant way?" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Consider the equivalence class of similar methods / features usually expected. Among them, what would in principle be the minimal set so that there's not a glaring omission in the offered features remaining? How much stuff would that be? Does including a representative one of them cover most use cases? Would it in principle sound reasonable to include everything from the minimal set in the module?" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Is what you are adding something that is well understood in the literature? If not, how sure are you that it will turn out well? Does the method perform well compared to other similar ones?" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Note that the twice-a-year release cycle and backward-compatibility policy makes correcting things later on more difficult." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The scopes of the submodules also vary, so it's probably best to consider each as if it's a separate project - \"numerical evaluation of special functions\" is relatively well-defined, but \"commonly needed optimization algorithms\" less so." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "SciPy development largely takes place on GitHub; this section describes the expected way of working for issues, pull requests and managing the main " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy" }, { "__type": "Text", "__tag": 4046, "value": " repository." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Each issue and pull request normally gets at least two labels: one for the topic or component (" }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy.stats" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "Documentation" }, { "__type": "Text", "__tag": 4046, "value": ", etc.), and one for the nature of the issue or pull request (" }, { "__type": "InlineCode", "__tag": 4051, "value": "enhancement" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "maintenance" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "defect" }, { "__type": "Text", "__tag": 4046, "value": ", etc.). Other labels that may be added depending on the situation:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "good-first-issue" }, { "__type": "Text", "__tag": 4046, "value": ": for issues suitable to be tackled by new contributors." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "needs-work" }, { "__type": "Text", "__tag": 4046, "value": ": for pull requests that have review comments that haven't been addressed." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "needs-decision" }, { "__type": "Text", "__tag": 4046, "value": ": for issues or pull requests that need a decision." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "needs-champion" }, { "__type": "Text", "__tag": 4046, "value": ": for pull requests that were not finished by the original author, but are worth resurrecting." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "backport-candidate" }, { "__type": "Text", "__tag": 4046, "value": ": bugfixes that should be considered for backporting by the release manager." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A milestone is created for each version number for which a release is planned. Issues that need to be addressed and pull requests that need to be merged for a particular release should be set to the corresponding milestone. After a pull request is merged, its milestone (and that of the issue it closes) should be set to the next upcoming release - this makes it easy to get an overview of changes and to add a complete list of those to the release notes." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When reviewing pull requests, please make use of pull request workflow features, see " }, { "__type": "CrossRef", "__tag": 4002, "value": "pull-request-workflow-features", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "dev:contributor:reviewing_prs" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When merging contributions, a committer is responsible for ensuring that those meet the requirements outlined in " }, { "__type": "CrossRef", "__tag": 4002, "value": "Contributing to SciPy", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "dev:hacking" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": ". Also check that new features and backwards compatibility breaks were discussed on the scipy-dev forum." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "New code goes in via a pull request (PR)." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Merge new code with the green button. In case of merge conflicts, ask the PR submitter to rebase (this may require providing some " }, { "__type": "InlineCode", "__tag": 4051, "value": "git" }, { "__type": "Text", "__tag": 4046, "value": " instructions)." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Backports and trivial additions to finish a PR (really trivial, like a typo or PEP8 fix) can be pushed directly." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For PRs that add new features or are in some way complex, wait at least a day or two before merging it. That way, others get a chance to comment before the code goes in." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Squashing commits or cleaning up commit messages of a PR that you consider too messy is OK. Make sure though to retain the original author name when doing this. Squashing is highly recommended whenever commit messages do not (roughly) follow the guidelines in " }, { "__type": "CrossRef", "__tag": 4002, "value": "writing-the-commit-message", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "dev:contributor:development_workflow" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Make sure that the labels and milestone on a merged PR are set correctly." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When you want to reject a PR: if it's very obvious, you can just close it and explain why. If it's not obvious, then it's a good idea to first explain why you think the PR is not suitable for inclusion in SciPy, then let a second committer comment or close." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "All pull requests (whether they contain enhancements, bug fixes or something else), should be made against main. Only bug fixes are candidates for backporting to a maintenance branch. The backport strategy for SciPy is to (a) only backport fixes that are important, and (b) to only backport when it's reasonably sure that a new bugfix release on the relevant maintenance branch will be made. Typically, the developer who merges an important bugfix adds the " }, { "__type": "InlineCode", "__tag": 4051, "value": "backport-candidate" }, { "__type": "Text", "__tag": 4046, "value": " label and pings the release manager, who decides on whether and when the backport is done. After the backport is completed, the " }, { "__type": "InlineCode", "__tag": 4051, "value": "backport-candidate" }, { "__type": "Text", "__tag": 4046, "value": " label has to be removed again." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A good strategy for a backport pull request is to combine several main branch pull requests, to reduce the burden on continuous integration tests and to reduce the merge commit cluttering of maintenance branch history. It is generally best to have a single commit for each of the main branch pull requests represented in the backport pull request. This way, history is clear and can be reverted in a straightforward manner if needed." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When a PR gets merged, consider if the changes need to be mentioned in the release notes. What needs mentioning: new features, backwards incompatible changes, deprecations, and \"other changes\" (anything else noteworthy enough, see older release notes for the kinds of things worth mentioning)." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Release note entries are maintained on " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "the wiki" } ], "url": "https://github.com/scipy/scipy/wiki", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ". The release manager will gather content from there and integrate it into the html docs. We use this mechanism to avoid merge conflicts that would happen if every PR touched the same file under " }, { "__type": "InlineCode", "__tag": 4051, "value": "doc/release/" }, { "__type": "Text", "__tag": 4046, "value": " directly." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Changes can be monitored (" }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "Atom feed" } ], "url": "https://github.com/scipy/scipy/wiki.atom", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ") and pulled (the wiki is a git repo: " }, { "__type": "InlineCode", "__tag": 4051, "value": "https://github.com/scipy/scipy.wiki.git" }, { "__type": "Text", "__tag": 4046, "value": ")." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "Cross-referencing:" } ] }, { "__type": "Text", "__tag": 4046, "value": " Cross-referencing issues and pull requests on GitHub is often useful. GitHub allows doing that by using " }, { "__type": "InlineCode", "__tag": 4051, "value": "gh-xxxx" }, { "__type": "Text", "__tag": 4046, "value": " or " }, { "__type": "InlineCode", "__tag": 4051, "value": "#xxxx" }, { "__type": "Text", "__tag": 4046, "value": " with " }, { "__type": "InlineCode", "__tag": 4051, "value": "xxxx" }, { "__type": "Text", "__tag": 4046, "value": " the issue/PR number. The " }, { "__type": "InlineCode", "__tag": 4051, "value": "gh-xxxx" }, { "__type": "Text", "__tag": 4046, "value": " format is strongly preferred, because it's clear that that is a GitHub link. Older issues contain " }, { "__type": "InlineCode", "__tag": 4051, "value": "#xxxx" }, { "__type": "Text", "__tag": 4046, "value": " which is about Trac (what we used pre-GitHub) tickets." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "PR naming convention:" } ] }, { "__type": "Text", "__tag": 4046, "value": " Pull requests, issues and commit messages usually start with a three-letter abbreviation like " }, { "__type": "InlineCode", "__tag": 4051, "value": "ENH:" }, { "__type": "Text", "__tag": 4046, "value": " or " }, { "__type": "InlineCode", "__tag": 4051, "value": "BUG:" }, { "__type": "Text", "__tag": 4046, "value": ". This is useful to quickly see what the nature of the commit/PR/issue is. For the full list of abbreviations, see " }, { "__type": "InlineRole", "__tag": 4003, "value": "writing the commit message ", "domain": null, "role": "ref", "inventory": null }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "SciPy is distributed under the " }, { "__type": "InlineRole", "__tag": 4003, "value": "modified (3-clause) BSD license", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ". All code, documentation and other files added to SciPy by contributors is licensed under this license, unless another license is explicitly specified in the source code. Contributors keep the copyright for code they wrote and submit for inclusion to SciPy." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Other licenses that are compatible with the modified BSD license that SciPy uses are 2-clause BSD, MIT and PSF. Incompatible licenses are GPL, Apache and custom licenses that require attribution/citation or prohibit use for commercial purposes." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "PRs are often submitted with content copied or derived from unlicensed code or code from a default license that is not compatible with SciPy's license. For instance, code published on StackOverflow is covered by a CC-BY-SA license, which is not compatible due to the share-alike clause. These contributions cannot be accepted for inclusion in SciPy unless the original code author is willing to (re)license their code under the modified BSD (or compatible) license. If the original author agrees, add a comment saying so to the source files and forward the relevant communication to the scipy-dev forum." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Another common occurrence is for code to be translated or derived from code in R, Octave (both GPL-licensed) or a commercial application. Such code also cannot be included in SciPy. Simply implementing functionality with the same API as found in R/Octave/... is fine though, as long as the author doesn't look at the original incompatibly-licensed source code." } ] }, { "__type": "Target", "__tag": 4061, "label": "modified (3-clause) BSD license" }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "SciPy version numbering complies with " }, { "__type": "InlineRole", "__tag": 4003, "value": "PEP 440", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ". Released final versions, which are the only versions appearing on " }, { "__type": "InlineRole", "__tag": 4003, "value": "PyPI", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", are numbered " }, { "__type": "InlineCode", "__tag": 4051, "value": "MAJOR.MINOR.MICRO" }, { "__type": "Text", "__tag": 4046, "value": " where:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "MAJOR" }, { "__type": "Text", "__tag": 4046, "value": " is an integer indicating the major version. It changes very rarely; a change in " }, { "__type": "InlineCode", "__tag": 4051, "value": "MAJOR" }, { "__type": "Text", "__tag": 4046, "value": " indicates large (possibly backwards-incompatible) changes." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "MINOR" }, { "__type": "Text", "__tag": 4046, "value": " is an integer indicating the minor version. Minor versions are typically released twice a year and can contain new features, deprecations and bug-fixes." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "MICRO" }, { "__type": "Text", "__tag": 4046, "value": " is an integer indicating a bug-fix version. Bug-fix versions are released when needed, typically one or two per minor version. They cannot contain new features or deprecations." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Released alpha, beta and rc (release candidate) versions are numbered like final versions but with postfixes " }, { "__type": "InlineCode", "__tag": 4051, "value": "a#" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "b#" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "rc#" }, { "__type": "Text", "__tag": 4046, "value": " respectively, with " }, { "__type": "InlineCode", "__tag": 4051, "value": "#" }, { "__type": "Text", "__tag": 4046, "value": " an integer. Development versions are postfixed with " }, { "__type": "InlineCode", "__tag": 4051, "value": ".dev0+" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Examples of valid SciPy version strings are " } ] }, { "__type": "Code", "__tag": 4050, "value": "0.16.0\n0.15.1\n0.14.0a1\n0.14.0b2\n0.14.0rc1\n0.17.0.dev0+ac53f09", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "An installed SciPy version contains these version identifiers " } ] }, { "__type": "Code", "__tag": 4050, "value": "scipy.__version__ # complete version string, including git commit hash for dev versions\nscipy.version.short_version # string, only major.minor.micro\nscipy.version.version # string, same as scipy.__version__\nscipy.version.full_version # string, same as scipy.__version__\nscipy.version.release # bool, development or (alpha/beta/rc/final) released version\nscipy.version.git_revision # string, git commit hash from which scipy was built", "execution_status": null }, { "__type": "Target", "__tag": 4061, "label": "PEP 440" }, { "__type": "Target", "__tag": 4061, "label": "PyPI" }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "There are various reasons for wanting to remove existing functionality: it's buggy, the API isn't understandable, it's superseded by functionality with better performance, it needs to be moved to another SciPy submodule, etc." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "In general, it's not a good idea to remove something without warning users about that removal first. Therefore, this is what should be done before removing something from the public API:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Propose to deprecate the functionality on the scipy-dev forum and get agreement that that's OK." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Add a " }, { "__type": "InlineCode", "__tag": 4051, "value": "DeprecationWarning" }, { "__type": "Text", "__tag": 4046, "value": " for it, which states that the functionality was deprecated, and in which release. For Cython APIs, see " }, { "__type": "CrossRef", "__tag": 4002, "value": "deprecating-public-cython-api", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "dev:contributor:public_cython_api" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": " for the practical steps." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Mention the deprecation in the release notes for that release." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Wait till at least 6 months after the release date of the release that introduced the " }, { "__type": "InlineCode", "__tag": 4051, "value": "DeprecationWarning" }, { "__type": "Text", "__tag": 4046, "value": " before removing the functionality." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Mention the removal of the functionality in the release notes." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The 6 months waiting period in practice usually means waiting two releases. When introducing the warning, also ensure that those warnings are filtered out when running the test suite so they don't pollute the output." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "It's possible that there is reason to want to ignore this deprecation policy for a particular deprecation; this can always be discussed on the scipy-dev forum." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Many parts of the SciPy codebase are maintained elsewhere, and vendored in SciPy. Some of these parts are vendored as git submodules, for example, " }, { "__type": "InlineCode", "__tag": 4051, "value": "xsf" }, { "__type": "Text", "__tag": 4046, "value": ", or are placed under the " }, { "__type": "InlineCode", "__tag": 4051, "value": "subprojects" }, { "__type": "Text", "__tag": 4046, "value": " directory (or both)." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Other parts are not vendored as git submodules or under the " }, { "__type": "InlineCode", "__tag": 4051, "value": "subprojects" }, { "__type": "Text", "__tag": 4046, "value": " directory, despite having a maintained upstream. This is usually either because:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "DefList", "__tag": 4033, "children": [ { "__type": "DefListItem", "__tag": 4037, "dt": { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "a subset of the upstream repo is vendored with a script." } ] }, "dd": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "(It is possible that these parts will be moved under " }, { "__type": "InlineCode", "__tag": 4051, "value": "subprojects" }, { "__type": "Text", "__tag": 4046, "value": " in the future.)" } ] } ] } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "DefList", "__tag": 4033, "children": [ { "__type": "DefListItem", "__tag": 4037, "dt": { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Code has been copied into SciPy and modified since." } ] }, "dd": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "(It is possible that some of these parts will see patches contributed upstream and become git submodules or be moved under " }, { "__type": "InlineCode", "__tag": 4051, "value": "subprojects" }, { "__type": "Text", "__tag": 4046, "value": " in the future.)" } ] } ] } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Maintainers should be careful to " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "not" } ] }, { "__type": "Text", "__tag": 4046, "value": " accept contributions (especially trivial changes) into parts of SciPy where the code is actively maintained upstream. Instead, they should direct contributors to the upstream repo." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Parts of the codebase which are vendored with a script include:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "PRIMA_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/_lib/pyprima" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Parts of the codebase which contain code copied from an upstream include:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "DIRECT_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/optimize/_direct" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "ARPACK_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/sparse/linalg/_eigen/arpack/ARPACK" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "SuperLU_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/sparse/linalg/_dsolve/SuperLU" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "trlib_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/optimize/_trlib" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "UNU.RAN_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/stats/_unuran" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "Cython/Tempita", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/_build_utils/tempita" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "fast_matrix_market", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/io/_fast_matrix_market" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "numpydoc/docscrape", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", at " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/_lib/_docscrape.py" } ] } ] } ] }, { "__type": "Target", "__tag": 4061, "label": "ARPACK" }, { "__type": "Target", "__tag": 4061, "label": "SuperLU" }, { "__type": "Target", "__tag": 4061, "label": "trlib" }, { "__type": "Target", "__tag": 4061, "label": "UNU.RAN" }, { "__type": "Target", "__tag": 4061, "label": "DIRECT" }, { "__type": "Target", "__tag": 4061, "label": "PRIMA" }, { "__type": "Target", "__tag": 4061, "label": "Cython/Tempita" }, { "__type": "Target", "__tag": 4061, "label": "fast_matrix_market" }, { "__type": "Target", "__tag": 4061, "label": "numpydoc/docscrape" }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Please refer to https://github.com/scipy/scipy/issues/21232 for further details and tracking of vendored code in the repository." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Distributing Python packages is nontrivial - especially for a package with complex build requirements like SciPy - and subject to change. For an up-to-date overview of recommended tools and techniques, see the " }, { "__type": "InlineRole", "__tag": 4003, "value": "Python Packaging User Guide", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ". This document discusses some of the main issues and considerations for SciPy." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Dependencies are things that a user has to install in order to use (or build/test) a package. They usually cause trouble, especially if they're not optional. SciPy tries to keep its dependencies to a minimum; the current required and optional build time dependencies can be seen in " }, { "__type": "InlineRole", "__tag": 4003, "value": "SciPy's configuration file", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": ". The only non-optional runtime dependency is " }, { "__type": "CrossRef", "__tag": 4002, "value": "NumPy", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "tutorial:index" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Furthermore, of course one needs C, C++ and Fortran compilers to build SciPy, but we don't consider those to be dependencies, and therefore they are not discussed here. For details, see " }, { "__type": "CrossRef", "__tag": 4002, "value": "building-from-source", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "building:index" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "When a package provides useful functionality and it's proposed as a new dependency, consider also if it makes sense to vendor (i.e. ship a copy of it with SciPy) the package instead. For example, " }, { "__type": "InlineRole", "__tag": 4003, "value": "decorator_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " is vendored in " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy._lib" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "There are some issues with how Python packaging tools handle dependencies reported by projects. Because SciPy gets regular bug reports about this, we go in a bit of detail here." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "SciPy reports its dependency on NumPy via " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " for build purposes, and SciPy also has a runtime check that an appropriate version of NumPy is available. SciPy no longer uses " }, { "__type": "InlineCode", "__tag": 4051, "value": "setup_requires" }, { "__type": "Text", "__tag": 4046, "value": " (which in the past invoked " }, { "__type": "InlineCode", "__tag": 4051, "value": "easy_install" }, { "__type": "Text", "__tag": 4046, "value": "); build dependencies are now handled only via " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": ". " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " relies on PEP 517; " }, { "__type": "InlineCode", "__tag": 4051, "value": "pip" }, { "__type": "Text", "__tag": 4046, "value": " has " }, { "__type": "InlineCode", "__tag": 4051, "value": "--no-use-pep517" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "--no-build-isolation" }, { "__type": "Text", "__tag": 4046, "value": " flags that may ignore " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " or treat it differently - if users use those flags, they are responsible for installing the correct build dependencies themselves." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For dependencies it's important to set lower and upper bounds on their versions. For " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "build-time" } ] }, { "__type": "Text", "__tag": 4046, "value": " dependencies, they are specified in " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " and the versions will " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "only" } ] }, { "__type": "Text", "__tag": 4046, "value": " apply to the SciPy build itself. It's fine to specify either a range or a specific version for a dependency like " }, { "__type": "InlineCode", "__tag": 4051, "value": "meson-python" }, { "__type": "Text", "__tag": 4046, "value": " or " }, { "__type": "InlineCode", "__tag": 4051, "value": "pybind11" }, { "__type": "Text", "__tag": 4046, "value": ". For NumPy we have to worry about ABI compatibility too. However, with NumPy " }, { "__type": "InlineCode", "__tag": 4051, "value": ">=2.0.0rc1" }, { "__type": "Text", "__tag": 4046, "value": " backwards compatibility is guaranteed as far back as the NumPy " }, { "__type": "InlineCode", "__tag": 4051, "value": "1.19" }, { "__type": "Text", "__tag": 4046, "value": " series so specification of a lowest supported version of NumPy at build time is no longer required in " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "run-time dependencies" } ] }, { "__type": "Text", "__tag": 4046, "value": " (currently only " }, { "__type": "InlineCode", "__tag": 4051, "value": "numpy" }, { "__type": "Text", "__tag": 4046, "value": "), we specify the range of versions in " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " and in " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/__init__.py" }, { "__type": "Text", "__tag": 4046, "value": ". Getting the upper bound right is slightly tricky. If we don't set any bound, a too-new version will be pulled in a few years down the line, and NumPy may have deprecated and removed some API that SciPy depended on by then. On the other hand if we set the upper bound to the newest already-released version, then as soon as a new NumPy version is released there will be no matching SciPy version that works with it. Given that NumPy and SciPy both release in a 6-monthly cadence and that features that get deprecated in NumPy should stay around for another two releases, we specify the upper bound as " }, { "__type": "InlineCode", "__tag": 4051, "value": "<2.xx+3.0" }, { "__type": "Text", "__tag": 4046, "value": " (where " }, { "__type": "InlineCode", "__tag": 4051, "value": "xx" }, { "__type": "Text", "__tag": 4046, "value": " is the minor version of the latest already-released NumPy)." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The " }, { "__type": "InlineRole", "__tag": 4003, "value": "Python_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " versions that SciPy supports are listed in the list of PyPI classifiers in " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": ", and mentioned in the release notes for each release. All newly released Python versions will be supported as soon as possible. For the general policy on dropping support for a Python or NumPy version, see " }, { "__type": "InlineRole", "__tag": 4003, "value": "NEP 29 ", "domain": null, "role": "ref", "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ". The final decision on dropping support is always taken on the scipy-dev forum." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The lowest supported " }, { "__type": "CrossRef", "__tag": 4002, "value": "NumPy", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "tutorial:index" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": " version for a SciPy version is mentioned in the release notes and is encoded in " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/__init__.py" }, { "__type": "Text", "__tag": 4046, "value": ". Typically the latest SciPy release supports ~5-7 minor versions of NumPy: up to 2.5 years' old NumPy versions, (given that the frequency of NumPy releases is about 2x/year at the time of writing) plus two versions into the future." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Supported versions of optional dependencies and compilers is documented in " }, { "__type": "CrossRef", "__tag": 4002, "value": "toolchain-roadmap", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "dev:toolchain" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": ". Note that not all versions of optional dependencies that are supported are tested well or at all by SciPy's Continuous Integration setup. Issues regarding this are dealt with as they come up in the issue tracker or forum." } ] }, { "__type": "Admonition", "__tag": 4056, "kind": "note", "base_type": "note", "children": [ { "__type": "AdmonitionTitle", "__tag": 4055, "children": [ { "__type": "Text", "__tag": 4046, "value": "note " } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "This section is only about building SciPy binary installers to " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "distribute" } ] }, { "__type": "Text", "__tag": 4046, "value": ". For info on building SciPy on the same machine as where it will be used, see " }, { "__type": "CrossRef", "__tag": 4002, "value": "building-from-source", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "docs", "path": "building:index" }, "kind": "exists" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "There are a number of things to take into consideration when building binaries and distributing them on PyPI or elsewhere." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "General" } ] } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A binary is specific for a single (major) Python version (because different major Python versions aren't ABI-compatible, at least up to Python 3.12)." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Build against NumPy " }, { "__type": "InlineCode", "__tag": 4051, "value": "2.0.0" }, { "__type": "Text", "__tag": 4046, "value": ", then it will work for all NumPy versions with the same major version number (NumPy does maintain backwards ABI compatibility), and as far back as NumPy " }, { "__type": "InlineCode", "__tag": 4051, "value": "1.19" }, { "__type": "Text", "__tag": 4046, "value": " series at the time of writing." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The easiest available toolchain for building portable SciPy binaries is our " }, { "__type": "InlineCode", "__tag": 4051, "value": "cibuildwheel" }, { "__type": "Text", "__tag": 4046, "value": " infrastructure for common platforms, with details available in our CI infrastructure code and available via the " }, { "__type": "InlineCode", "__tag": 4051, "value": "cibuildwheel" }, { "__type": "Text", "__tag": 4046, "value": " command on Windows, Linux, and MacOS, albeit with some extra external dependencies required in some cases" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "Windows" } ] } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For 64-bit Windows installers built with a free toolchain, use the method documented at https://github.com/numpy/numpy/wiki/Mingw-static-toolchain. That method will likely be used for SciPy itself once it's clear that the maintenance of that toolchain is sustainable long-term. See the " }, { "__type": "InlineRole", "__tag": 4003, "value": "MingwPy_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " project and " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "this thread" } ], "url": "https://mail.python.org/pipermail/numpy-discussion/2015-October/074056.html", "title": "" }, { "__type": "Text", "__tag": 4046, "value": " for details." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The other way to produce 64-bit Windows installers is with " }, { "__type": "InlineCode", "__tag": 4051, "value": "icc" }, { "__type": "Text", "__tag": 4046, "value": ", " }, { "__type": "InlineCode", "__tag": 4051, "value": "ifort" }, { "__type": "Text", "__tag": 4046, "value": " plus " }, { "__type": "InlineCode", "__tag": 4051, "value": "MKL" }, { "__type": "Text", "__tag": 4046, "value": " (or " }, { "__type": "InlineCode", "__tag": 4051, "value": "MSVC" }, { "__type": "Text", "__tag": 4046, "value": " instead of " }, { "__type": "InlineCode", "__tag": 4051, "value": "icc" }, { "__type": "Text", "__tag": 4046, "value": "). For Intel toolchain instructions see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "this article" } ], "url": "https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl", "title": "" }, { "__type": "Text", "__tag": 4046, "value": " and for (partial) MSVC instructions see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "this wiki page" } ], "url": "https://github.com/numpy/numpy/wiki/Building-with-MSVC", "title": "" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Older SciPy releases contained a .exe \"superpack\" installer. Those contain 3 complete builds (no SSE, SSE2, SSE3), and were built with https://github.com/numpy/numpy-vendor. That build setup is known to not work well anymore and is no longer supported. It used g77 instead of gfortran, due to complex DLL distribution issues (see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-2829" } ], "url": "https://github.com/scipy/scipy/issues/2829", "title": "" }, { "__type": "Text", "__tag": 4046, "value": "). Because the toolchain is no longer supported, g77 support isn't needed anymore and SciPy can now include Fortran 90/95 code." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "Linux" } ] } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "PyPI-compatible Linux wheels can be produced via the " }, { "__type": "InlineRole", "__tag": 4003, "value": "manylinux_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " project, which is used under the hood by our " }, { "__type": "InlineCode", "__tag": 4051, "value": "cibuildwheel" }, { "__type": "Text", "__tag": 4046, "value": " infrastructure." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Other Linux build-setups result in PyPI incompatible wheels, which would need to be distributed via custom channels, e.g. in a " }, { "__type": "InlineRole", "__tag": 4003, "value": "Wheelhouse_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ", see at the " }, { "__type": "InlineRole", "__tag": 4003, "value": "wheel_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineRole", "__tag": 4003, "value": "Wheelhouse_", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " docs." } ] }, { "__type": "Target", "__tag": 4061, "label": "SciPy's configuration file" }, { "__type": "Target", "__tag": 4061, "label": "NumPy" }, { "__type": "Target", "__tag": 4061, "label": "Python" }, { "__type": "Target", "__tag": 4061, "label": "nose" }, { "__type": "Target", "__tag": 4061, "label": "asv" }, { "__type": "Target", "__tag": 4061, "label": "matplotlib" }, { "__type": "Target", "__tag": 4061, "label": "Pillow" }, { "__type": "Target", "__tag": 4061, "label": "scikits.umfpack" }, { "__type": "Target", "__tag": 4061, "label": "mpmath" }, { "__type": "Target", "__tag": 4061, "label": "pooch" }, { "__type": "Target", "__tag": 4061, "label": "Cython" }, { "__type": "Target", "__tag": 4061, "label": "pybind11" }, { "__type": "Target", "__tag": 4061, "label": "setuptools" }, { "__type": "Target", "__tag": 4061, "label": "wheel" }, { "__type": "Target", "__tag": 4061, "label": "pip" }, { "__type": "Target", "__tag": 4061, "label": "Python Packaging User Guide" }, { "__type": "Target", "__tag": 4061, "label": "Wheelhouse" }, { "__type": "Target", "__tag": 4061, "label": "MingwPy" }, { "__type": "Target", "__tag": 4061, "label": "Sphinx" }, { "__type": "Target", "__tag": 4061, "label": "PyData Sphinx theme" }, { "__type": "Target", "__tag": 4061, "label": "Sphinx-Design" }, { "__type": "Target", "__tag": 4061, "label": "six" }, { "__type": "Target", "__tag": 4061, "label": "decorator" }, { "__type": "Target", "__tag": 4061, "label": "manylinux" }, { "__type": "Target", "__tag": 4061, "label": "threadpoolctl" }, { "__type": "Target", "__tag": 4061, "label": "MyST-NB" }, { "__type": "Target", "__tag": 4061, "label": "Hypothesis" }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Examples within docstrings can be made interactive using the " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "jupyterlite-sphinx extension" } ], "url": "https://jupyterlite-sphinx.readthedocs.io/en/stable/", "title": "" }, { "__type": "Text", "__tag": 4046, "value": ", which implements the " }, { "__type": "InlineCode", "__tag": 4051, "value": ".. try_examples::" }, { "__type": "Text", "__tag": 4046, "value": " directive. When used in the \"Examples\" section of a docstring, this directive will create a button that allows users to open the example in an interactive JupyterLite window, or in a new tab as a Jupyter notebook. For SciPy, this directive is automatically added to all docstrings with an \"Examples\" section through the " }, { "__type": "InlineCode", "__tag": 4051, "value": "global_enable_try_examples" }, { "__type": "Text", "__tag": 4046, "value": " configuration option in " }, { "__type": "InlineCode", "__tag": 4051, "value": "doc/source/conf.py" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "To hide the buttons that turn docstring examples into embedded notebooks, after building the html documentation, edit the " }, { "__type": "InlineCode", "__tag": 4051, "value": "try_examples.json" }, { "__type": "Text", "__tag": 4046, "value": " runtime configuration file after " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/doc/build/html/" }, { "__type": "Text", "__tag": 4046, "value": " and add the url paths of the files to be ignored to the " }, { "__type": "InlineCode", "__tag": 4051, "value": "ignore_patterns" }, { "__type": "Text", "__tag": 4046, "value": " list. This list expects elements to be JavaScript Regex patterns. The initial version of this file in a clean documentation build is" } ] }, { "__type": "Code", "__tag": 4050, "value": "{\n \"global_min_height\": \"400px\",\n \"ignore_patterns\": [\".*\"]\n}", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The buttons for converting examples sections into embedded interactive notebooks are hidden by default on clean docs builds." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "See the documentation for the " }, { "__type": "InlineCode", "__tag": 4051, "value": "jupyterlite-sphinx" }, { "__type": "Text", "__tag": 4046, "value": " " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "TryExamples directive" } ], "url": "https://jupyterlite-sphinx.readthedocs.io/en/stable/directives/try_examples.html#", "title": "" }, { "__type": "Text", "__tag": 4046, "value": " for more information." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "At the highest level, this is what the release manager does to release a new SciPy version:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": true, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Propose a release schedule in the SciPy forum at https://discuss.scientific-python.org/." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Create the maintenance branch for the release." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Tag the release." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Build all release artifacts (sources, installers, docs)." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Upload the release artifacts." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Announce the release." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Port relevant changes to release notes and build scripts to main." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "In this guide we attempt to describe in detail how to perform each of the above steps. In addition to those steps, which have to be performed by the release manager, here are descriptions of release-related activities and conventions of interest:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "backporting", "domain": null, "role": "ref", "inventory": null } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "labels-and-milestones", "domain": null, "role": "ref", "inventory": null } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "version-numbering", "domain": null, "role": "ref", "inventory": null } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "supported-py-numpy-versions", "domain": null, "role": "ref", "inventory": null } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineRole", "__tag": 4003, "value": "deprecations", "domain": null, "role": "ref", "inventory": null } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A typical release cycle looks like:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Create the maintenance branch" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Release a beta version" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Release a \"release candidate\" (RC)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If needed, release one or more new RCs" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Release the final version once there are no issues with the last release candidate" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "There's usually at least one week between each of the above steps. Experience shows that a cycle takes between 4 and 8 weeks for a new minor version. Bug-fix versions don't need a beta or RC, and can be done much quicker." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Ideally the final release is identical to the last RC, however there may be minor difference - it's up to the release manager to judge the risk of that. Typically, if compiled code or complex pure Python code changes then a new RC is needed, while a simple bug-fix that's backported from main doesn't require a new RC." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "To propose a schedule, send a list with estimated dates for branching and beta/rc/final releases to the SciPy forum at https://discuss.scientific-python.org/. In the same message, ask everyone to check if there are important issues/PRs that need to be included and aren't tagged with the Milestone for the release or the \"backport-candidate\" label." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Before branching, ensure that the release notes are updated as far as possible. Include the output of " }, { "__type": "InlineCode", "__tag": 4051, "value": "tools/gh_lists.py" }, { "__type": "Text", "__tag": 4046, "value": " and " }, { "__type": "InlineCode", "__tag": 4051, "value": "tools/authors.py" }, { "__type": "Text", "__tag": 4046, "value": " in the release notes." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Maintenance branches are named " }, { "__type": "InlineCode", "__tag": 4051, "value": "maintenance/..x" }, { "__type": "Text", "__tag": 4046, "value": " (e.g. 0.19.x). To create one, simply push a branch with the correct name to the scipy repo. Immediately after, push a commit where you increment the version number on the main branch and add release notes for that new version. Send an email to the SciPy forum at https://discuss.scientific-python.org/ to let people know that you've done this." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The version switcher dropdown needs to be updated with the new release information on the " }, { "__type": "InlineCode", "__tag": 4051, "value": "main" }, { "__type": "Text", "__tag": 4046, "value": " branch only." } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "doc/source/_static/version_switcher.json" }, { "__type": "Text", "__tag": 4046, "value": ": add the new release, the new development version, and transfer " }, { "__type": "InlineCode", "__tag": 4051, "value": "\"preferred\": true" }, { "__type": "Text", "__tag": 4046, "value": " from the old release to the new release." } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "In main we do not set upper bounds, because we want to test new releases or development versions of dependencies there. In a maintenance branch however, the goal is to be able to create releases that stay working for years. Hence correct upper bounds must be set. The following places must be updated after creating a maintenance branch:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "DefList", "__tag": 4033, "children": [ { "__type": "DefListItem", "__tag": 4037, "dt": { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": ": all build-time dependencies, as well as supported Python" } ] }, "dd": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "and NumPy versions" } ] } ] } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "scipy/__init__.py" }, { "__type": "Text", "__tag": 4046, "value": ": for NumPy version check" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Each file has comments describing how to set the correct upper bounds." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "First ensure that you have set up GPG correctly. See https://github.com/scipy/scipy/issues/4919 for a discussion of signing release tags, and https://keyring.debian.org/creating-key.html for instructions on creating a GPG key if you do not have one. Note that on some platforms it may be more suitable to use " }, { "__type": "InlineCode", "__tag": 4051, "value": "gpg2" }, { "__type": "Text", "__tag": 4046, "value": " instead of " }, { "__type": "InlineCode", "__tag": 4051, "value": "gpg" }, { "__type": "Text", "__tag": 4046, "value": " so that passwords may be stored by " }, { "__type": "InlineCode", "__tag": 4051, "value": "gpg-agent" }, { "__type": "Text", "__tag": 4046, "value": " as discussed in https://github.com/scipy/scipy/issues/10189. When preparing a release remotely, it may be necessary to set " }, { "__type": "InlineCode", "__tag": 4051, "value": "pinentry-mode loopback" }, { "__type": "Text", "__tag": 4046, "value": " in " }, { "__type": "InlineCode", "__tag": 4051, "value": "~/.gnupg/gpg-agent.conf" }, { "__type": "Text", "__tag": 4046, "value": " because use of " }, { "__type": "InlineCode", "__tag": 4051, "value": "gpg2" }, { "__type": "Text", "__tag": 4046, "value": " will otherwise proceed via an inaccessible graphical password prompt." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "To make your key more readily identifiable as you, consider sending your key to public key servers, with a command such as " } ] }, { "__type": "Code", "__tag": 4050, "value": "gpg --send-keys ", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Check that all relevant commits are in the branch. In particular, check issues and PRs under the Milestone for the release (https://github.com/scipy/scipy/milestones), PRs labeled \"backport-candidate\", and that the release notes are up-to-date and included in the html docs." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Then update the " }, { "__type": "InlineCode", "__tag": 4051, "value": "version" }, { "__type": "Text", "__tag": 4046, "value": " in " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " and commit the change with a message like " }, { "__type": "InlineCode", "__tag": 4051, "value": "REL: set version to " }, { "__type": "Text", "__tag": 4046, "value": ". Don't push this commit to the SciPy repo yet." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Finally tag the release locally with " }, { "__type": "InlineCode", "__tag": 4051, "value": "git tag -s " }, { "__type": "Text", "__tag": 4046, "value": " (the " }, { "__type": "InlineCode", "__tag": 4051, "value": "-s" }, { "__type": "Text", "__tag": 4046, "value": " ensures the tag is signed). If " }, { "__type": "InlineCode", "__tag": 4051, "value": "gpg2" }, { "__type": "Text", "__tag": 4046, "value": " is preferred, then " }, { "__type": "InlineCode", "__tag": 4051, "value": "git config --global gpg.program gpg2" }, { "__type": "Text", "__tag": 4046, "value": " may be appropriate. Continue with building release artifacts (next section). Only push the release commit to the scipy repo once you have built the sdists and docs successfully. Then continue with building wheels. Only push the release tag to the repo once all wheels have been built successfully on TravisCI and Appveyor (if it fails, you have to move the tag otherwise - which is bad practice). Finally, after pushing the tag, also push a second commit which increments the version number and appends " }, { "__type": "InlineCode", "__tag": 4051, "value": ".dev0" }, { "__type": "Text", "__tag": 4046, "value": " for " }, { "__type": "InlineCode", "__tag": 4051, "value": "version:" }, { "__type": "Text", "__tag": 4046, "value": " and sets " }, { "__type": "InlineCode", "__tag": 4051, "value": "ISRELEASED" }, { "__type": "Text", "__tag": 4046, "value": " to False again. This also applies with new release candidates, and for removing the " }, { "__type": "InlineCode", "__tag": 4051, "value": "rc" }, { "__type": "Text", "__tag": 4046, "value": " affix when switching from release candidate to release proper." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Here is a complete list of artifacts created for a release:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "sdist (" }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy-x.y.y.tar.gz" }, { "__type": "Text", "__tag": 4046, "value": ", for both PyPI and GitHub Releases)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Binary wheels for Windows, Linux and macOS" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Documentation (html)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A " }, { "__type": "InlineCode", "__tag": 4051, "value": "README.txt" }, { "__type": "Text", "__tag": 4046, "value": " file" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "A " }, { "__type": "InlineCode", "__tag": 4051, "value": "Changelog" }, { "__type": "Text", "__tag": 4046, "value": " file" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "An " }, { "__type": "InlineCode", "__tag": 4051, "value": "sdist" }, { "__type": "Text", "__tag": 4046, "value": " is generated by running " }, { "__type": "InlineCode", "__tag": 4051, "value": "python -m build --sdist" }, { "__type": "Text", "__tag": 4046, "value": " (note: we still need to move this into a CI job!), and the Changelog and README are built by running " }, { "__type": "InlineCode", "__tag": 4051, "value": "spin notes" }, { "__type": "Text", "__tag": 4046, "value": " (with tags, see " }, { "__type": "InlineCode", "__tag": 4051, "value": "spin notes --help" }, { "__type": "Text", "__tag": 4046, "value": ") in the repo root, and end up in " }, { "__type": "InlineCode", "__tag": 4051, "value": "REPO_ROOT/release/" }, { "__type": "Text", "__tag": 4046, "value": ". Do this after you've created the signed tag locally. If this completes without issues, push the release commit (not the tag, see section above) to the scipy repo." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "To build wheels, push a commit containing the text " }, { "__type": "InlineCode", "__tag": 4051, "value": "[wheel build]" }, { "__type": "Text", "__tag": 4046, "value": " to the branch used for the current release. This triggers " }, { "__type": "InlineCode", "__tag": 4051, "value": "cibuildwheel" }, { "__type": "Text", "__tag": 4046, "value": " builds for all needed Python versions and platforms. The appropriate version pins for NumPy and other dependencies should have been updated in " }, { "__type": "InlineCode", "__tag": 4051, "value": "pyproject.toml" }, { "__type": "Text", "__tag": 4046, "value": " just after branching. If the wheel builds reveal issues that need to be fixed with backports on the maintenance branch, you may remove the local tags (for example " }, { "__type": "InlineCode", "__tag": 4051, "value": "git tag -d v1.2.0rc1" }, { "__type": "Text", "__tag": 4046, "value": ") and restart with tagging above on the new candidate commit." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The " }, { "__type": "InlineCode", "__tag": 4051, "value": "cibuildwheel" }, { "__type": "Text", "__tag": 4046, "value": " infrastructure runs the tests from the built wheels and if they pass, uploads the wheels to https://anaconda.org/multibuild-wheels-staging/scipy." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "From there you can download them for uploading to PyPI. This can be done in an automated fashion using " }, { "__type": "InlineCode", "__tag": 4051, "value": "tools/download-wheels.py" }, { "__type": "Text", "__tag": 4046, "value": " " } ] }, { "__type": "Code", "__tag": 4050, "value": "$ python tools/download-wheels.py 1.5.0rc1 -w REPO_ROOT/release/installers", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "After this, we want to regenerate the README file, in order to have the MD5 and SHA256 checksums of the just downloaded wheels in it. Run " }, { "__type": "InlineCode", "__tag": 4051, "value": "spin notes" }, { "__type": "Text", "__tag": 4046, "value": " again." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For a release there are currently five places on the web to upload things to:" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "PyPI (sdist, wheels)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "GitHub Releases (sdist, release notes, Changelog)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "scipy.org (an announcement of the release)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "docs.scipy.org (html docs)" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "PyPI:" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Upload first the wheels and then the sdist " } ] }, { "__type": "Code", "__tag": 4050, "value": "twine upload REPO_ROOT/release/installers/*.whl\ntwine upload REPO_ROOT/release/installers/scipy-1.x.y.tar.gz", "execution_status": null }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "Github Releases:" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Use GUI on https://github.com/scipy/scipy/releases to create release and upload all release artifacts. At this stage, it is appropriate to push the tag and associate the new release (candidate) with this tag in the GUI. For example, " }, { "__type": "InlineCode", "__tag": 4051, "value": "git push upstream v1.2.0rc1" }, { "__type": "Text", "__tag": 4046, "value": ", where " }, { "__type": "InlineCode", "__tag": 4051, "value": "upstream" }, { "__type": "Text", "__tag": 4046, "value": " represents " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/scipy" }, { "__type": "Text", "__tag": 4046, "value": ". It is useful to check a previous release to determine exactly which artifacts should be included in the GUI upload process. Also, note that the release notes are not automatically populated into the release description on GitHub, and some manual reformatting to markdown can be quite helpful to match the formatting of previous releases on the site. We generally do not include Issue and Pull Request lists in these GUI descriptions." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "scipy.org:" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Sources for the site are in https://github.com/scipy/scipy.org. Update the News section in " }, { "__type": "InlineCode", "__tag": 4051, "value": "content/en/news.md" }, { "__type": "Text", "__tag": 4046, "value": " via a PR. This is only for proper releases, not release candidates." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "docs.scipy.org:" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "First build the scipy docs, by running " }, { "__type": "InlineCode", "__tag": 4051, "value": "make dist" }, { "__type": "Text", "__tag": 4046, "value": " in " }, { "__type": "InlineCode", "__tag": 4051, "value": "scipy/doc/" }, { "__type": "Text", "__tag": 4046, "value": ". Verify that they look OK, then upload them to the doc server with " }, { "__type": "InlineCode", "__tag": 4051, "value": "make upload USERNAME=rgommers RELEASE=0.19.0" }, { "__type": "Text", "__tag": 4046, "value": ". Note that SSH access to the doc server is needed; ask @pv (server admin), @tylerjereddy or @rgommers (can upload) if you don't have that." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The sources for the website itself are maintained in https://github.com/scipy/docs.scipy.org/. Add the new SciPy version in the table of releases in " }, { "__type": "InlineCode", "__tag": 4051, "value": "index.rst" }, { "__type": "Text", "__tag": 4046, "value": ". Push that commit, then do " }, { "__type": "InlineCode", "__tag": 4051, "value": "make upload USERNAME=yourusername" }, { "__type": "Text", "__tag": 4046, "value": ". This is only for proper releases, not release candidates." } ] }, { "__type": "Admonition", "__tag": 4056, "kind": "note", "base_type": "note", "children": [ { "__type": "AdmonitionTitle", "__tag": 4055, "children": [ { "__type": "Text", "__tag": 4046, "value": "note " } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The version switcher on the deployed docs will not consider the new release as the stable release until the changes to " }, { "__type": "InlineCode", "__tag": 4051, "value": "version_switcher.json" }, { "__type": "Text", "__tag": 4046, "value": " have been merged into the " }, { "__type": "InlineCode", "__tag": 4051, "value": "main" }, { "__type": "Text", "__tag": 4046, "value": " branch (see " }, { "__type": "Link", "__tag": 4049, "children": [ { "__type": "Text", "__tag": 4046, "value": "gh-22305" } ], "url": "https://github.com/scipy/scipy/pull/22305", "title": "" }, { "__type": "Text", "__tag": 4046, "value": "). Hence forward porting the release notes and version switcher changes to " }, { "__type": "InlineCode", "__tag": 4051, "value": "main" }, { "__type": "Text", "__tag": 4046, "value": " should ideally be done at the same time as deploying the docs for the new release." } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Send a message announcing the release to https://discuss.scientific-python.org/c/announcements/." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "For beta and rc versions, ask people to test (run the scipy tests and test against their own code) and report issues on Github or Discourse." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "After the final release is done, port relevant changes to release notes, build scripts, author name mapping in " }, { "__type": "InlineCode", "__tag": 4051, "value": "tools/authors.py" }, { "__type": "Text", "__tag": 4046, "value": " and any other changes that were only made on the maintenance branch to main." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Enable interactive examples by editing the runtime configuration file, " }, { "__type": "InlineCode", "__tag": 4051, "value": "try_examples.json" }, { "__type": "Text", "__tag": 4046, "value": ", in the root folder of the uploaded documentation on the release server. One must remove the regular expression pattern " }, { "__type": "InlineCode", "__tag": 4051, "value": "\".*\"" }, { "__type": "Text", "__tag": 4046, "value": " from the " }, { "__type": "InlineCode", "__tag": 4051, "value": "ignore_patterns" }, { "__type": "Text", "__tag": 4046, "value": " list." } ] }, { "__type": "Code", "__tag": 4050, "value": "$ ssh your-username@docs.scipy.org\n$ cd /srv/docs_scipy_org/doc/scipy-1.13.1\n$ vim try_examples.json # edit the ignore list to remove: \".*\"", "execution_status": null } ], "title": [ { "__type": "Text", "__tag": 4046, "value": "SciPy Core Developer Guide" } ], "level": 0, "target": "core-dev-guide" } ], "local_refs": [] }