Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dependency groups to pyproject.toml #37446

Merged
merged 1 commit into from
Dec 8, 2024

Conversation

tobiasdiez
Copy link
Contributor

@tobiasdiez tobiasdiez commented Feb 24, 2024

Following PEP 735, add a few groups of dependencies that one usually needs while developing sage (e.g. for tests or docs).

This info will then be used to generate the conda lock files and once support for PEP 735 is more widespread other tools will be able to use the metadata as well. Eg, once pypa/pip#12963 is implemented one can install all test dependencies with pip install --groups test.

📝 Checklist

  • The title is concise, informative, and self-explanatory.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation accordingly.

⌛ Dependencies

@dimpase
Copy link
Member

dimpase commented Feb 26, 2024

#37482 - is a peculiar PR, just cherry-picking commits from here, a PR on which I can't even comment, and positively reviewed by the PR author himself...

@tobiasdiez
Copy link
Contributor Author

#37482 - is a peculiar PR, just cherry-picking commits from here, a PR on which I can't even comment, and positively reviewed by the PR author himself...

It's not even a cherry-picked commit. Matthias copied a subset of the code changes of this PR, put's it into a different place that is more to his liking and slaps a "positive review" label on it. He did the same with #37452, and insisted that it's "positively review" even after I've removed that label (and explained in #36503 (comment) why the new location is not working for me).

@roed314 @williamstein sorry to bother you with this, but my understanding is that this is a clear misuse of the "positive review" label and, since done multiple times on purpose and not by accident, an abuse of his admin/triage rights. Speaking of abuse of admin rights, Matthias continues to flag my review comments as "resolved" - sometimes without even a response to the question (#37351 (review)). Normal users without admin rights cannot mark them as "unresolved" again, effectively hiding them from other reviewers. This problematic conduct has been mentioned to Matthias at multiple instances before (and if my memory serves me well even by members of the sage abuse team).

@dimpase dimpase removed the disputed PR is waiting for community vote, see https://groups.google.com/g/sage-devel/c/IgBYUJl33SQ label Feb 27, 2024
@dimpase
Copy link
Member

dimpase commented Feb 27, 2024

@mkoeppe has blocked me on GH - while this is sorted out, we can proceed. And if it's not sorted out, we can block him, why not...

@mkoeppe mkoeppe added the disputed PR is waiting for community vote, see https://groups.google.com/g/sage-devel/c/IgBYUJl33SQ label Feb 27, 2024
@roed314
Copy link
Contributor

roed314 commented Mar 14, 2024

The Sage Code of Conduct Committee is setting this back to needs review for now based on it being part of @dimpase's reaction to our decision on #36181, #36561, #36676, #36951, #36964, #37351, #36999, and #36941. It can, of course, be granted positive review at a future date, and will be subject to the policy on disputed PRs as normal.

@dimpase
Copy link
Member

dimpase commented Mar 14, 2024

is this still a disputed PR?

@mkoeppe
Copy link
Contributor

mkoeppe commented Mar 14, 2024

This PR has obviously neither seen sufficient development, nor any meaningful review. It cannot be merged in this form.

It creates a pyproject.toml in the SAGE_ROOT, which I have objected to elsewhere.

  • it declares a build-backend = "setuptools.build_meta", which obviously does not work -- there is nothing that backs it, neither a setup.cfg, setup.py, nor [tools.setuptools] information.
  • it duplicates version information, violating our long-standing policy and practice of single-sourcing version information.

To summarize: -1, this is deliberate noise. please close.

@mkoeppe
Copy link
Contributor

mkoeppe commented Mar 14, 2024

The part that declares external dependencies according to draft PEP 725 is valuable and interesting. I am hoping to use it with the skeleton generator for the modularized Sage packages in pyodide, #34539. I have split it out as

@dimpase
Copy link
Member

dimpase commented Mar 14, 2024

we don't really practice "single source version information", it can't therefore "violate" it.

@mkoeppe
Copy link
Contributor

mkoeppe commented Mar 14, 2024

we don't really practice "single source version information", it can't therefore "violate" it.

@dimpase That's a meaningless comment. It seems that you are still confused about our version files even after the documentation improvements in

@mkoeppe mkoeppe requested a review from kiwifb March 14, 2024 23:31
@mkoeppe
Copy link
Contributor

mkoeppe commented Mar 14, 2024

For anyone interested in improving how we source versions: This requires refactoring, as done in:

@mkoeppe mkoeppe requested a review from saraedum March 17, 2024 17:19
@saraedum
Copy link
Member

I see merit in having such a list of dependencies for use cases such as #37447. I also understand that it's not ideal to have such a file at the top level. Maybe there's a version of this that allows #37447 without putting a pyproject.toml into the top-level directory?

vbraun pushed a commit to vbraun/sage that referenced this pull request Apr 25, 2024
… 'external' section according to draft PEP 725

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

As done in sagemath#37482 (unbundled from sagemath#37446) for **sagemath-standard**.

- This information will be used in the skeleton generator in
pyodide/pyodide#4438

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#37486
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
@mkoeppe
Copy link
Contributor

mkoeppe commented Jul 27, 2024

This should be closed as a duplicate.

vbraun pushed a commit to vbraun/sage that referenced this pull request Jul 31, 2024
…t PEP 725 (unbundled from sagemath#37446)

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

This is aspirational decoration for future use by skeleton generators by
distributions such as conda, sage-the-distribution, pyodide.

Split out from the disputed sagemath#37446, where it is bundled with a number of
other changes, including: creating a `pyproject.toml` file in
`SAGE_ROOT`, hardcoding versions of Python packages instead of using the
existing `sage_bootstrap` infrastructure, etc. @roed314 @vbraun

**The scope of PRs should be chosen to minimize friction, not to
maximize friction.**
sagemath#36726 (comment)

Author: @mkoeppe, based on @tobiasdiez's sagemath#37446.

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [X] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [X] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#37482
Reported by: Matthias Köppe
Reviewer(s):
vbraun pushed a commit to vbraun/sage that referenced this pull request Aug 2, 2024
…t PEP 725 (unbundled from sagemath#37446)

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

This is aspirational decoration for future use by skeleton generators by
distributions such as conda, sage-the-distribution, pyodide.

Split out from the disputed sagemath#37446, where it is bundled with a number of
other changes, including: creating a `pyproject.toml` file in
`SAGE_ROOT`, hardcoding versions of Python packages instead of using the
existing `sage_bootstrap` infrastructure, etc. @roed314 @vbraun

**The scope of PRs should be chosen to minimize friction, not to
maximize friction.**
sagemath#36726 (comment)

Author: @mkoeppe, based on @tobiasdiez's sagemath#37446.

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [X] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [X] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#37482
Reported by: Matthias Köppe
Reviewer(s):
vbraun pushed a commit to vbraun/sage that referenced this pull request Aug 3, 2024
…t PEP 725 (unbundled from sagemath#37446)

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

This is aspirational decoration for future use by skeleton generators by
distributions such as conda, sage-the-distribution, pyodide.

Split out from the disputed sagemath#37446, where it is bundled with a number of
other changes, including: creating a `pyproject.toml` file in
`SAGE_ROOT`, hardcoding versions of Python packages instead of using the
existing `sage_bootstrap` infrastructure, etc. @roed314 @vbraun

**The scope of PRs should be chosen to minimize friction, not to
maximize friction.**
sagemath#36726 (comment)

Author: @mkoeppe, based on @tobiasdiez's sagemath#37446.

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [X] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [X] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#37482
Reported by: Matthias Köppe
Reviewer(s):
@tobiasdiez tobiasdiez changed the title Specify all dependencies of sagelib in its pyproject.toml Add dependency groups to pyproject.toml Oct 28, 2024
@tobiasdiez tobiasdiez removed the disputed PR is waiting for community vote, see https://groups.google.com/g/sage-devel/c/IgBYUJl33SQ label Oct 28, 2024
@tobiasdiez
Copy link
Contributor Author

The scope of this PR is now largely reduced, only adding a few dependency groups at the end of the pyproject.toml file. I've thus removed the "disputed" flag. (I can also close this PR and open a new one if this is preferred)

@dimpase
Copy link
Member

dimpase commented Oct 30, 2024

I'll be slow to respond in the coming 5-6 days or so, as we're moving across the pond tomorrow.

@dimpase
Copy link
Member

dimpase commented Nov 21, 2024

lgtm

@dimpase
Copy link
Member

dimpase commented Nov 21, 2024

ought to be squashed into 1 commit.

@tobiasdiez
Copy link
Contributor Author

lgtm

Thanks!

ought to be squashed into 1 commit.

Done now.

vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 6, 2024
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Following [PEP 735](https://peps.python.org/pep-0735/#canonical-names-
of-dependencies-and-dev-el-split-packages), add a few groups of
dependencies that one usually needs while developing sage (e.g. for
tests or docs).

This info will then be used to generate the conda lock files and once
support for PEP 735 is more widespread other tools will be able to use
the metadata as well. Eg, once pypa/pip#12963
is implemented one can install all test dependencies with `pip install
--groups test`.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->


<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37446
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik
vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 8, 2024
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Following [PEP 735](https://peps.python.org/pep-0735/#canonical-names-
of-dependencies-and-dev-el-split-packages), add a few groups of
dependencies that one usually needs while developing sage (e.g. for
tests or docs).

This info will then be used to generate the conda lock files and once
support for PEP 735 is more widespread other tools will be able to use
the metadata as well. Eg, once pypa/pip#12963
is implemented one can install all test dependencies with `pip install
--groups test`.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->


<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37446
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik
@vbraun vbraun merged commit d05bafa into sagemath:develop Dec 8, 2024
21 of 22 checks passed
vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 11, 2024
…da lock files

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Using the metadata added in sagemath#37446,
we automatically generate the conda environment files. This no longer
makes any reference to the `conda.txt` files contained in sage-the-
distribution. Thus sage-on-top-of-conda is now completely independent of
sage-the-distribution (only relying on information specified by sage-
the-library). In particular, after this PR is merged the `conda.txt`
files could be deleted from sage-the-distribution.

In particular, we no longer need to maintain the mapping of pypi
packages to conda packages but instead can rely on the offical mappings
maintained by the conda team (https://github.com/regro/cf-graph-
countyfair/tree/master/mappings/pypi).

To test:
```
pip install grayskull conda-lock
python tools/update-conda.py
```

The updated conda files are committed here as well.

Moreover, the `environment-dev` files have been deleted as there was
only little difference between these files and some people rightfully
complained that having too many files in the root folder is distracting.

As a byproduct, this fixes sagemath#34626.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

- sagemath#37446: specifies additional
requirements in the pyproject.toml which are used to generate the conda
env files
- sagemath#38983: for the update of numpy
- sagemath#38982: to fix meson build

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37447
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik, Julian Rüth, Tobias Diez
vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 12, 2024
…da lock files

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Using the metadata added in sagemath#37446,
we automatically generate the conda environment files. This no longer
makes any reference to the `conda.txt` files contained in sage-the-
distribution. Thus sage-on-top-of-conda is now completely independent of
sage-the-distribution (only relying on information specified by sage-
the-library). In particular, after this PR is merged the `conda.txt`
files could be deleted from sage-the-distribution.

In particular, we no longer need to maintain the mapping of pypi
packages to conda packages but instead can rely on the offical mappings
maintained by the conda team (https://github.com/regro/cf-graph-
countyfair/tree/master/mappings/pypi).

To test:
```
pip install grayskull conda-lock
python tools/update-conda.py
```

The updated conda files are committed here as well.

Moreover, the `environment-dev` files have been deleted as there was
only little difference between these files and some people rightfully
complained that having too many files in the root folder is distracting.

As a byproduct, this fixes sagemath#34626.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

- sagemath#37446: specifies additional
requirements in the pyproject.toml which are used to generate the conda
env files
- sagemath#38983: for the update of numpy
- sagemath#38982: to fix meson build

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37447
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik, Julian Rüth, Tobias Diez
vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 13, 2024
…da lock files

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Using the metadata added in sagemath#37446,
we automatically generate the conda environment files. This no longer
makes any reference to the `conda.txt` files contained in sage-the-
distribution. Thus sage-on-top-of-conda is now completely independent of
sage-the-distribution (only relying on information specified by sage-
the-library). In particular, after this PR is merged the `conda.txt`
files could be deleted from sage-the-distribution.

In particular, we no longer need to maintain the mapping of pypi
packages to conda packages but instead can rely on the offical mappings
maintained by the conda team (https://github.com/regro/cf-graph-
countyfair/tree/master/mappings/pypi).

To test:
```
pip install grayskull conda-lock
python tools/update-conda.py
```

The updated conda files are committed here as well.

Moreover, the `environment-dev` files have been deleted as there was
only little difference between these files and some people rightfully
complained that having too many files in the root folder is distracting.

As a byproduct, this fixes sagemath#34626.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

- sagemath#37446: specifies additional
requirements in the pyproject.toml which are used to generate the conda
env files
- sagemath#38983: for the update of numpy
- sagemath#38982: to fix meson build

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37447
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik, Julian Rüth, Tobias Diez
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants