Instructions for running tox with docker don't work

[bmispelon]

As mentionned by Pat on Discord (not sure what their github username is), our current instructions for running tox with docker don't seem to work:

Full log

``` $ docker compose exec web tox py312-tests: install_deps> python -I -m pip install -r /usr/src/app/requirements/tests.txt Collecting django-push@ git+https://github.com/brutasse/django-push.git@22fda99641cfbd2f3075a723d92652a8e38220a5 (from -r /usr/src/app/requirements/common.txt (line 6)) Cloning https://github.com/brutasse/django-push.git (to revision 22fda99641cfbd2f3075a723d92652a8e38220a5) to /tmp/pip-install-xbk4s6l8/django-push_869cd076d70d473da4c86b60e8cecf7d Resolved https://github.com/brutasse/django-push.git to commit 22fda99641cfbd2f3075a723d92652a8e38220a5 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting Babel==2.16.0 (from -r /usr/src/app/requirements/common.txt (line 1)) Downloading babel-2.16.0-py3-none-any.whl.metadata (1.5 kB) Collecting django-contact-form==5.1.1 (from -r /usr/src/app/requirements/common.txt (line 2)) Downloading django_contact_form-5.1.1-py3-none-any.whl.metadata (2.7 kB) Collecting django-countries==7.6.1 (from -r /usr/src/app/requirements/common.txt (line 3)) Downloading django_countries-7.6.1-py3-none-any.whl.metadata (31 kB) Collecting django-hosts==5.1 (from -r /usr/src/app/requirements/common.txt (line 4)) Downloading django_hosts-5.1-py3-none-any.whl.metadata (4.7 kB) Collecting django-money==3.5.3 (from -r /usr/src/app/requirements/common.txt (line 5)) Downloading django_money-3.5.3-py3-none-any.whl.metadata (18 kB) Collecting django-read-only==1.18.0 (from -r /usr/src/app/requirements/common.txt (line 7)) Downloading django_read_only-1.18.0-py3-none-any.whl.metadata (7.1 kB) Collecting django-recaptcha==4.0.0 (from -r /usr/src/app/requirements/common.txt (line 8)) Downloading django_recaptcha-4.0.0-py3-none-any.whl.metadata (13 kB) Collecting django-registration-redux==2.13 (from -r /usr/src/app/requirements/common.txt (line 9)) Downloading django_registration_redux-2.13-py2.py3-none-any.whl.metadata (5.1 kB) Collecting Django==5.0.9 (from -r /usr/src/app/requirements/common.txt (line 10)) Downloading Django-5.0.9-py3-none-any.whl.metadata (4.1 kB) Collecting docutils==0.21.2 (from -r /usr/src/app/requirements/common.txt (line 11)) Downloading docutils-0.21.2-py3-none-any.whl.metadata (2.8 kB) Collecting feedparser==6.0.11 (from -r /usr/src/app/requirements/common.txt (line 12)) Downloading feedparser-6.0.11-py3-none-any.whl.metadata (2.4 kB) Collecting Jinja2==3.1.4 (from -r /usr/src/app/requirements/common.txt (line 13)) Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB) Collecting libsass==0.23.0 (from -r /usr/src/app/requirements/common.txt (line 14)) Downloading libsass-0.23.0-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (4.2 kB) Collecting Markdown==3.7 (from -r /usr/src/app/requirements/common.txt (line 15)) Downloading Markdown-3.7-py3-none-any.whl.metadata (7.0 kB) Collecting Pillow==11.0.0 (from -r /usr/src/app/requirements/common.txt (line 16)) Downloading pillow-11.0.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (9.1 kB) Collecting psycopg==3.2.3 (from psycopg[c]==3.2.3->-r /usr/src/app/requirements/common.txt (line 17)) Downloading psycopg-3.2.3-py3-none-any.whl.metadata (4.3 kB) Collecting Pygments==2.18.0 (from -r /usr/src/app/requirements/common.txt (line 18)) Downloading pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB) Collecting pykismet3==0.1.1 (from -r /usr/src/app/requirements/common.txt (line 19)) Downloading pykismet3-0.1.1-py2.py3-none-any.whl.metadata (2.9 kB) Collecting requests==2.32.3 (from -r /usr/src/app/requirements/common.txt (line 20)) Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB) Collecting sorl-thumbnail==12.11.0 (from -r /usr/src/app/requirements/common.txt (line 21)) Downloading sorl_thumbnail-12.11.0-py3-none-any.whl.metadata (10 kB) Collecting Sphinx==8.1.3 (from -r /usr/src/app/requirements/common.txt (line 22)) Downloading sphinx-8.1.3-py3-none-any.whl.metadata (6.4 kB) Collecting stripe==3.1.0 (from -r /usr/src/app/requirements/common.txt (line 23)) Downloading stripe-3.1.0-py2.py3-none-any.whl.metadata (2.6 kB) Collecting time-machine==2.16.0 (from -r /usr/src/app/requirements/common.txt (line 24)) Downloading time_machine-2.16.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB) Collecting django-debug-toolbar==4.4.6 (from -r /usr/src/app/requirements/dev.txt (line 2)) Downloading django_debug_toolbar-4.4.6-py3-none-any.whl.metadata (3.9 kB) Collecting isort==5.13.2 (from -r /usr/src/app/requirements/dev.txt (line 3)) Downloading isort-5.13.2-py3-none-any.whl.metadata (12 kB) Collecting pre-commit~=4.0.1 (from -r /usr/src/app/requirements/dev.txt (line 4)) Downloading pre_commit-4.0.1-py2.py3-none-any.whl.metadata (1.3 kB) Collecting watchdog==6.0.0 (from -r /usr/src/app/requirements/dev.txt (line 5)) Downloading watchdog-6.0.0-py3-none-manylinux2014_x86_64.whl.metadata (44 kB) Collecting coverage==7.6.9 (from -r /usr/src/app/requirements/tests.txt (line 2)) Downloading coverage-7.6.9-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.2 kB) Collecting requests-mock==1.12.1 (from -r /usr/src/app/requirements/tests.txt (line 3)) Downloading requests_mock-1.12.1-py2.py3-none-any.whl.metadata (4.1 kB) Collecting tox<5.0.0 (from -r /usr/src/app/requirements/tests.txt (line 4)) Downloading tox-4.23.2-py3-none-any.whl.metadata (3.7 kB) Collecting asgiref (from django-countries==7.6.1->-r /usr/src/app/requirements/common.txt (line 3)) Downloading asgiref-3.8.1-py3-none-any.whl.metadata (9.3 kB) Collecting typing-extensions (from django-countries==7.6.1->-r /usr/src/app/requirements/common.txt (line 3)) Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB) Collecting setuptools (from django-money==3.5.3->-r /usr/src/app/requirements/common.txt (line 5)) Using cached setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB) Collecting py-moneyed<3.1,>=2.0 (from django-money==3.5.3->-r /usr/src/app/requirements/common.txt (line 5)) Downloading py_moneyed-3.0-py3-none-any.whl.metadata (3.2 kB) Collecting sqlparse>=0.3.1 (from Django==5.0.9->-r /usr/src/app/requirements/common.txt (line 10)) Downloading sqlparse-0.5.3-py3-none-any.whl.metadata (3.9 kB) Collecting sgmllib3k (from feedparser==6.0.11->-r /usr/src/app/requirements/common.txt (line 12)) Downloading sgmllib3k-1.0.0.tar.gz (5.8 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting MarkupSafe>=2.0 (from Jinja2==3.1.4->-r /usr/src/app/requirements/common.txt (line 13)) Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB) Collecting charset-normalizer<4,>=2 (from requests==2.32.3->-r /usr/src/app/requirements/common.txt (line 20)) Downloading charset_normalizer-3.4.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (34 kB) Collecting idna<4,>=2.5 (from requests==2.32.3->-r /usr/src/app/requirements/common.txt (line 20)) Downloading idna-3.10-py3-none-any.whl.metadata (10 kB) Collecting urllib3<3,>=1.21.1 (from requests==2.32.3->-r /usr/src/app/requirements/common.txt (line 20)) Downloading urllib3-2.2.3-py3-none-any.whl.metadata (6.5 kB) Collecting certifi>=2017.4.17 (from requests==2.32.3->-r /usr/src/app/requirements/common.txt (line 20)) Downloading certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB) Collecting sphinxcontrib-applehelp>=1.0.7 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading sphinxcontrib_applehelp-2.0.0-py3-none-any.whl.metadata (2.3 kB) Collecting sphinxcontrib-devhelp>=1.0.6 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading sphinxcontrib_devhelp-2.0.0-py3-none-any.whl.metadata (2.3 kB) Collecting sphinxcontrib-htmlhelp>=2.0.6 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl.metadata (2.3 kB) Collecting sphinxcontrib-jsmath>=1.0.1 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl.metadata (1.4 kB) Collecting sphinxcontrib-qthelp>=1.0.6 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading sphinxcontrib_qthelp-2.0.0-py3-none-any.whl.metadata (2.3 kB) Collecting sphinxcontrib-serializinghtml>=1.1.9 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl.metadata (2.4 kB) Collecting snowballstemmer>=2.2 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading snowballstemmer-2.2.0-py2.py3-none-any.whl.metadata (6.5 kB) Collecting alabaster>=0.7.14 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading alabaster-1.0.0-py3-none-any.whl.metadata (2.8 kB) Collecting imagesize>=1.3 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading imagesize-1.4.1-py2.py3-none-any.whl.metadata (1.5 kB) Collecting packaging>=23.0 (from Sphinx==8.1.3->-r /usr/src/app/requirements/common.txt (line 22)) Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB) Collecting python-dateutil (from time-machine==2.16.0->-r /usr/src/app/requirements/common.txt (line 24)) Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) Collecting psycopg-c==3.2.3 (from psycopg[c]==3.2.3->-r /usr/src/app/requirements/common.txt (line 17)) Downloading psycopg_c-3.2.3.tar.gz (598 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 598.9/598.9 kB 3.2 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... error

Running command git clone --filter=blob:none --quiet https://github.com/brutasse/django-push.git /tmp/pip-install-xbk4s6l8/django-push_869cd076d70d473da4c86b60e8cecf7d
Running command git rev-parse -q --verify 'sha^22fda99641cfbd2f3075a723d92652a8e38220a5'
Running command git fetch -q https://github.com/brutasse/django-push.git 22fda99641cfbd2f3075a723d92652a8e38220a5
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
running dist_info
creating /tmp/pip-modern-metadata-qwn9zhkf/psycopg_c.egg-info
writing /tmp/pip-modern-metadata-qwn9zhkf/psycopg_c.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-modern-metadata-qwn9zhkf/psycopg_c.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-modern-metadata-qwn9zhkf/psycopg_c.egg-info/top_level.txt
writing manifest file '/tmp/pip-modern-metadata-qwn9zhkf/psycopg_c.egg-info/SOURCES.txt'
couldn't run 'pg_config' --includedir: [Errno 2] No such file or directory: 'pg_config'
error: [Errno 2] No such file or directory: 'pg_config'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
py312-tests: exit 1 (21.41 seconds) /usr/src/app> python -I -m pip install -r /usr/src/app/requirements/tests.txt pid=28
py312-tests: FAIL ✖ in 21.53 seconds
py312-flake8: commands[0]> flake8
py312-flake8: OK ✔ in 2.4 seconds
py312-black: OK ✔ in 0.02 seconds
py312-isort: commands[0]> make isort-check
python -m isort --check accounts aggregator blog contact dashboard djangoproject docs foundation fundraising legacy members releases svntogit tracdb
py312-tests: FAIL code 1 (21.53 seconds)
py312-flake8: OK (2.39=setup[0.01]+cmd[2.38] seconds)
py312-black: OK (0.01 seconds)
py312-isort: OK (0.87=setup[0.01]+cmd[0.85] seconds)
evaluation failed :( (24.98 seconds)

</details

[pbratkowski]

The [edit: first part of the] issue occurs due to tox attempting to install requirements in isolation, but the docker build process purging required packages.

djangoproject.com/Dockerfile

Lines 37 to 41 in 75f3fde

	&& apt-get purge --assume-yes --auto-remove \
	gcc \
	libc6-dev \
	libpq-dev \
	zlib1g-dev \

This creates a smaller image, which is good for production but does affect development.

I think the most elegant solution would be to use Docker's multi-stage builds, with base, dev, and production targets. Package installation could then be done in the base image, while unnecessary packages could be purged in production.