ci: use much faster D: drive for TEMP on Windows

[ichard26]

This is apparently an inherent limitation of Azure (which powers GHA) which uses a slow C: drive for the OS (read-optimized) and a fast D: drive for working space.

I first thought a Dev Drive would be even faster (#13123), after a lot of testing, it seems to be slower than simply moving TEMP to the D: drive. To prove this, I did a run where the entire test suite was run back to back three times:

• First run: TEMP=D:/Temp
• Second run: TEMP on ReFS/Dev Drive
• Third run: TEMP=C:/Temp aka the status quo on main

[ichard26]

For reference, here are the average CI runtimes for the last 100 successful runs on main:

[3.8 (1)] mean: 0:17:35 min: 0:16:38
[3.8 (2)] mean: 0:15:00 min: 0:13:46
[3.13 (1)] mean: 0:11:15 min: 0:09:57
[3.13 (2)] mean: 0:10:44 min: 0:09:22

[pfmoore]

The script tools/ci/devdrive.ps1 creates the dev drive at C:/pip_dev_drive.vhdx, which is the slow disk (and there's a message saying dev drives aren't supported, which might indicate C: is unsuitable in other ways as well). Is it faster if you have New-VHD -Path D:/pip_dev_drive.vhdx?

[ichard26]

@pfmoore Sorry there's a lot of context I forgot to include. The Dev Drive PR is out of date. The final Dev Drive patch I tested is included on this branch: https://github.com/pypa/pip/tree/refs/heads/d-drive (specifically 483859b).

In that version, the VHDX is created on the D: drive as best for performance.

pip/tools/ci/devdrive.ps1

Lines 25 to 26 in 483859b

	$OSVersion = [Environment]::OSVersion.Version
	$Partition = New-VHD -Path D:/pip_dev_drive.vhdx -SizeBytes $size |

As well, it is true that Dev Drives aren't supported on the current Windows 2019 runners. I wrote in that check 🙂. However, I was talking to Steve Dower and he said that simply using a ReFS drive should get you 90% of the way there:

You might also be on an OS that doesn't have it yet and so it's disabled (ReFS on a separate drive should give you most of the benefits though - all Dev Drive really adds on top of that is reducing Windows Defender's impact, but that'll be turned off on GHA already).

If you look at the run I linked earlier (which is hard to read admittedly), you'll notice that last unit/integration test steps are done on the ReFS drive, while the first test steps are done on the D: drive. The D: drive beats the ReFS drive on D: handily.