[Problem] Weekly builds "freecad_version.txt" content does not match version number in bundle

[Murmele]

Describe the bug
The version in the "freecad_version.txt" is quite actual. But the version number of the bundles is much lower and when installing the version it can be also seen the number of the bundle.

• FreeCad bundle version number matches the version number when opening FreeCad
• FreeCad bundle version number does not match the version number of the "freecad_version.txt" file

Content of the "freecad_version.txt":

rev_number: 38693 (Git)
branch_name: main
commit_date: 2024/09/10 18:06:05
commit_hash: 85ade21436d4ce618691720688b299c9b6902590
remote_url: https://github.com/FreeCAD/FreeCAD main

OS: Windows 11 build 22621
Word size of FreeCAD: 64-bit
Version: 1.0.0RC1.38643 (Git)
Build type: Release
Branch: main
Hash: fe40748c10503eed367840239db3e0f81873e359
Python 3.11.9, Qt 5.15.13, Coin 4.0.2, Vtk 9.2.6, OCC 7.7.2
Locale: English/United States (en_US)

[oursland]

The file freecad_version.txt refers to the version of the freecad_source.tar.gz file. I do understand the confusion; I had to look up how these files were created to figure it out.

[Murmele]

Is it possible to create bundles from the version in the freecad_version.txt instead of just a weekly? A nightly would be already nice, because my PR got merged a few days ago but now I have to wait for a complete week to get a bundle to give it a friend.

[oursland]

Currently the Bundle is built once a week. I agree that nightly builds would be preferable, but I am unfamiliar with the costs of doing so.

The source is generated every 6 hours, or 4 times a day.

@adrianinsaval A couple questions:

1. Would nightly builds be possible, and if so, are there any costs to doing so?
2. Why is the source generated out-of-sync with the built artifacts?

[adrianinsaval]

1 - there is not a monetary cost to us, we use github's free runners. The builds are compute intensive though because they are clean builds with no caching so they take up to 2.5h. Doing this daily for little benefits seems a bit abusive IMO.

In any case while we call them weeklies because we kept the original name, nowadays they happen twice a week actually on monday and friday nights, this was deemed a good compromise in the past. It is worth nothing that github seems to limit how many ci runs you run concurrently. I noticed that even the source generation workflow took a while the other day because main's ci wasn't using caching properly so there where many long workflows running at the same time. Making these happen daily might slow down other more important workflows. One more thing to consider is that storage space in the conda repo is limited and currently old builds are not cleaned automatically, If builds where to happen daily this will fill up much more often than now so this should be automated.

So I have to ask, is there real benefit in having daily builds as opposed to twice a week? I'm not opposed to the idea though.

2 - loooo's idea IIRC was that these source files could be used by other packagers that would like to do dev releases and wanted an up to date source package without using git, with proper version info and nowadays they also include submodules. The source package creation workflow is fairly short so it's not much of an issue to run it multiple times, this workflow also pushes the source with version info and submodules to a dummy repo in salsa.debian.org that is used for the ubuntu ppa daily builds.

[adrianinsaval]

as for the OP, this is expected. I'm open to suggestions about how to document this to avoid confusion but due to the order in which these files are created even if we did match their schedules it might be the case that the version won't match, for example when a build fails. Version info is displayed in the filename already so users should not pay attention to freecad_version.txt.

We could also just remove freecad_version.txt as I think it's not really required for anything in particular.

[adrianinsaval]

ah one more thing, the arm bundles are not currently done by github runners but by cirrus-ci https://cirrus-ci.com/github/FreeCAD/FreeCAD-Bundle which is also free but I believe does have a limit on how many minutes per month you can use, so that probably shouldn't be moved to a daily schedule.
I think it should be doable to port this back to github though, by using qemu for the little portion that needs to run an arm binary or eliminating the need to run the binary at all. Also github is supposedly going to eventually support linux arm too.

[oursland]

Thanks, @adrianinsaval ! That does clear up a lot of things for me.

I've been informed that some developers do not built locally, which is really bizarre to me. Instead they rely upon committing code, getting through code review, and waiting for the weekly build. I find this kinda wild, but I can see utility in having frequent builds to have access to bugfixes and other user-impacting functionality available to those users and devs who do not build locally.

The limitation on concurrent runners is a serious concern, so I'm not going to suggest increasing the frequency of builds until that limitation is removed.

[adrianinsaval]

I don't know what the actual limit is, there certainly can be several jobs at the same time. But I have noticed a few times that when there are too many running jobs new jobs take long to be assigned a runner

[adrianinsaval]

I've been informed that some developers do not built locally, which is really bizarre to me.

Indeed quite bizarre, one possible way to help with that could be to upload a compiled package in the ci workflow as an artifact with a conda environment file that would make it easy to reproduce the environment locally. That way devs that don't compile could download the artifact after pushing a commit and then install and test locally, that would also mean that a PR would get tested before merge

[oursland]

Someone has requested this in the past for PRs. I was under the impression that there's space limitations for the GitHub storage. What would the lifespan of such an artifact be?

[Murmele]

I've been informed that some developers do not built locally, which is really bizarre to me.

Indeed quite bizarre, one possible way to help with that could be to upload a compiled package in the ci workflow as an artifact with a conda environment file that would make it easy to reproduce the environment locally. That way devs that don't compile could download the artifact after pushing a commit and then install and test locally, that would also mean that a PR would get tested before merge

Yes this would be also quite nice, so everybody can test a pullrequest before it gets merged, without having to compile the project by him self.

[luzpaz]

So I have to ask, is there real benefit in having daily builds as opposed to twice a week? I'm not opposed to the idea though.

I think during the RC process we should consider doubling that (4 times a week). Reasons:

• Fixes are merged during the week as well. Testers and engaged users need to wait 1-5 days in order to test and provide feedback
• Weekend activity is peak opensource volunteering.
• Some commits need to follow-up fixes and so more waiting happens till next build to test changes
• During the RC process more eyeballs and participation is happening. We can take that engagement into consideration
• Higher potential to on-board more devs, testers, triagers, and volunteers to the community because of the higher output and potentially streamlined build workflow

Edit: Case in point: FreeCAD/FreeCAD#16479 (comment)