hold off on packages until dependencies are built #281

Open
opened 2025-11-27 12:45:29 +01:00 by Riedler · 6 comments

I've been having problems every now and then when I happen to upgrade my system while builds are running - e.g. currently I seem to have some problem regarding qt6 having some packages upgraded, but not others, causing various programs to just crash immediately.

image.png

since it's quite annoying to fix this manually (need to temporarily swap priorities between ALHP and normal arch packages, and manually upgrade packages, which also means I won't have the ALHP variant until the next upgrade of those packages) I would really welcome it if the build process kept co-dependent packages (incl. optdepends) from appearing in repos until all of them are built.

The easiest way to do this would be to ignore dependencies completely and just batch it the same way upstream does (i.e. complete compiling all upstream changes and only then push packages to prod) but that might result in unnecessarily long delays before upgrades trickle down, as well as increased storage requirements (though, I don't think either downside is that bad compared to everyones' systems breaking occasionally)

I've been having problems every now and then when I happen to upgrade my system while builds are running - e.g. currently I seem to have some problem regarding qt6 having some packages upgraded, but not others, causing various programs to just crash immediately. ![image.png](/attachments/f06e14d1-b89f-4815-8a51-aed100d7663e) since it's quite annoying to fix this manually (need to temporarily swap priorities between ALHP and normal arch packages, and manually upgrade packages, which also means I won't have the ALHP variant until the next upgrade of those packages) I would really welcome it if the build process kept co-dependent packages (incl. optdepends) from appearing in repos until all of them are built. The easiest way to do this would be to ignore dependencies completely and just batch it the same way upstream does (i.e. complete compiling all upstream changes and only then push packages to prod) but that might result in unnecessarily long delays before upgrades trickle down, as well as increased storage requirements (though, I don't think either downside is that bad compared to everyones' systems breaking occasionally)
1.2 MiB

look closely - the situation appears because you are upgrading packages that are taken from the Arch repositories because they are either ANY packages or packages that failed before or packages that are blacklisted - TLDR: packages that are not present in ALHP - that is all done while the packages present in ALHP are still being built.

ALHP can't hold your packages on your system if they come from the Arch repositories. That's up to you.

look closely - the situation appears because you are upgrading packages that are taken from the Arch repositories because they are either ANY packages or packages that failed before or packages that are blacklisted - TLDR: packages that are not present in ALHP - that is all done while the packages present in ALHP are still being built. ALHP can't hold your packages on your system if they come from the Arch repositories. That's up to you.
Author

how the hell did that happen? 😕 I'm 100% sure I never manually installed the non-ALHP qt6-base, and I know pacman doesn't use a lower-prio repo if a higher-prio repo has the requested package, even if the lower-prio repo has a newer version. And I can see that ALHP does have qt6-base built so I have no idea how this happened.

Please keep this issue open and I will report back next time something like this happens with more info.

how the hell did that happen? 😕 I'm 100% sure I never manually installed the non-ALHP `qt6-base`, and I know pacman doesn't use a lower-prio repo if a higher-prio repo has the requested package, even if the lower-prio repo has a newer version. And I can see that ALHP does have `qt6-base` built so I have no idea how this happened. Please keep this issue open and I will report back next time something like this happens with more info.

You can uncomment the line #VerbosePkgLists in your /etc/pacman.conf (to read VerbosePkgLists) to make it easier to spot non-ALHP updates.
It can happen that another Arch - package pulls in an Arch package as a dependency although it does have a corresponding package in ALHP but that one does not have the required version yet - but that would be a modification in pacman or the way that Arch handles dependencies, not something that ALHP can change.

That being said, it might be possible to write a pacman hook that checks if packages are still being built in ALHP when trying to update - but I won't have time to help with that, not sure how @anonfunc sees that

You can uncomment the line `#VerbosePkgLists` in your `/etc/pacman.conf` (to read `VerbosePkgLists`) to make it easier to spot non-ALHP updates. It can happen that another Arch - package pulls in an Arch package as a dependency although it does have a corresponding package in ALHP but that one does not have the required version yet - but that would be a modification in pacman or the way that Arch handles dependencies, not something that ALHP can change. That being said, it might be possible to write a pacman hook that checks if packages are still being built in ALHP when trying to update - but I won't have time to help with that, not sure how @anonfunc sees that
Author

It can happen that another Arch - package pulles in an Arch package as a dependency although it does have a corresponding package in ALHp but that one does not have the required version yet

this could theoretically be fixed by serving non-ALHP packages like a normal arch mirror instead of relying on pacman to fall back on regular arch mirrors. This way you could remove upstream arch from the list of repos and eliminate the chance of this happening.

> It can happen that another Arch - package pulles in an Arch package as a dependency although it does have a corresponding package in ALHp but that one does not have the required version yet this could theoretically be fixed by serving non-ALHP packages like a normal arch mirror instead of relying on pacman to fall back on regular arch mirrors. This way you could remove upstream arch from the list of repos and eliminate the chance of this happening.
Owner

It can happen that another Arch - package pulles in an Arch package as a dependency although it does have a corresponding package in ALHp but that one does not have the required version yet

this could theoretically be fixed by serving non-ALHP packages like a normal arch mirror instead of relying on pacman to fall back on regular arch mirrors. This way you could remove upstream arch from the list of repos and eliminate the chance of this happening.

Yea I though about that, but that would mean we would need to freeze the mirror packages to the same build-cycles the buildbot runs at. certainly doable, but not sure I have time for that rn. But I'll have another look.

> > It can happen that another Arch - package pulles in an Arch package as a dependency although it does have a corresponding package in ALHp but that one does not have the required version yet > > this could theoretically be fixed by serving non-ALHP packages like a normal arch mirror instead of relying on pacman to fall back on regular arch mirrors. This way you could remove upstream arch from the list of repos and eliminate the chance of this happening. Yea I though about that, but that would mean we would need to freeze the mirror packages to the same build-cycles the buildbot runs at. certainly doable, but not sure I have time for that rn. But I'll have another look.
Owner

That being said, it might be possible to write a pacman hook that checks if packages are still being built in ALHP when trying to update - but I won't have time to help with that, not sure how @anonfunc sees that

This is more or less already possible by utilizing https://somegit.dev/ALHP/ALHP.utils, you would just need to wire that up into a hook. But this also produces files-positives, since not every update may behave breaking in that sense.

> That being said, it might be possible to write a pacman hook that checks if packages are still being built in ALHP when trying to update - but I won't have time to help with that, not sure how @anonfunc sees that This is more or less already possible by utilizing https://somegit.dev/ALHP/ALHP.utils, you would just need to wire that up into a hook. But this also produces files-positives, since not every update may behave breaking in that sense.
Sign in to join this conversation.