ALHP/ALHP.GO
6
43
Fork 9
Code Issues 25 Pull Requests Projects Releases Wiki Activity

The database files in the source have invalid signatures #279

New Issue
Closed
opened 2025-10-22 09:38:42 +02:00 by w568w · 7 comments
w568w commented 2025-10-22 09:38:42 +02:00
Copy Link

Since today, I found I can no longer update with ALHP. The error reported is as follows:

$ LANG=C sudo pacman -Syu
error: core-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: extra-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: core-x86-64-v3: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: extra-x86-64-v2: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: multilib-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
:: Synchronizing package databases...
<redacted>

error: core-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: extra-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: core-x86-64-v3: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: extra-x86-64-v2: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: multilib-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid
error: failed to synchronize all databases (unexpected error)

After doing some research and following community guidance, I have ruled out all issues related to local configuration and I am certain the problem lies with the source itself (though I'm willing to provide other information as you ask).

You can reproduce the verification error using the steps below:

$ wget https://cdn.alhp.dev/extra-x86-64-v4/os/x86_64/extra-x86-64-v4.db
$ wget https://cdn.alhp.dev/extra-x86-64-v4/os/x86_64/extra-x86-64-v4.db.sig
# Import the public key into the personal GPG store. You can skip this step if you've already done so, or if you prefer to use `sudo gpg --homedir /etc/pacman.d/gnupg`.
$ gpg --auto-key-locate keyserver --locate-keys buildbot@alhp.dev

Then, running gpg --verify extra-x86-64-v4.db.sig yields the following output:

gpg: assuming signed data in 'extra-x86-64-v4.db'
gpg: Signature made Wed Oct 22 09:21:44 2025 CST
gpg:                using EDDSA key BC3993A9EBDD40E5C242D72F0FE58E8D1B980E51
gpg: BAD signature from "ALHP Buildbot <buildbot@alhp.dev>" [unknown]

Are the database files (extra-x86-64-v4.db) or the .sig files corrupted?

Since today, I found I can no longer update with ALHP. The error reported is as follows: ``` $ LANG=C sudo pacman -Syu error: core-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: extra-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: core-x86-64-v3: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: extra-x86-64-v2: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: multilib-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid :: Synchronizing package databases... <redacted> error: core-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: extra-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: core-x86-64-v3: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: extra-x86-64-v2: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: multilib-x86-64-v4: signature from "ALHP Buildbot <buildbot@alhp.dev>" is invalid error: failed to synchronize all databases (unexpected error) ``` After doing some research and following community guidance, I have ruled out all issues related to local configuration and I am certain the problem lies with the source itself (though I'm willing to provide other information as you ask). You can reproduce the verification error using the steps below: ```bash $ wget https://cdn.alhp.dev/extra-x86-64-v4/os/x86_64/extra-x86-64-v4.db $ wget https://cdn.alhp.dev/extra-x86-64-v4/os/x86_64/extra-x86-64-v4.db.sig # Import the public key into the personal GPG store. You can skip this step if you've already done so, or if you prefer to use `sudo gpg --homedir /etc/pacman.d/gnupg`. $ gpg --auto-key-locate keyserver --locate-keys buildbot@alhp.dev ``` Then, running `gpg --verify extra-x86-64-v4.db.sig` yields the following output: ``` gpg: assuming signed data in 'extra-x86-64-v4.db' gpg: Signature made Wed Oct 22 09:21:44 2025 CST gpg:                using EDDSA key BC3993A9EBDD40E5C242D72F0FE58E8D1B980E51 gpg: BAD signature from "ALHP Buildbot <buildbot@alhp.dev>" [unknown] ``` Are the database files (`extra-x86-64-v4.db`) or the `.sig` files corrupted?
w568w commented 2025-10-22 11:53:46 +02:00
Author
Copy Link

Seems that it's the .sig files that were broken, and now they've been corrected:

$ sha256sum extra-x86-64-v4.db extra-x86-64-v4.db.1
9f2bc1258f17c691373d13fe8fd5ba281037784f585719c0edcf6f821497f74d  extra-x86-64-v4.db
9f2bc1258f17c691373d13fe8fd5ba281037784f585719c0edcf6f821497f74d  extra-x86-64-v4.db.1

$ sha256sum extra-x86-64-v4.db.sig extra-x86-64-v4.db.sig.1 
269f45572ec76c9dfe7b357bd19169ffc9515a8e549a949be10bad7dbb8b8fe4  extra-x86-64-v4.db.sig
b62a987d7c1b87336b73a416cc18301898dbf7cc0cdeb350ab5f1be2e8dfbd04  extra-x86-64-v4.db.sig.1

I will keep this issue open in the hope that it could remind maintainers to check for potential pitfalls in the signature process.

Seems that it's the `.sig` files that were broken, and now they've been corrected: ```bash $ sha256sum extra-x86-64-v4.db extra-x86-64-v4.db.1 9f2bc1258f17c691373d13fe8fd5ba281037784f585719c0edcf6f821497f74d extra-x86-64-v4.db 9f2bc1258f17c691373d13fe8fd5ba281037784f585719c0edcf6f821497f74d extra-x86-64-v4.db.1 $ sha256sum extra-x86-64-v4.db.sig extra-x86-64-v4.db.sig.1 269f45572ec76c9dfe7b357bd19169ffc9515a8e549a949be10bad7dbb8b8fe4 extra-x86-64-v4.db.sig b62a987d7c1b87336b73a416cc18301898dbf7cc0cdeb350ab5f1be2e8dfbd04 extra-x86-64-v4.db.sig.1 ``` I will keep this issue open in the hope that it could remind maintainers to check for potential pitfalls in the signature process.
anonfunc commented 2025-10-23 01:03:38 +02:00
Owner
Copy Link

Hi! You probably saw a desync between db's and signatures on the CDN. This was most likely not a signing problem but a cache invalidation one. I'll have a look later on why this may have happened.

Hi! You probably saw a desync between db's and signatures on the CDN. This was most likely not a signing problem but a cache invalidation one. I'll have a look later on why this may have happened.
w568w commented 2025-10-23 10:33:24 +02:00
Author
Copy Link

Hi! You probably saw a desync between db's and signatures on the CDN. This was most likely not a signing problem but a cache invalidation one.

Thanks for your response and maintenance effort. I've already run into desync issues with cdn.alhp.dev at least three (update: four) times over yesterday and today. (It actually only managed to sync updates properly right as I was writing this reply!) So, it might be worth looking into. ;)

> Hi! You probably saw a desync between db's and signatures on the CDN. This was most likely not a signing problem but a cache invalidation one. Thanks for your response and maintenance effort. I've already run into desync issues with `cdn.alhp.dev` at least three (update: four) times over yesterday and today. (It actually only managed to sync updates properly right as I was writing this reply!) So, it might be worth looking into. ;)
anonfunc commented 2025-10-23 18:57:53 +02:00
Owner
Copy Link

Hi! You probably saw a desync between db's and signatures on the CDN. This was most likely not a signing problem but a cache invalidation one.

Thanks for your response and maintenance effort. I've already run into desync issues with cdn.alhp.dev at least three (update: four) times over yesterday and today. (It actually only managed to sync updates properly right as I was writing this reply!) So, it might be worth looking into. ;)

For sure, especially with this frequency. Can you give me an approximate region so I can check with the CDN provider if there maybe some incident going on with cache validation?

> > Hi! You probably saw a desync between db's and signatures on the CDN. This was most likely not a signing problem but a cache invalidation one. > > Thanks for your response and maintenance effort. I've already run into desync issues with `cdn.alhp.dev` at least three (update: four) times over yesterday and today. (It actually only managed to sync updates properly right as I was writing this reply!) So, it might be worth looking into. ;) For sure, especially with this frequency. Can you give me an approximate region so I can check with the CDN provider if there maybe some incident going on with cache validation?
w568w commented 2025-10-23 19:35:35 +02:00
Author
Copy Link

For sure, especially with this frequency. Can you give me an approximate region so I can check with the CDN provider if there maybe some incident going on with cache validation?

My physical location is in mainland China (AS17621), for testing purposes, I also tried VPN providers from Taiwan (AS215672) and Hong Kong (AS48266). So basically East Asia region.

> For sure, especially with this frequency. Can you give me an approximate region so I can check with the CDN provider if there maybe some incident going on with cache validation? My physical location is in mainland China (AS17621), for testing purposes, I also tried VPN providers from Taiwan (AS215672) and Hong Kong (AS48266). So basically East Asia region.
👍 1
anonfunc commented 2025-10-24 17:32:56 +02:00
Owner
Copy Link

Thanks for the helpful info. I think I could narrow it down: Seems like the higher latency to the SEA/China POPs caused a cache invalidation issue for the databases and signatures (probably some propagation race condition).

I now enforce a stricter revalidation for those files, hopefully fixing this.

Please keep an eye on if this happens again.

Thanks for the helpful info. I think I could narrow it down: Seems like the higher latency to the SEA/China POPs caused a cache invalidation issue for the databases and signatures (probably some propagation race condition). I now enforce a stricter revalidation for those files, hopefully fixing this. Please keep an eye on if this happens again.
👍 1
w568w commented 2025-10-25 06:13:54 +02:00
Author
Copy Link

Thanks for the helpful info. I think I could narrow it down: Seems like the higher latency to the SEA/China POPs caused a cache invalidation issue for the databases and signatures (probably some propagation race condition).

I now enforce a stricter revalidation for those files, hopefully fixing this.

Please keep an eye on if this happens again.

Thanks! I tried again about an hour after your reply, and the signature was still incorrect at that time. However, it seems to be working correctly now.

Since the issue appears to be resolved, I'll close it for now. I will continue to monitor it in case the problem recurs.

> Thanks for the helpful info. I think I could narrow it down: Seems like the higher latency to the SEA/China POPs caused a cache invalidation issue for the databases and signatures (probably some propagation race condition). > > I now enforce a stricter revalidation for those files, hopefully fixing this. > > Please keep an eye on if this happens again. Thanks! I tried again about an hour after your reply, and the signature was still incorrect at that time. However, it seems to be working correctly now. Since the issue appears to be resolved, I'll close it for now. I will continue to monitor it in case the problem recurs.
👍 1
w568w closed this issue 2025-10-25 06:13:54 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked upstream
bug
Something is not working
 build-failure
package build fails
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 informational
Provides information about something
 invalid
Something is wrong
 invalid/corrupt package
Package needs to be rebuild
 packaging issue
priority: high
High priority
 question
More information is needed
 support
wontfix
This won't be fixed
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
anonfunc
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ALHP/ALHP.GO#279
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?