Zed editor / what to do about packages that fail due to OOM in build #252

New Issue

Open

opened 2024-07-16 14:52:12 +02:00 by Ralim · 4 comments

Ralim commented

2024-07-16 14:52:12 +02:00

Hello,
I really hope this isn't a duplicate.

I've noticed the zed package (text editor in rust, so likely a pain to build) is failing due to OOM (memory limit exceeded).
Should this package be blacklisted/dropped; or is there some better way to handle these failures (such as forcing less build concurrency for larger rust builds)?

Hello, I _really_ hope this isn't a duplicate. I've noticed the `zed` package (text editor in rust, so likely a pain to build) is failing due to OOM (`memory limit exceeded`). Should this package be blacklisted/dropped; or is there some better way to handle these failures (such as forcing less build concurrency for larger rust builds)?

anonfunc commented

2024-07-16 19:20:13 +02:00

Owner

Hi 👋.

Would be cool to build those, yeah. Probably need a different build profile for that, but it would be possible. My main concern is that this heavy memory usage occurs during linking, where there would be no parallel building anyway, and -j1 for example would have no effect. Building without LTO would have an effect.

Hi 👋. Would be cool to build those, yeah. Probably need a different build profile for that, but it would be possible. My main concern is that this heavy memory usage occurs during linking, where there would be no parallel building anyway, and `-j1` for example would have no effect. Building without LTO would have an effect.

Ralim commented

2024-07-17 13:13:32 +02:00

Author

Ack, good point that its the link time exploding.

Would it be preferable to drop LTO for these packages (automatically) or do something else for them?
I'm happy to try and help but not sure what would be most preferable for the project.

Ack, good point that its the link time exploding. Would it be preferable to drop LTO for these packages (automatically) or do something else for them? I'm happy to try and help but not sure what would be most preferable for the project.

BS86 commented

2024-07-19 19:13:08 +02:00

Personally, I have very good experience with Zram and the zram-generator method.
I only have 32GB RAM and with the default config it adds 16GB as Swap.
It sounds like it would lead to performance issues because of it's CPU usage, but it actually doesn't. I can perfectly compile packages on all 16 threads here with the compilation process using 40-45 GB and the machine is still perfectly usable for regular desktop usage like watching videos or surfing the web.
That should help with adding some emergency buffer for RAM-hungry compilations.

Personally, I have very good experience with [Zram](https://wiki.archlinux.org/title/Zram) and the [zram-generator](https://wiki.archlinux.org/title/Zram#Using_zram-generator) method. I only have 32GB RAM and with the default config it adds 16GB as Swap. It sounds like it would lead to performance issues because of it's CPU usage, but it actually doesn't. I can perfectly compile packages on all 16 threads here with the compilation process using 40-45 GB and the machine is still perfectly usable for regular desktop usage like watching videos or surfing the web. That should help with adding some emergency buffer for RAM-hungry compilations.

anonfunc commented

2024-07-24 21:05:18 +02:00

Owner

The build machine has zswap enabled (up to 50% pool size), so zram should not make a significant difference. I think disabling LTO might be something we can try. I'll try that on zed as soon as I find the time.

The build machine has zswap enabled (up to 50% pool size), so zram should not make a significant difference. I think disabling LTO might be something we can try. I'll try that on `zed` as soon as I find the time.