Skip to content

Adds Ruby 4.0.0#183

Open
rus-max wants to merge 1 commit intomainfrom
ruby-4.0.0
Open

Adds Ruby 4.0.0#183
rus-max wants to merge 1 commit intomainfrom
ruby-4.0.0

Conversation

@rus-max
Copy link
Collaborator

@rus-max rus-max commented Dec 26, 2025

@rus-max
Copy link
Collaborator Author

rus-max commented Dec 26, 2025

Hi!
Could you please help me with the failed pipeline?
@FooBarWidget

@FooBarWidget
Copy link
Member

FooBarWidget commented Dec 29, 2025

@rus-max I think Github Actions is failing due to running out of disk space. I see this in attempt 4, there was an error saying "no space left on device". Looks like Github Actions runner itself fails due to running out of disk space which is why you don't even see any logs after the event.

This is a known limitation of the current approach, and one of my fears with #178, which really increases disk space stress. We haven't invested in a structural solution for a long time but it seems we've finally hit the limit.

A real, robust, long-term solution would be to switch to an APT management system that doesn't rely on the local disk and can operate directly on cloud storage. I don't think such a solution exists, which means we'll have to write our own (not for the faint of heart).

Short-term workarounds might be possible, but we don't know until we try. They don't really solve the underlying limitation, but kick the can down the road, which means we'll run into the problem some time in the future again.

  • Clean Github Actions runner disk space more aggressively. I don't know how though: I think we're already cleaning everything we can.
  • Remove unnecessary packages from the APT repository. Some thoughts here:
    • We can remove all but the latest package revisions.
      • For example if there's fullstaq-ruby-2.5.7-jemalloc_0 and fullstaq-ruby-2.5.7-jemalloc_1, we can get rid of the _0 revision since it has absolutely no value anymore. Nobody's going to use an older package revision.
      • This requires a surgical modification on the APT repo directly. This means downloading the APT repo version folder from Google Cloud Storage, using aptly to remove the relevant packages from the database and publishing the changes to a local folder, then uploading the result back to Google Cloud Storage as a new APT repo version folder.
    • We can remove the malloctrim variant for Ruby >= 3.3 since Ruby natively has it.
      • This means people who installed the malloctrim variant for Ruby >= 3.3 won't get auto-upgrades anymore, and their existing 'apt-get install' automations will fail, but I think that's an acceptable tradeoff if that can give us more breathing space. We'll just have to communicate this to users well.
      • This will involve surgically removing relevant packages from Google Cloud Storage, like what I described above.
      • It also means modifying the build system to no longer build malloctrim variant for Ruby >= 3.3.
  • Some other workaround that gives us more disk space? Maybe there's some block storage we can mount? Or maybe we can mount Azure Blob Storage as an ephemeral local filesystem through FUSE, and hope that Aptly works properly with that? For example something like JuiceFS. I expect performance to be pretty bad though.

I would really like to work on the long-term solution, but unfortunately my health doesn't allow me to at the moment.

@FooBarWidget
Copy link
Member

FooBarWidget commented Dec 29, 2025

@rus-max So let me know how you would like to proceed and whether you need anything more from me. Access can be arranged. With my current health, I am in no condition to do any of that work myself. Or perhaps you can rally the community for help.

@rus-max
Copy link
Collaborator Author

rus-max commented Dec 30, 2025

I'm ready to take responsibility for starting with the simple approach — freeing up space for new builds.

Get well soon and come back stronger!

@FooBarWidget

@FooBarWidget
Copy link
Member

Thanks! I've sent you an email with additional setup instructions.

@FooBarWidget
Copy link
Member

Oh looks like your Git email address is not valid. What's your email address? If you don't want to post it here in public, you can email me instead. See https://github.com/fullstaq-ruby/server-edition/blob/main/dev-handbook/members.md

@rus-max
Copy link
Collaborator Author

rus-max commented Dec 30, 2025

Oh looks like your Git email address is not valid. What's your email address? If you don't want to post it here in public, you can email me instead. See https://github.com/fullstaq-ruby/server-edition/blob/main/dev-handbook/members.md

russs.max@gmail.com
Thanks!

@FooBarWidget
Copy link
Member

I just thought of another strategy:
Move all EOL'ed packages to a separate APT repo (let's call it an "archive" repo) that we won't touch in the CI. This means users will have to add two APT source entries, one for the normal repo and one for the archive repo. This will buy us a lot of time and this strategy can be used again and again in the future.

@ferblape
Copy link

Hi @FooBarWidget thanks for this. Could you finally separate the APT repositories?

@FooBarWidget
Copy link
Member

Hi @ferblape, due to health issues I'm not working on Fullstaq Ruby at the moment. I've entrusted the matter to @rus-max.

Max, have you been able to make any progress? Do you need any advice?

@rus-max
Copy link
Collaborator Author

rus-max commented Jan 14, 2026

Hi!
I'm currently studying the materials. I'll provide an answer in the coming days.

@FooBarWidget

@Webics
Copy link

Webics commented Jan 29, 2026

@rus-max Any updates so far?

@eis-ioki
Copy link

eis-ioki commented Feb 2, 2026

After a lot of research I ended up not using fullstaq ruby anymore, this is in my Dockerfile now:

FROM ruby:4.0.1-slim-trixie

RUN apt-get install -y libtcmalloc-minimal4
ENV LD_PRELOAD libtcmalloc_minimal.so.4
  1. Fullstaq ruby malloc_trim variant is not needed for ruby >= 3.3, it's already a part of the ruby code as Hongli notes in the ruby lang ticket.
  2. Fullstaq ruby jemalloc variant can be replaced by preloading it on linux:
    RUN apt-get install libjemalloc2
    ENV LD_PRELOAD libjemalloc.so.2
  3. jemalloc is not maintained anymore, tcmalloc is a great alternative also well suited for long running server processes, which can also be transparently preloaded.

The amount of committed memory on our servers has decreased drastically. Now I don't know if this is because I switched from ruby 3.4.8 to 4.0.1 or if it is because I switched from jemalloc to tcmalloc. In either case I'm happy and would like to thank the fullstaq ruby project for it's great work at the time. Nowadays I think fullstaq ruby is outdated and you can just switch to the official ruby docker image.

PS: I realize that the fullstaq ruby project hinges on the idea that jemalloc 3.6 outperforms 5.4 and that compiling it with ruby makes a difference. I don't think either is true (anymore) and the evidence from 2018 with ruby 3.0 when fullstaq ruby was formed would need to be updated for ruby 4.0 in 2026. In general I do believe that a different memory allocator will help with fragmentation and speed though, so I am going with tcmalloc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants