[WIP] Trying to add ZFS with compression support

[xylonzinho]

• Experimenting with ZFS and real-time file decompression images.
• makezfs.sh contains a script to create the images files, and bruteforce the best free space marging for the application.

This branch has conflicts with the latest main branch. I will work on solving the conflicts soon.

I still didnt check if it is working or not in the target console. Pehaps this will never work; we will see :)

But the image file creation of .ffzfs files is working and has good compression results (given the fact that zfs needs free space to operate).

Without Space Optimization

• Input size (raw files): 6306749913 bytes (~6014 MB)
• Free space margin: 268435456 bytes (~256 MB, fixed in --no-optimize mode)
• Final image: 6575620096 bytes (~6271 MB, -4% reduction)

With Space Optimization (10%)

• Input size (raw files): 6306749913 bytes (~6014 MB)
• Compressed data: 3951005696 bytes (~3767 MB, 37% reduction)
• Free space margin: 395100569 bytes (~376 MB, 10%)
• Final image: 4346347520 bytes (~4145 MB, 31% reduction)

With Space Optimization And Brute Force mode (85)

• Input size (raw files): 6306749913 bytes (~6014 MB)
• Compressed data: 3954462720 bytes (~3771 MB, 37% reduction)
• Free space margin: 355901644 bytes (~339 MB, 8% actual)
• Final image: 4310695936 bytes (~4111 MB, 31% reduction)

FYI, @drakmor.

[drakmor]

PS5 kernel doesn't support ZFS

[xylonzinho]

PS5 kernel doesn't support ZFS

Haha, good to know :D
Any other FS that you migth think it supports with compression support?

[drakmor]

PPR-PFS and PFS (PS4)

[xylonzinho]

@drakmor are you saying that the PFS support that is already being worked on the project already can support file compression? Anything specific you need help with?
Adiditonally, can you please point me to some direction on how to create the ffpfs files?

[drakmor]

There is another type of PFS (savedata) and it works very slowly.

[drakmor]

https://github.com/drakmor/ShadowMountPlus/actions/runs/23715002617 u can test it )

[xylonzinho]

https://github.com/drakmor/ShadowMountPlus/actions/runs/23715002617 u can test it )

Yes, I tried. The main logic works but the mount command fails.
Need to find another route :D

[15:13:15]   [IMG] Mounting image (zfs): /data/homebrew/PPSA07923-app.ffzfs -> /mnt/shadowmnt/PPSA07923-app_9645c939
[15:13:15]   [IMG][LVD] attach backend selected for /data/homebrew/PPSA07923-app.ffzfs
[15:13:15]   [IMG][LVD] attach try: ver=0 sec=4096 sec2=4096 raw=0x9 flags=0x1c img=0
[15:13:15]   [IMG][LVD] attach returned unit=1
[15:13:15]   [IMG][LVD] Attached as /dev/lvd1
[15:13:15]   [IMG][LVD] nmount rdonly failed: Operation not supported by device
[15:13:15] NOTIFY: Image mount failed: 0x00000013 (Operation not supported by device)
/data/homebrew/PPSA07923-app.ffzfs

[xylonzinho]

Now I am thinking on an Hybrid approach. (I have no idea if this makes sense)

What if we use usf for the main files, but replace some large files with symbolic/hard links to a different folder, that could be mounted using PFS.

ABC1234-app/    < mounted using USF)
 - a_small_file.txt
 - a_big_file_that_is_almost_never_used.xyz (Symbolic link to  ./ABC1234-app-compressed/a_big_file_that_is_almost_never_used.xyz
  
 ABC1234-app-compressed/    < mounted using the slow FS, but it supports file decompression
 - a_big_file_that_is_almost_never_used.xyz. (real file here)

The idea here is that important and frequently accessed files would be loaded at USF speeds and unecessary files like extra languages and videos could be put inside the compressed mount.

Again, I have no idea if this would work :)
But I need to learn how PSF works.

[RenanGBarreto]

Even before that, I wil try PSF, but I have no idea how to create the image correctly.