# Compressed r/w FS or alternatives to aufs?

## dE_logics

Thunderbird mail storage directory becomes HUGE in size, most of the content being text; so I was looking forward towards a solution.

A long time ago, I compiled fusecompress, at that time it was at version 2.

Now I cant get any of available fusecompress ebuilds to compile, all cause of boost errors.

Thunderbird stores mails in a single large text file which's the main content to be compressed; with aufs (+squashfs), if one of those files changes by a bit, the whole file is copied to the r/w location by aufs, making it effectively consume more space with every run of Thunderbird.

:sight:

Are there any options remaining?

----------

## 666threesixes666

after seeing 100s of episodes of hoarders, i decided to unsubscribe to everything, start monitoring my email traffic and snap it up when it comes.  i read and purge it.

did you try to unmask the very latest boost and compile that?  or go back to previous versions of boost?  id just start purging and unsubscribing, only keeping the very most important mail.

----------

## dE_logics

It doesn't matter. It's not about the new mails, it's about the mails I already have in the inbox and various other places.

Speaking of which, I'll see to all that.

Since this's a very much critical requirements, I'll try the new boost.

----------

## dE_logics

Upgrading boost didn't work.

I think the project is dead; it never compiles and doesn't have a homepage.

----------

## xming

btrfs?

----------

## khayyam

 *dE_logics wrote:*   

> A long time ago, I compiled fusecompress, at that time it was at version 2.

 

dE_logics ... looking at the useflags for sys-fs/fusecompress::foo-overlay it uses bzip2, lzma, lzo, and zlib for compression, so it may be the file is simply compressed with one or other of these compression types and that fusecompress then fuse mounts it ... you might run 'file' and see it this is the case, as you may then be able to access (via bzip2, xz, etc) the data without the need of fusecompress.

 *dE_logics wrote:*   

> Thunderbird stores mails in a single large text file which's the main content to be compressed; with aufs (+squashfs), if one of those files changes by a bit, the whole file is copied to the r/w location by aufs, making it effectively consume more space with every run of Thunderbird.

 

Sounds like it uses 'mbox' and that the mailboxes are compressed on the fly ... not sure why more space would be consumed by the process, other than the fact that the data is copied, it should however compact the data into whatever compression method is used once Thunderbird quits. That said, given the nature of the application it may be that deleted mail is not actually deleted but stored in a Trash mailbox that needs to be explicitly purged.

 *dE_logics wrote:*   

> Are there any options remaining?

 

Not having ever used Thunderbird I can't say, but given that mail is a standard format (mbox, or maildir, most likely) an archive should be maintainable whatever application is used. My archived mail is stored in gzip'ed mbox, with current mail in maildir, archived mail can be accessed, and appended too from within mutt with the following hooks:

```
# gzip

open-hook \\.gz$ "gzip -cd %f > %t"

close-hook \\.gz$ "gzip -c %t > %f"

append-hook \\.gz$ "gzip -c %t >> %f"

# bzip2:

# no appending possible with bzip2

open-hook \\.bz2$ "bzip2 -cd %f > %t"

close-hook \\.bz2$ "bzip2 -c %t > %f"
```

I've been maintaining this setup for ages, and given the compression format is standard, easily managed, and accessable outside of a mail client, its fairly future proof.

best ... khay

----------

## dE_logics

 *khayyam wrote:*   

>  *dE_logics wrote:*   A long time ago, I compiled fusecompress, at that time it was at version 2. 
> 
> dE_logics ... looking at the useflags for sys-fs/fusecompress::foo-overlay it uses bzip2, lzma, lzo, and zlib for compression, so it may be the file is simply compressed with one or other of these compression types and that fusecompress then fuse mounts it ... you might run 'file' and see it this is the case, as you may then be able to access (via bzip2, xz, etc) the data without the need of fusecompress.
> 
>  *dE_logics wrote:*   Thunderbird stores mails in a single large text file which's the main content to be compressed; with aufs (+squashfs), if one of those files changes by a bit, the whole file is copied to the r/w location by aufs, making it effectively consume more space with every run of Thunderbird. 
> ...

 

They don't appear to be compressed, it's plane text which I can read using a text editor and highly compressible, and yet it's mbox, but mbox is not a compressed format.

 *khayyam wrote:*   

> 
> 
>  *dE_logics wrote:*   Are there any options remaining? 
> 
> Not having ever used Thunderbird I can't say, but given that mail is a standard format (mbox, or maildir, most likely) an archive should be maintainable whatever application is used. My archived mail is stored in gzip'ed mbox, with current mail in maildir, archived mail can be accessed, and appended too from within mutt with the following hooks:
> ...

 

But thunderbird needs plane/abstract rw access on the FS.

----------

## khayyam

 *dE_logics wrote:*   

> They don't appear to be compressed, it's plane text which I can read using a text editor and highly compressible, and yet it's mbox, but mbox is not a compressed format.

 

dE_logics ... I had read your initial message as suggesting you were using fusecompress for mail storeage, and that as you were now unable to compile it, were unable to access the data. That's somewhat the result of your presentation ... but I think what your actually asking is how you might setup a compressed fs, or whats on offer in that regard. Anyhow, squash_dir (sys-fs/squash_dir::mv) is one possible example. Perhaps, as has been suggested, using btrfs on a loop/file might also be a possible solution, but I'm not sure how much you'll save in terms of diskspace space.  

 *dE_logics wrote:*   

> But thunderbird needs plane/abstract rw access on the FS.

 

Yes, but it wasn't altogether clear from your inital post what you we're doing or trying to achieve. I posted the mutt hooks meerly as an example of how deal with it.

best ... khay

----------

## dE_logics

What I'm trying to compress here is the ~/.thunderbird; that's exactly not a 'mail directory' as you may be thinking; it's the place where thunderbird also stores settings and all, and among that are mails in mbox format.

Also squash_dir uses some kind of union FS which has issues with my setup (as stated in the main question). BUT the RDEPEND in the squash_dir ebuild has provided hints to aufs alternatives which I'll try. Thanks for that!

----------

## dE_logics

None of those work as per my expectations.

In the mean time I delete those mbox files if it takes too much space.

----------

