# Problems with TMPFS space not getting freed

## Eversor

Hi all,

I've been using TMPFS as a way to have most caches sitting on a RAM based filesystem to avoid wearing out ssds (mostly /tmp, /var/tmp/portage and xdg-caches) but I've seen to bumped into a bug where space on TMPFS isn't freed after the files have been deleted (even after accounting for a supposed 15min delay I read about somewhere).

So, let's say I have my /tmp at 4GiB and then copy a 1GiB file into there, then delete it after it is done copying. I wait a while and do a "df -h" and I get the same used space, the 1GiB, as before deleting the file. I wait for days and the space is never freed. This obviously makes it a rather big issue to using TMPFS for any kind of filesystem temp dir.

lsof for open files in /tmp shows nothing that is taking up the space and while rebooting the system, during shutdown, the init scripts also claim that /tmp is being used but fuser claims no files in /tmp - this is another bug, /tmp is always in use, even in console mode where it isn't using any files there, which I can see with lsof.

Am I the only one witnessing this? Any help is welcomed.

Kernel is gentoo-sources-3.6.11, don't know what else is relevant.

----------

## blain3

whats your fstab look like? i was just looking into TRIM and found a few things, i see a lot of this: 

 tmpfs defaults,noatime,mode=1777, as im using a ssd as well im going to edit in discard and nodiratime to reduce writes(for root and var partitions of mine on SSD), the discard is trim. i have my /var/tmp set at 12G with 16G of ram, as one of the mods posted and have yet to have a problem.

Edit: also are you using only a SSD and no other drive? and are you using a swap partition? might help those smarter then me to help u to know this

----------

## _______0

for portage, the proper way of using RAM is to add these two in make.conf:

```
PORTAGE_TMPFS="/dev/shm"

PORTAGE_TMPDIR="/dev/shm"

```

then you'll see /dev/shm/portage/ being used. This way is automatic no need to manually mount a directory and no need to fiddle with mount options.

----------

## Eversor

 *_______0 wrote:*   

> for portage, the proper way of using RAM is to add these two in make.conf:
> 
> ```
> PORTAGE_TMPFS="/dev/shm"
> 
> ...

 

I was using

PORTAGE_TMPDIR="/tmp"

DISTDIR="/tmp/distfiles"

PORTAGE_TMPFS doesn't seem to be documented anywhere. What does it do?

I also have interest in using TMPFS /tmp for xdg-cache, as chrome is one of the biggest culprits of disk IO activity. xsession-errors is another one, and I think I will be adding some init scripts to move /var/log to tmpfs at boot and shutdown to restore them between shutdowns.

----------

## Eversor

 *blain3 wrote:*   

> whats your fstab look like? i was just looking into TRIM and found a few things, i see a lot of this: 
> 
>  tmpfs defaults,noatime,mode=1777, as im using a ssd as well im going to edit in discard and nodiratime to reduce writes(for root and var partitions of mine on SSD), the discard is trim. i have my /var/tmp set at 12G with 16G of ram, as one of the mods posted and have yet to have a problem.
> 
> Edit: also are you using only a SSD and no other drive? and are you using a swap partition? might help those smarter then me to help u to know this

 

These are my relevant sections of fstab:

none                    /tmp                    tmpfs           size=7G                         0 0

# glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for

# POSIX shared memory (shm_open, shm_unlink).

# (tmpfs is a dynamically expandable/shrinkable ramdisk, and will

#  use almost no memory if not populated with files)

shm                     /dev/shm                tmpfs           nodev,nosuid,noexec             0 0

I tried _______0's suggestion and files created by portage are correctly freed. Guess it's either a mount option or I should not be using "none" but "shm". Thoughts?

EDIT:

About TRIM, I've added that to the FSTAB entry, completely forgot about that.

----------

## Eversor

Forget that it works... Somehow it is working on the console but once I get X server running and it starts writing files to xdg-cache dir, writing a big file and deleting it will not free up the space :/

Tried everything, can't get this to work ok.

----------

## eccerr0r

Did you try kernel 3.5.7?  Might be a kernel regression too.

I have three SSD machines that I run portage in tmpfs and have not run into this issue... "can't reproduce" on 3.5.7 yet...

Also can you unmount the tmpfs and remount it?  Though lsof should tell you if you can or not...

(just to cover all bases, it's not a swapfile on tmpfs right?)

----------

## Eversor

 *eccerr0r wrote:*   

> Did you try kernel 3.5.7?  Might be a kernel regression too.
> 
> I have three SSD machines that I run portage in tmpfs and have not run into this issue... "can't reproduce" on 3.5.7 yet...
> 
> Also can you unmount the tmpfs and remount it?  Though lsof should tell you if you can or not...
> ...

 

No swap of any kind ATM. And if this was working ok, I would rarely need it. Mostly only use it when compiling stuff like open office or turning some big VMs on. Then I turn off swap afterwards.

Can I ask you to dd from /dev/zero 2GB into a file in your tmpfs, while you're compiling there? The trigger for me seems to be extra open files/I-O activity.

I was on 3.2 before, upgraded to 3.6 and just see the same behavior.

----------

## eccerr0r

My tmpfs is only 2GB large on this machine I'm using now, so if I made a 2G file the emerge will fail... A 1G file seems to have no effect: I tried making a 1GB file and deleting it repeatedly and it hasn't failed to free the space yet...

This is a 4G RAM machine...

----------

## Hu

 *_______0 wrote:*   

> for portage, the proper way of using RAM is to add these two in make.conf:
> 
> ```
> PORTAGE_TMPFS="/dev/shm"
> 
> ...

 That is a way of using RAM, but I would not call it the proper way.  Many users will mount /dev/shm as noexec.  Portage temporary files must be written to a filesystem that permits execution.

----------

## Eversor

 *eccerr0r wrote:*   

> My tmpfs is only 2GB large on this machine I'm using now, so if I made a 2G file the emerge will fail... A 1G file seems to have no effect: I tried making a 1GB file and deleting it repeatedly and it hasn't failed to free the space yet...
> 
> This is a 4G RAM machine...

 

That was what I was saying, I can't reproduce until I let xorg load and two things happen:

- /tmp/xsession-errors is open and written to

- /tmp/xdg-cache is written with various files from xdg compatible programs, like chromium

After this, any files being written, including caches from chromium in xdg-cache, are not getting freed :/

----------

## eccerr0r

 *Eversor wrote:*   

> That was what I was saying, I can't reproduce until I let xorg load and two things happen:
> 
> - /tmp/xsession-errors is open and written to
> 
> - /tmp/xdg-cache is written with various files from xdg compatible programs, like chromium
> ...

 

Wait, what are you saying now?  This seems contradictory, or are these not the files causing issues?  If you have X11 open and it's writing log files and keeping the handles open, deleting them will not free the space by design, until the writing program exits.  But generally the files are small - are they growing to GB-sized files due to some other issue?  If this is the case, this is the underlying problem.  Perhaps Chromium is writing GB sized files there but these should clear up once you quit Chromium.

So what exactly are you doing?  Does the space free up once you quit X11?  As said, I cannot reproduce the problem by creating a file and deleting it over and over again - if this had been able to cause the problem, then this is a real kernel bug.  But it's not... please give a more detailed method of how to reproduce the issue.

BTW, I don't do xsession-errors or xdg-cache move on any of my SSD-only machines (32G, 128G, and an 180G).  Only portage tmp is moved to tmpfs when possible, I do nothing else special.  So far I have negligible wear on my SSDs, the major endurance killers currently are just multi-GB software installs.

Though I can't see why xsession-errors or xdg-cache moves would cause problems, could you restore them to the default locations and is the problem reproduceable in the default setup?

----------

## blain3

 *Quote:*   

> These are my relevant sections of fstab:
> 
> none                    /tmp                    tmpfs           size=7G                         0 0
> 
> # glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for
> ...

 

Correct me if im wrong but should anything running off your ram be this:

tmpfs                   /tmp                     tmpfs            noatime,size=7G        0 0

notice that its not labeled as "none", duno if thats going to make any difference, also i believe it defaults at 8G without the "size" variable? i could be wrong. Also how much ram do you have exactly? And where is your /var/tmp mounted? because /var/tmp is a much longer lasting tmp system then the others from what i understand. Personally even with 16G of 1600 DDR3 i have all of my tmp systems using ram, I also keep a 32G swap partition on my second HD, and i can emerge on several terminals at once, and my system rarely uses the swap from what i can see with system monitor. I have also read, i/o scheduler's can bottleneck resources if not managed correctly. One last thing to look at possibly? is this: http://www.redips.net/linux/tmpwatch-delete-files/

Hope this helps, alas im not very good with all this yet so please, if im wrong by all means, correct me. cheers

----------

## eccerr0r

Tmpfs defaults to half physical memory size if none is specified.  I actually specified 1.5GB on my 2GB netbook to fit more of portage temp in RAM without overflowing the tmpfs (1GB isn't quite enough sometimes).

The labelling shouldn't matter, it's just for decorative purposes/making it clear what the temp filesystem is...

As for the OP - no NFSD running?  I had some some strange behavior due to nfsd.

----------

## blain3

 *eccerr0r wrote:*   

> Tmpfs defaults to half physical memory size if none is specified.  I actually specified 1.5GB on my 2GB netbook to fit more of portage temp in RAM without overflowing the tmpfs (1GB isn't quite enough sometimes).
> 
> The labelling shouldn't matter, it's just for decorative purposes/making it clear what the temp filesystem is...
> 
> As for the OP - no NFSD running?  I had some some strange behavior due to nfsd.

 

Yes i realized that soon after i posted and forgot, 50% of mine is 8G so yeah my bad on that. Only reason i brought up the labeling is because if i edit fstab in vim it shows up a different color if i use none, and for some reason even shm is colored over in red, so i truly wasn't sure on that, just thinking out loud.. because if i use tmpfs instead of none its the default color and not white. As for clearing ram / tmp files yscou could easily script that into a single command i would assume, name it, chmod a+x, mv to /bin and run as needed or even run on a timer if all else fails. Im no expert but i'v made a few bash scripts with Geany and used them, iirc something like cron daemon would run it on a timer or some such? worth looking into. ( and no im not sure if its cron or something else ). good luck, hope you figure it out.

----------

