# Using tmpfs for /var/{log,lock,...}

## cwng

This is original tips/tricks post.  As you can see, the code get very simplified.  :Smile:   You don't even need any undo script, because now /etc/init.d/vartmp stop undo itself.

----------

Hello all,

The thread in Using tmpfs for /tmp prompted me to experiment with using tmpfs for some /var directories that has frequent disk read/write, such as /var/log.  I imagine doing so will have certain advantages:

- Speed: it speeds up the logging process

- I/O: it reduces disk I/O

The following are the scripts I have conjured, to see if anyone brave enough to try them.  I have been using them for a while, no problems so far.

/etc/init.d/vartmp - This is the startup script:

```

#!/sbin/runscript

#

# This init script mounts various /var sub-directories into RAM-based filesystem

# (i.e. tmpfs).  There are two main advantages in doing so:

#   - speed: its speeds up logging processes

#   - disk-sync: it reduces the need for constant disk syncs

#

# Pre-requisite: you need to have the following entry in your /etc/fstab

#   none    ${VARTMP_MOUNT}    tmpfs    defaults,size=256m    0 0

#

# and make sure the ${VARTMP_MOUNT} directory is created and mounted before executing

# this script

depend() {

   need localmount

   before bootmisc

}

start() {

   ebegin "Populating /mnt/vartmp "

   for d in ${VARTMP_DIRS}; do

      unpack_vartmp ${d}

   done

   eend $?

}

stop() {

   ebegin "Storing /mnt/vartmp "

   for d in ${VARTMP_DIRS}; do

      pack_vartmp ${d}

   done

   eend $?

}

# This function load previously compressed /var diretories into the tmpfs

unpack_vartmp() {

   ebegin "Unpacking /var/${1} to /mnt/vartmp/${1} "

   #

   # Check if previously stored compression of/var diretory exists

   #

   if [ ! -e /var/.vartmp.${1}.tgz ]; then

      #

      # Creates the archives for the very first time

      #

      if [ -d /var/${1} ]; then

         ewarn "Creating packed version of /var/${1}"

         cd /var && tar -czf /var/.vartmp.${1}.tgz ${1}

      else

         ewarn "No original /var/${1}"

      fi

   fi

   #

   # Check existence of ${VARTMP_MOUNT}

   #

   if [ -z ${VARTMP_MOUNT} ]; then

      mkdir /dev/shm/vartmp

      VARTMP_MOUNT=/dev/shm/vartmp

   fi

   if [ ! -d ${VARTMP_MOUNT} ]; then

      mkdir /dev/shm/vartmp

      mount none -t tmpfs -o defaults,size=256m ${VARTMP_MOUNT}

   fi

   #

   # Actual populating of previous /var contents

   #

   if [ -e /var/.vartmp.${1}.tgz ]; then

      cd ${VARTMP_MOUNT} && tar -xzf /var/.vartmp.${1}.tgz

   else

      if [ ! -d ${VARTMP_MOUNT}/${1} ]; then

         rm -f ${VARTMP_MOUNT}/${1}

         mkdir ${VARTMP_MOUNT}/${1}

         chmod 775 ${VARTMP_MOUNT}/${1}

      fi

   fi

   #

   # link /var to ${VARTMP_MOUNT}

   #

   if [ -e /var/${1} ]; then

      rm -fR /var/${1}

   fi

   ln -sf ${VARTMP_MOUNT}/${1} /var/${1}

   eend $?

}

# This function store current /var directories into a compressed archive for later use

pack_vartmp() {

   ebegin "Packing ${VARTMP_MOUNT}/${1} "

   if [ -d ${VARTMP_MOUNT}/${1} ]; then

      cd ${VARTMP_MOUNT} && tar -czf /var/.vartmp.${1}.tgz ${1}

   fi

   eend $?

}

```

/etc/conf.d/vartmp - This is the configuration file:

```

# /etc/conf.d/vartmp

#

# VARTMP_MOUNT defines where the tmpfs base directory for /var is

VARTMP_MOUNT="/mnt/vartmp"

# VARTMP_DIRS defines which of the /var sub-directories resides on the tmpfs

VARTMP_DIRS="log lock run"

```

Steps - Here is what I did:

First, edit /etc/fstab to contain the line:

```
none      ${VARTMP_MOUNT}      tmpfs      defaults,size=256m      0 0

```

The size I chose is 256M, its up to you to vary.  You can use a higher size since tmpfs has the nice feature of not actually using any memory until it is populated.

Next, mount the tmpfs

```
mkdir /mnt/vartmp

mount /mnt/vartmp

```

Then, edit /etc/conf.d/vartmp to your liking (particularly the VARTMP_DIRS="" variable that specify which of the subdirectories to move to tmpfs.  You definitely don't want to move all of them,  unless you have obscene amount of RAM, like 32G  :Smile: ) and run the init script:

```
rc-update add vartmp boot

/etc/init.d/vartmp start

```

For me, I will see this:

```
bach cwng # rc-update add vartmp boot

 * vartmp added to runlevel boot

 * Caching service dependencies...

 * rc-update complete.

bach cwng # /etc/init.d/vartmp start

 * Populating /mnt/vartmp ...

 * Unpacking /var/log to /mnt/vartmp/log ...

 * Creating packed version of /var/log                                           [ok ]

 * Unpacking /var/lock to /mnt/vartmp/lock ...

 * Creating packed version of /var/lock                                         [ ok ]

 * Unpacking /var/run to /mnt/vartmp/run ...

 * Creating packed version of /var/run                                          [ ok ]

bach cwng # ls -l /var

total 0

drwxr-xr-x   6 root root  74 Dec 21 00:21 cache

drwxr-xr-x   3 root root  16 Oct 28 14:36 db

drwxr-xr-x   2 root root  18 Jan  4 11:10 empty

drwxrwx--T   2 root gdm   18 Jan 10 22:45 gdm

drwxr-xr-x  13 root root 150 Dec 29 19:40 lib

lrwxrwxrwx   1 root root  16 Jan 25 12:12 lock -> /mnt/vartmp/lock

lrwxrwxrwx   1 root root  15 Jan 25 12:12 log -> /mnt/vartmp/log

lrwxrwxrwx   1 root root  15 Dec 20 19:06 mail -> /var/spool/mail

lrwxrwxrwx   1 root root  15 Jan 25 12:12 run -> /mnt/vartmp/run

drwxr-xr-x   8 root root  92 Jan 14 16:25 spool

drwxr-xr-x   2 root root  18 Oct 28 14:23 state

drwxrwxrwt   5 root root  93 Jan 24 15:05 tmp

```

Well, all seems fine.  If for any reason you wish to revert back to the original setting, I have also prepared the following script vartmp-undo.sh to undo the work:

```
#!/bin/bash

#

# This script undo the changes caused by the switch to using tmpfs for directories in /var

if [ ! -e /etc/conf.d/vartmp ]; then

   echo "* No /etc/conf.d/vartmp!"

   exit 1

fi

source /etc/conf.d/vartmp

if [ -z ${VARTMP_MOUNT} ]; then

   VARTMP_MOUNT=/dev/shm/vartmp

fi

if [ ! -d ${VARTMP_MOUNT} ]; then

   echo "* VARTMP_MOUNT=${VARTMP_MOUNT} does not exist!"

   exit 1

fi

for d in ${VARTMP_DIRS}; do

   if [ -d ${VARTMP_MOUNT}/${d} ]; then

      if [ -e /var/${d} ]; then

         rm -f /var/${d}

      fi

      mv ${VARTMP_MOUNT}/${d} /var

   fi

done

if [ -x /etc/init.d/vartmp ]; then

   /etc/init.d/vartmp zap

fi

rm -fR /var/.vartmp.*.tgz

```

Just remember to do a 

```
rc-update del vartmp
```

after running the undo-script.

EDIT:  Split off the original material/replies.  --pjp

----------

## vladgrigorescu

This was really helpful, thanks!

----------

## Config

Just as a side-note - putting /var/log on a ramdisk is not a good idea. If something crashes, your logs won't be available on next reboot.

----------

## adsmith

 *Config wrote:*   

> Just as a side-note - putting /var/log on a ramdisk is not a good idea. If something crashes, your logs won't be available on next reboot.

 

This is VERY true, but what I do is separate logs by message level:

warn and worse go to /var/log/messages (or similar), which is on HD.

but info or debug and worse and go /tmp/log/debug, which is tmpfs.  This way, I can sift through all the extra info if I want, but nothing important is lost on hard reboot.

I also broadcast logs between machines on the lan, so each machine is watching the other machines' (info and worse) logs on /tmp/log/remote (tmpfs).  Hence, if one machine goes down (but not all of them), I can quickly see what happened by just going to another workstation.

redundancy is good.

----------

## Cazzantonio

Hi. I didn't know about this tip and I created a my own initscript to to do the same things.

This script generates a tar of /var/log, /var/run and /var/lock at shutdown and populate them at reboot. You have to add it to boot runlevel.

It also does some rotating of logs so you don't have to use logrotate. You can set a maximum file size for the log tar archive and the script will create a backup (and will delete the old logs) when the size is reached.

You have to have lines like these in fstab to mount those directories in tmpfs

```
varlog          /var/log/               tmpfs   mode=1755,size=100m            0 0

varrun          /var/run/               tmpfs   mode=1755,size=10m             0 0

varlock         /var/lock/              tmpfs   mode=1775,size=10m             0 0
```

This is the initscript:

```
#!/sbin/runscript

depend() {

        after localmount

        before bootmisc hostname clock logger

}

#Directory for /var/log, /var/run and /var/lock backups

DATA_DIR_BACKUP=/var

#Masx size of /var/log before rotation occurs

MAX_LOG_SIZE=50

backup() {

    cd /var

    tar cvpf $DATA_DIR_BACKUP/log_bk.tar log/ &>/dev/null

    LOGSIZE=$(du -m $DATA_DIR_BACKUP/log_bk.tar|cut -f1)

    echo "Log size = $LOGSIZE M, Rotating at $MAX_LOG_SIZE"

    if [ $LOGSIZE -gt $MAX_LOG_SIZE ]; then

        echo "Log size limit reached. Rotating logs"

        /bin/rm `find /var/log -type f|grep -v "\.keep"`

        /bin/mv $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar_old -f

        tar cvzpf $DATA_DIR_BACKUP/log_bk.tar log/ &>/dev/null

    fi

    tar cvpf $DATA_DIR_BACKUP/run_bk.tar run/ &>/dev/null

    tar cvpf $DATA_DIR_BACKUP/lock_bk.tar lock/ &>/dev/null

}

restore() {

    tar xvpf $DATA_DIR_BACKUP/log_bk.tar -C /var/ &>/dev/null

    touch /var/log/{btmp,lastlog,wtmp}

    chgrp utmp /var/log/{btmp,lastlog,wtmp}

    chmod 664 /var/log/{btmp,lastlog,wtmp}

    tar xvpf $DATA_DIR_BACKUP/run_bk.tar -C /var/ &>/dev/null

    touch /var/run/utmp                             

    chgrp utmp /var/run/utmp                 

    chmod 664 /var/run/utmp 

    tar xvpf $DATA_DIR_BACKUP/lock_bk.tar -C /var/ &>/dev/null

    chmod 775 /var/lock

    chown root:uucp /var/lock

}

start() {

        ebegin "Restoring /var/log, /var/run and /var/lock"

        restore

        eend $? 

}

stop() {

        ebegin "Backup structure of /var/log, /var/run and /var/lock"

        backup

        eend $?

}
```

----------

## mv

 *Cazzantonio wrote:*   

> Hi. I didn't know about this tip and I created a my own initscript to to do the same things.

 

Is there a particular reason for these chown and chrgrp commands? Aren't they automatically executed by tar?

Moreover, why is it necessary to touch utmp? (Does something go crazy if utmp has "old" timestamp?)

----------

## Cazzantonio

 *mv wrote:*   

>  *Cazzantonio wrote:*   Hi. I didn't know about this tip and I created a my own initscript to to do the same things. 
> 
> Is there a particular reason for these chown and chrgrp commands? Aren't they automatically executed by tar?
> 
> Moreover, why is it necessary to touch utmp? (Does something go crazy if utmp has "old" timestamp?)

 Just to make sure those files are there... They must exists for linux to behave correctly. The rest of the tar archive could become corrupted and all other files get lost, but at least you want to boot and get a working console.

----------

## Cazzantonio

Hi all. I stumbled on this old script of mine and I gave it an update. Now it's divided into an init script and a log daemon. The name of the files is important, so if you want to change it you have to change also something in the two scripts.

It does a backup of the directory structure of /var/log, /var/run, /var/lock and /var/spool, restoring necessary files and subdirectories each reboot and providing a rotation of logs in case the tar file becomes too big (it preserves emerge.log).

It is compatible with baselayout-2 and openrc, at least up to version 0.7.0. It should behave also with parallel boot, but don't take it for granted...

Basically you have to put this into fstab:

```
varlog      /var/log/       tmpfs   mode=1755,size=100m            0 0

varrun      /var/run/      tmpfs   mode=1755,size=1m            0 0

varlock      /var/lock/      tmpfs   mode=1775,size=1m            0 0

varspool          /var/spool   tmpfs   mode=1750,size=2m            0 0
```

put this initscript in the boot runlevel:

/etc/init.d/manage_log :

```
#!/sbin/runscript

depend() {

        after localmount root

        before bootmisc logger cron

}

#Directory for backups of /var/log, /var/run and /var/lock

DATA_DIR_BACKUP=/var

#Maximum size of logs in the tar archive before rotating

MAX_LOG_SIZE=50

#If you want the script to act like a daemon and rotate logs before the system restart, then set this variable to 1

DEMONIZED=0

#If DEMONIZED=1, then select the time interval between checks (s,m,h,d are the suffixes for seconds, minutes, hours, days)

CHECK_TIME=2h

#The daemon is the script located in /usr/local/sbin/log_daemon.sh

backup() {

    cd /var

    tar cvpf $DATA_DIR_BACKUP/log_bk.tar log/ &>/dev/null

    chmod 600 $DATA_DIR_BACKUP/log_bk.tar

    LOGSIZE=$(du -m $DATA_DIR_BACKUP/log_bk.tar|cut -f1)

    if [ $LOGSIZE -gt $MAX_LOG_SIZE ]; then

        cd /var

        echo "Log size limit reached. Rotating logs"

        /bin/mv $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar_old -f

        tar cvzpf $DATA_DIR_BACKUP/log_bk.tar log/ &>/dev/null

        chmod 600 $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar

        /bin/mv $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar_old -f

        /bin/rm `find /var/log -type f|grep -v "\.keep"|grep -v "emerge.log"`

        touch /var/run/utmp; chgrp utmp /var/run/utmp; chmod 664 /var/run/utmp

        tar cvzpf $DATA_DIR_BACKUP/log_bk.tar log/ &>/dev/null

        chmod 600 $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar

    else

        echo "Log size = $LOGSIZE M, Rotating at $MAX_LOG_SIZE"

    fi

    /bin/rm `find /var/run -type f -iname *.pid` -rf

    echo "" > /var/run/utmp

    tar cvpf $DATA_DIR_BACKUP/run_bk.tar run/ &>/dev/null

    chmod 600 $DATA_DIR_BACKUP/run_bk.tar

    tar cvpf $DATA_DIR_BACKUP/lock_bk.tar lock/ &>/dev/null

    chmod 600 $DATA_DIR_BACKUP/lock_bk.tar

    tar cvpf $DATA_DIR_BACKUP/spool_bk.tar spool/ &>/dev/null

    chmod 600 $DATA_DIR_BACKUP/spool_bk.tar

}

restore() {

    tar xvpf $DATA_DIR_BACKUP/log_bk.tar -C /var/ &>/dev/null

    tar xvpf $DATA_DIR_BACKUP/run_bk.tar -C /var/ &>/dev/null

    test -e /var/run/utmp || touch /var/run/utmp && chgrp utmp /var/run/utmp && chmod 664 /var/run/utmp

    tar xvpf $DATA_DIR_BACKUP/lock_bk.tar -C /var/ &>/dev/null

    chmod 775 /var/lock && chown root:wheel /var/lock

    tar xvpf $DATA_DIR_BACKUP/spool_bk.tar -C /var/ &>/dev/null

    chmod 755 /var/spool

    if [ $DEMONIZED -eq 1 ]; then

        /usr/local/sbin/log_daemon.sh &

    fi

}

start() {

        ebegin "Recover structure of /var/log, /var/run, /var/lock and /var/spool"

        restore

        eend $?  

}

stop() {

        ebegin "Backup structure of /var/log, /var/run, /var/lock and /var/spool"

        backup

        eend $?

}
```

and this is the daemon in case you want to daemonize the rotation of the logs:

/usr/local/sbin/log_daemon.sh :

```
#!/bin/sh

DEMONIZED=`cat /etc/init.d/manage_log|grep -v "#" |grep DEMONIZED|cut -d"=" -f2|head -n1`

DATA_DIR_BACKUP=`cat /etc/init.d/manage_log|grep -v "#" |grep DATA_DIR_BACKUP|cut -d"=" -f2|head -n1`

MAX_LOG_SIZE=`cat /etc/init.d/manage_log|grep -v "#" |grep MAX_LOG_SIZE|cut -d"=" -f2|head -n1`

CHECK_TIME=`cat /etc/init.d/manage_log|grep -v "#" |grep CHECK_TIME|cut -d"=" -f2|head -n1`

while [ $DEMONIZED -eq 1 ]; do

    LOGSIZE=$(du -ms /var/log|cut -f1)

    if [ $LOGSIZE -gt $MAX_LOG_SIZE ]; then

        cd /var

        echo "Log size limit reached. Rotating logs"

        /etc/init.d/syslog-ng stop

        /bin/mv $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar_old -f

        tar cvzpf $DATA_DIR_BACKUP/log_bk.tar log/ &>/dev/null

        chmod 600 $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar

        /bin/mv $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar_old -f

        /bin/rm `find /var/log -type f|grep -v "\.keep"|grep -v "emerge.log"`

        touch /var/run/utmp; chgrp utmp /var/run/utmp; chmod 664 /var/run/utmp

        tar cvzpf $DATA_DIR_BACKUP/log_bk.tar log/ &>/dev/null

        chmod 600 $DATA_DIR_BACKUP/log_bk.tar $DATA_DIR_BACKUP/log_bk.tar

        /etc/init.d/syslog-ng start

    fi

    sleep $CHECK_TIME

done
```

----------

## slick

I found some apps they create its own directory in /var/log or /var/run (during install?) . If mount /var/log or /var/run with tmpfs this apps will not start because missing directories. Can anybody reproduce this? This will avoid to mount this directories empty with tmpfs in /etc/fstab, without backup and restore the structure?

----------

## tclover

Here will follow my flavour of the above scripts, the only extra script is a little 'pack' script which is called by crontab to pack in a regular basis in order to minimize data loss.

 *Quote:*   

> # /etc/conf.d/vartmp
> 
> VTMP_MNT="/var/tmp/vartmp"
> 
> VTMP_DIR="mail"
> ...

 

 *Quote:*   

> #!/sbin/runscript
> 
> # Copyright 1999-2011 Gentoo Foundation
> 
> # Distributed under the terms of the GNU General Public License v2
> ...

 

 *Quote:*   

> #!/bin/sh
> 
> #~/vtmp-pack.sh
> 
> . /etc/conf.d/vartmp
> ...

 

You have to add something like this: */20 * * * *    root    $HOME/vtmp-pack.sh log

in crontab for this to work.

----------

## d-fens

hi,

i used cwng's script to put samba files into tmpfs, so the hdds do spin down on my nas, i had do change a bit in order to allow also nested directories to work, here is my diff:

```
47c47,49

<          cd /var && tar -czf /var/.vartmp.${1}.tgz ${1}

---

>          path=$1

>        escaped=${path/\//\\}

>        cd /var && tar -czf /var/.vartmp.${escaped}.tgz ${1}

66,67c68,69

<    if [ -e /var/.vartmp.${1}.tgz ]; then

<       cd ${VARTMP_MOUNT} && tar -xzf /var/.vartmp.${1}.tgz

---

>    if [ -e /var/.vartmp.${escaped}.tgz ]; then

>       cd ${VARTMP_MOUNT} && tar -xzf /var/.vartmp.${escaped}.tgz

89c91,93

<       cd ${VARTMP_MOUNT} && tar -czf /var/.vartmp.${1}.tgz ${1}

---

>       path=$1

>       escaped=${path/\//\\}

>       cd ${VARTMP_MOUNT} && tar -czf /var/.vartmp.${escaped}.tgz ${1}

```

so now you can use

```

VARTMP_DIRS="log/samba cache/samba lib/samba"

```

in /etc/conf.d/vartmp

----------

## seneca

quick question.  With the changes in udev >=200, do these tmpfs references have to be changed to devtmpfs?

----------

## khayyam

 *seneca wrote:*   

> quick question.  With the changes in udev >=200, do these tmpfs references have to be changed to devtmpfs?

 

seneca ... no, devtempfs is a temporary filesystem for device nodes, so /dev, tmpfs is a more general temporary filesystem, the former hasn't replaced the latter.

```
# grep TMPFS /usr/src/linux/.config

CONFIG_DEVTMPFS=y

CONFIG_DEVTMPFS_MOUNT=y

CONFIG_TMPFS=y

CONFIG_TMPFS_POSIX_ACL=y

CONFIG_TMPFS_XATTR=y
```

best ... khay

----------

## seneca

Thank you, Khay

Oh, new to this... is it ok to thank people for their help, as now, or does this clutter the forums?  will refrain in the future based on your advice

----------

## khayyam

 *seneca wrote:*   

> Thank you, Khay

 

seneca ... you're welcome :)

 *seneca wrote:*   

> Oh, new to this... is it ok to thank people for their help, as now, or does this clutter the forums?  will refrain in the future based on your advice

 

I generally think its good form, if it solves the issue then its some clue that the provided information worked, though tagging the thread [SOLVED] is also fine. I most often refrain from replying with a "you're welcome" unless there is something further to add :)

best ... khay

----------

## ppurka

 *seneca wrote:*   

> Oh, new to this... is it ok to thank people for their help, as now, or does this clutter the forums?  will refrain in the future based on your advice

 Even better. Quote the correct reply and say "thank you"  :Smile:  Then some future wanderer will stumble upon the thread and know exactly which advice worked.

----------

## roscop

Hi, 

Does anyone else had to modify initscript to handle run and lock changes or have I missed something in my last install ? 

https://bugs.gentoo.org/show_bug.cgi?id=342049

https://bugs.gentoo.org/show_bug.cgi?id=332633

Regards

----------

