# Swap on a CF Card

## linga

I have an idea about how I could make my system a bit more responsive when using swap. If I make a swap partition on a fast CF-card then my main harddrive won't get trashed and reading files will stay fast. 

I have a few questions though:

* How long will a memorycard last when using it as swap?

* Should I use a PATA(IDE) <-> CF converter or a SATA <-> CF converter? I know SATA is faster but the signals have to be converted between CF and SATA. This is not the case with PATA, however PATA is much slower when it comes to transfer the data. The memory card I will use is a fast one: 20MB/sec or even 40MB/sec if i choose the more expensive alternative. Will a standard SATA <-> CF converter cope with this speed?

* Is this a bad idea?

----------

## NeddySeagoon

linga,

FLASH memory is much slower than your hard drive but you will gain on the latency due to head movements. How much, will depend on what the hard drive is doing ... it may actually be slower.

If you have to buy the FLASH memory, invest in more RAM instead, so you don't need swap space.

----------

## linga

I have never seen my hd peek more than maybe 42-45MB/sec in reading, and that must have been a pretty ideal situation.

BTW isn't access times the big problem when it comes to swap?

Also, having swap on a memorycard doesn't keep the harddrive busy while swapping.

About memory: I am actually planning on buying a second 1GB stick, but not even then will I get rid of all the swapping(think emerge eclipse-sdk + doing something at the same time) I think.

EDIT: should mention that 2GB of memory max for my mainboard.

----------

## NeddySeagoon

linga,

You will be lucky to get 8Mb/sec from FLASH.  The problem with swap is the speed compared to main memory.

To execute something in swap, first, some real RAM must be made available by writing something else to swap.

Having made some free RAM, the thing you want to run can be read in from swap and executed.

You will drop the read/write speed from 45Mb/sec to 8Mb/sec and gain the head movement latencey.

If you are doing emerges and actively using swap with 1Gb RAM, your setup is not right.

Note the actively above. Its quite normal for a few Mb, not needed until shutdown to be written to swap.

Post your emerge --info please, I'll look at your settings.

----------

## i92guboj

As NeddySeagoon said, a flash device is not the right thing for a swap device. Yes, latency should be lesser, but when you notice the effects of swap, it is not because of latency or access times. When you notice that your system lags and lock, it is because there is a need to transfer a big thing to/from swap to the main memory. When this happens, having a big access time is not a real problem, nor anything noticeable. Having to move big chuncks from the swap to the memory, involves -probably- moving an equally big chunk from the ram to swap (to free some space), and this is the problem.

It is important to have a good transfer ratio. Access times remain on a secondary position. And flash devices sucks at that.

You should look at niceness and if you are using 1gb swap, check that you don't have MAKEOPTS="-j<big number>". Also, check if this happens only on given packages. On kmail or wine, openoffice, etc etc is usual to get hight memory usages. In kde programs you can have these problems if you USE="kdeenablefinal".

----------

## linga

 *emerge --info wrote:*   

> 
> 
> Portage 2.1.3.19 (default-linux/amd64/2007.0, gcc-4.1.2, glibc-2.6.1-r0, 2.6.23-gentoo-r3-6 x86_64)
> 
> =================================================================
> ...

 

Note the LD-flags which are not standard nor recommended. I haven't so far seen any troubles with them. The same goes for -fno-ident, but it feels as if things are a little faster than without them.

Also, I have quite many USE-flags enabled since I want some kind of consistency across the system.

I tend to use quite a lot of program as well, for example monodevelop, banshee, beagle, liferea, firefox and OpenOffice Writer, all at the same time. Note that real thrashing is not that common, more like general slugginess when reading back from swap. Also, linking eclipse uses LOADS of memory. Not that I do NOT do that every day, but i have had crashes during linking eclipse.

I have also made a small test with the memorycard from DSLR camera: Canon 400D. The card is a Sandisk Extreme III which is meant to be capable of 20 MB/sec for both reading and writing. I tested copying a 3,8 GB folder with pictures á ~4MB. The results were clear: write: ca 8MB/sec (8,4 actually) read: ca 12 MB/sec. It would probably be slower on smaller files as well. So, even if there is a newer generation capable of 40MB/sec they won't do that good in real life, so maybe the raw read speed isn't enough, as you said.

But, then I can't help wondering why Windows Vista has ReadyBoost (or whatever it's called), which does the same thing but with a USB memories instead. Maybe just a PR-trick  :Confused: 

EDIT: There seems to be aproblem with line-wrapping here, inserted newlines

----------

## NeddySeagoon

linga,

That all looks sane. I would be interested to see the first six lines from top when the system is sluggish.

If you want to separate swap, its most effective if you put it on another drive, maybe with things you don't use very often.

----------

## i92guboj

 *linga wrote:*   

> 
> 
> I tend to use quite a lot of program as well, for example monodevelop, banshee, beagle, liferea, firefox and OpenOffice Writer, all at the same time. Note that real thrashing is not that common, more like general slugginess when reading back from swap. Also, linking eclipse uses LOADS of memory. Not that I do NOT do that every day, but i have had crashes during linking eclipse.
> 
> 

 

There programs are memory eaters. I don't think your problem is with emerge. Emerge is just another drop in the glass, which makes it spills out. You are using lots of heavy programs, as NeddySeagoon said, check your top (or better, htop) output when working, leave it open and look at it. You can press F6 on htop and then select RES on the left to sort the output by memory usage. That will give you a better overview on what's happening.

 *Quote:*   

> 
> 
> But, then I can't help wondering why Windows Vista has ReadyBoost (or whatever it's called), which does the same thing but with a USB memories instead. Maybe just a PR-trick 
> 
> EDIT: There seems to be aproblem with line-wrapping here, inserted newlines

 

I don't know anything about that "ReadyBoost" technology at all. But those devices can't really do any better. They are slow, such is the nature of the flash device itself. They are not great in trasfer rates because flash devices were never designed to handle massive storage. They were born just to fill a gap, where there was a need for a rom memory that could be rewriten under certain circunstances. The contents was small, and the access times should be low. That is, the tipical behaviour of a rom bios (except for the write-able part).

This is why these devices will never be that fast (unless something great happens). And this is why they are equally slow, regardless of the interconnection interface we choose (may it be usb, scsi, sata or whatever).

I studied and designed some sample memories in the university, but it has never been my field, so it's all fuzzy in my mind. But if I am not mistaken, the flash chips are NAND devices (this is due to the logical base used to implement them, just maths). And NAND devices are by nature very slow to read. NOR based chips are maybe faster to read, but they are very much slower to write or erase. This is due to a big part based on the intrinsic logic that each kind of chips uses, but also based on other factors, for example: I think that sdd controlers which are the responsibles for reading the flash stuff need a strong error checking and some other stuff that makes them slower.

----------

## linga

I think I need to upgrade that 'general slugginess' to diskthrashing. I had banshee, monodevelop, writer, firefox and liferea open at the same time as I was re-emergeing Scribus and the system quickly became unusable. At that point there was quite high load on the system, so the memory requirements were high. I would think that the only way to get rid of the problem is to get more ram.

Anyway:

 *top wrote:*   

> 
> 
> top - 20:53:42 up  7:22,  4 users,  load average: 8.28, 5.09, 2.47
> 
> Tasks: 131 total,   1 running, 130 sleeping,   0 stopped,   0 zombie
> ...

 

Ideally this kind of load wouldn't be too much of a problem, but I'll have to buy mor ram then.

----------

## linga

I looked up ReadyBoost: http://www.microsoft.com/windows/products/windowsvista/features/details/readyboost.mspx

I was basically right in my previous post about it. It also uses something called SuperFetch which preloads commonly used applications into ram which sounds a bit like swap prefetch that I read about somewhere.

----------

## NeddySeagoon

linga

```
load average: 8.28, 5.09, 2.47

Cpu(s): 8.5%us, 3.2%sy, 4.3%ni, 0.0%id, 83.1%wa, 0.3%hi, 0.7%si, 0.0%st 
```

Thats odd 8.28 (average) processes in the ready to run state and 83.1% of the CPU time spent waiting.

Still with 581740k of swap used, its probably waiting for swapping to happen. Another 1G of RAM would give you almost 500M of spare RAM at the time this snapshot was taken. However you do swap, you have the wasted swap read/write time, so the right way to address this issue to to remove or reduce the need for swap.

----------

## think4urs11

 *linga wrote:*   

> But, then I can't help wondering why Windows Vista has ReadyBoost (or whatever it's called), which does the same thing but with a USB memories instead. Maybe just a PR-trick 

 

From anything i've read so far about it thats exactly what it is, even when using the new HD's with additional 'ready boost flash' onboard. In best case there will be a measurable improvement by ~1-2%, nothing one will really notice in a daily live situation.

----------

## linga

 *Think4UrS11 wrote:*   

> 
> 
> From anything i've read so far about it thats exactly what it is, even when using the new HD's with additional 'ready boost flash' onboard. In best case there will be a measurable improvement by ~1-2%, nothing one will really notice in a daily live situation.
> 
> 

 

Well, they gotta do something to survive...

----------

## linga

Oops! I didn't see your post NeddySeagoon.

 *NeddySeagoon wrote:*   

> 
> 
> Thats odd 8.28 (average) processes in the ready to run state and 83.1% of the CPU time spent waiting.
> 
> Still with 581740k of swap used, its probably waiting for swapping to happen. Another 1G of RAM would give you almost 500M of spare RAM at the time this snapshot was taken. However you do swap, you have the wasted swap read/write time, so the right way to address this issue to to remove or reduce the need for swap.

 

I'm probably going to buy more ram, before it's too late. Last time I checked I had troubles finding the right kind.

----------

## KShots

Regardless of the speed, using flash for swap is a bad thing because flash is not designed for continuous rewrites. Unlike a standard hard drive, a flash drive has a limited number of writes to any given sector before that sector will no longer accept writes. Once that happens, your flash is toast.

I'd been looking into using flash for other projects (like a vehicle computer) and eventually decided that to make the thing run as smoothly as possible for as long as possible, I'd use flash + tmpfs + unionfs (unifying the flash and the RAM). In my setup, I did away with swap entirely because of this problem (I don't want to wear out my drive). In your case, I agree with the others - get yourself some more RAM. Otherwise, stick with swap on your hard drive.

----------

## ewaller

Kshots is absolutely correct.  In my day job, we use CF for embedded systems that are installed in some nasty environmental conditions.  They most definitely will wear out with repeated write cycles.

Since CF have a finite number of write cycles, some CF have  clever algorithms that notice when certain areas of the device are being over exercised  (say a swap area) vs other areas that rarely change (say the kernel), and when they do, they will automatically remap those portions of the drive to move the things that rarely change into the parts that are wearing out and move the things that change often into the blocks that have suffered fewer write cycles.

This is all well and good, but it tends to make the entire drive start to fail all at once.  More importantly, most CFs react VERY, badly to power failures that occur during a write cycle (whether caused by the OS or due to internal housekeeping described above).  We deliberately try to kill CF by killing the power while writing to the drive.  We have a large collection of drives that cannot be read, formated, fdisk'ed, or will even fail to identify themselves on the bus.  

By the way (and I do not have any financial interest in them) Silicon Systems makes a very good CF that is tolerant of power failures.

----------

## Monkeh

RAM is cheap. Just get more of it.

----------

