# tar playing up re space used [resolved]

## idella4

I had a /mnt/suse in /dev/sda1, a 50 gig partition.  It had about 34 gig of space used on it.  I made a tar file of /mnt/suse, /mnt/gentoo/suse.img.tar.bz2.

It took hours and I noted that once finished the tar.bz2 file was about 14 gig in size.  Later after reformatting and re-using /dev/sda1 a couple of times, I went to explode suseimg.tar.bz2 to /mnt/suse on /dev/sda1.  To my amazement, the file read 

```

idella@genny /mnt/gentoo $ ls -ld /mnt/gentoo/suse.img.tar.bz2

-rw-r--r-- 1 root root 140 Nov  5 12:09 /mnt/gentoo/suse.img.tar.bz2

```

140 bytes of data.  I considered the system lost. Now, in btrfs,

```

idella@genny /mnt/gentoo $ ls -l /mnt/gentoo

total 4

drwxrwsr-x 1 root portage 139494 Nov  8 06:18 distfiles

dr-xr-xr-x 1 root root        10 Jan  1  1970 ext2_saved

drwx------ 1 root root         0 Nov  5 05:48 lost+found

drwxr-xr-x 1 root root        28 Nov  5 10:21 packages

-rw-r--r-- 1 root root       140 Nov  5 12:09 suse.img.tar.bz2

```

distfiles is 7 gig, 

ext2_saved is a file made by btrfs from converting /mnt/gentoo from ext4 to btrfs and is required to convert it back so I'm not shifting or deleting it.

It reports 44.9 gig in size by dolphin.  This is kind of spurious since it is btrfs, which is experimental and has flaws everywhere.  The size can't really be 44,9 gig since the partition is about 46 gig and the other content  added to 44.9 way surpasses 46 gig.  It's a folder that records data to be used for converting btrfs back to ext4.

lost+found is empty, then there is suse.img.tar.bz2 at 140 bytes.

I have had /mnt/gentoo in ext4.  While in ext4, I made a tar of /mnt/gentoo to /mnt/suse/mnt-gentoo-Nov8.tar.bz2.

In its current btrfs state, I made a tar of /mnt/gentoo to /mnt/suse/gentoo-Nov-btrfs.tar.bz2.

In addition, I made an image of /mnt/gentoo using  a partition imaging program partimage.

```

idella@genny /mnt/gentoo $ ls -l /mnt/suse

total 36740540

-rw-r--r-- 1 root   root  15057665299 Nov  8 21:16 gentoo-Nov-btrfs.tar.bz2

drwx------ 2 root   root        16384 Nov  5 11:56 lost+found

-rw-r--r-- 1 root   root  15058034260 Nov  8 14:54 mnt-gentoo-Nov8.tar.bz2

-rw------- 1 root   root   2135905534 Nov  8 18:37 mnt-gentoo.img.000

-rw------- 1 root   root   2135966483 Nov  8 18:40 mnt-gentoo.img.001

-rw------- 1 root   root   2135942270 Nov  8 18:43 mnt-gentoo.img.002

-rw------- 1 root   root   1098741306 Nov  8 18:45 mnt-gentoo.img.003

drwxr-xr-x 2 idella users        4096 May  6  2010 openSUSE-11.2-DVD-x86_64-iso

```

/mnt/suse is formatted in ext4.

```

idella@genny /mnt/gentoo $ sudo umount /mnt/suse

Password: 

idella@genny /mnt/gentoo $ sudo fsck.ext4 /dev/sda1

e2fsck 1.41.12 (17-May-2010)

suse-11: clean, 21/3278576 files, 10572921/13109032 blocks

```

Putting btrfs formatted state aside, /mnt/gentoo in ext4 contains

```

idella@genny /mnt/gentoo $ ls -l /mnt/gentoo

total 4

drwxrwsr-x 1 root portage 139494 Nov  8 06:18 distfiles

drwx------ 1 root root         0 Nov  5 05:48 lost+found

drwxr-xr-x 1 root root        28 Nov  5 10:21 packages

-rw-r--r-- 1 root root       140 Nov  5 12:09 suse.img.tar.bz2

```

distfiles comes to 7 gig.  lost+found is empty.  packages comes to 64 kb.  suse.img.tar.bz2 comes to 140 bytes.

Total of 7 gig rounding it off.

Now 

```

idella@genny /mnt/gentoo $ ls -l /mnt/suse

total 36740540

-rw-r--r-- 1 root   root  15057665299 Nov  8 21:16 gentoo-Nov-btrfs.tar.bz2

drwx------ 2 root   root        16384 Nov  5 11:56 lost+found

-rw-r--r-- 1 root   root  15058034260 Nov  8 14:54 mnt-gentoo-Nov8.tar.bz2

-rw------- 1 root   root   2135905534 Nov  8 18:37 mnt-gentoo.img.000

-rw------- 1 root   root   2135966483 Nov  8 18:40 mnt-gentoo.img.001

-rw------- 1 root   root   2135942270 Nov  8 18:43 mnt-gentoo.img.002

-rw------- 1 root   root   1098741306 Nov  8 18:45 mnt-gentoo.img.003

drwxr-xr-x 2 idella users        4096 May  6  2010 openSUSE-11.2-DVD-x86_64-iso

```

-rw-r--r-- 1 root   root  15058034260 Nov  8 14:54 mnt-gentoo-Nov8.tar.bz2 represents the tar file of /mnt/gentoo in ext4.  Its size is 15.05 gig.

-rw------- 1 root   root   2135905534 Nov  8 18:37 mnt-gentoo.img.000 - 3 represent the image file of /mnt/gentoo in ext4.  Total size is around 7.4 gig.

Both are compressed files containing the content of /mnt/gentoo.

-rw-r--r-- 1 root   root  15057665299 Nov  8 21:16 gentoo-Nov-btrfs.tar.bz2 represents the tar file of /mnt/gentoo in btrfs. Its size is 15.05 gig.

 of mnt-gentoo-Nov8.tar.bz2 & gentoo-Nov-btrfs.tar.bz2, storing a partition of used space of 7 G, is 2-3 gig.

The expected total size of mnt-gentoo.img.000 - 3 is probably 1-2 gig.

This begs the question, why do I have three separate compressed partition storing files of that size from a partition of content of 7 gig?

The file suse.img.tar.bz2 is the likely source.  Its size should be around 14 gig.  If you add that 14 gig to 7 gig, then the resultant file sizes in /mnt/suse begin to add up.

So, is  the file suse.img.tar.bz2 being mis-reported in the line

```

-rw-r--r-- 1 root root       140 Nov  5 12:09 suse.img.tar.bz2

```

How on earth do you go about examining it and rectifying it?

----------

## pianosaurus

I have no idea what causes your problem, but if the size is being misrepresented, you could always just check the contents instead. Just run 

```
cat suse.img.tar.bz2 | wc -c
```

If this file is really just 140 bytes, it should finish in a split microsecond. If is is several gigabytes, it will take some time, but that in itself is an indication the file size is being incorrectly reported.

----------

## 1clue

You did all this on the same partition?

I suspect you may have run out of room on the file system.

Look at df -h and see if any of the partitions you used are or could have been full at the time.  That goes for creating the archive and extracting it.  Also keep in mind that the system keeps a certain amount of space on each partition which is reserved for root.

----------

## idella4

1clue, 

all the copying?  I described in the description.  I copied the content of /mnt/gentoo [around 10 gig] to the root of /mnt/suse, then freshly formatted it, initially in ext4.  It is 48 gig.  I then created suse.img.tar.bz2 

I then acquired / created /distfiles.  I then

 *Quote:*   

> 
> 
> I suspect you may have run out of room on the file system.
> 
> 

 

Definitely no way. It was empty when I made it.  No need, I noted their content using df and I cited those content measurements.

```

idella@genny /mnt/gentoo $ cat suse.img.tar.bz2 | wc -c

140

```

```

idella@genny /mnt/gentoo $ df -h /mnt/gentoo

Filesystem            Size  Used Avail Use% Mounted on

/dev/sda10             45G  7.9G   25G  25% /mnt/gentoo

idella@genny /mnt/suse $ sudo umount /mnt/gentoo

idella@genny ~ $ sudo btrfs-convert -r /dev/sda10

rollback complete.

idella@genny ~ $ df -h /dev/sda10

Filesystem            Size  Used Avail Use% Mounted on

udev                   10M  340K  9.7M   4% /dev

idella@genny ~ $ sudo mount -t ext4 /dev/sda10 /mnt/gentoo

idella@genny ~ $ df -h /mnt/gentoo

Filesystem            Size  Used Avail Use% Mounted on

/dev/sda10             45G  7.2G   35G  18% /mnt/gentoo

It's all over the place. In btrfs it had the extra folder /ext2_saved/

Converting it back to ext4, /dev/sda10 is 4%.  This rids it of the folder.
```

Mounting /mnt/gentoo, it's then 18%, so the modern ext4 is offering conflicting measurements.

This looks like a job for hexedit.  Does anyone know how to use it to look for the file markers and content?

----------

## 1clue

I'm afraid I can't help then.

A few years back I ran into the file size limits (at that time) of tar and gzip.  At that point there wasn't much you could do about it, so I started using a removable drive which was several times the size of my system's storage.  Makes a great backup, you just make an uncompressed copy of everything into a folder.

Everything I used to know about tar dribbled out of my ear shortly after that.

All I can think is that you ran into some sort of maximum size somewhere in the chain and that maybe you wrapped around the index and wrote an EOF somewhere near the front.  Actually it would be an EOF inside the header record.  I've never used ext4, so I don't know if there's a maximum file size there.  Tar no longer has any practical limit in my understanding, but I don't know about bzip2.  In every case I can think of, there should be some sort of error message rather than a quiet failure.

Good luck.

----------

## idella4

 *1clue wrote:*   

> 
> 
> maybe you wrapped around the index and wrote an EOF somewhere near the front. Actually it would be an EOF inside the header record. 
> 
> 

 

yes....

----------

## pianosaurus

bzip2 just operates on a stream, and has no limits on file size.

If you are going to search raw partition data for the original file, you could use a tool such as foremost (it's in portage). I don't know if it supports bz2-files or not. If there is any way you can tell it what to look for, here is the bzip2 format:

```
.magic:16                       = 'BZ' signature/magic number

.version:8                      = 'h' for Bzip2 ('H'uffman coding), '0' for Bzip1 (deprecated)

.hundred_k_blocksize:8          = '1'..'9' block-size 100 kB-900 kB

.compressed_magic:48            = 0x314159265359 (BCD (pi))

.crc:32                         = checksum for this block

.randomised:1                   = 0=>normal, 1=>randomised (deprecated)

.origPtr:24                     = starting pointer into BWT for after untransform

.huffman_used_map:16            = bitmap, of ranges of 16 bytes, present/not present

.huffman_used_bitmaps:0..256    = bitmap, of symbols used, present/not present (multiples of 16)

.huffman_groups:3               = 2..6 number of different Huffman tables in use

.selectors_used:15              = number of times that the Huffman tables are swapped (each 50 bytes)

*.selector_list:1..6            = zero-terminated bit runs (0..62) of MTF'ed Huffman table (*selectors_used)

.start_huffman_length:5         = 0..20 starting bit length for Huffman deltas

*.delta_bit_length:1..40        = 0=>next symbol; 1=>alter length

                                                { 1=>decrement length;  0=>increment length } (*(symbols+2)*groups)

.contents:2..∞                  = Huffman encoded data stream until end of block

.eos_magic:48                   = 0x177245385090 (BCD sqrt(pi))

.crc:32                         = checksum for whole stream

.padding:0..7                   = align to whole byte
```

Everything is bit aligned. So basically "BZh" followed by a digit followed by 0x314159265359 followed by everything until you find 0x177245385090, the next four bytes pluss a few padding bits to make it a full byte at the end.

----------

## idella4

pianosaurus,

[quote="pianosaurus"] *Quote:*   

> 
> 
> you could use a tool such as foremost (it's in portage)
> 
> 

 

Sure, I've emerged hexedit which does exactly what is required. and I try to minimise emerging packages  so as to not blow the system out over time.

Here is a hexedit view of the file suse.img.tar.bz2

```

00000000   42 5A 68 39  31 41 59 26  53 59 D9 06  1A AC 00 00  BZh91AY&SY......

00000010   67 FB 84 C8  80 00 40 40  09 EF 80 08  0 165 05 9E   g.....@@.....e..

00000020   00 00 00 88  08 20 00 92  84 A8 4F 29  A0 68 00 0D   ..... ....O).h..

00000030   A4 0A A1 53  D4 32 19 00  D0 D3 4D 33  5C C1 5A 90  ...S.2....M3\.Z.

00000040   4D 90 88 74  72 DB BC C5  7A CD C7 14  A2 0A 44 A9  M..tr...z.....D.

00000050   9A 97 EB AF  BE AB A0 C2  43 CE 57 A2  D2 B5 4D 26  ........C.W...M&

00000060   3C 36 4E 59  51 4C 5A AA  B9 EA B1 41  20 91 80 62   <6NYQLZ....A ..b

00000070   7D D7 34 D8  26 C0 CC 5C  4C 48 8F 0A  39 F4 14 88  }.4.&..\LH..9...

00000080   3F 8B B9 22  9C 28 48 6C  83 0D 56 00                       ?..".(Hl..V.

```

This is it, all of it.  This tells according to this description of a bzip2 structure that it is totally wrong;

BX i followed by a 'h' not a digit.

0x314159265359 followed by everything until you find 0x177245385090, the next four bytes pluss a few padding bits to make it a full byte at the end

is clearly totally absent.  

However;

```

idella@genny /mnt/suse $ ls -ld /mnt/suse/gentoo-Nov-btrfs.tar.bz2

-rw-r--r-- 1 root root 15057665299 Nov  8 21:16 /mnt/suse/gentoo-Nov-btrfs.tar.bz2

idella@genny /mnt/suse $ sudo hexedit

........................................

00000000   42 5A 68 39  31 41 59 26  53 59 E2 44  B7 F0 01 33  BZh91AY&SY.D...3

00000010   BE 7F FF FF  FF FF FF FF  FF FF FF FF  FF FF FF FF  ................

00000020   FF FF FF FF  FF FF FF FF  FF FF FF FF  FF FF FF FF  ................

00000030   FF FF FF E8  BF BF 01 D6  A1 F5 89 D7  D7 AD E6 EB  ................

00000040   5A EE BD BB  DE FB B6 67  CF BD ED EF  74 FB 9B D8  Z......g....t...

00000050   1E BA 5F 6F  6C 75 7B EE  B5 F7 D9 F3  E7 6D F5 7B  .._olu{......m.{

00000060   3A 76 E6 BD  F7 DE 94 AF  5F 67 BD 78  F4 7B B7 BB  :v......_g.x.{..

00000070   C7 6A 8D 4F  77 1D 16 B5  B3 AB DD DE  F6 7B D8 57  .j.Ow........{.W

00000080   A6 F6 D5 7A  6D EE 48 F4  EB DD B7 63  BE D6 F7 39  ...zm.H....c...9

00000090   37 CD F4 5F  57 79 ED 57  B6 FA E8 DE  C6 BC AA DF  7.._Wy.W........

000000A0   7B 9B EF 83  A0 DA B0 33  EE EB ED F4  51 BB 7D AE  {......3....Q.}.

000000B0   EB B5 37 D8  75 D1 AE 2F  7B AE 3D BB  DE 7B B3 AE  ..7.u../{.=..{..

000000C0   BD 3A 19 B0  D7 BE CF A7  DF 77 AF 14  EF 56 AB 61  .:.......w...V.a

000000D0   D2 D8 1E BB  B3 8A 9D F6  0D F7 BB AF  AF A1 D7 77  ...............w

000000E0   5D CD F7 DE  1C F7 CA F6  DF 7D E8 ED  BA F4 53 6C  ]........}....Sl

000000F0   FB 7A F2 F3  E3 CE EA 76  A9 A4 A3 D0  D1 F7 37 77  .z.....v......7w

00000100   B9 D7 47 BE  EE FA 7A F9  3A B7 DC B7  B9 5A 7A 5B  ..G...z.:....Zz[

00000110   1A 1A 15 4D  6D 99 0A 5B  DC F5 7B 3D  B6 D8 0F 5A  ...Mm..[..{=...Z

```

The above is the view of the firsr few lines of /mnt/suse/gentoo-Nov-btrfs.tar.bz2, a 15 gig file.

On doing a search for the asci string 0x314159265359 yields

```

                                          not found

                                       (press any key)

```

On doing a search for the asci string 0x314159265359 yields the same.  

So I have no idea what to make of your well intentioned advice on looking for the documented content structure of a bzip file.

Any  suggestions to follow?

----------

## pianosaurus

 *idella4 wrote:*   

> BX i followed by a 'h' not a digit.

 

Actually, I said BZh followed by a digit, so it looks just fine. The reason you are not finding the end is that it may not be byte aligned. Hang on, let me calculate some possible values for the end. I'll post again in a few minutes.

Edit: Also, when you are searching for pi, you must be doing something wrong. It's right there at the top, right after BZh9. Look at the hex portion, not the ascii portion.

----------

## pianosaurus

Possible end values (on the HEX side):

```
17 72 45 38 50 90

2E E4 8A 70 A0 ??

5C C8 14 E0 40 ??

B8 90 28 C0 80 ??

70 20 50 80 00 ??

E0 40 A0 00 00 ??

C0 80 40 00 00 ??

80 00 80 00 00 ??
```

I don't know what the padding is, so I can't calculate the final byte. Just make sure to copy it, whatever it is.

Edit 1: Oops, forgot the checksum. Meaning, if you find any of the above, you need to include the ??-byte and the next 4 bytes.

Edit 2: Then again, the file you are looking at now is a bz2-file. Is it corrupted? If not, looking for bz2-markers isn't going to do much good. I thought you were going to try to find the missing data on the raw partition instead.

----------

## idella4

 *pianosaurus wrote:*   

> 
> 
> I said BZh followed by a digit, so it looks just fine. 
> 
> The reason you are not finding the end is that it may not be byte aligned. 

 

ok, good.

 *pianosaurus wrote:*   

> 
> 
> when you are searching for pi, you must be doing something wrong
> 
> 

 

pi,???  don't follow.  It take it pi somehow must refer to the long 0x string  So the first 4 entires match.

Is a single byte one single hex entry?

Edit 1:  

 *pianosaurus wrote:*   

> looking for bz2-markers isn't going to do much good.
> 
> I thought you were going to try to find the missing data on the raw partition instead.
> 
> 

 

1. bz2 markers are a start.  /mnt/gentoo/suse.img.tar.bz2 is worth examining to check its structure.

My first effort missed because I missed the h in the first entry, but I still don't follow the long 0x string.

I shall now check on the entry you calculated in hex for the end marker in the ?? truncated sue.img.tar.bz2

2. Yes, can you help there?  Here's a start, a not too promising start.

On hexedit searching in    /mnt/suse/gentoo-Nov-btrfs.tar.bz2  I searched for the ascii string image.

Remember  /mnt/suse/gentoo-Nov-btrfs.tar.bz2 is 15 gig, so I am postulating my suse system is in there.

The suse system had at least two image entries;

/var/lib/xen/images/

/image  

 which I created in suse's root.

I got not found.  So not good.  I've never really used a hex editor before because you need to know what you're doing looking at raw data.

Here's an example of I don't know what I'm doing

```

idella@genny /mnt/gentoo $ sudo hexedit ./suse.img.tar.bz2

                                   Hexa string to search: 17 72 45 38 50 90

.................

                                     Invalid hexa string

                                       (press any key)

```

Now I tried a hex search using two hex byte content and it did it.  I enter the full line from the calculated hex entry and I get  Invalid hexa string.

Who knows!!

----------

## pianosaurus

 *idella4 wrote:*   

> pi,???  don't follow.  It take it pi somehow must refer to the long 0x string  So the first 4 entires match.
> 
> Is a single byte one single hex entry?

 

The 0x notation simply means the following digits are hex digits (meaning you won't find it doing an ascii search). Your hex editor lists the ascii version on the right, and the hex in the middle. So when you are looking for 0x31415926, it means you should look for 31 41 59 26 in the middle portion (which happens to be the same as 1AY& in ascii). You will find both at the top of your files, right after BZh9.

I called it pi because it is the digits from the constant pi: 3.14...

 *idella4 wrote:*   

> 1. bz2 markers are a start.  /mnt/gentoo/suse.img.tar.bz2 is worth examining to check its structure.
> 
> My first effort missed because I missed the h in the first entry, but I still don't follow the long 0x string.
> 
> I shall now check on the entry you calculated in hex for the end marker in the ?? truncated sue.img.tar.bz2

 

Well, it looks like suse.img.tar.bz2 is the first 140 bytes of a valid bz2 archive, but you won't find anything else useful there.

 *idella4 wrote:*   

> 2. Yes, can you help there?  Here's a start, a not too promising start.
> 
> On hexedit searching in    /mnt/suse/gentoo-Nov-btrfs.tar.bz2  I searched for the ascii string image.
> 
> Remember  /mnt/suse/gentoo-Nov-btrfs.tar.bz2 is 15 gig, so I am postulating my suse system is in there.

 

I was a bit confused about that first post. Let me first see if I understood this correctly.

A) gentoo-Nov-btrfs.tar.bz2 is 15 gig.

B) Extracting it yields 7 gig.

C) You think the missing data from suse.img.tar.bz2 could be inside gentoo-Nov-btrfs.tar.bz2, since it is way to big for it's content.

Did I get that right?

----------

## idella4

 *pianosaurus wrote:*   

> 
> 
> I was a bit confused about that first post. Let me first see if I understood this correctly.
> 
> A) gentoo-Nov-btrfs.tar.bz2 is 15 gig.
> ...

 

Yes, close enough to get by.  gentoo-Nov-btrfs.tar.bz2 is 15 gig.

 Extracting it yields 7 gig.  If I were to extract it, it should yield the content of /mnt/gentoo.

/mnt/gentoo is freshly reformatted and has only been issued with 

```

idella@genny ~/bin $ ls -ls /mnt/gentoo

total 4

0 drwxrwsr-x 1 root portage 139494 Nov  8 06:18 distfiles

4 dr-xr-xr-x 1 root root        10 Jan  1  1970 ext2_saved

0 drwx------ 1 root root         0 Nov  5 05:48 lost+found

0 drwxr-xr-x 1 root root        28 Nov  5 10:21 packages

0 -r--r--r-- 1 root root       140 Nov  5 12:09 suse.img.tar.bz2

```

ok, the ext2_saved folder is there because I have currently reconverted the /mnt/gentoo to btfs, because I'm tinkering with btrfs.

Considering all I have described, my tinkering currently consists only of 

```

btrfs convert /dev/sda10                                     converting it to btrfs

                                                          &

btrfs convert -r /dev/sda10                                  converting it back to ext4.

```

So that folder is present in order to btrfs -r back to ext4, so for this post, ignore it.  On converting it back, the folder is deleted from the /mnt/gentoo root.

So in order to make a valid attempt of the retrieval of the suse system from /mnt/gentoo, I am no longer writing to it until this is sorted.

That said, I may have mucked it up anyway because I made the suse image first thing after re-formatting /mnt/gentoo, and secondly wrote 7 gig of data to distfiles, but that's done so  I can only continue & try to examine this suse image.  I discovered the file size discrepancy in the course of making

```

.idella@genny ~/bin $ ls -l /mnt/suse

total 36740540

-rw-r--r-- 1 root   root  15057665299 Nov 10 03:19 gentoo-Nov-btrfs.tar.bz2                   

drwx------ 2 root   root        16384 Nov  5 11:56 lost+found

-rw-r--r-- 1 root   root  15058034260 Nov  8 14:54 mnt-gentoo-Nov8.tar.bz2

-rw------- 1 root   root   2135905534 Nov  8 18:37 mnt-gentoo.img.000

-rw------- 1 root   root   2135966483 Nov  8 18:40 mnt-gentoo.img.001

-rw------- 1 root   root   2135942270 Nov  8 18:43 mnt-gentoo.img.002

-rw------- 1 root   root   1098741306 Nov  8 18:45 mnt-gentoo.img.003

```

As I said in the initial post [re-read], all these three forms of saving the current CONTENT OF /MNT/GENTOO  report file sizes that do not reflect the reported 7 gig of data that /mnt/gentoo is reported to contain by adding the content of its folders. 

They reflect a file size of /mnt/gentoo IF /mnt/gentoo/suse.img.tar.bz2 were to be around 12 - 14 gig in size. 

12 - 14 gig in size is what it would be if it contained the compressed suse system.

So as NeddySeagoon puts it, you go fishing in the raw data with hexedit.  There are not one, not two, not three. but four sources of data that may contain the compressed suse system in the tarball; /mnt/gentoo itself, & the files of /mnt/suse's saved forms of /mnt/gentoo.

A /mnt/gentoo of data of size 7 gig ought result in a tarball of maybe 3 gig.  Three saves of /mnt/gentoo contradict that size.

Clear?

----------

## pianosaurus

Got it!

If the extra data is in there, you first need to find out if it is the tar archive or the bzip2 file which has the extra data. You can do that by first running bunzip2 on the file. You may want to do this on a copy, as this will replace your .tar.bz2-file with a bare .tar-file. The .tar-file has no compression, so this file will be the full size of the volume. If this file is the truncated 7 gig, you know that all that extra data is not in the tar file, and that it is lost when you bunzip2 it.

Find out which one it is first, and we'll see what we can come up with.

----------

## idella4

 *pianosaurus wrote:*   

> 
> 
> You may want to do this on a copy, as this will replace your .tar.bz2-file with a bare .tar-file. The .tar-file has no compression,   and that it is lost when you bunzip2 it.
> 
> Find out which one it is first, and we'll see what we can come up with.
> ...

 

ok, do you mean copy the suse.img.tar.bz2 from the current /mnt/gentoo. to another partition?  I don't want to miss maybe the one chance of working that file.  I suppose if it were ruined, I have 3 copies of it in the files of /mnt/suse.

Actually, I don't know which file you are suggesting; /mnt/gentoo/suse.img.tar.bz2 or one of the  tarred files of /mnt/suse/.

Copying one if those means copying a 15 gig file and I shall have to go  organize 15 gig of free space, which will just mean some re-shuffling and some more time.

----------

## pianosaurus

No, I mean copy the big one (gentoo-Nov-btrfs.tar.bz2) to some place where you can safely work. Extract the copy using bunzip2 to get a tar file, and see if the tar file is big enough to contain suse.img.tar.bz2, or if it is just 7 gig.

----------

## idella4

ok, I was editing my post as you replied.  Actually, my /mnt/gentoo does have the available 15 gig, but I suppose I should keep that quarantined.

ok, a /mnt/karmic64 is a white elephant.  That will do.  I shall blow it away, reformat to btrfs just to carry on tinkering, and if it blows up that's fine since it's a copy of the source file of /mnt/suse and can just repeat in ext4.

EDIT; A non impacting change of plan, just making an image of my karmic64, then reformat, then copy it over, then bunzip the copy.

The other btrfs version and the mnt/suse image file(s) are backups that can be used for the same purpose.

Given the circumstances, I shall use dd to make the copy.  When using tar, it was very slooooow.

Need about 20 minutes.

Using bunzip is a change. I made them with tar, and used tar -x -f to untar it which output 140 bytes of suse.

----------

## pianosaurus

Yes, it's unusual to use bzip2 directly. It might be handy to know that tar -jxvf file.tar.bz2 is equivalent to bunzip2 file.tar.bz2 && tar -xvf file.tar, except that the latter leaves an uncompressed tar file behind instead of the original archive.

The same goes for compression. You can make .tar.bz2 archives using tar -cf file.tar * && bzip2 file.tar.

In other words, tar puts all files into one file (but with no compression). Bzip compresses single files (it cannot put multiple files into one, unlike regular zip). It's a typical Un*x way of doing things. Since bundling and compressing are two different operations, they should be performed by two different utilities.

----------

## idella4

pianosaurus,

thanks I'm getting there.  My /mnt/karmic64 was only 20 gig, thought it was bigger.  I shall finally delete my gen2.i,g of 48 gig in /mnt/fedora, which has been the source of another post.  It's a broken btrfs.  

At this stage, attempting to unzip the file started, but filled up my formatted /mnt/karmic64. 

Just have to send the file to /mnt/fedora/store, and re-post.  The fact that it filled the 20gig partition says it was on its way to unzippimg a very large amount.  

Stay tuned.

```

genny karmic64 # bunzip2 ./mnt-gentoo-Nov8.tar.bz2 /mnt/fedora/store/

idella@genny /mnt/suse $ df /mnt/fedora/store

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda9             55754616   3421236  49501124   7% /mnt/fedora

```

in motion.  I'd say that 7 % is going to jump to about 40-50% in one step once the mnt-genoo-Nov8.tar is finalised

hmmm, have to shift the whole thing.  bunzip is insisting on making the output folder in the same directory.

More time.   /mnt/fedora is about 56 gig.

```

genny karmic64 # mv ./mnt-gentoo-Nov8.tar.bz2 /mnt/fedora/store

genny karmic64 # cd /mnt/fedora/store/

.......... bunzip2 ./mnt-gentoo-Nov8.tar.bz2 

```

eventually.

EDIT: It is taking an age, just like tar took hours, all night, to make the thing in the first place, it's going to take all night to unzip it.

```

idella@genny /mnt/suse $ df /mnt/fedora/store

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda9             55754616   3421236  49501124   7% /mnt/fedora

idella@genny /mnt/suse $ df /mnt/fedora      

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda9             55754616   8389688  44532672  16% /mnt/fedora

idella@genny /mnt/suse $ df /mnt/fedora

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda9             55754616  25289640  27632720  48% /mnt/fedora

idella@genny /mnt/suse $ df /mnt/fedora

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda9             55754616  33611628  19310732  64% /mnt/fedora

```

pianosaurus I suspect I shall catch you tomorrow.

----------

## pianosaurus

Yeah, (un)zipping is the slow part. Tar is a lot faster, and you can extract particular files from it, you don't need to pull out everything. I have to get some sleep now, but I'll check back tomorrow. Let me know what you find out.

----------

## idella4

Hey this looks promising, very promising

```

idella@genny /mnt/fedora/store $ ls -l  /mnt/fedora/store/

total 46533040

-rw------- 1 root root 32591790080 Nov 10 08:11 mnt-gentoo-Nov8.tar

-rw-r--r-- 1 root root 15058034260 Nov 10 06:31 mnt-gentoo-Nov8.tar.bz2        

```

Right on target, it's on its way to making an output of around 34 gig.

It bombed out soon after making this entry.  Re-doing it using tar directly, then it won't go and delete the output.

This ensures it will take all night.

pianosaurus,

well, I have it figured.  The folder /ext2_saved, the one I told you to ignore turned out to be the source.

At the time, for some reason when I saved /mnt/gentoo in ext4, the folder /ext_saved was still there. It contains an image file which is like a snapshot of the data in the root, and is used to convert it back to ext4.  The image in the folder was huge.  It happened to be around a similar size to the should have been suse.img.tar.bz2.  The file suse.img.tar.bz2 came out at the same 140 bytes.  So the chase was in vain, but that happens.

Shall delete the invalid /mnt/suse/mnt-gentoo-Nov8.tar.bz2 and remake it for what it's worth in its valid form.

thanks anyway.

----------

## pianosaurus

Oh well... At least you figured it out. I've lost a drive or two to accidents myself.

----------

## idella4

pianosaurus,

hey, having done all that, I've made another blew, lost a partition table.  Got them back but not fully.  I placed in portage & programming.

It's a similar ask to the contents of this, so if you find this, please consider sorting out the other post.

https://forums.gentoo.org/viewtopic-t-852087.html

----------

