# strange things with md5sum

## SarahS93

in my system i have a disk only for downloads (sdc), in the past i download a lot of stuff to this drive and extract it (with unrar) to my data hdd (sdg).

i extract from sdc with unrar to my backup drive sdh too, and then i run md5sum on both files and become not the same checksum, how can that be possible?

with smartctl i check sdc, sdg and sdh

sdg and sdh are new drives, only two weeks old, all smartctl infos looks good.

but my download/temp drive sdc looks very ungly

```
smartctl --all /dev/sdc

smartctl 6.4 2015-06-04 r4109 [x86_64-linux-4.12.12-gentoo] (local build)

Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===

Model Family:     Seagate NAS HDD

Device Model:     ST4000VN000-1H4168

Serial Number:    XXXXXXXX

LU WWN Device Id: 5 000c50 067120e15

Firmware Version: SC44

User Capacity:    4.000.787.030.016 bytes [4,00 TB]

Sector Sizes:     512 bytes logical, 4096 bytes physical

Rotation Rate:    5900 rpm

Form Factor:      3.5 inches

Device is:        In smartctl database [for details use: -P show]

ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b

SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)

Local Time is:    Sun Dec 10 02:15:54 2017 CET

SMART support is: Available - device has SMART capability.

SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===

SMART overall-health self-assessment test result: PASSED

General SMART Values:

Offline data collection status:  (0x00) Offline data collection activity

                                        was never started.

                                        Auto Offline Data Collection: Disabled.

Self-test execution status:      (   0) The previous self-test routine completed

                                        without error or no self-test has ever

                                        been run.

Total time to complete Offline

data collection:                (  117) seconds.

Offline data collection

capabilities:                    (0x73) SMART execute Offline immediate.

                                        Auto Offline data collection on/off support.

                                        Suspend Offline collection upon new

                                        command.

                                        No Offline surface scan supported.

                                        Self-test supported.

                                        Conveyance Self-test supported.

                                        Selective Self-test supported.

SMART capabilities:            (0x0003) Saves SMART data before entering

                                        power-saving mode.

                                        Supports SMART auto save timer.

Error logging capability:        (0x01) Error logging supported.

                                        General Purpose Logging supported.

Short self-test routine 

recommended polling time:        (   1) minutes.

Extended self-test routine

recommended polling time:        ( 521) minutes.

Conveyance self-test routine

recommended polling time:        (   2) minutes.

SCT capabilities:              (0x10bd) SCT Status supported.

                                        SCT Error Recovery Control supported.

                                        SCT Feature Control supported.

                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10

Vendor Specific SMART Attributes with Thresholds:

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE

  1 Raw_Read_Error_Rate     0x000f   108   099   006    Pre-fail  Always       -       20486048

  3 Spin_Up_Time            0x0003   094   091   000    Pre-fail  Always       -       0

  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       85

  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0

  7 Seek_Error_Rate         0x000f   074   060   030    Pre-fail  Always       -       27766222

  9 Power_On_Hours          0x0032   066   066   000    Old_age   Always       -       30079

 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0

 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       73

184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0

187 Reported_Uncorrect      0x0032   077   077   000    Old_age   Always       -       23

188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0

189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0

190 Airflow_Temperature_Cel 0x0022   067   051   045    Old_age   Always       -       33 (Min/Max 21/33)

191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0

192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       30

193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       85

194 Temperature_Celsius     0x0022   033   049   000    Old_age   Always       -       33 (0 18 0 0 0)

197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       8

198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       8

199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0

SMART Error Log Version: 1

ATA Error Count: 23 (device log contains only the most recent five errors)

        CR = Command Register [HEX]

        FR = Features Register [HEX]

        SC = Sector Count Register [HEX]

        SN = Sector Number Register [HEX]

        CL = Cylinder Low Register [HEX]

        CH = Cylinder High Register [HEX]

        DH = Device/Head Register [HEX]

        DC = Device Command Register [HEX]

        ER = Error register [HEX]

        ST = Status register [HEX]

Powered_Up_Time is measured from power on, and printed as

DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,

SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 23 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)

  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:

  ER ST SC SN CL CH DH

  -- -- -- -- -- -- --

  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:

  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name

  -- -- -- -- -- -- -- --  ----------------  --------------------

  60 00 08 ff ff ff 4f 00  43d+05:50:41.678  READ FPDMA QUEUED

  e5 00 00 00 00 00 00 00  43d+05:50:41.678  CHECK POWER MODE

  ef 10 02 00 00 00 a0 00  43d+05:50:41.678  SET FEATURES [Enable SATA feature]

  27 00 00 00 00 00 e0 00  43d+05:50:41.678  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]

  ec 00 00 00 00 00 a0 00  43d+05:50:41.677  IDENTIFY DEVICE

Error 22 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)

  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:

  ER ST SC SN CL CH DH

  -- -- -- -- -- -- --

  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:

  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name

  -- -- -- -- -- -- -- --  ----------------  --------------------

  60 00 08 ff ff ff 4f 00  43d+05:50:37.606  READ FPDMA QUEUED

  ef 10 02 00 00 00 a0 00  43d+05:50:37.606  SET FEATURES [Enable SATA feature]

  27 00 00 00 00 00 e0 00  43d+05:50:37.606  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]

  ec 00 00 00 00 00 a0 00  43d+05:50:37.605  IDENTIFY DEVICE

  ef 03 46 00 00 00 a0 00  43d+05:50:37.605  SET FEATURES [Set transfer mode]

Error 21 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)

  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:

  ER ST SC SN CL CH DH

  -- -- -- -- -- -- --

  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:

  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name

  -- -- -- -- -- -- -- --  ----------------  --------------------

  60 00 08 ff ff ff 4f 00  43d+05:50:33.608  READ FPDMA QUEUED

  60 00 08 ff ff ff 4f 00  43d+05:50:33.608  READ FPDMA QUEUED

  60 00 08 ff ff ff 4f 00  43d+05:50:33.607  READ FPDMA QUEUED

  60 00 08 ff ff ff 4f 00  43d+05:50:33.607  READ FPDMA QUEUED

  60 00 08 ff ff ff 4f 00  43d+05:50:33.607  READ FPDMA QUEUED

Error 20 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)

  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:

  ER ST SC SN CL CH DH

  -- -- -- -- -- -- --

  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:

  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name

  -- -- -- -- -- -- -- --  ----------------  --------------------

  60 00 00 ff ff ff 4f 00  43d+05:50:29.447  READ FPDMA QUEUED

  60 00 80 ff ff ff 4f 00  43d+05:50:29.446  READ FPDMA QUEUED

  60 00 00 ff ff ff 4f 00  43d+05:50:29.271  READ FPDMA QUEUED

  60 00 80 ff ff ff 4f 00  43d+05:50:29.270  READ FPDMA QUEUED

  60 00 00 ff ff ff 4f 00  43d+05:50:28.319  READ FPDMA QUEUED

Error 19 occurred at disk power-on lifetime: 25208 hours (1050 days + 8 hours)

  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:

  ER ST SC SN CL CH DH

  -- -- -- -- -- -- --

  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:

  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name

  -- -- -- -- -- -- -- --  ----------------  --------------------

  60 00 08 ff ff ff 4f 00  43d+04:30:46.995  READ FPDMA QUEUED

  ef 10 02 00 00 00 a0 00  43d+04:30:46.994  SET FEATURES [Enable SATA feature]

  27 00 00 00 00 00 e0 00  43d+04:30:46.994  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]

  ec 00 00 00 00 00 a0 00  43d+04:30:46.994  IDENTIFY DEVICE

  ef 03 46 00 00 00 a0 00  43d+04:30:46.994  SET FEATURES [Set transfer mode]

SMART Self-test log structure revision number 1

No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1

 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS

    1        0        0  Not_testing

    2        0        0  Not_testing

    3        0        0  Not_testing

    4        0        0  Not_testing

    5        0        0  Not_testing

Selective self-test flags (0x0):

  After scanning selected spans, do NOT read-scan remainder of disk.

If Selective self-test is pending on power-up, resume after 0 minute delay.
```

does unrar check the files for corruption?

[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]

----------

## eccerr0r

Yes, rar does have checksums embedded during decompression.  Most of the time, if there was a problem reading the file, unrar should flag this. 

Usually if there is corruption read from the hard drive, a bad sector warning will pop up during read as the hard drive also has error detection, but the OS will be notified before passing the corrupt data to your application.  It does look like your hard drive had problems reading a few sectors so you're rightly inclined to blame it, though it should have reported an i/o error to the application (unrar).

 If it didn't flag any errors during decompression possibly it was corrupted in ram buffers, make sure your RAM doesn't have problems (memtest).

----------

## SarahS93

Memtest is a good idea. I have memtest86+ installed, what lines do i add by hand to my grub.cfg file (grub2) ?

Or is there any way to test the RAM at a running system?

Or how can i disable buffering into RAM by extacting with unrar?

Or is there a way to see i there goes anything wrong in the RAM by extracting files, so i know it and can try to extract it again new.

----------

## eccerr0r

There is sys-apps/memtester so you don't need to reboot, but it's not as thorough.

You can keep on uncompressing to the same disk and you get different checksums each time, another red flag.

You should be able to run memtest86+ the same way as any other Linux kernel with grub2.  I would imagine grub-mkconfig should automatically pick up memtest86+ for you.  Mine automatically used the linux16 directive to run it, the binary was installed to [BOOT]/memtest86plus/memtest .

----------

## Akkara

 *eccerr0r wrote:*   

> There is sys-apps/memtester so you don't need to reboot, but it's not as thorough.

 

I've seen cases where sys-apps/memtester finds problems that memtest86+ misses.  Probably has something to do with the fact that it runs on the live system while other things are going on, so it can spot rare failures that a pure pattern-based tester might not "tickle".

Running sys-apps/memtester + emerge makes a good pair if you suspect hardware problems.  Ideally do it in /tmp or somewhere unimportant.  I've had a machine hardlock while running that.  It had been unnerving enough that I've since moved to ECC RAM.  Never had a problem after that.

----------

## eccerr0r

I was more concerned with the memory that's locked into position by either the kernel or another program, and also it's a bit harder to map a failure to a physical address/specific module due to MMU maps (though it may not be that easy with memtest86+ either).

Either program will at least indicate existence of a problem location if it can access it.

----------

## NeddySeagoon

SarahS93,

```
smartctl --all /dev/sdc 

...

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE 

197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       8 
```

That drive is scrap.  It has 8 used sectors that it can no longer read.

If it could read them, even once, they would be relocated.

That's only the failed sectors it knows about too.

Check your warranty status.

----------

## SarahS93

i know, the drive i very scrap.

is there a way to unrar into md5sum on the fly?

so i must not write the file first and read it than to become the md5sum

----------

## eccerr0r

unrar supports the "t" command to use its internal checksum to test the rar file:

```
$ unrar t file.rar
```

You could also use the p command

```
$ unrar p file.rar|md5sum
```

though I don't know what it would do with an archive that contains more than one file...

I rarely if ever use rar, this is just a guess...

----------

## pjp

Moved from Multimedia to Kernel & Hardware. Not Multimedia related and appears to be a hardware issue.

----------

