# Filesystem "sda3": XFS internal error xfs_da_do_buf

## apryan

Getting buffer errors shortly after running cron tests. I previously got these errors on an older kernel 2 months ago and assumed it was possibly a bad kernel. Not sure if this indicates drive failure or something else. Any suggestions?

Currently running 2.6.23-hardened-r2 x86_64

```
Mar 19 03:10:01 [cron] (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )

Mar 19 03:10:03 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

Mar 19 03:10:03 [kernel] Filesystem "sda3": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c.  

  Caller 0xffffffff8030b708

Mar 19 03:10:03 [kernel] Call Trace:

Mar 19 03:10:03 [kernel]  [<ffffffff8030b5bb>]                  

Mar 19 03:10:03 [kernel]  [<ffffffff8030b708>]                               

Mar 19 03:10:03 [kernel]  [<ffffffff80362733>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030b708>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e5dc>]

                - Last output repeated twice -

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]       

Mar 19 03:10:03 [kernel]  [<ffffffff8027c5fd>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 19 03:10:03 [kernel]  [<ffffffff80302e11>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030df98>]

Mar 19 03:10:03 [kernel]  [<ffffffff8033969b>]

Mar 19 03:10:03 [kernel]  [<ffffffff8033fd1a>]

Mar 19 03:10:03 [kernel]  [<ffffffff80288be5>]    

                - Last output repeated twice -

Mar 19 03:10:03 [kernel]  [<ffffffff80288a86>]

Mar 19 03:10:03 [kernel]  [<ffffffff80288d4d>]

Mar 19 03:10:03 [kernel]  [<ffffffff8020b5ee>]
```

```
Portage 2.1.4.4 (hardened/amd64/multilib, gcc-3.4.6, glibc-2.7-r1, 2.6.23-hardened-r2 x86_64)

=================================================================

System uname: 2.6.23-hardened-r2 x86_64 Dual-Core AMD Opteron(tm) Processor 2214 HE

Timestamp of tree: Sat, 01 Mar 2008 09:46:01 +0000

app-shells/bash:     3.2_p33

dev-lang/python:     2.4.4-r6, 2.5.1-r5

dev-python/pycrypto: 2.0.1-r6

sys-apps/baselayout: 1.12.11.1

sys-apps/sandbox:    1.2.18.1-r2

sys-devel/autoconf:  2.13, 2.61-r1

sys-devel/automake:  1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1

sys-devel/binutils:  2.18-r1

sys-devel/gcc-config: 1.4.0-r4

sys-devel/libtool:   1.5.26

virtual/os-headers:  2.6.24

ACCEPT_KEYWORDS="amd64 ~amd64"

CBUILD="x86_64-pc-linux-gnu"

CFLAGS="-O2 -march=k8 -fomit-frame-pointer -pipe"

CHOST="x86_64-pc-linux-gnu"

CONFIG_PROTECT="/etc /var/bind"

CONFIG_PROTECT_MASK="/etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"

CXXFLAGS="-O2 -march=k8 -fomit-frame-pointer -pipe"

DISTDIR="/usr/portage/distfiles"

FEATURES="autoconfig distlocks loadpolicy metadata-transfer sandbox sfperms strict unmerge-orphans userfetch"

GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"

LDFLAGS="-Wl,-O1 -Wl,--enable-new-dtags -Wl,--sort-common -s"

MAKEOPTS="-j12"

PKGDIR="/usr/portage/packages"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage"

SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"

USE="X acpi aim amd64 apache2 apm avi berkdb bzlib cairo caps cracklib crypt curl encode exim extensions fam flash ftp gd gif gnutls gstreamer gtk2 hardened icq imagemagick imap innodb ithreads jpeg justify lcms mbox midi ming mng mpeg msql multilib mysql nls nptl nptlonly ogg oggvorbis opengl pam pic postgres python quicktime quotas readline sqlite ssl szip tcltk tcpd theora threads tiff tokenizer truetype urandom usb vhosts xml xorg xsl xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i810 mach64 mga neomagic nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"

Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
```

thanks

----------

## apryan

```
Mar 19 03:10:03 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

Mar 19 03:10:03 [kernel] Filesystem "sda3": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c.  Caller 0xffffffff8030b708

Mar 19 03:10:03 [kernel] Call Trace:

Mar 19 03:10:03 [kernel]  [<ffffffff8030b5bb>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030b708>]

Mar 19 03:10:03 [kernel]  [<ffffffff80362733>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030b708>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e5dc>]

                - Last output repeated twice -

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 19 03:10:03 [kernel]  [<ffffffff8027c5fd>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 19 03:10:03 [kernel]  [<ffffffff80302e11>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 19 03:10:03 [kernel]  [<ffffffff8030df98>]

Mar 19 03:10:03 [kernel]  [<ffffffff8033969b>]

Mar 19 03:10:03 [kernel]  [<ffffffff8033fd1a>]

Mar 19 03:10:03 [kernel]  [<ffffffff80288be5>]

                - Last output repeated twice -

Mar 19 03:10:03 [kernel]  [<ffffffff80288a86>]

Mar 19 03:10:03 [kernel]  [<ffffffff80288d4d>]

Mar 19 03:10:03 [kernel]  [<ffffffff8020b5ee>]

Mar 14 03:10:04 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

Mar 14 03:10:04 [kernel] Filesystem "sda3": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c.  Caller 0xffffffff8030b708

Mar 14 03:10:04 [kernel] Call Trace:

Mar 14 03:10:04 [kernel]  [<ffffffff8030b5bb>]

Mar 14 03:10:04 [kernel]  [<ffffffff8030b708>]

Mar 14 03:10:04 [kernel]  [<ffffffff80362733>]

Mar 14 03:10:04 [kernel]  [<ffffffff8030e352>]

Mar 14 03:10:04 [kernel]  [<ffffffff8030b708>]

Mar 14 03:10:04 [kernel]  [<ffffffff8030e5dc>]

                - Last output repeated twice -

Mar 14 03:10:04 [kernel]  [<ffffffff8030e352>]

Mar 14 03:10:04 [kernel]  [<ffffffff8027c5fd>]

Mar 14 03:10:04 [kernel]  [<ffffffff8030e352>]

Mar 14 03:10:04 [kernel]  [<ffffffff80302e11>]

Mar 14 03:10:04 [kernel]  [<ffffffff8030e352>]

Mar 14 03:10:04 [kernel]  [<ffffffff8030df98>]

Mar 14 03:10:04 [kernel]  [<ffffffff8033969b>]

Mar 14 03:10:04 [kernel]  [<ffffffff8033fd1a>]

Mar 14 03:10:04 [kernel]  [<ffffffff80288be5>]

                - Last output repeated twice -

Mar 14 03:10:04 [kernel]  [<ffffffff80288a86>]

Mar 14 03:10:04 [kernel]  [<ffffffff80288d4d>]

Mar 14 03:10:04 [kernel]  [<ffffffff8020b5ee>]

Mar 15 03:10:08 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

Mar 15 03:10:08 [kernel] Filesystem "sda3": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c.  Caller 0xffffffff8030b708

Mar 15 03:10:08 [kernel] Call Trace:

Mar 15 03:10:08 [kernel]  [<ffffffff8030b5bb>]

Mar 15 03:10:08 [kernel]  [<ffffffff8030b708>]

Mar 15 03:10:08 [kernel]  [<ffffffff80362733>]

Mar 15 03:10:08 [kernel]  [<ffffffff8030e352>]

Mar 15 03:10:08 [kernel]  [<ffffffff8030b708>]

Mar 15 03:10:08 [kernel]  [<ffffffff8030e5dc>]

                - Last output repeated twice -

Mar 15 03:10:08 [kernel]  [<ffffffff8030e352>]

Mar 15 03:10:08 [kernel]  [<ffffffff8027c5fd>]

Mar 15 03:10:08 [kernel]  [<ffffffff8030e352>]

Mar 15 03:10:08 [kernel]  [<ffffffff80302e11>]

Mar 15 03:10:08 [kernel]  [<ffffffff8030e352>]

Mar 15 03:10:08 [kernel]  [<ffffffff8030df98>]

Mar 15 03:10:08 [kernel]  [<ffffffff8033969b>]

Mar 15 03:10:08 [kernel]  [<ffffffff8033fd1a>]

Mar 15 03:10:08 [kernel]  [<ffffffff80288be5>]

                - Last output repeated twice -

Mar 15 03:10:08 [kernel]  [<ffffffff80288a86>]

Mar 15 03:10:08 [kernel]  [<ffffffff80288d4d>]

Mar 15 03:10:08 [kernel]  [<ffffffff8020b5ee>]

Mar 16 03:10:07 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

Mar 16 03:10:07 [kernel] Filesystem "sda3": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c.  Caller 0xffffffff8030b708

Mar 16 03:10:07 [kernel] Call Trace:

Mar 16 03:10:07 [kernel]  [<ffffffff8030b5bb>]

Mar 16 03:10:07 [kernel]  [<ffffffff8030b708>]

Mar 16 03:10:07 [kernel]  [<ffffffff80362733>]

Mar 16 03:10:07 [kernel]  [<ffffffff8030e352>]

Mar 16 03:10:07 [kernel]  [<ffffffff8030b708>]

Mar 16 03:10:07 [kernel]  [<ffffffff8030e5dc>]

                - Last output repeated twice -

Mar 16 03:10:07 [kernel]  [<ffffffff8030e352>]

Mar 16 03:10:07 [kernel]  [<ffffffff8027c5fd>]

Mar 16 03:10:07 [kernel]  [<ffffffff8030e352>]

Mar 16 03:10:07 [kernel]  [<ffffffff80302e11>]

Mar 16 03:10:07 [kernel]  [<ffffffff8030e352>]

Mar 16 03:10:07 [kernel]  [<ffffffff8030df98>]

Mar 16 03:10:07 [kernel]  [<ffffffff8033969b>]

Mar 16 03:10:07 [kernel]  [<ffffffff8033fd1a>]

Mar 16 03:10:07 [kernel]  [<ffffffff80288be5>]

                - Last output repeated twice -

Mar 16 03:10:07 [kernel]  [<ffffffff80288a86>]

Mar 16 03:10:07 [kernel]  [<ffffffff80288d4d>]

Mar 16 03:10:07 [kernel]  [<ffffffff8020b5ee>]

Mar 17 03:10:03 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

Mar 17 03:10:03 [kernel] Filesystem "sda3": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c.  Caller 0xffffffff8030b708

Mar 17 03:10:03 [kernel] Call Trace:

Mar 17 03:10:03 [kernel]  [<ffffffff8030b5bb>]

Mar 17 03:10:03 [kernel]  [<ffffffff8030b708>]

Mar 17 03:10:03 [kernel]  [<ffffffff80362733>]

Mar 17 03:10:03 [kernel]  [<ffffffff8025b3fa>]

Mar 17 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 17 03:10:03 [kernel]  [<ffffffff8030b708>]

Mar 17 03:10:03 [kernel]  [<ffffffff8030e5dc>]

                - Last output repeated twice -

Mar 17 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 17 03:10:03 [kernel]  [<ffffffff8025b7df>]

Mar 17 03:10:03 [kernel]  [<ffffffff80302e11>]

Mar 17 03:10:03 [kernel]  [<ffffffff8030e352>]

Mar 17 03:10:03 [kernel]  [<ffffffff8030df98>]

Mar 17 03:10:03 [kernel]  [<ffffffff8033969b>]

Mar 17 03:10:03 [kernel]  [<ffffffff8033fd1a>]

Mar 17 03:10:03 [kernel]  [<ffffffff80288be5>]

                - Last output repeated twice -

Mar 17 03:10:03 [kernel]  [<ffffffff80288a86>]

Mar 17 03:10:03 [kernel]  [<ffffffff80288d4d>]

Mar 17 03:10:03 [kernel]  [<ffffffff8020b5ee>]

Mar 18 03:10:07 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

Mar 18 03:10:07 [kernel] Filesystem "sda3": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c.  Caller 0xffffffff8030b708

Mar 18 03:10:07 [kernel] Call Trace:

Mar 18 03:10:07 [kernel]  [<ffffffff8030b5bb>]

Mar 18 03:10:07 [kernel]  [<ffffffff8030b708>]

Mar 18 03:10:07 [kernel]  [<ffffffff80362733>]

Mar 18 03:10:07 [kernel]  [<ffffffff8025b3fa>]

Mar 18 03:10:07 [kernel]  [<ffffffff8030e352>]

Mar 18 03:10:07 [kernel]  [<ffffffff8030b708>]

Mar 18 03:10:07 [kernel]  [<ffffffff8030e5dc>]

                - Last output repeated twice -

Mar 18 03:10:07 [kernel]  [<ffffffff8030e352>]

Mar 18 03:10:07 [kernel]  [<ffffffff8025b7df>]

Mar 18 03:10:07 [kernel]  [<ffffffff80302e11>]

Mar 18 03:10:07 [kernel]  [<ffffffff8030e352>]

Mar 18 03:10:07 [kernel]  [<ffffffff8030df98>]

Mar 18 03:10:07 [kernel]  [<ffffffff8033969b>]

Mar 18 03:10:07 [kernel]  [<ffffffff8033fd1a>]

Mar 18 03:10:07 [kernel]  [<ffffffff80288be5>]

                - Last output repeated twice -

Mar 18 03:10:07 [kernel]  [<ffffffff80288a86>]

Mar 18 03:10:07 [kernel]  [<ffffffff80288d4d>]

Mar 18 03:10:07 [kernel]  [<ffffffff8020b5ee>]
```

Just noticed they all seem to occur at around 03:10 am each night... weird.

----------

## snIP3r

hi!

one option is to run "xfs_repair" to check your filesystem/partition for filesystem errors.

it looks like the error is happening after a cron job is starting.

 *Quote:*   

> 
> 
> Mar 19 03:10:01 [cron] (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )
> 
> Mar 19 03:10:03 [kernel] 0x0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> ...

 

you can check your crontab for running a job that might not be doing well.

the error does not occur at another time?? if not, you can deactivate all running cron jobs at this time step by step and see if it happens again...

HTH

snIP3r

----------

## apryan

 *snIP3r wrote:*   

> 
> 
> one option is to run "xfs_repair" to check your filesystem/partition for filesystem errors.

 

```

# xfs_repair /dev/sda3

xfs_repair: /dev/sda3 contains a mounted filesystem

fatal error -- couldn't initialize XFS library
```

So I guess I have to boot into livecd and try running a repair?

 *snIP3r wrote:*   

> it looks like the error is happening after a cron job is starting.
> 
> you can check your crontab for running a job that might not be doing well.
> 
> the error does not occur at another time?? if not, you can deactivate all running cron jobs at this time step by step and see if it happens again...

 

I went through all crons and none that I saw were odd. Only the defaults are enabled. The weird thing is that it doesn't occur at every time the cron for run-crons is executed. It only seems to happen at 3am. Weird. I will try sifting through this more thou.

thanks!

----------

## snIP3r

hi!

i guess that /dev/sda3 contains the root filesystem. in this case - yes, you have to boot with a live cd and then do the repair command.

does it _always_ run at 3am?? then something must be done at this time, i only assumed that it might be a cron-job.

HTH

snIP3r

----------

