# ssh running but bash broken because of missing GLIBC_2.11

## henri

Hi folks,

yesterday night, I was upgrading an old server from 2006 step by step which worked pretty well.

Unfortunately the network connection broke down while transferring glibc-2.11.3.tbz2 to extract it there for a further gcc upgrade.

Now ssh is still running and accepting connections but because glibc-2.11 is missing, bash is broken now and will not execute commands.

What I need to do is to upload glibc-2.11.3.tbz2 to / and extract it there.

Any hints how to achieve this withount rebooting with a live-CD and chrooting?

Many thanks in advance for every hint,

    yours Henri

----------

## dE_logics

 *Quote:*   

> What I need to do is to upload glibc-2.11.3.tbz2 to / and extract it there.

 

It's a source package, it needs to be compiled first.

I suggest you boot a a Gentoo live CD (like sysrescuecd), and change the ROOT variable in make.conf of the live CD to point to the mounted FS on the HDD.

----------

## henri

 :Smile:  No, ist's not a source package! It's a compiled version I created on a similar machine with...

```
emerge --buildpkgonly ...
```

The issue is that I'm advised not to stop the machine at the moment for different reasons by the management of the owning company.

I would need a hint how to achive uploading and extracting with ssh.

Btw.: rsync is not an option because it's broken for the same reason.

Many thanks

----------

## dE_logics

Do you have another box at hand, as a replacement?

What sort of server is it?

----------

## phajdan.jr

 *henri wrote:*   

> yesterday night, I was upgrading an old server from 2006 step by step which worked pretty well.

 

Could you post more details how you did the upgrade? I'd be very surprised if "emerge -uDNa world" resulted in broken bash complaining about incompatible glibc. But you mentioned the server was very old, so the upgrade was most probably more complicated.

I guess you should still be able to transfer files from/to the server using ssh. That might allow you to replace binaries like /bin/bash, or change the root shell to something like busybox, or just change bash to something that works...

By the way, it is extremely important to safeguard against network problems when doing updates. Did you use screen or tmux? Maybe we should update the docs to mention it... which guide did you look at?

----------

## henri

Allright, I solved the problem:

I called one of the technicians of the company who's server it is and asked him if he could take a look if the server was still in a state with the root user logged in locally.

Sometimes you need a little luck... it was  :Smile: 

So I copied my glibc package to an usb-stick, drove to the company and was able to copy and unpack it whithot having to reboot with CD and chown into the system.

Te server was unchanged since 2006, so it was a horrible time of trial and error to at least get it from pyton 2.3 or 2.3 , very old emerge/portage, glibc 0 to a state where I could at least use emerge again.

A big help was an old 2007 and 2008 portage-snapshot, but even finding those old required sources somewhere took me hours and hours of searching.

No NuDvp world was possible! - Even in the state where I got it now a lot of things and dependencies are still broken.

I beg you: Upgrade your servers at least once a year or take about 5000 €/$ for such an upgrade.

Absolutely horrible if you have no chance to stop the system, boot from CD, mount it, cover it with an actual stage3, then chroot into it and compile everything until the system is in the new stable state.

My tip for administrators: Collect even very old portage snapshots and all sources you ever used. - You will probabely need them some day when a customer comes along with such an old machine.

Expecially keep old install CDs to make an installation in a VirtualBox to be able to compile intermediate versions of libs or programs in it and package them.

Thanks for your attention,

    yours Henri

----------

## Ant P.

It might be nice if gentoo.org kept snapshots of every 6 months for that reason. I'm assuming 6 months is close enough...

----------

## dE_logics

I would prefer a cluster of at least 2 in such mission critical systems.

----------

## phajdan.jr

 *Ant P. wrote:*   

> It might be nice if gentoo.org kept snapshots of every 6 months for that reason. I'm assuming 6 months is close enough...

 

Feel free to file a bug if there isn't one already. I think it is a good idea.

----------

