# Updating to GCC 3.4/4? Want to use 2006.1 profile? READ THIS

## Guenther Brunthaler

Hi all,

(You are reading version 1.19 of this guide.)

ABSTRACT: How to recompile your entire system / toolchain with MINIMUM PROCESSING TIME EFFORT.

If you plan to do a major compiler update for your Gentoo box, such as from GCC 3.3 to GCC 3.4, or from GCC 3.3/3.4 to GCC 4, then read on. (This will typically be the case when upgrading your Gentoo installation to the new 2006.1 profile.)

The problem is that in those cases the C++ ABI of the GCC changes, which requires recompiling most of your system.

In other words: You have in fact to recompile your ENTIRE SYSTEM in order to be sure no "undesired" side-effects may arise from your compiler upgrade.

This is especially true when using KDE/QT and other programs/libraries which extensively make use of the C++ language.

And here is the reason for this posting: While I have found many postings how to best recompile your entire Gentoo system in such cases, or how to rebuild your "toolchain" best, I found none of them being optimal.

The problem: Actually all of them require much more time to execute then is really necessary.

Because in my opinion the amazing truth is: It is in fact UNNECESSARY to recompile any package, including the new GCC, more than exactly once when rebuilding your entire system! (For those who cannot believe this, see my argumentation in the related article https://forums.gentoo.org/viewtopic-p-3548628.html#3548628.)

Using my guide (and the accompanying utility script) presented here, you will be able to rebuild your entire Gentoo system with an absolute minimum of effort.

Compared to other guides which suggest recompiling the entire system up to 6 times "to be sure" (I will prove them wrong - their approach is not safer than my approach presented here, it only takes more time), my approach can save you days or even weeks of processing time!

And as weeks of unnecessary processing time equates days/weeks of unnecessary power consumption as well, this will certainly save you a few bucks on your next electicity bill.

I feel obliged to mention at this point that badpenguin has also written a guide how to recompile your entire system, taking about the same running time. Although not for the faint of heart (because it requires a stage 1 reinstallation of your system), I consider it to be a viable hardcore alternative to my guide. You can find it at http://badpenguins.com/gentoo-build-test/.

OK; let's be short: Here is my guide as of 2006-12-05:

===================

Recompile Entire System Helper

Copy the script from the end of this article and save it to a text file. (Go to the section "HOW TO OBTAIN THE SCRIPT" of this article to get it.) Be sure to follow the instructions below before you use it!

Script version as of $Date: 2006-09-26T04:22:55.640944Z $

Written in 2006 by Guenther Brunthaler

This script will generate another script to be run by you. That other script will then recompile each and every package of your whole current system in the correct order.

This will typically be required on a major GCC upgrade.

IMPORTANT: Do not execute this script before all of the following prerequisites are met:

 Portage tree is up-to-date (emerge --sync)

 Your system is up-to-date (emerge --ask --update --deep --newuse world)

 gentoolkit is available. (The script uses it.) If you are unsure, just do an 'emerge --update gentoolkit' and it will be emerged unless it is already installed.

 OPTIONAL. Review the /var/lib/portage/world list of installed packages and consider unmerging packages you are no longer using. It's a good time cleaning up your packages: The less packages are installed, the faster the recompilation of the entire system will be finished.

 OPTIONAL. Now it might also be a good time to run "emerge --ask --depclean" in order to uninstall packages representing outdated old dependencies no longer used. Be sure to read the manual page for emerge before trying this!

 Run "revdep-rebuild". If there are problems found and fixed, repeat running "revdep-rebuild" until no more problems are reported.

 OPTIONAL. If you want to change your Gentoo system profile to a new one using "eselect profile", it is now also the right time to do it. Also do an env-update after changing the profile. (Be sure your kernel version is supported by the new profile! If necessary, upgrade the kernel, install it, reboot and continue following this guide only after that.)

 Do an "emerge --oneshot --update --newuse gcc". It is quite common that emerge will detect that the current GCC is already up-to-date at this point, and will therefore emerge nothing. (But for the less common situations, this step will prevent troubles later.)

 At this point, the new compiler you want should already be *installed*. (No packages need have to be recompiled with it yet. It also need not be the currently selected default compiler version yet.) As GCC allows multislot installations, it is not a problem in Gentoo to have both your current and a new compiler be installed at the same time. This guide even depends on that fact!

 If all the above conditions are met, and no more packages need to be compiled in order to have an up-to-date system, set the desired new system compiler as the new system default compiler using "gcc-config". (This will make the new GCC 3.4 or GCC 4 the new system compiler.)

 For those who have never used gcc-config before: "gcc-config --list-profiles" displays a list of all GCC versions which are currently installed on your box. The compiler you are currently using is marked with an asterisk. In order to change this "current" compiler to a different one from the list, remember the number between the brackets on the left side of the line which contains the compiler version you want to use. Then use "gcc-config <number>" to change the system default compiler to the list entry <number>, where <number> is the the number you remembered in the previous step.

 Do an "env-update". Then run "gcc-config --get-current-profile". Verify that the new compiler version is really displayed as the current system compiler.

 REBOOT! Many troubles when recompiling the system with a new compiler are due to the fact that the updated environment variables may not have been propagated to all active shell windows which you might be using for command input. A reboot ensures that every process in the system now uses the updated environment variables.

 Are you using ebuilds which install kernel modules? Typical examples are special ALSA drivers, closed source video drivers, virtual machine networking drivers, add-on encryption engines. Not sure? Then better assume this is the case, and do this: Optionally update your kernel first. Then recompile your kernel with the (newly installed) GCC. Or, if you are using genkernel, just re-emerge genkernel. This will eliminate potential build failures later when the kernel module ebuilds will be emerged as part of the system rebuild. For instance, Karsten1973 reported build failures for the nvidia drivers when following an older version of this guide which did not include this point.

 Download my utility script now if not already done. The link can be found at the beginning of this article. A good place where to put the script after downloading may be /usr/local/sbin/. (This is only a suggestion. Any other place will do as well.)

 Run it!

 The script will do nothing dangerous on its own. Not yet. Instead, it will use gentoolkit's services in order to construct a list of packages to be re-emerged when recompiling your entire system, and also determines the correct order in which those packages must be emerged. It also filters out unnecessary or duplicate emerges.

 Then it will construct a shell script which, when run by you, will emerge all the packages in the right order until every package in your system has been recompiled as required.

 The generated script will also allow to be aborted and resumed at any point. That is, you do not need to keep it running for days or weeks. The script remembers the last package successfully emerged, and will skip any packages already recompiled when run again later. This allows incrementally rebuilding your system. You can even shut down your system and continue recompiling the other day!

 Run the generated script to actually start recompiling your entire system.

 Run "revdep-rebuild". If there are problems found and fixed, repeat running "revdep-rebuild" until no more problems are reported. (Thanks go to devsk who identified possible problems when omitting this step.)

 Reboot. (Just to be sure. It's not strictly necessary.)

OK, that's the guide so far.

If my guide served you well, you might perhaps also be interested in my article https://forums.gentoo.org/viewtopic-t-495997-highlight-.html where a promising new approach to prelinking is discussed.

Now some warnings and comments about the above guide. Although not necessary, it may be wise to read them also.

 The script is now tolerant when packages fail to compile. In this case, the failing packages will be skipped and their names will be recorded in a file. Then, after all remaining packages have been recompiled, the script attempts to recompile those failing packages again. This will be repeated until all packages have been recompiled, or until none of the remaining packages could be recompiled successfully. This new feature allows an interruption-free rebuild of the system, without a need for the administrator to periodically monitor the build progress. The idea for this very useful feature was contributed by count_zero. Thanks a lot!

 A different log file containing all of the screen output from the generated script will be created each time it is run. Together with the new interruption-free rebuild behaviour, this effectively turns system rebuilds into long-running batch-jobs.

 The generated script assumes your portage tree will remain in a consistent state during the rebuild process. So please don't do an "emerge --sync" again before your system has been recompiled completely! (This is because the generated script contains specific package version numbers which were extracted at the time the script was generated. If the portage tree was updated before the script finished, package dependencies might change and the assumptions of the script regarding package dependendies might thus become void.)

 In former versions of this guide, I strongly recommended running this script from the text mode interface, without an X-server running. Due to the experiences reported by Lloeki in his posting https://forums.gentoo.org/viewtopic.php?p=3553978#3553978, I consider this no longer to be a prerequisite, as you follow the general guidelines from the referenced posting.

 Repeating from that post, those guidelines are: If you are heavily using KDE or other C++ based applications, start all such applications you may ever need during the update before running the script from my guide. Keep those applications running and use them until the update is finished. But don't ever close them, or there are chances you won't be able to start them again until the update is complete! Then, after the update, quit all applications, log out and reboot. (The trick here is that shared libraries which are already loaded will be kept im memory, even if their on-disk files are being replaced by different versions during the update. As long as you don't close those applications, the old versions will remain in memory, and all will remain consistent.)

 The above trick allows you to keep working with your applications using C++ shared libraries during the update, which normally would run you into troubles. If you have a second machine to work on during the update, it may still be better to shut down the X server during the update and keep the machine running in text mode.

 If you don't know how to get to the text mode, here is a short guide: First log out of your X session if necessary, so that you see the X login-widget of your kdm, xdm or gdm. Instead of entering your credentials here, press [Ctrl]+[Alt]+[F1] now. This should switch from the X login screen to the text console where you can log in as root. Then run "/etc/init.d/xdm stop" in order to shut down the X server.

 Although the approach presented in the above guide greatly reduces the amount of time required to rebuild your entire system in comparison to all the other guides I am aware of, rebuilding your system will nevertheless take bloody ages. (At least unless you have a distcc compiler farm at your disposal. Or unless you are a "Bot-Master", controlling thousands of hijacked Windows PCs serving as distcc-slaves...  :Wink:  )

So far for my comments - for now.

HOW TO OBTAIN THE SCRIPT

Here are the instructions how to get the utility script mentioned in the text of the article above.

 First select the text between (not including) the "---BEGIN SCRIPT---" and "---END SCRIPT---" lines (can be found at the very end of this article text) with the mouse.

 Then copy the selected lines into a text editor, using the mouse or copy/paste.

 Save the copied text to some file, such as "genscript.sh".

 Launch a command shell and go to the directory where you saved the text file.

 Run the command 

```
sh genscript.sh
```

 This should decompress the actual script and leave it in the current directory as a file "recompile-entire-system-20060926".

Final remarks:

I wrote this guide not only as a small contribution to the Gentoo community, but also to the environment.

Yes, to the environment! Thousands and thousands of Gentoo users, all wasting weeks worth of processing power when following inefficient "recompile entire system"-style compiler upgrade guides, will waste an enormous amount of kilowatt hours of electrical power.

This energy not only costs money, but will also have to be produced, where it will contribute to even more pollution of air or water.

And that additional amount of completely unnecessary environmental pollution can easily be avoided by following the above guide... so that's why I wrote it.

Greetings from Vienna/Austria,

Guenther Brunthaler

==============================

```

---BEGIN SCRIPT---

#! /bin/bash

# This is file "genscript.sh".

die() { echo "ERROR: $1" >& 2; exit 1; }

echo "Decoding attachment - please standby."

which openssl > /dev/null 2>& 1 || {

emerge --oneshot --update openssl || {

die "I need openssl for base64 decoding!"; } }; {

while read LINE; do test "$LINE" = "exit" && break; done

openssl enc -d -a; } < "$0" > script.ar || {

die "Error in base64-encoded text!"; }

ar -x script.ar || die "Cannot extract archive!"

N="recompile-entire-system-20060926"; md5sum -c $N.bz2.md5 || {

die "Checksum error in extracted bzip2 file!"; }

bunzip2 -f $N.bz2 || die "Corrupted bzip2 archive!"

echo "Script extracted successfully!"

rm script.ar $N.bz2.md5

exit

ITxhcmNoPgovLyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg

ICAgICAgICA4MCAgICAgICAgYApyZWNvbXBpbGUtZW50aXJlLXN5c3RlbS0yMDA2

MDkyNi5iejIvCnJlY29tcGlsZS1lbnRpcmUtc3lzdGVtLTIwMDYwOTI2LmJ6Mi5t

ZDUvCi8wICAgICAgICAgICAgICAxMTU5MjQ1NDAwICAwICAgICAwICAgICAxMDA3

NTUgIDQ0MjggICAgICBgCkJaaDkxQVkmU1kWFB8ZAAULX4BSeX///7//79//////

/2AS/3tHS7sWlYVI7DsqwG5umjOihrIhOWRUtmUdAykKVLbG5g3YcNNECmTQYjRG

0Jpo0mARqeNFTwUZoYptkiHqGIwGgQARDTVPTIJqfqYp5TR5RoA000AAA0AM1NBw

NNNNBoaGhkaAZAGhoDTRkAADCYgNBISECJqabU9QnhU/RT1HpMjTR6aTTRtQAGg/

VPSGjQbUBwNNNNBoaGhkaAZAGhoDTRkAADCYgNBIkE0BAmI0TAEylPwk1D9U2mSa

GmaTRoPIgehHqaWomCAaCqSbGwGNtsTGmMTBgwbGJpgDGm0MbE29fm/nPWQ9lHG/

ro5F/yfGKvAseigl8leYkAqHMC9lYAPlo/3B6cxuIfudv/T6LWorn3Mw/avQ/nkd

nhtnXUtJIs5q5CQjTcYNpxQbbTf2qqkQY5HHBjGyONsbDjxtZOzcG2zaUevnylrM

b1P9/PvXKDDXejGoTNScvPn+nbuPFO0/L/da3voQZz3PhBDJ7IPFJ0sklmU0Nm7h

Y2sXNdZfszv1cL4MZ0t8Ii7NVVmVpeBX0SvimzNTtZEerzJIEujVCQoCsZAi/OAE

z+3/G7cdmqGPv+IfVV4yok5d6SjfKf4wiU0oK33Y3ard+y8unaWYSa72rwG0Nq60

SZJlWpHP/jvrmYNqCCCTJbjO9ldM0TwB0gsSL1b/hee93Dr8pTVKIimwZ7aVMDCw

p5uJnu2CUSRQEAyR0Ne4GvcnHCFVYxuPZRME3GZhSpVeZk68FuYYbUKeQ+7ojnYV

lEtpn2eUKhh486CObEvTmXCmgPcNp3znCE4V1jSyEv9blHHWLJQDkIkDHLB2dOTf

1cuumHUjcux9uim76W9B1xEAvxa14dT0CRrpXr7u2p/Ltfr7fz686rFA1YafnIlm

r1tygx7eOSk6fQKrooIk2OzPlaYVJD8xuB9+952wsvuzRwlHqMgpRc5XgezGzht1

psZwWezWFaAKzp0KNIcN6MQLiu4VG4FYsWmdrVxTO+QVZ2Ae0nsMD9HQcBsflu+F

U3pUWLU14ILxOa+9pSLhlTwwT1KIiMo/u4MTCMzEXmsh2rc6OQ0ikAamULgwY6r0

+sj57wrjZS3+k4MGNEjBMrVwwc0kHrZP4atkRmZXQ0QkWHGBOiIoktOgBM+keY9Q

D4vMac5qDtMOg4G6K1uLdDeGmKkmwIUeSyqM8sPBcdy7FKVSSfic7b3XBdZ6JCTq

Uy7tacVOKjS6ScJfRK6/gP3xk1mFxsnrrrJgKI6/F10sDUFlnWWbcR8dQ1URhRrT

/dAkdjQyHqh6QCWRZDU65x1I55WgdmJ1dKja2/lp8LT9BDfkmGZUpvkUPwOavoxQ

d1rDFOk81CsIDEFx5M9hjnnuAcQrm3IkPUm25SocsApBwEYs+JuSq+Od1B7SMJZ1

uhI2EiYaGxENq98DZDSwcYY8hpjpGqyAPwuhm/g1Mpo10CNdTxZequbQkk5iOg2y

AElg5AHClcHaEZ78DoZthwX1Qg8eRlxxFGvSftA5vklDQratCgzbLbMtz54clQit

Ys6h5OvXHI3YrU5eUFZrNqSIDyAxPZ+GiYcAHIaUF9JLZT9iV2dqMAHPXrI2yA2z

6B3AidRW2836MFA4iL27p0VfUpltly2uyTzd2US6fKTgTXo6nYWrYZZ3L4Rex06o

tNRj53LPH9ueckAlEKEAEgAEoASDlWJ7tj2NjenFYcBcbxFp7bt+xRjNK30CDUxD

aFd+pEDBNgLs9LE0B0NVaC5x688l7fx2PGWIiEihlS5DDe1cG0wY2i40iAwYxjSY

vxCmK/ows/+HyxsytB5wX8cERiOj/Uj8hAaMoOzQVbuH/rjNNh3PFn0O2s5go0aK

4EabZQ4D2Grlob4Npigu7gRbBqPLXoSVfztav9tuNQc/CWLB/hrOzrKI2sET7tjX

/q+cA429m8HJsmo1dzuwkw/adF1pVxsg0ERKQP41L7MaeoDrHrelrkUffSLyNOL0

vDv+fz3e24PMRgI4/l624LVGEjcDDYIupKILCFI9gz0HG1r23Z7vl7t0eeAR8pvU

wxSeo+y59up/x4rmnsonWbD0zJsGkbLBuceLHhW7sNi5N4Mx78OSNnb4rfFhq/pp

sDbAMa9nY8UFIW+1U0yueqVl8La2FgChAK4ifrZRJ90FwubRvRckbgvZUcZkiIJ7

4wTm+PCOTnkIB0Pt04+Ad33Zk9nXtaqtQL3in0RZApg+AWA9F9FHRiiyD60Nw+XR

CyeZ4ZhvlNzCtFB7+yl4zU9/BTbPM9LWquBbo2QU39lCj6luScHaRyXq4J3z4ekQ

J9+CFNSbOkunQGn46bZ8Tdbq/Td2xwjqf1cMBPw64ynQwhxnXwMIoHqjeQ4BepH+

3Wsu3YMGTmgjcJDmQC3mCdKRCV7qf6SKe9CPGdQVKKHsPxeU+9hWqKyF7FgtFI8L

td2OdTuXxW8HjtVQXo4QPpgSASlNFKNQUvCdUk3HxIOMgf/5/Y5N8HRoXVewPkb6

YEjdpguf0yZevzcmg9Mpamgamh2rKZwWed7quSW2iEG24XagcsR6D31dAZ73qduP

NRLxJ1nvMJrRK62SgbN4ScUQGt/meVVfOWTqAI+WjwVonKAD7ejWnwJQpv1KE2kf

dzYgGsxZt3ftG257yyLwrIzJn3xLMG1b/AJJHvmvQ5pBdb7xOoeRSvUvj0MNcDi/

JsyvZ7wnvIG6QcgU7CuEHS7016r6Yf1PzCQ2AxgNfYQH6394LtvDmxPvD2LnUw/G

Xma9fzSUTOIvhEDf70H7KdK/bnYL2xpVvPpabObmWNO7vrVRhINokBkRAZE/Yolk

FxrWf2oIBWDEJroy1PJP5sFAw04oX/syTMliN7UiRk1vLw4OU9/QhfeCxoHyPPiC

4oNSdMLvnlqSzLmSNNgIkKCQZyVqInfjYL7gMyLgSaz2Guu0MAxoa6QYoxaLTdpg

gZ8jJLMLcHRszRig6LukqtVwjJY/3UxpD1PFaLswNmZYuGT60vwQck5UHrSu5jYI

wpXAiOKOTJk7gsguslRU5VU0SNWGQFD4/hI/wXPTPREA1rXHBZd0GciOlG0NTEcj

d2t15Q7azmCKD5bDfp+lVAcovk2slIIII2KPD3M7CPDeeu2XV67jm3e8a+pdpHff

AM2zWz2I/KNH/AMw+byXYnCcfFUPrPqAZFpdeYi/PDy/hOh5Aev/VHutATNXEvmh

EaB/BfKw+FoV9540dxDRGhvo6Z+ZXDci9ls6XrKN7bosnViLuQQjXpRptIopTXpx

RzSD7XpvvL1SihbBrfZTJGlMbDMPqUgnbQ+6tHNNYHO8Ubgtdor0VO338ZJI6XZb

amD4/uXz3f94ZJc6XNQRjxkNVx/YQGFOsNA3VcIb0zMLWpDaZkp6ft+9uARYSPFh

jWllGJtjf6IQgwI+g87Q2hFGzzTMsGTo3veOzMB7zUV2uuBKZSB0wbE2NMuzDOkY

rr/2S9LWve0mhvLsZsrYRAV5YhQq99lRAbGZsW5O5Fx923IHw4y7FLDIvQpDXeW8

adffxTZiQ7YzMIHXq5BTGrDfqxcRStS3OR4WEhJ5OiEqFiowghB1bIIScJzR1pUd

ASgEB2YB4WiAkJHLZmo+U2QLE2pMOlIB9H4tOVVswTNUs2zAYPHRr0usa4RWxr6Q

2BPE3piI0X5Gigoer/keJHjflegwRGLBg9TBThpWlugLbccu87VDMydik/DIZkxU

ebIdHJZtclEJMTVCRr3LDA27NzioZIzaDFS8pElu98l01uA3ByG6KJShw2SiCwfH

ejcg5b06NjgXUsrrpqpgMiHkEleZPFVfBnFliZxgKEpcRWqerufvXOi6rs0MNabJ

rXgm3vga0onzTpdbPRXDJMq2ElXdMwUi+4mLHY1rhcDI6eRM8IiZo2c3OeGcYl7c

qkEcZ8cUabBluh3RGiQgD6YRjZqH3osty+otD9WXMNLGQp222KGixEf2yn95xJmH

rCmep1HfiaaREEIBs5VLz+OuWtlAYVMVIaUIE13z1VGD+NKX8/eFbigD31QgGsYo

BXTy7kxqEUYNsciHwiPgssg5kee9GrcEUxsG0NpD9K6j5DhuMIMPy9ziBqpuoMMV

Cm0yFpRcWJBLAzauwa0tz6p9wrkIjyLoY4D2hv0DHNHmIMKNemo7vDVESfgOP4y8

qtmointYuRsc+O+35DiccIwUEgQJ+xwuNsWJxEEh83w6PBaIqMDzv9F1zfbDThbE

aCYEszp6oDxb0rGwl7PxTV1taMfczC/LtTuAaMzXWAwaBjZ79o7kJTOHQYIyQdGQ

0mw1raazRc4xlVTifoOzRMmQcpFNwEDpvigByZRkI0Z8A3LVseAg5HPDY5zR4ME2

XRANh1I2BKJjKPS/wgtVZILrySR1tGpmtot+QvkkSF4Ns0Utq1GG2AyOFruBCKsJ

W98lYwChsaRAx/jZCewGiZ52sHzPnDatdjCBRemS76KjNGV4TRmx2gMGfJw0ygNU

HVQsHcm2SCp9SnFLjyu3rqrgVhFUqiQ4C94Wuve35vdLu93UTkiJxMfZSZijU1sj

0yDYNXCWZSZcOFi9mXYXd3RaDTVOgfohZ2VsWVqJ8QMuyqsvcsaBbSuzLY+zFfVh

4GiLYFYHcmEMWVwoAlM2PVImkizKoGHJi07WGKwGd2vpGlkDR3XIGMhpvRjYmBBi

30aqlaMtVk0M8mP3FGYdq6kdbFwYhQSkz01f1yRp9kFR5udvBacO9eJlMSnQP1wH

8lff9PwVDImTYYpI5liMG083CzYag2britCBYGrXnqu8Uwr6KEd4G0W4l8mI3bu/

3EzFzLNo/mpwDbyD4quvcG3SyAmdK5u3VRv4eaVtPYBnP/y+petwZ67mvV3l5D6p

kRTsqtoon1JVeFiuPGwWxyXMI9LF2ubjExquoxSzfhY0GjKe8ySC1DQDGZiJmV5U

9j9D64kPX9IRgWSvYDYGiYtROSJpge7tx5M3GvMaBGme/A0FzxQ4AyDA3uRobId6

TD/7aXeVaEhhiu0fgIhhQml3wLmlhjochwVrUihejpPMdtjsYdkiNtVtjvlxqwbo

MJEqOPWZUUwizXeKR4HF6RxL2u/kG5sMQxAzQlu+tOtLuZhJZ3sIvF5GUUrVVS2I

DIFQucVuGOEkt+KUiAkkcrmsZTdMrVFZG2AmSEDDXBu04VVaOtpMmK+HLDbHpzWo

r0Ztee31kT3yPCq+Xj2Uw3atEePbAl3CXE215whQOQMpLioWBOZvhmVaV3UJHKZ9

eEBZMe3tlHl9uhUSxGsgctJkyInpf7+SUKnBKFFlvUhZkQeW3E/L00P2soRlkubB

h4at7cBzQzYFc0IVAVgWsIokTCdwaXeyoSmQqsXTfCTxzMSCdgjhaDymYfC7B0EI

pmplMwKTCeOxoKqQUlFzUptv1sWFcTuk4h4PBVFGVbMBf7jA0MFCNiMp6pK5qSvz

uiawGel2GAwY2O/eeYpLmLYrJWWC1LevkzR0GSGy5ZAkNTzDVNBrcxyyKnjWmq6d

XpaxkUE2mQcNDuYYqb5NQxaNHXKDo1kj26UOFFtSxEnxZ9FeuW83DDbcjcrhgV0G

3sS3wC/Q1wpN/mXz+SVvPqCUy+iBHOOBwLJECdB/YGPHxcrqdKKi0XFQMJWmm1Ay

djskdRnhOk78zyIeB1+Pj1GFMuk1urtpooJUWvVDxd1Q39hEWFD1te9hHmEBVDDx

dGBRlRqWBC2mA7H3TiYQXVzPH9Oua2MIYuHRzCDk5BAdI5nEtNTp9Dp6zkOtbBKV

H9l3LYBiQL2tH8LbQ8+osulggvA5z9ww/5+TE6BlEiSX+JZB7V6h2dUT+eJB4gC3

4BZc0seSQiogcPkt8SBh7Zf+LuSKcKEgLCg+Mi8zOCAgICAgICAgICAgICAxMTU5

MjQ1NDcwICAwICAgICAwICAgICAxMDA2NDQgIDcxICAgICAgICBgCjJiZmQ1Mjdl

NTQyY2MzY2Y5MDY0MTBiNDhiZGRmNDBkICpyZWNvbXBpbGUtZW50aXJlLXN5c3Rl

bS0yMDA2MDkyNi5iejIKCg==

---END SCRIPT---

```

Last edited by Guenther Brunthaler on Tue Dec 05, 2006 6:11 pm; edited 19 times in total

----------

## Nick C

can you not just post the script in plain text inside code bb tags? Would save anyone wanting to follow your guide a bit of hassle.  :Smile: 

----------

## Guenther Brunthaler

 *Nick C wrote:*   

> can you not just post the script in plain text inside code bb tags? Would save anyone wanting to follow your guide a bit of hassle. 

 

Sorry for the hassle!

Initially I wanted to do just as you suggested.

But then I had concerns that the whitespace and indentation of my script could be ruined by the BBCode formatting and/or careless copying/pasting.

There were also data integrity concerns: What if only part of the script was copied? This script does work vital to the future health of the system. It must be assured that its contents are intact before starting! (So I added an MD5 checksum file as part of the base-64 encoded text.)

Another problem is the script length: It is relatively large in uncompressed form, so I wanted to bzip2 it in order to get much smaller.

The optimum solution would certainly be to host it on some web server, and only provide a download link in the article.

But unfortulately I'm already quite a bit over quota with my ISP, so I wouldn't like to put it on my own web page.

However, if someone reading this has some bandwith left to waste, he/she is invited to host the decoded script at their web site and send me the download URL.

I would then thankfully included it within my article.

----------

## devsk

please post a text version, no matter how long it is. A useful thing is never too long to cut&paste.... :Smile: 

I was hoping someone actually wrote such a script. Thanks for doing that.

----------

## Nick C

aslong as you use the code tags whitespace will be preserved

----------

## Earthwings

```
$ ./move.pl

Encoding, please wait...

Generating move script... done.

$ ./move.bash

Moved from Installing Gentoo to Documentation, Tips & Tricks.
```

----------

## Guenther Brunthaler

 *devsk wrote:*   

> please post a text version, no matter how long it is. A useful thing is never too long to cut&paste....

 

Actually, the attached base64-encoded "attachment" is already a script!  :Smile: 

Look more closely at it: Although it undeniably contains a bunch of base64-lines at its end, there is bash code at the beginning which decodes the base-64 stuff at the end of the script, decompresses it and verifies the md5 checksum.

So, actually you have a script which you can copy/paste, and run.

You don't have to decode the base-64 stuff by hand - the script will do it for you.

If I posted the uncompressed script instead, you would have to do exactly the same: Copying, pasting into some text editor, saving as a file, running the script.

Only that the text to copy would be longer than it is now (because it is compressed).

Plus there is the the security bonus: The current version verifies its own data integrity. This would be hard to achieve with a fully expanded text version.

 *devsk wrote:*   

> I was hoping someone actually wrote such a script. Thanks for doing that.

 

Actually I wanted the script to get even better; namely an interactive, dialogue-based version which asks the user questions and performs most of the "preparation"-steps of my guide semi-automatically.

But then I could not get it finshed in time, and decided to choke on the interactivity: I definitively wanted to post the article before the majority of Gentoo users have switched to 2006.1, because otherwise they would have to use the old, inefficient guides that are available at that time.

----------

## MaBu-Gentoo

I have put the script on pastebin. I think it is a great script, but I have one question, when should I update glibc?

I konw have sys-libs/glibc-2.3.6-r4 and the new one is 2.4-r3.

----------

## devsk

 *Guenther Brunthaler wrote:*   

> 
> 
> So, actually you have a script which you can copy/paste, and run.
> 
> You don't have to decode the base-64 stuff by hand - the script will do it for you.
> ...

 I understand all that... :Smile:  this is not the first time I am seeing something like this. But with the whole script in the page, I as a dev, can easily look at it and make a case for whether I want it or not. I can review it without copying it and then decide that Mr Guenther has done a good job with writing it, has handled errors and corner cases well. Even make suggestions to the parts which I think could be done better or in a different way. Its just that I am too lazy. And trust me there will be others with same problem.

code tags do preserve sanity and this is the standard way of copying scripts around here on these forums.

----------

## devsk

 *MaBu-Gentoo wrote:*   

> I have put the script on pastebin. I think it is a great script, but I have one question, when should I update glibc?
> 
> I konw have sys-libs/glibc-2.3.6-r4 and the new one is 2.4-r3.

 thanks for doing that....  :Very Happy: 

----------

## Guenther Brunthaler

 *Guenther Brunthaler wrote:*   

> Actually, the attached base64-encoded "attachment" is already a script! 
> 
> 

 

For the sake of clarity, I have edited my original article and removed all references to the fact that the script is internally base-64 encoded.

Those references were actually completely useless, as the fact that parts of the script are base-64 encoded is completely transparent to the user: The script itself handles the decoding; the user never comes in touch with the base-64 part of the script.

Also, it seems obvious to me now, that the mere mentioning of the term "base-64" has the potential to scare away half of the potential users right from the start  :Wink:  - which really wasn't what I intended to  happen.

----------

## Guenther Brunthaler

 *devsk wrote:*   

>  *MaBu-Gentoo wrote:*   I have put the script on pastebin. I think it is a great script, 

 

Thank you very much for this! I will edit my article and add your link there as soon as possible.

 *devsk wrote:*   

>  *MaBu-Gentoo wrote:*   but I have one question, when should I update glibc?
> 
> I konw have sys-libs/glibc-2.3.6-r4 and the new one is 2.4-r3. thanks for doing that.... 

 

This is easy: You don't have to. Recompiling the glibc will be one of the first things my generated script will do, and it will use the "current" version of the glibc as of the time when you ran your last "emerge --sync" before my script (which is one of the steps in my upgrade guide, btw).

Even if you decide to update the profile to 2006.1 (and not only update the GCC) this will be true: In this case, the newly activated profile will tell my script to use the new glibc version rather than the one from the old 2006.0 profile. (My upgrade guide also shows at what step one should upgrade to the new profile).

In other words: Dont bother; my script will take care of all the details.

Just be sure to follow all steps outlined in my guide exactly as described!

----------

## Guenther Brunthaler

 *MaBu-Gentoo wrote:*   

> I have put the script on pastebin.

 

Thanks for your help, but unfortunately, the URL does not work for me!

But on the other hand, I have changed the "ATTACHMENT" from my original article into a script in the new version of the article.

So, the old version is outdated anyway.

----------

## Guenther Brunthaler

 *devsk wrote:*   

> But with the whole script in the page, I as a dev, can easily look at it and make a case for whether I want it or not. I can review it without copying it

 

I agree; this is a comprehensible concern indeed.

On the other hand, I am not willing to sacrifice the MD5 integrity check on my script for security considerations. (There can too much go wrong with copy/paste, escpecially for inexperienced users.)

But in this case the solution is simple: I will post an expanded version of my script for review purposes only, but the download version will remain to be the MD5-protected version.

 *devsk wrote:*   

> and then decide that Mr Guenther has done a good job

 

I bet you would have been happier if you never saw that messy code... well, it's Perl code; and as such a good candidate for some code obfuscation contest... especially if written by me  :Wink: 

On the plus side is that the messy Perl script only generates the script to do the recompilation.

And the generated script is a really plain and simple bash script without any blows and whistles.

----------

## Guenther Brunthaler

Due to several requests, here is a line-numbered copy of the generator script which will be generated if the script from my article at the top is copied/pasted, saved and then executed.

This version is for review purposes only, please use the version from the article at the top if you actually want to download it.

The reason: Much can easily go wrong with copy/paste; and this is especially dangerous in a language like Perl where a single missing period can cause a script to reformat your hardisk instead of doing some useful job  :Smile: 

Which basically means, just carelessly copying/pasting can be dangerous. One must also be sure that the pasted text is identical (and not just similar) to the original text.

Which can best be achieved with checksums.

For this reason, the original script from my article contains a copy of the script below (in compressed form), and also an MD5 checksum. When the script is run, it expands the script below to your current directory, verifies its MD5 checksum and will warn you if the checksum does not match.

So please use the code below for review purposes only, but for the sake of security, use the version from the article at the top for downlading.

```

     1   #! /usr/bin/perl -w

     2   #

     3   # Generate a script which when run recompiles each and every package

     4   # in the Gentoo system.

     5   # This will typically be required on a major GCC upgrade.

     6   #

     7   # $HeadURL: /caches/xsvn/trunk/usr/local/sbin/recompile-entire-system $

     8   # $Author: root $

     9   # $Date: 2006-09-26T04:22:55.640944Z $

    10   # $Revision: 390 $

    11   #

    12   # Written in 2006 by Guenther Brunthaler

    13   

    14   

    15   use strict;

    16   use File::Temp ':POSIX';

    17   

    18   

    19   # Change this to any name you like.

    20   my $script= "recompile-remaining-packages";

    21   

    22   

    23   my $script_header= << '.';

    24   #!/bin/bash

    25   #

    26   # Run this script repeatedly (if interrupted)

    27   # until no more packages will be compiled.

    28   #

    29   # $Date: 2006-09-26T04:22:55.640944Z $

    30   # $Revision: 390 $

    31   # Written in 2006 by Guenther Brunthaler

    32   

    33   

    34   SCRIPT_VERSION="1.13"

    35   

    36   

    37   die() {

    38      echo "ERROR: $*" >& 2; exit 1

    39   }

    40   

    41   RM() {

    42      rm "$1" || die "Could not remove file '$1': $!!"

    43   }

    44   

    45   MV() {

    46      mv "$1" "$2" || die "Could not rename file '$1' into '$2': $!!"

    47   }

    48   

    49   CHMOD() {

    50      chmod $* || die "Could not set permissions 'chmod $*': $!!"

    51   }

    52   

    53   save_progress() {

    54      {

    55         echo "$VERSION"; echo "$OURGCC"

    56         echo "$PROGRESS $OK $FAILED"

    57      } > "$STATE_FILE"

    58   }

    59   

    60   item() {

    61      test "$PROGRESS" -ge "$1" && return

    62      echo "Emerging package # $1 ('$2')..."

    63      local RC; emerge --oneshot --nodeps "$2"; RC=$?

    64      if [ $RC = 0 ]; then

    65         echo "Package # $1 rebuild complete."

    66         (( ++OK ))

    67      elif [ $RC -gt 100 ]; then

    68         echo "Emerge failed return code $?. Aborting on user request."

    69         exit 1

    70      else

    71         echo "Emerge failed return code $? (will be retried later)."

    72         (( ++FAILED ))

    73         echo "$2" >> "$FAILURES_FILE"

    74      fi

    75      echo; PROGRESS="$1"; save_progress

    76   }

    77   

    78   

    79   BASENAME="${0##*/}"

    80   STATE_FILE="$HOME/.$BASENAME.state"

    81   FAILURES_FILE="$HOME/.$BASENAME.failures"

    82   LOG_FILE="$HOME/${BASENAME}_$(date '+%Y-%m-%dT%T').log"

    83   REPEAT="--4ydzd3yuhmsynbjr644bbfzx5"

    84   if [ "$1" != "$REPEAT" ]; then

    85      exec > >(

    86         echo "Note: Logging to file '$LOG_FILE'."

    87         tee "$LOG_FILE"

    88         echo

    89         echo "Note: A log has been written to '$LOG_FILE'."

    90      )

    91   fi

    92   OURGCC="`gcc-config --get-current-profile`" || die "gcc-config failed!"

    93   VERSION=; LASTGCC=; PROGRESS=; OK=; FAILED=

    94   {

    95      read VERSION; read LASTGCC; read PROGRESS OK FAILED

    96   }  2> /dev/null < "$STATE_FILE"

    97   if [ "$VERSION" != "$SCRIPT_VERSION" -o "$OURGCC" != "$LASTGCC" ]; then

    98      VERSION="$SCRIPT_VERSION"; true > "$FAILURES_FILE"

    99      PROGRESS=0; OK=0; FAILED=0; save_progress

   100   fi

   101   

   102   # Now the list of packages to be recompiled follows.

   103   .

   104   

   105   my $script_tail= << '.';

   106   # End of package list.

   107   

   108   echo

   109   if [ $FAILED = 0 ]; then

   110      echo "Success! All packages have been re-compiled."

   111      echo "Your system is now up-to-date with respect to $OURGCC!"

   112   elif [ $OK = 0 ]; then

   113      echo "Giving up: Could not compile any more packages in the current"

   114      echo "recompilation phase."

   115      echo "Look into the 'item'-lines of script '$0'"

   116      echo "to see which packages could not be recompiled."

   117   else

   118      echo "Partial success: The current recompilation phase has"

   119      echo "finished."

   120      echo

   121      echo "However, $FAILED packages were not compiled successfully"

   122      echo "in this phase."

   123      echo

   124      echo "Therefore, another phase will be started now, where an attempt"

   125      echo "will be made to recompile the remaining packages."

   126      echo

   127      echo "Note that this will *not* lead into an infinite loop:"

   128      echo "In the last phase $OK packages have been compiled"

   129      echo "successfully, and thus the total number of remaining"

   130      echo "packages has been diminished already."

   131      echo

   132      ME="$0"

   133      if [ ! -e "$ME" ]; then

   134         ME="$(which "$ME")"

   135         test -e "$ME" || die "Cannot locate '$0'"

   136      fi

   137      NEW="${ME}_$$.tmp"

   138      (

   139         IFS=$'\n'

   140         while read -r LINE; do

   141            test "$LINE" != "${LINE#item }" && break

   142            echo "$LINE"

   143         done

   144         {

   145            while read -r LINE; do

   146               echo "item $(( ++PROGRESS )) $LINE"

   147            done

   148         } < "$FAILURES_FILE"

   149         while read -r LINE; do

   150            if [ "$LINE" = "${LINE#item }" ]; then

   151               echo "$LINE"

   152               break

   153            fi

   154         done

   155         while read -r LINE; do echo "$LINE"; done

   156      ) < "$ME" > "$NEW"

   157      CHMOD --reference="$ME" "$NEW"; RM "$ME"; MV "$NEW" "$ME"

   158      VERSION="NONE"; save_progress

   159      exec "$ME" "$REPEAT"

   160   fi

   161   RM "$FAILURES_FILE"; RM "$STATE_FILE"

   162   .

   163   

   164   

   165   # Remove the largest common whitespace prefix from all lines

   166   # of the first argument.

   167   # (Empty lines or lines containing only whitespace are skipped

   168   # by this operation and will be replaced by

   169   # completely empty lines.)

   170   # The first argument must either be a reference to a multiline

   171   # string containing newline characters or a reference to an

   172   # array of single line strings (without newline characters).

   173   # Then optionally indent all resulting lines with the prefix

   174   # specified as the argument to the -first option.

   175   # For all indented lines do the same, but use the argument

   176   # to option -indent as the value of the -first option then.

   177   # If option -wrap <number> is specified, contiguous non-empty

   178   # lines of the same indentation depth are considered paragraphs,

   179   # and will be word-wrapped on output, resulting in a maximum

   180   # total line length of <number> characters.

   181   # The word-wrappin will occur on whitespaces, which can be

   182   # protected by a backslash.

   183   sub normalize_indentation {

   184      my($tref, %opt)= @_;

   185      my(@t, $t, $p, $pl);

   186      $opt{-first}||= '';

   187      $opt{-indent}||= ' ';

   188      $t= ref($tref) eq 'ARRAY' ? $tref : [split /\n/, $$tref];

   189      foreach (@$t) {

   190         s/^\s+$//;

   191         next if $_ eq '';

   192         if (defined $pl) {

   193            for (;;) {

   194               substr($p, $pl= length)= '' if length() < $pl;

   195               last if substr($_, 0, $pl) eq $p;

   196               substr($p, --$pl)= '';

   197            }

   198         } else {

   199            ($p)= /^(\s*)/;

   200            $pl= length $p;

   201         }

   202      }

   203      substr($_, 0, $pl)= '' foreach grep $_ ne '', @$t;

   204      if (exists $opt{-wrap}) {

   205         my $width= $opt{-wrap} - length $opt{-first};

   206         my $i;

   207         my $wrap= sub {

   208            my($tref, $aref, $iref, $w)= @_;

   209            my $buf;

   210            my $insert= sub {

   211               my($tref, $aref, $iref)= @_;

   212               splice @$aref, $$iref++, 0, $$tref if defined $$tref;

   213               undef $$tref;

   214            };

   215            return unless $$tref;

   216            foreach (split /(?:(?<!\\)\s)+/, $$tref) {

   217               s/\\\s/ /gs;

   218               if (length($buf || '') + length > $w) {

   219                  &$insert(\$buf, $aref, $iref);

   220               }

   221               if (defined $buf) {$buf.= " $_"} else {$buf= $_}

   222            }

   223            &$insert(\$buf, $aref, $iref);

   224            undef $$tref;

   225         };

   226         $width= 1 if $width < 1;

   227         undef $p;

   228         for ($i= 0; $i < @$t; ) {

   229            if ($t->[$i] =~ /^(?:\s|$)/) {

   230               &$wrap(\$p, $t, \$i, $width);

   231               ++$i;

   232            } else {

   233               if (defined $p) {$p.= ' '} else {$p= ''}

   234               $p.= $t->[$i];

   235               splice @$t, $i, 1;

   236            }

   237         }

   238         &$wrap(\$p, $t, \$i, $width);

   239      }

   240      for (my $i= 0; $i < @$t; ) {

   241         if ($t->[$i] =~ /^\s/) {

   242            push @t, splice @$t, $i, 1;

   243            next;

   244         }

   245         if (@t) {

   246            &normalize_indentation(\@t, %opt, -first => $opt{-indent});

   247            splice @$t, $i, 0, @t;

   248            $i+= @t;

   249            @t= ();

   250         }

   251         ++$i;

   252      }

   253      if (@t) {

   254         &normalize_indentation(\@t, %opt, -first => $opt{-indent});

   255         push @$t, @t;

   256      }

   257      substr($_, 0, 0)= $opt{-first} foreach grep $_ ne '', @$t;

   258      $$tref= join '', map "$_\n", @$t if ref($tref) ne 'ARRAY';

   259   }

   260   

   261   

   262   sub wrap0(@) {

   263      my $text= join ' ', @_;

   264      normalize_indentation \$text, -indent => '    ', -wrap => 79;

   265      return \$text;

   266   }

   267   

   268   

   269   sub pwrap(@) {

   270      print ${wrap0 @_};

   271   }

   272   

   273   

   274   $ENV{LC_ALL}= "C";

   275   my $home= $ENV{HOME};

   276   unless ($home && -d $home) {

   277      die 'Please set $HOME to your home directory';

   278   }

   279   $home =~ s!/*$!/!;

   280   substr($script, 0, 0)= $home;

   281   if (-e $script) {

   282      die "Please remove the existing '$script'.\nIt is in the way";

   283   }

   284   pwrap "$0 -", << '.';

   285   Recompile Entire System Helper

   286   

   287   Script version as of $Date: 2006-09-26T04:22:55.640944Z $

   288   

   289   Written in 2006 by Guenther Brunthaler

   290   

   291   This script will generate another script to be run by you. That other script

   292   will then recompile each and every package in the whole system in the correct

   293   order.

   294   

   295   This will typically be required on a major GCC upgrade.

   296   

   297   When the generated script will be run, it will log all of its screen output

   298   to a log file.

   299   

   300   Furthermore, the generated script will automatically skip failing packages and

   301   automatically attempt to recompile them again after all the other packages

   302   have been recompiled.

   303   

   304   IMPORTANT: Do not execute this script without also following the steps of the

   305   accompanying usage guide.

   306   

   307   The guide can be found at

   308   http://forums.gentoo.org/viewtopic-t-494331-highlight-.html

   309   

   310   Press [Ctrl]+[C] now in order to abort processing if you are not following the

   311   guide step by step. Come back and re-run this script after you have managed to

   312   get the guide.

   313   

   314   Press [Enter] now to continue if you dare.

   315   

   316   .

   317   <STDIN>;

   318   my $tmp= tmpnam or die "No temporary file names left";

   319   print "Collecting list of packages and evaluating installation order...\n";

   320   my @head= qw(

   321      sys-kernel/linux-headers

   322      sys-devel/gcc

   323      sys-libs/glibc

   324      sys-devel/binutils

   325   );

   326   my $r= join '|', map quotemeta, @head;

   327   $r= qr/ ^ (?: $r ) - \d /x;

   328   open OUT, (

   329      '| sort -k1,1 | sort -suk3,3 | sort -nk2,2 | sort -sk1,1 '

   330      . '| cut -d" " -f3 >> "' . $tmp . '"'

   331   ) or die "Cannot open output pipe: $!";

   332   my $n= 0;

   333   foreach my $f (qw/system world/) {

   334      open IN, "emerge -pe $f |" or die "Cannot open input pipe for '$f': $!";

   335      while (defined($_= <IN>)) {

   336         if (/]\s+(.*?)\s/) {

   337            (my $t, $_)= ($f, $1);

   338            if (/$r/o) {

   339               for (my $i= @head; $i--; ) {

   340                  my $L= length $head[$i];

   341                  if (length >= $L && substr($_, 0, $L) eq $head[$i]) {

   342                     print OUT "begin $i";

   343                     goto field3;

   344                  }

   345               }

   346            }

   347            print OUT "$t ", ++$n;

   348            field3:

   349            print OUT " =$_\n";

   350         }

   351      }

   352      close IN or die $!;

   353   }

   354   close OUT or die $!;

   355   open IN, '<', $tmp or die "Cannot open file '$tmp': $!";

   356   open OUT, '>', "$script" || die "Could not create '$script': $!";

   357   print OUT $script_header;

   358   $n= 1;

   359   while (defined($_= <IN>)) {

   360      next if m!^=sys-devel/gcc!; # It's already up to date!

   361      print OUT "item $n $_"; ++$n;

   362   }

   363   print OUT $script_tail;

   364   close OUT or die "Could not finish writing '$script': $!";

   365   close IN or die $!;

   366   unlink($tmp) == 1 or warn "Could not remove temorary file '$tmp': $!";

   367   unless (chmod(0755, $script) == 1) {

   368      die "Could not set permissions for '$script': $!";

   369   }

   370   pwrap << ".";

   371   Done.

   372   

   373   Script "$script" has been generated.

   374   

   375   Run this script in order to recompile each and every package in the

   376   system!

   377   

   378   By the way, the generated script will do this in a recoverable way:

   379   It can be aborted at any time by you, and will continue where it left off

   380   when you re-run it. (The package where the script was interrupted will

   381   have to be compiled again from its beginning, though.)

   382   .

```

Last edited by Guenther Brunthaler on Tue Sep 26, 2006 4:26 am; edited 2 times in total

----------

## Abit667

Wish I had seen this a little earlier..now that I'm halfway into that stupid emerge -eav system and world crap from the guide.  That always seemed so insanely inefficient to me...

----------

## staffan

 *Abit667 wrote:*   

> Wish I had seen this a little earlier..now that I'm halfway into that stupid emerge -eav system and world crap from the guide.  That always seemed so insanely inefficient to me...

 

I know the feeling...

Personally I was just about to start that crap again but thought I'd browse around in a few random forums for a bit first. And I just happened to see this thread and grabbed the script :D 

The recompile is running right now (I'm posting this from my window$ machine) and it looks good so far. Of course I have a rather old CPU (an Athlon-XP-something (I always forget)) and a lot of packages so it will still take a while.

I especially like the abort and resume features. We've had a lot of thunderstorms around here recently so sometimes you have to shut everything down very suddenly.

----------

## stahlsau

hi,

thanks for the script, i think it's really valuable in this world of "i-recompile-my-system-a-thousand-times-to-be-uptodate" people. I can't say i'd need it since i've never encountered severe problems after updating gcc (i'm running 4.1.1 atm), but i think many people will find this useful.

----------

## KD-120RD

Hi

I have a question here: What is the purpose of switching the profile?

I thought gentoo is a meta-distribution. Just update your packages and configs (ie. baselayout) and there you are.

Besides I'm using gentoo for about two years now and as I recall I have not manually swicthed my profile.

When I take a look at my /etc/make.profile it is linked against  ../usr/portage/profiles/default-linux/x86/2006.0/

Is that possible?

----------

## grimm26

i don't understand the significance of my system profile matching my kernel.  It's just default use flags and stuff, right?

----------

## jure1873

This script seems very interresting to me, but how much does this approximately take? days?

I mean, I have and sempron 3000 with 1gb of ram but if I remember installing gentoo from stage3 wasn't all that fast (I mean to get to kde) 

Upgrading gcc probably recompiles everything so this is basically a stage1 install? Could I grab a newer gentoo cd and get binary packages for some of the files from that cd and recompile the other files to make it faster?

----------

## CPUguy387

Hi,

I performed all the steps in the instructions before running the script, and everything worked great!  Now I'm trying to run your script, but it quits quickly with an error:

```

MATT-LAPTOP mstanisz # ./recompile-entire-system-20060901

./recompile-entire-system-20060901 - Recompile Entire System Helper

Script version as of $Date: 2006-09-01T14:15:49.548823Z $

Written in 2006 by Guenther Brunthaler

This script will generate another script to be run by you. That other script

will then recompile each and every package in the whole system in the correct

order.

This will typically be required on a major GCC upgrade.

IMPORTANT: Do not execute this script before all of the following prerequisites

are met:

* Portage tree is up-to-date (emerge --sync)

* Your system is up-to-date (emerge --ask --update --deep --newuse world)

* gentoolkit is available. (The script uses it.) If you are unsure, just do an

'emerge --update gentoolkit' and it will be emerged unless it is already

installed.

* The new compiler you want is already *installed*. (No packages need have to be

recompiled with it yet. It also need not be the currently selected default

compiler version yet.) As GCC allows multislot installations, it is not a

problem in Gentoo to have both your current and a new compiler be installed at

the same time.

* If all the above conditions are met, and no more packages need to be compiled

in order to have an up-to-date system, set the new compiler as the new system

default compiler using "gcc-config".

* If you want to change your Gentoo system profile to a new one using

eselect profile, it is now also the right time to do it.

* Only then continue running this script!

Press [Ctrl]+[C] now in order to abort processing if some of the above

preconditions are not met. Come back and re-run this script after you have

managed to establish all the preconditions as specified.

Press [Enter] now to continue if you dare.

Collecting list of packages and evaluating installation order...

Died at ./recompile-entire-system-20060901 line 450.

MATT-LAPTOP mstanisz # 

```

Could you help me?  Thanks!

Matt

----------

## Guenther Brunthaler

 *KD-120RD wrote:*   

> I have a question here: What is the purpose of switching the profile?

 

A Gentoo profile sets lower and upper margins for the versions of some important tools (in the list of installed "system" packages), such as the GCC.

Packages which require a compiler (or other system tool) outside of the version range defined by your current Gentoo profile will be masked. That is, "emerge --update" will skip such versions. And "emerge --search" or "esearch" (if installed) will not show such versions to you.

 *KD-120RD wrote:*   

> I thought gentoo is a meta-distribution. Just update your packages and configs (ie. baselayout) and there you are.

 

This is correct. But the catch is that updates will just not be performed for packages which are outside the scope of your current profile...  :Wink: 

But what are profiles good for at all?

Profiles protect users from changes that are so dramatic, that their entire system (or at least large portions of it) would have to be recompiled.

One such a change would be an upgrade from GCC 3 to GCC 4.

Without profiles, as soon as GCC4 has been marked as "stable", all Gentoo users running "emerge --update" would find themselves recompiling their whole system, which takes days if not weeks to complete.

Certainly, not all users would be happy then...

In order to avoid this, there are profiles: They allow the users to decide themselves when it is the right time to take such a big step and update their profile to the latest one. (And my guide might be very helpful in this case.)

 *KD-120RD wrote:*   

> Besides I'm using gentoo for about two years now and as I recall I have not manually swicthed my profile.

 

You are not alone! I am also still using the 2006.0 profile. I actually used my guide to update from GCC 3.3 to GCC 3.4 - and not to GCC 4. But it's all the same game.

 *KD-120RD wrote:*   

> When I take a look at my /etc/make.profile it is linked against  ../usr/portage/profiles/default-linux/x86/2006.0/
> 
> Is that possible?

 

It is perfectly possible.

In fact, nearly everyone who has not yet updated to this newest 2006.1 profile will see the same.

Note: You can update your profile by using "eselect profile".

----------

## Guenther Brunthaler

 *grimm26 wrote:*   

> i don't understand the significance of my system profile matching my kernel.  It's just default use flags and stuff, right?

 

The dependency between the system profile and your kernel is the linux-headers package.

This package contains a copy of selected header files which are taken from some linux kernel version (at the time the respective linux-headers package was assembled).

The header files in the linux-headers package are then used as the basis on which your glibc will be built.

The most important information which glibc gets from linux-headers is the syscall numbers.

Have you ever wondered how glibc, being a user-mode library, can actually output a character on your console - which typically belongs to the system, not to you (your user account)?

Glibc implements its write() function by stuffing the write() arguments into processor registers, loading the syscall-number for "write" into another register, and then performing an "int 0x80" processor instruction.

This instruction transfers control to the Linux kernel, which will verify the parameters and perform the write operation if your process is allowed to do so.

And the "syscall"-Number comes from one of the header files in linux-headers.

Now you can also see the relation between your current kernel version and glibc: If the version of the linux-headers package and your actual kernel drift too far apart, it may be that your new kernel provides syscall numbers that are not yet part of your (older) linux-headers package.

Admittedly, syscall numbers nearly never change, and new syscall numbers are also added only very rarely.

But *sometimes* they change, and then the linux-headers (and thus glibc) must be updated in order to take advantage of this.

Besides this, there are also different things in linux-headers which might be of interest for application programs (via glibc): IOCTL numbers. As more and more devices are supported by the Linux kernel, more and more IOCTL numbers (and related structs) are also added to the linux-headers.

And applications which want to use such IOCTLs might thus require newer versions of the linux-headers package.

On the other hand, it certainly would be a problem if linux-headers referred to a newer version of the Linux kernel than is actually installed on the machine.

In this case, the header files would contain IOCTLs that are not yet supported by your current kernel.

So, profiles are again the solution: The profile defines the *oldest* linux kernel that must be available, and linux-headers will have copies from the header files of that kernel.

As the profile ensures that your actual kernel will be at least as current as the headers in the linux-headers package, those header files will never contain anything your kernel does not support.Last edited by Guenther Brunthaler on Tue Sep 19, 2006 9:48 pm; edited 1 time in total

----------

## Guenther Brunthaler

 *CPUguy387 wrote:*   

> 
> 
> ```
> 
> Collecting list of packages and evaluating installation order...
> ...

 

Hmmm - this is very strange: Line 450 closes a the pipe which was used for reading the packages... For READING!

I could understand if there was an error OPENING it - but closing it?!

 *CPUguy387 wrote:*   

> Could you help me?  Thanks!

 

I'll try my best.

As I am unable to figure out the cause of the problem, you need to provide me with more information.

It may be easiest to perform some of the actions in the script manually, so we can find out which of the steps does not work.

First see whether the following commands work:

# emerge -pe system > system.list

# emerge -pe world > world.list

When running these commands, they should not print any error messages.

Then, do a

# less system.list

and after that a

# less world.list

and see if both files contain a list of packages, or whether there are obvious problems in the contents of those files.

My script assumes both files are non-empty contain lists of packages.

----------

## Guenther Brunthaler

 *jure1873 wrote:*   

> This script seems very interresting to me, but how much does this approximately take?

 

It takes bloody ages.

However, it is still the fastest way to recompile your entire system: The other guides (I am aware of) take even bloodier ages...  :Wink: 

 *jure1873 wrote:*   

> gcc probably recompiles everything so this is basically a stage1 install?

 

Yes.

 *jure1873 wrote:*   

> Could I grab a newer gentoo cd and get binary packages for some of the files from that cd and recompile the other files to make it faster?

 

It depends on whether or not you

1.) Want to use USE flags which are different from the ones used by compiling the Gentoo CD.

2.) Want to use different compiler flags than were used for compiling the Gentoo CD.

For instance, I have an Athlon-1333, and wanted to use the following compiler flags in my make.conf

```
CFLAGS="-march=athlon-tbird -O3 -DNDEBUG -pipe -fomit-frame-pointer -fno-stack-check"
```

in order to fully optimize Gentoo for my machine.

If want you use such settings, which are quite different from the ones used by compiling the Gentoo CD, you have to recompile everything anyway, so using a stage 3 CD will not help.

If you are happy with the defaults used on the CD, however, I certainly would be much faster to make a stage 3 reinstall. (But I have tweaked and twisted my installation so much, that I won't be happy with what has been compiled on the stage3 CD.)

----------

## jure1873

 *Guenther Brunthaler wrote:*   

> 
> 
> It takes bloody ages.
> 
> 

 

that's what I thought...  :Sad:  I'll think a little more before doing this. The problem is that while this is being upgraded it's wise if X is not running, but I need my computer almost every day so I think I'll wait some more time for this. I'll check how much free space do I have for another chroot and how could I grab some stuff from the packages cd.

----------

## CPUguy387

Thanks for the reply.  

My system.list generated fine with no apprent errors.

It turns out that in the world.list file, it was showing that ati-drivers and ati-drivers-extra were masked for some odd reason (probably used ACCEPT_KEYWORDS instead of adding it to my package.keywords file!).  I added them to my /etc/portage/package.keywords file and then it generated world.list.

I had a block at the top of my list.  Pam login was blocking shadow.  To fix that, see this thread: https://forums.gentoo.org/viewtopic-t-443022.html, or the short answer, run this command:

 *Quote:*   

> To solve "emerge -C pam-login && emerge -1 shadow", this is safe to do

 

Now my system.list and world.list files are correct, and when I ran your script, it generated the script to recompile everything, so I'm back on track.

Thanks Guenther!

Matt

----------

## Erlend

 *Quote:*   

> 
> 
> I strongly recommend running this script from the text mode interface, without an X-server running. There are too many things that can go wrong if trying to replace an X server by a recompiled version while it is running. (But on the other hand, I have never tried it. Perhaps it may actually work. But I would not count on it.) 

 

I've always recompiled xorg from within xorg and I don't remember having any problems - I'm not saying you won't have problems just that any problems I had were so minor I never noticed.

Why is the script more efficient than just doing "emerge -e world" once after upgrading to gcc 4.1.1?

Is it possible, in general, to tell what compiler version a package was compiled with?  I already compiled some packages with emerge -e system (as per http://www.gentoo.org/doc/en/gcc-upgrading.xml) before reading about this script, but your script tries to compile these again (at least as far as I can see)... I think it would be useful if it wouldn't.

----------

## Guenther Brunthaler

 *Erlend wrote:*   

> I've always recompiled xorg from within xorg and I don't remember having any problems - I'm not saying you won't have problems just that any problems I had were so minor I never noticed.

 

That's good news!

But - what Desktop Environment are you using?

I suspect some environments to be more fragile than others when recompiling them while they are running.

 *Erlend wrote:*   

> Why is the script more efficient than just doing "emerge -e world" once after upgrading to gcc 4.1.1?

 

Well - at least it will not recompile GCC a second time  :Smile: 

But to be a bit more serious: Just go and consider the output of

```
# emerge -ep world | less
```

Now notice the top entries of the generated list. On my system this are:

```

[ebuild  N    ] sys-libs/zlib-1.2.3  USE="-build" 415 kB

[ebuild  N    ] sys-libs/gpm-1.20.1-r5  559 kB

[ebuild  N    ] sys-libs/ncurses-5.5-r2  USE="gpm unicode -bootstrap -build -debug -doc -minimal -nocxx" 2,259 kB

[ebuild  N    ] app-shells/bash-3.1_p16  USE="nls -afs -bashlogger -build" 2,514 kB

[ebuild  N    ] sys-libs/readline-5.1_p4  1,986 kB

[ebuild  N    ] virtual/libiconv-0  0 kB

[ebuild  N    ] sys-devel/gettext-0.14.5  USE="emacs nls -doc -nocxx" 6,939 kB

[ebuild  N    ] sys-apps/diffutils-2.8.7-r1  USE="nls -static" 1,037 kB

```

and finally, in line 75:

```
[ebuild  N    ] sys-libs/glibc-2.4-r3  USE="nls nptl nptlonly -build -glibc-omitfp -hardened -profile" 0 kB 
```

That's why. The order is not optimal. glibc should not be the 75th package to be compiled, but one of the first ones.

Basically, my script does nothing else than run an emerge -e world, only that it reorders items, and excludes GCC from the list, because GCC has already been installed.

"Is 'emerge -e' flawed then?" one may ask.

No. But "emerge -e" assumes there is absolutely nothing there when it starts its job.

Which is not the reality if you are running an emerge from an installed gentoo System.

For instance, you may already have the new GCC, binutils etc. installed.

Exploiting this fact, you can optimize the order in which "emerge -e" does things, and even skip some of the packages (GCC).

Besides that, my script is identical to "emerge -e" world.

It does nothing what you couldn't do manually. (In fact, I did all the steps manually before writing the script.)

So my script is only by one packet faster than what "emerge -e" does.

However - "emerge -e" has not yet done the full job!

Remember those 75 packages that have been compiled before glibc was updated?

"Just to be sure" one should recompile those 75 packages again after the emerge finishes, so that all of them are linked against the new glibc (although I doubt this will be necessary... but that's another story).

My script compiles glibc as the first package (emerges it as the second package; the first package is only header files), and thus makes any this sort of headache obsolete from the beginning.

 *Erlend wrote:*   

> Is it possible, in general, to tell what compiler version a package was compiled with?  I already compiled some packages with emerge -e system (as per http://www.gentoo.org/doc/en/gcc-upgrading.xml) before reading about this script, but your script tries to compile these again (at least as far as I can see)... I think it would be useful if it wouldn't.

 

Well, that's the problem.

Actually, it is not even necessary to recompile all packages which have been compiled with the old compiler, because the C ABI is pretty stable (see my posting with the explanations for my guide for more of this).

It would be sufficient to recompile only those source files, which have been written in the C++ language by the old compiler.

Unfortunately, it's hard if not even possible to find that out in a safe, automated way (see also my explanation posting).

So we better recompile everything, just to be sure not to miss one of those C++ sources.

Plus we have the benefit that then everything is compiled using the presumably better code generator of the new compiler.

----------

## grimm26

re system profile and glibc.  So, I should recompile glibc after setting my system profile?  Also, when I did an eselect profile list, it didn't show one as selected.  It also did not show 2006.1/server as a choice, only desktop.

----------

## dalek

It doesn't show up here either but it is there.  It is here:  /usr/portage/profiles/default-linux/x86/2006.1/server  In the same place as desktop no less.

What's up with that??

 :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy: 

----------

## Guenther Brunthaler

 *grimm26 wrote:*   

> re system profile and glibc.  So, I should recompile glibc after setting my system profile?  Also, when I did an eselect profile list, it didn't show one as selected.  It also did not show 2006.1/server as a choice, only desktop.

 

Then your system seems not to be up to date, or something is screwed. When I run

```
$ eselect profile list

```

I get

```
Available profile symlink targets:

  [1]   default-linux/x86/2006.1

  [2]   default-linux/x86/no-nptl

  [3]   default-linux/x86/no-nptl/2.4

  [4]   default-linux/x86/2006.1/desktop

  [5]   hardened/x86

  [6]   hardened/x86/2.6

```

So you see it's profile # 1 on my box.

In order to be sure your system is really up to date, run the following

```
emerge --sync && emerge --update --deep --newuse world && revdep-rebuild
```

It that won't help then I'm also clueless...

----------

## Erlend

 *Guenther Brunthaler wrote:*   

>  *Erlend wrote:*   I've always recompiled xorg from within xorg and I don't remember having any problems - I'm not saying you won't have problems just that any problems I had were so minor I never noticed. 
> 
> That's good news!
> 
> But - what Desktop Environment are you using?
> ...

 

I'm currently using Xfce.  I previously used Gnome.

 *Guenther Brunthaler wrote:*   

>  *Erlend wrote:*   Is it possible, in general, to tell what compiler version a package was compiled with?  I already compiled some packages with emerge -e system (as per http://www.gentoo.org/doc/en/gcc-upgrading.xml) before reading about this script, but your script tries to compile these again (at least as far as I can see)... I think it would be useful if it wouldn't. 
> 
> Well, that's the problem.
> 
> 

 

Actually I think it could be done if you ask the user when they changed their compiler to (say) gcc 4.1.1.  Just look up emerge.log for any packages done after this date (although they would probably not be linked against the new glibc if you do this).

 *Guenther Brunthaler wrote:*   

> 
> 
> Plus we have the benefit that then everything is compiled using the presumably better code generator of the new compiler.

 

Off topic, but I can't wait until icc works as a drop in replacement for gcc (if ever)!

----------

## Naib

talk abt postcount++++++++

----------

## Erlend

Hey I just happened to spot your script re-emerging =sys-libs/libstdc++-v3-3.3.6 ... surely I don't need that program any more?

Thanks,

Erlend

----------

## Guenther Brunthaler

 *Erlend wrote:*   

> I'm currently using Xfce.

 

Hmm that's also one of the leaner ones. The odds of "surviving" a recompile without crashing are certainly quite better with such a DTE than for KDE!

Furthermore, can you tell whether Xfce is written in C++ or C?

The maximum damage is done only if C++ Shared Objects are heavily used (as plugin DLLs).

I also assume Gnome to be much less effected by a recompile than KDE for that reason.

 *Erlend wrote:*   

> Actually I think it could be done if you ask the user when they changed their compiler to (say) gcc 4.1.1.  Just look up emerge.log for any packages done after this date (although they would probably not be linked against the new glibc if you do this).

 

Yes, that might actually work.

And it can rather easily be included within my guide: After the recompilation script has been generated, remove any lines from it which refer to packages found in the output of emerge.log since the new compiler was emerged and was made the system default compiler. (Because otherwise it will not have been used although it was installed.)

A small Perl/Python/Ruby/awk script will do.

Unfortunately, it will only work for people who have enabled portage logging - it's still not the default, I'm afraid. (Is that correct?)

 *Erlend wrote:*   

> Off topic, but I can't wait until icc works as a drop in replacement for gcc (if ever)!

 

Arrrgh! A non-free compiler... <paranoid>Will I ever trust code generated by such a compiler? Who knows what it embeds into my object code? 50 % Intel code, 50 % NSA code? Shiver...</paranoid>

Don't mind!  :Smile: 

----------

## Lloeki

it took me 2 days, but my kde laptop survived while using it extensively at work. I even hibernated 2 times.

sure kde is plenty of c++, but once a shared lib is loaded, it stays in mem until unused, so you can safely use your computer while emerging world. just be sure not to close c++ programs you wish to use (and start them beforehand of course). then, at the end of the upgrade, restart X, or better even, reboot, so the previous libs are flushed and latest libs reloaded.

I basically didn't noticed it up until I accidentally closed kmail, which then complained about the ABI mismatch. but who cares, I waited next day to check my mail again. just a fine, hassle-free upgrade.

----------

## Guenther Brunthaler

 *Lloeki wrote:*   

> just be sure not to close c++ programs you wish to use (and start them beforehand of course). then, at the end of the upgrade, restart X, or better even, reboot, so the previous libs are flushed and latest libs reloaded.

 

This sounds very promising, indeed.

I have taken the liberty to update my guide accordingly and refer to your post!

Thanks a lot for sharing your experience with us!

----------

## Lloeki

you're welcome.

do you know about app-portage/genlop?

I didn't use your script, but  I thought it would be a nice addition to your script to output the result of 'emerge -p `cat list_of_packages` | genlop -p' (i.e total estimated merge time) before actually running 'emerge `cat list_of_packages`', and have a yes/no question like 'emerge -a'.[/url]

----------

## Guenther Brunthaler

 *Lloeki wrote:*   

> do you know about app-portage/genlop?

 

Well, all I was using it for until now was

```
genlop -l | tail -30
```

and

```
genlop -c
```

I skimmed quickly over the other options when first reading the manual, but didn't notice anything else which caught my immediate interest.

However, I'll give it another try now (after having read your comments).

 *Lloeki wrote:*   

> i.e total estimated merge time

 

Actually, when writing my script I was thinking about some "ETA" display. But then I realized that date/time arithmetic is a bit outside the scope of a humble shell script, and so I let it be.

I was also uncertain how to best calculate that ETA, because my script is incremental and may be stopped and restarted at any time.

Futhermore, users may run different processes during the running time of my script, which makes it hard to give a good estimate.

But ultimately I was just too lazy to do it!  :Wink: 

----------

## vicaya

Interesting. But I'd recommend install ccache (make sure it's enabled in FEATURES in make.conf) and do it the usual way: emerge -e system and then world. It's probably about 30min longer than the "optimal" script. But you'll be 100% sure, everything is done correctly.

----------

## devsk

 *Guenther Brunthaler wrote:*   

> 
> 
> I was also uncertain how to best calculate that ETA, because my script is incremental and may be stopped and restarted at any time.

 you need to pipe the output of emerge -p (or equivalent generated by your script) to genlop -pt. Try this:

```
emerge -pe system|genlop -pt
```

----------

## Guenther Brunthaler

 *vicaya wrote:*   

> Interesting. But I'd recommend install ccache (make sure it's enabled in FEATURES in make.conf) and do it the usual way

 

Without question, ccache is a great tool. I'm using it all the time.

And it works perfectly with my script. (At least I did not encounter any problems. However, I manually cleared the cache of ccache after using gcc-config to set the new compiler. Just to be sure there are no cached object files left originating from the old compiler.)

 *vicaya wrote:*   

> emerge -e system and then world. It's probably about 30min longer than the "optimal" script. But you'll be 100% sure, everything is done correctly.

 

I that only were so easy ... I would never needed to write my script.

If you just do a "emerge -e world", you will eventually find yourself with a recompiled system where about 75 packages have been linked against the old version of glibc ... see https://forums.gentoo.org/viewtopic.php?p=3555009#3552919 for more of this.

So, if you want to be "100 % sure", "emerge -e world" won't provide you with what you want.

My script does the same as an "emerge -e world", only in the correct order for recompiling a system.

Also note that dispite of this, "emerge -e world" is neither flawed nor broken! It's just not intended to be used for recompiling the whole system, but for rebuilding a non-existent system from scratch. Which is not the same!

My script, in contrary, has been designed for recompiling your entire system.

So it does a similar, but not the same job.Last edited by Guenther Brunthaler on Mon Sep 04, 2006 9:01 pm; edited 2 times in total

----------

## Erlend

 *vicaya wrote:*   

> Interesting. But I'd recommend install ccache (make sure it's enabled in FEATURES in make.conf) and do it the usual way: emerge -e system and then world.

 

ccache won't help in this instance because the compiler version has changed.

 *vicaya wrote:*   

> It's probably about 30min longer than the "optimal" script. But you'll be 100% sure, everything is done correctly.

 

You won't be 100% sure, because as stated above 70-or-so packages won't be linked against the newly compiled glibc... this is why some people say you should do

```
emerge -e system

emerge -e world

emerge -e world
```

The optimisation this script makes is to place glibc at the beginning of the compile run, so everything thereafter is compiled against it and running "emerge -e world" twice is definitely unnecessary.

 *Guenther Brunthaler wrote:*   

> 
> 
> Yes, to the environment! Thousands and thousands of Gentoo users, all wasting weeks worth of processing power when following inefficient "recompile entire system"-style compiler upgrade guides, will waste an enormous amount of kilowatt hours of electrical power. 
> 
> 

 

How much more electricity does it use?  The processor will go to 100%, and I guess the hard drives will thrash?  Hard drives are always spinning though, so I don't think they'll really use significantly more power whem read/writing - it'll just kill the disks.  The fans will probably be going quite fast too?

----------

## Erlend

 *Quote:*   

> Also note that dispite of this, "emerge -e world" is neither flawed nor broken! It's just not indended to be used for recompiling the whole system, but for rebuilding a non-existent system from scratch. Which is not the same! 

 

There should be some way of doing what your script does from within portage... like emerge --rebuild-upgrade world.

----------

## Guenther Brunthaler

 *Erlend wrote:*   

> Hey I just happened to spot your script re-emerging =sys-libs/libstdc++-v3-3.3.6 ... surely I don't need that program any more?

 

Well, my script recompiles any package that is installed at the time it is generated. It doesn't care about whether the packages are up-to-date or outdated. (That's why I want users to run "emerge --update --newuse --deep world" first, before starting my script.)

So it seems your package is simply left from an old compiler and never has been uninstalled.

OK, now it is too late... but as soon as all is finished, run a

emerge --ask --depclean

and see what portage suggests for uninstallation. (And don't forget to run revdep-rebuild in order to fix any packages which might have been broken by those uninstalls.)

Another common problem are old slot installations.

For instance, GCC is not replaced when you emerge a newer version, but installed additionally.

Same for gentoo-sources, by the way.

If you want to see what slots of a package are actually installed, try

```
equery list gcc

equery list gentoo-sources

```

Then use "emerge --unmerge" to uninstall the slot versions you do not need. But be sure to specify the exact version number for the unmerge, or else the newest version will be removed instead!

----------

## Erlend

 *Quote:*   

> 
> 
>  *Quote:*   
> 
> Hey I just happened to spot your script re-emerging =sys-libs/libstdc++-v3-3.3.6 ... surely I don't need that program any more? 
> ...

 

Sorry I meant in general... I don't think I need libstdc++v3 any more - I think I emerged it to fix any problems that might occur in the 3.5 -> 3.6 upgrade.

----------

## Guenther Brunthaler

 *Erlend wrote:*   

> You won't be 100% sure, because as stated above 70-or-so packages won't be linked against the newly compiled glibc...

 

Yes!  :Smile: 

 *Erlend wrote:*   

> How much more electricity does it use?

 

It certainly depends on the processor and its power-saving capabilities.

And also on the OS.

Unless disabled, Linux will perform a HLT instruction on the processor when no processes need running.

Then the processor "sleeps" until the next interrupt, requiring only very little power.

But when running under full load, such as recompiling QT libraries - I'm quite sure the processor needs much more power then, in comparison to "sleeping". (You can even hear it work when listening to the variable fan noise.)

And also don't forget: Some people turn off their machines at night! But for the updates they have to let them running for a week or so. Certainly this also uses more power than usual.

 *Erlend wrote:*   

> and I guess the hard drives will thrash?

 

At least they will if those horror-stories from GCC 4 are actually true - I've read reports where the machines were running out of swap space, because GCC4 needed more than 2 gigs to compile a single C++ file.

I can only hope those guys were exaggerating a bit... because one of my machines has only 512 megs.

But for now I'll stay with 2006.0 for a while - I have updated my GCC from 3.3 to 3.4 not more than two months ago - it's no fun recomiling everything again so soon ;-(

 *Erlend wrote:*   

> Hard drives are always spinning though

 

Yes, but there are 2 motors in a hard drive: The normal one

 which makes it spin, and the stepper motor. At least in older hard disks there was one. Newer hard disk have something else (I think its called "actuator" or similarly), but it certainly needs also more power if its moving the heads, than if not moving them.

 *Erlend wrote:*   

> The fans will probably be going quite fast too?

 

You can bet on that! Especially on my notebook here - "you can hear the sound"...  :Wink: Last edited by Guenther Brunthaler on Mon Sep 04, 2006 10:17 pm; edited 1 time in total

----------

## Erlend

 *Quote:*   

> At least they will if those horror-stories from GCC 4 are actually true - I've read reports where the machines were running out of swap space, because GCC4 needed more than 2 gigs to compile a single C++ file. 

 

Ah that explains it... my swapfile is going through the roof (and I can only remember 3-4 occassions when it's been used before).

But it's not just the swapping that'll use disk space, there's also the fast creation of hundreds of small files in /var/tmp, although I think PORTAGE_TMPFS="/dev/shm" in make.conf is meant to lower this.

----------

## Guenther Brunthaler

 *Erlend wrote:*   

> Ah that explains it... my swapfile is going through the roof

 

Then it seems those guys were actually not exaggerating... Which makes me rather unhappy.

I think I'll stay with 2006.0 for yet a long while! 

 *Erlend wrote:*   

> and I can only remember 3-4 occassions when it's been used before).

 

I remember when I had the same problem with an 128 meg machine.

What I did to attenuate the situation somewhat, was replacing the -j2 in the MAKEOPTS in /etc/make.conf by a -j1. The memory requirements of the emerges went down by a factor of 2 immediately.

 *Erlend wrote:*   

> hundreds of small files in /var/tmp, although I think PORTAGE_TMPFS="/dev/shm" in make.conf is meant to lower this.

 

Which won't help much in your situation, because /dev/shm is of type tmpfs, which is a RAM disk backed up by the swap device - which lets your system swap even more if there is too little RAM...

OK, enough for today. It's midnight here; I'm off till tomorrow. Cheers!

----------

## Lloeki

tip: 'emerge --resume -p |genlop -p' in another terminal will give you an ETA while emerging.

as for gcc4 rocket high swap usage, I have 1.5G ram and 3G swap (just to be sure, for hibernation), and as far as a I recall, it barely hit swap, even when compiling kde. 

and I was using kde/xorg + kdevelop + mysql + lighty + 6 prespawned fcgi php + some java (sun jdk 1.5) at the same time. smooth as butter.

so those 'horror stories' are a bit surprising to me.

for the record it took me two days to compile 700+ ebuilds on that pentium m dothan 1.6GHz w/1.5G ram and a 5400rpm udma5 ide hd. I just wonder on which half-decent hardware (anything >=pentium III) it would take weeks. or are those p-m that code-crunching-able ?

----------

## hazza

Cheers for this script Guenther - it's made updating the gaggle of servers we have here at work an absolute breeze!

IMHO this should become part of the official Gentoo docs as it's worked flawlessly on each of the fourteen machines it's been run on so far.

 *Quote:*   

> for the record it took me two days to compile 700+ ebuilds on that pentium m dothan 1.6GHz w/1.5G ram and a 5400rpm udma5 ide hd. I just wonder on which half-decent hardware (anything >=pentium III) it would take weeks. or are those p-m that code-crunching-able ?

 

I just started my laptop (Pentium M Dothan 1.7GHz, 512MB RAM) and workstation (Dual Athlon MP 1900+, 1GB RAM) doing this simultaneously. They both have near-as-damn-it the same software installed and the laptop is comprehensively out-compiling the desktop! You can clearly see that Pentium-M is the precursor to the current Core and Core-2 series...

All the best,

--Harry

----------

## Lloeki

Indeed I was always amazed to see how my 1.6GHz/mobility x600 fared in games like quake 4, fear and such, supposedly requiring a 2+GHz computer/insane graphics card... but had nothing to compare to (except a celeron m (which is the same minus some C states) and another pentium m)... you just enlightened me!

----------

## zxy

@Guenther Brunthaler

Great job. This script is the optimal solution I have found so far. 

@Erlend Look down and try the tip. With 850M of tmpfs almost everything execept openoffice gets compiled. If you put 1G for sure it's even safer. Swap is faster than tons of small files. And everything gets untared to RAM + swap. The gain using the tip is quite observable. Mostly things just fly-by.

----------

## Erlend

Yeah, I've read about that trick before but a lot of people say it doesn't really speed things up in their own tests.  I also only have 512MB RAM.  

Maybe I should use device-mapper...

run 

```
echo "0 $size1 linear /dev/shm 0 $size1 $totalSize linear /dev/hdaX 0" | dmsetup create compiledrive

mkfs.ext2 /dev/mapper/compiledrive

mount -t ext2 /dev/mapper/compiledrive /var/portage/tmp

```

This way you can have a 5GB /var/portage/tmp with the first $size1 (say, 300MB) as tmpfs, and the rest slower disk to allow larger compiles to work as normal.

The only thing that bothers me about compiling from source is that it can't be good for your hardware!  Possibly significantly shortens the life of your computer.

----------

## devsk

 *Quote:*   

> Because the amazing truth is: It is in fact UNNECESSARY to recompile any package, including the new GCC, more than exactly once when rebuilding your entire system!
> 
> 

 

@Guenther: There is a problem. BIG problem! The approach of doing emerge once will not work with the current portage. Consider this:

```

$ emerge -pe gpm

These are the packages that would be merged, in order:

[ebuild  N    ] sys-libs/ncurses-5.5-r3

[ebuild  N    ] sys-libs/gpm-1.20.1-r5

$ emerge -pe ncurses

These are the packages that would be merged, in order:

[ebuild  N    ] sys-libs/gpm-1.20.1-r5

[ebuild  N    ] sys-libs/ncurses-5.5-r3

```

As you can see gpm and ncurses are locked in a cycle, one depends on the other. If emerge -e gpm, were to upgrade both ncurses and gpm to a version which breaks ABI, after a single pass (i.e. after a single emerge -e gpm), your system is broken because ncurses is using older gpm. Either:

```
emerge -e gpm && emerge -e gpm
```

 needs to be done or portage needs to take care of the cycles like this:

```
$ emerge -pe gpm

These are the packages that would be merged, in order:

[ebuild  N    ] sys-libs/ncurses-5.5-r3

[ebuild  N    ] sys-libs/gpm-1.20.1-r5

[ebuild  N    ] sys-libs/ncurses-5.5-r3

```

 I don't know why portage doesn't do it this way, but that's how things are. This is just a small example (and probably a bad one). Much longer cycles exist in portage right now. Unless portage fixes it with the second approach, the only safe approach is to do 'emerge -e system' twice.

----------

## hielvc

Guenther Brunthaler here is the thread which I belive is granddady of the emerge wrapers aimed at 

1 If the TC, toolchain, is going to be modified, updated or re-built, do it first

2 break large emerges into smaller chunks. TC, system minus TC, world minus system minus TC

3 with an exclude option you can prevent building x11, gnome, or kde until later.

 An emerge wrapper for more correctly building the toolchain

Your explanation presented here is far better than mine and Im glad to see someone more knowledgeable present it. Thanks. 

 :Embarassed:   I mean here  Why multiple "emerge -e world" are actually useless

Heres a tip as a replacement to emerge --depclean world, " emerge udept " which is ecatmur's dep script. Here is the thread  Clean out your world file

Hiel

----------

## zxy

@Erlend I don't really want to mix it in this topic, but I used it on three different computers amd64, athlon, pentium II and it felt much better. My experience. Others might have different.

----------

## Guenther Brunthaler

 *devsk wrote:*   

> There is a problem. BIG problem! The approach of doing emerge once will not work with the current portage.

 

Basically, you are right.

But I used a few tricks in my script to avoid such problems...

 *devsk wrote:*   

> 
> 
> ```
> [ebuild  N    ] sys-libs/ncurses-5.5-r3
> 
> ...

 

First of all, cirular dependencies won't bother my script. Where circular dependencies exist, an arbitrary order will be chosen. (Actually, this will be decided by "emerge -e world" itself, not by my script.)

But the ultimate trick is that all my emerges use the following scheme: Instead of doing

```
$ emerge ncurses
```

my script will do a

```
$ emerge --oneshot --nodeps ncurses
```

Which means nothing else than to completely disable Portage's dependency tracking calculation!

When used this way, emerge will do exactly as commanded, and not try to resolve dependencies by itself.

That is, it will never detect or warn about what it otherwise may think to be missing or circular dependencies.

Also note that we are recompiling the system, which means both packages are already there, although perhaps not yet in a recompiled version.

But that won't affect the build process: All header files etc. of both packages are already there and installed, so the build process of either package can find the header files of the other package and will thus compile successfully - no matter in which order both packages are recompiled.

At runtime, problems were possible at that moment: When recompiling the first package, the shared libraries of the second package are still the ones compiled with the old compiler.

But: Who cares?

We are just recompiling the packages, not running them!

And at the time when my script finishes, both packages will have been recompiled - no matter in which order they have actually been recompiled.

You see - although it may look so, there is not really a problem.

----------

## outspoken

Your guide should read more like this:

 *Guenther Brunthaler wrote:*   

> 
> 
> [*] If all the above conditions are met, and no more packages need to be compiled in order to have an up-to-date system, set the desired new system compiler as the new system default compiler using "gcc-config". (This will make the new GCC 3.4 or GCC 4 the new system compiler.)
> 
> [*] For those who have never used gcc-config before: "gcc-config --list-profiles" displays a list of all GCC versions which are currently installed on your box. The compiler you are currently using is marked with an asterisk. In order to change this "current" compiler to a different one from the list, remember the number between the brackets on the left side of the line which contains the compiler version you want to use. Then use "gcc-config <number>" to change the system default compiler to the list entry <number>, where <number> is the the number you remembered in the previous step.
> ...

 

I switched the gcc-config to be in front of the emerge. following your guide step-by-step led me to a brick wall in the pre-requisites when trying to emerge glibc 2.4. I found this link. which helped me learn that I needed to switch my gcc-config and resource my /etc/profile.

----------

## Guenther Brunthaler

 *hielvc wrote:*   

> Guenther Brunthaler here is the thread which I belive is granddady of the emerge wrapers aimed at 
> 
> 1 If the TC, toolchain, is going to be modified, updated or re-built, do it first
> 
> 2 break large emerges into smaller chunks. TC, system minus TC, world minus system minus TC
> ...

 

Yes, steps 1 and 2 are a rather accuracte description what my script is actually doing, I only determine things in the reverse order: Instead of starting with the TC, I start with the output of "emerge -ep system" and "emerge -ep world".

Then I remove the items which are both in world and system from world, which means all from system comes first, followed from what is still missing from world.

And then I'm moving the TC packages to the front of the system list, effectively making it a TC prefix list.

So the effective order will indeed be exactly what you have described in step 2. (Except that I exclude GCC from the TC because it's up to date already. BTW, I do all the sorting using the standard sort command. The Perl script only parses the output of the emerges and puts them in a form suited for the sort commands. Oh yes and it helps moving the TC packages to the beginning of the list.)

Anyway, it really seems my script has got a granddaddy now!

Too bad I didn't find it when I searched for it; it could have saved me from writing its grandson...  :Wink: 

But I was so tired of permanently reading those "recompile your system a zillion times"-postings; I just had to do something fast before the big migration to GCC 4 started.

 *hielvc wrote:*   

> Your explanation presented here is far better than mine and Im glad to see someone more knowledgeable present it. Thanks.

 

I'm glad you liked it. I'm also happy to have found an obvious "brother in mind" (at least when relating to this issue).

And let's be glad that we are not at SCO here: We immediately had to sue each other...  :Wink: 

Because if there is a right way to do things, given enough time, everyone will eventually find that way.

 *hielvc wrote:*   

> Heres a tip as a replacement to emerge --depclean world, " emerge udept "

 

Thanks - I'll check it out soon!

----------

## devsk

 *Guenther Brunthaler wrote:*   

> 
> 
> First of all, cirular dependencies won't bother my script. Where circular dependencies exist, an arbitrary order will be chosen. (Actually, this will be decided by "emerge -e world" itself, not by my script.)

 what I am saying is that this arbitrary order is what will cause the problem. And its not your mistake but portage's. It needs to do the right thing (which is to emerge all pkgs except the last one in the random list generated from the cycle should be emerged twice i.e. 2*N - 1 emerges. You just can't bypass this if you want a sane system) and not just select a random order to merge. Note that 2*N - 1 is just for the cycles (of length N) and not for the whole system. So, an 'emerge -e system' which handles cycles will not have 199 pkgs to emerge if a single emerge -e system which doesn't handle cycles emerges 100 pkgs. It will probably do, say, 120 but that would be the safe thing to do for portage.

----------

## Bad Penguin

 *Guenther Brunthaler wrote:*   

> Because the amazing truth is: It is in fact UNNECESSARY to recompile any package, including the new GCC, more than exactly once when rebuilding your entire system! (For those who cannot believe this, see my argumentation in the related article https://forums.gentoo.org/viewtopic-p-3548628.html#3548628.)
> 
> Using my guide (and the accompanying utility script) presented here, you will be able to rebuild your entire Gentoo system with an absolute minimum of effort.
> 
> Compared to other guides which suggest recompiling the entire system up to 6 times "to be sure" (I will prove them wrong - their approach is not safer than my approach presented here, it only takes more time), my approach can save you days or even weeks of processing time!

 

I have read this thread and your arguments in the other thread.  Amazingly enough, I have yet to see any proof that what you are so articulately expressing is actually true.  I am very curious to know how you have proven your assertion that it not necessary (or safer) to recompile any package more than exactly once.  Would you be willing to bet your job on that assertion?  I will wait anxiously for your proof, I assume you will post it here?

As far as your second assertion, that your method is somehow faster, wrong again.  You ask the reader to do an "emerge --update --deep --newuse world" before running your scheme...  A complete waste of time if there is any toolchain update involved in complete rebuild of the system.

As I detailed here, the quickest and safest method is to start with a stage1, bootstrap, then emerge -e system.  This method is quicker because the bootstrap builds the toolchain with a minimal set of USE flags, which is slightly faster than doing the equivalent "emerge -e system && emerge -e system".

If you want to quickly and safely upgrade your box, with resorting to idiotic methods like revdep-rebuild and fix_libtools.sh, build from stage1 in a chroot with FEATURES="buildpkg", then upgrade your existing box from the binary packages.  I have yet to see a tested, documented method to refute my conclusion.

----------

## Erlend

What would be faster is not reinstalling packages that don't need recompiling... like ut2004, q3demo, java applications...

----------

## Guenther Brunthaler

Hi devsk,

I'm afraid I can still not see the problem!

I won't allege there is none, but only that I don't understand what should actually break.

 *devsk wrote:*   

> 
> 
> As you can see gpm and ncurses are locked in a cycle, one depends on the other. If emerge -e gpm, were to upgrade both ncurses and gpm to a version which breaks ABI, after a single pass (i.e. after a single emerge -e gpm), your system is broken because ncurses is using older gpm.

 

How can the recompilation order break the ABI?

Don't forget: We already have the up-to-date versions of both packages installed when the script runs. There are no "old" versions around, and after recompilation there will be the same versions as before.

They will then only be compiled by a different compiler both.

But all the header files, library names - just all which could break compilation - is after the recompilation exactly the same as before.

All except for the contents of the shared libraries themselves, which however are dynamically linked. Which means they are only loaded at runtime, not at link time.

Hmmm.

Is this right?

I admit, I haven't tought too much about that yet.

In my own programs, I was always using static libraries.

But how is linking done in shared libraries?

Is it necessary to access the shared library in order to obtain the list of symbols it provides, or are there other sources of information the linker has access to?

Frankly, I still know too little about how the linker actually works to answer that.

So - perhaps there might indeed be the potential for ABI breakage, if it's that what you meant.

But luckily for my script (and all those who were already using it), this won't have any effect in this particular case:

gpm and ncurses are both C libraries, so they won't be affected by the compiler switch at all! (See my explanation posting https://forums.gentoo.org/viewtopic-p-3548628.html#3548628 for more details why.)

It is actually even unnecessary to recompile any of them; this is only necessary for C++ shared libraries and executables using them as such.

And even then there is only a problem if the header files do not export the declarations as extern "C".

The only reason why still all packages are recompiled, is the problem of determining this automatically. (And of course the new GCC will generate better code, so recompilation might be a good thing anyway.)

So, as long as there are no cycles in one of the "system" packages which generate shared C++ libraries, the problem does not apply anyway.

BTW: Is there any C++ shared library in the "system" list at all? (Except for the new GCC itself, which is the first package compiled anyway.)

Perhaps this even explains why such cycles still exist in the portage tree: The package maintainers know that the compilation order cannot break the ABI, because no C++ is involved. And so they just don't care.

----------

## devsk

gpm/ncurses was just an example... :Smile:  longer and more complicated cycles (involving some c++ packages e.g.) can be found by perusing the output of 'emerge -pe' of some of the more complex packages. e.g. try looking at emerge -pe qt and emerge -pe cups with both qt and cups use flags. 'emerge -uDN world' before doing your thing helps, but it doesn't completely eliminate the possibility of runtime breakage. Its a portage deficiency, and people work around it with couple of ways e.g. some do what Bad Penguin does, while others are content with emerge -e twice. For me, I would like to see portage cycle handling improved before I jump onto your method.

----------

## devsk

expat is not a c++ library (it is part of the 'system') but it famously broke many systems single handedly (including 2 of mine, there is a sticky somewhere here on forums)...So, its wrong to assume that only c++ progs can break ABI. A simple data structure change and a overlook from the developer can break ABI. And a person updating should not leave his system in an inconsistent state during such mistakes(which we are more prone to compared to lets say binary distro like fedora).

----------

## Guenther Brunthaler

Dear Penguin,

(can a Penguin be bad at all? Not for me. I like penguins!)

 *Bad Penguin wrote:*   

> I have read this thread and your arguments in the other thread.  Amazingly enough, I have yet to see any proof that what you are so articulately expressing is actually true.

 

But you are not the first one to find a flaw in my guide. Hey, I'm only human! I'm not perfect, everyone can make as mistake and I'm no exception. That's why we discuss ideas like this in the forums, don't we?

For instance, devsk has already pinned down a potential problem related to circular dependencies, although it seems that (by coincidence I must admit) my script will not be affected by it. See his posting for more.

Regarding your desire for a proof (my exact words were "see my argumentation", not "see my proof", btw), I'm afraid there is none.

I'm not a mathematician. Proofs are not my domain. I have not even an idea how to prove the correctness of a computer program with as far-reaching consequences as the application of my script. Perhaps if I wrote it in ADA instead as a BASH script...  :Wink: 

So, all I could do, and have done, was to write down my ideas and arguments and let others review it.

See it not as a proof, see it as a theory.

And a theory can be proven wrong.

So, if you found a flaw in my guide, it's OK.

I'll carefully consider your arguments, and if you found a way how to make my guide work better or fix some bugs in it, I'll be pleased to improve it accordingly (and to mention your contribution, of course).

 *Bad Penguin wrote:*   

> As far as your second assertion, that your method is somehow faster, wrong again.

 

Here we obviously have a simple misunderstanding: When I wrote  *Quote:*   

> Compared to other guides which suggest recompiling the entire system up to 6 times

  I didn't mean I my guide was faster than all the guides which may exist, only faster than those which recommend excessive recompilation of the system over and over.

Which, by the way, was the primary motivation for writing my guide: 6 times was too much!

So I started to think "but how often then?"

And the more I was thinking it over, the fewer the less it got.

Until finally I reached the point where only a single recompilation pass was left!

 *Bad Penguin wrote:*   

> You ask the reader to do an "emerge --update --deep --newuse world" before running your scheme...  A complete waste of time if there is any toolchain update involved in complete rebuild of the system.

 

Yes, you are right.

Actually, I considered only recommending doing an "emerge --update --deep --newuse system" at first.

However, I finally decided to stick with the "world" variant.

The reason is that the system and all installed libraries are in a consistent state then.

As you have read my argumentation and how my script works, you might have noticed that it builds a list of all packages to be recompiled when the generated build script is run.

This list countains all the packages which will be rebuilt in the correct order, including any dependencies.

But if the system was not up to date at the moment when the script was generated, then outdated package versions might have become be part of the list. Which I wanted to avoid, because then those outdated versions would also be recompiled in the rebuild phase of my guide, wasting even more time.

I also assumed that most users willing to update their GCC also update their systems regularly, which means an "emerge --update --deep --newuse world" won't have to update too many packages anyway.

And users who won't care about updates at all will also very likely ignore my guide anyway.

 *Bad Penguin wrote:*   

> the quickest and safest method is to start with a stage1, bootstrap, then emerge -e system.

 

I won't deny that it is the safest way.

And there are good chances it will even be the only way (besides installing a cross TC) to do a global update if the "C" ABI should ever change (that is, on a switch from glibc 2.x to 3.x).

My script won't work in such extreme situations. (Like all the the other scripts I've seen.) Only your script, plus the cross TC variant, would work then.

But an update like the 2006.1 switch is mere overkill for the power of your script, where my simple guide is obviously sufficient for such minor cases.

I also suppose doing a stage 1 reinstall unless absolutely necessary will seem to be a bit extreme in the opinion of most users (but I may be wrong).

Nevertheless, I absolutely agree that your script will work in a larger number of possible scenarios than mine.

But will it also be necessarily the quickest way to do the job?

I'd say: It depends.

OK, the "update world" is something your script can avoid. But, as outlined above, I think that overhead will be tolerable on the average and only a few packages, if any, will actually be emerged.

On the other hand, if the system is already up to date, then this step has nearly zero overhead.

 *Bad Penguin wrote:*   

> This method is quicker because the bootstrap builds the toolchain with a minimal set of USE flags, which is slightly faster than doing the equivalent "emerge -e system && emerge -e system".

 

(I assume you meant "world" not "system" in your second emerge. Because if you actually emerged system twice, my guide would be definitively faster. Well, unless that unnecessary overhead triggered by my emerge --update happens to be the openoffice package...)

But what do you mean when you write "minimal USE flags"? If you mean that there will not be any SSE, MMX etc use flags set when the TC was compiled into the stage 1 as shipped, then yes.

There will also be no compiler flags tuned to your CPU, i. e. no "-march". The stage 1 compiler will instead be shipped as built with some generic compiler flags.

That's all correct.

But why should such a compiler actually be faster than the compiler emerged at the beginning of my guide, which fully honors all MMX/SSE flags and also uses the full optimization settings as defined in /etc/make.conf?

And from then on, my compiler has to recompile all remaining packages in the system exactly once, according to my guide.

How can your script, starting with a slower compiler, do the same in a faster way?

So your script/guide has to recompile at least exactly the same number of packages as my guide does.

The "overhead" of my guide is the initial "update world", and the overhead of your guide is the overhead of a stage1 reinstall.

Mine is thus somewhat easier to apply, while yours has the bonus of even working in especially rare situations where my guide as well as most others will fail.

So, from my point of view, the guides are pretty much equivalent in their overall performance, but with different focuses.

I depends on the situation and user preference which one to actually use.

----------

## Guenther Brunthaler

 *devsk wrote:*   

> Its a portage deficiency, and people work around it with couple of ways e.g. some do what Bad Penguin does, while others are content with emerge -e twice. For me, I would like to see portage cycle handling improved before I jump onto your method.

 

Hmmm. And what about simply running revdep-rebuild at the end of the recompile? Shouldn't that track down the broken libraries and recompile the affected packages automatically?

Does revdep-rebuild only check for the presence of referenced libraries, or also for the presence of the referenced entry points?

In the latter case, this should solve our problem.

If you agree, I'll just add that to the end of my guide then, and we'll all be safe again.

----------

## devsk

revdep-rebuild can check only the library linkages.

but I agree, a revdep-rebuild at the end will further solidify your method. (but still not make it bullet proof... :Wink:  )

----------

## Guenther Brunthaler

 *devsk wrote:*   

> wrong to assume that only c++ progs can break ABI. A simple data structure change and a overlook from the developer can break ABI.

 

Don't you rather mean API than ABI? (Structures have typically no effect on the ABI because they will be passed as pointers anyway.)

Also, we are talking of the linker here only, not of the compiler: All packages will be recompiled anyway, so all the source files will use the same and newest header files no matter of the package build order.

However, if the problem you were referring to in your last post actually applies (I still don't know if I got you right - or how the linker actually works), then the ABI can break if the linker creates a new executable which refers to an old C++ shared library.

In this case, the linkage will fail, because the linker cannot find the requested name in the old library.

The reason is that the new executable uses a different name mangling scheme than the old compiler which created the old library, so the referenced entry point won't be found and a linkage error occurs.

But only C++ does name mangling!

How could expat, being a C library, be affected by C++'s name mangling at all?

There is only ONE reason I could tell, and that's version scripts. That is, if the library author explicitly required version incompatibility with the old version.

Then, and only then, yes, it is possible that a C library has exactly the same problems as a C++ shared C++ library.

But wait.

That can't be!

A different version script can't be the result of a compiler update, but only the result of a package version update!

And we did an "emerge --update --deep --newuse world" at the beginning...

...which won't help detect incompatible library updates. Agreed.

However, a revdep-rebuild should track such problems down.

I think, I really have to add it to my guide!

----------

## Guenther Brunthaler

 *devsk wrote:*   

> revdep-rebuild can check only the library linkages.

 

When you say "linkages", does that include checking for the referenced the entry points, or just checking for the existence of the library file as a whole?

 *devsk wrote:*   

> but I agree, a revdep-rebuild at the end will further solidify your method. (but still not make it bullet proof... )

 

OK, I have updated the guide then. Your contribution has been mentioned, of course.

----------

## Guenther Brunthaler

 *Bad Penguin wrote:*   

> As I detailed here, the quickest and safest method is to start with a stage1, bootstrap, then emerge -e system

 

Having acknowledged your guide as a working alternative to my guide, I've taken the liberty to recommend your guide as a viable alternative to my own guide in its introduction for those who like it the hardcore way.

It's certainly always good for the interested ones to have alternatives available.

----------

## devsk

 *Guenther Brunthaler wrote:*   

> 
> 
> When you say "linkages", does that include checking for the referenced the entry points, or just checking for the existence of the library file as a whole?

 I think it uses ldd to determine if all the executables/libraries have there dependent libs present. And that's it.

----------

## Guenther Brunthaler

 *outspoken wrote:*   

> Your guide should read more like this:

 

OK, so you are suggesting to do the gcc-config before the emerge --update then.

It should certainly not a be problem changing this in my guide, but first I have to understand what exactly the problem was.

From what I've seen in the link you provided, they were actually getting a "gcc too old"-sort of error message.

But in our case, GCC will be up to date when the glibc will finally be compiled by my script, so I am uncertain on what step of my guide the problem actually occurred in your case!

So far, my the logic of my guide is:

The initial emerge --update brings all packages up-to-date, including glibc and the old GCC as well as all the tools required for rebuilding the system.

Then the new GCC is emerged. Now both GCCs are installed, and still all packages are up-to-date.

Now the used runs gcc-config and sets the new GCC as the system default compiler. But the user does not use it to compile anything with it yet.

Instead the user starts my script, which will generate a package list of all packages in the system, and also determine the correct order in which they should be recompiled. This includes glibc.

The user starts the generated script. glibc should be emerged as the second package of all, and the first one to be actually compiled. This will already be done with the new GCC, which is the system C compiler now.

If there is an error in this flow of logic, please be  more specific and tell me at what point of the above list the problem occurs.

I may then be able to realize the origin of the problem. (Let's hope!)

----------

## dalek

Well, I did mine the hard way.  I didn't see this guide before I started.  I have ran revdep-rebuild about 3 times now.  It wants to rebuild gcc each time and it just goes in circles.  Do you have any reason why it does this??  Info:

```
[I--] [  ] app-portage/portage-utils-0.1.20 (0)
```

Which I seem to recall has revdep-rebuild in it.

```
root@smoker / # emerge --info

Portage 2.1-r2 (default-linux/x86/2006.1/desktop, gcc-4.1.1, glibc-2.4-r3, 2.6.16-gentoo-r7 i686)

=================================================================

System uname: 2.6.16-gentoo-r7 i686 AMD Athlon(tm) XP 2500+

Gentoo Base System version 1.12.4

ccache version 2.3 [enabled]

app-admin/eselect-compiler: [Not Present]

dev-lang/python:     2.3.4-r1, 2.4.3-r1

dev-python/pycrypto: 2.0.1-r5

dev-util/ccache:     2.3

dev-util/confcache:  [Not Present]

sys-apps/sandbox:    1.2.17

sys-devel/autoconf:  2.13, 2.59-r7

sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2

sys-devel/binutils:  2.16.1-r3

sys-devel/gcc-config: 1.3.13-r3

sys-devel/libtool:   1.5.22

virtual/os-headers:  2.6.17

ACCEPT_KEYWORDS="x86"

AUTOCLEAN="yes"

CBUILD="i686-pc-linux-gnu"

CFLAGS="-march=athlon-xp -O2 -fomit-frame-pointer -momit-leaf-frame-pointer -fno-ident -pipe"

CHOST="i686-pc-linux-gnu"

CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/"

CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"

CXXFLAGS="-march=athlon-xp -O2 -fomit-frame-pointer -momit-leaf-frame-pointer -fno-ident -pipe"

DISTDIR="/usr/portage/distfiles"

FEATURES="autoconfig buildpkg ccache distlocks metadata-transfer parallel-fetch sandbox sfperms strict"

GENTOO_MIRRORS=  < snip >

LDFLAGS="-Wl,-z,now"

LINGUAS="en"

MAKEOPTS="-j2"

PKGDIR="/usr/portage/packages"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage"

SYNC="rsync://rsync.gentoo.org/gentoo-portage"

USE="x86 3dnow X acl acpi alsa amd arts artswrappersuid avi berkdb bitmap-fonts bzip2 cairo cdr chroot cli crypt cups dbus dlloader doc dri dvd dvdr eds emboss encode esd exif fam fdftk fortran gaim gcj gdbm gif gimp gimpprint gkrellm gphoto2 gpm gstreamer gtk hal hbci ipv6 isdnlog java javascript jbig jpeg jpeg2k justify kde ldap libg++ mad mikmod mmx mp3 mpeg ncurses nls nptl nptlonly nsplugin offensive ofx ogg opengl pam pcre pdflib perl png postgres ppds pppd python qt3 qt4 quicktime readline reflection samba scanner sdl seamonkey session spell spl sqlite sse ssl tcltk tcpd tiff tk truetype truetype-fonts type1-fonts udev unicode usb vorbis win32codecs wmf xml xmms xorg xprint xv yahoo zlib elibc_glibc input_devices_keyboard input_devices_mouse kernel_linux linguas_en userland_GNU video_cards_nvidia"

Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

root@smoker / #               
```

I ran into the same thing with the old compiler before the upgrade.  It just wants to do both now like this:

```
root@smoker / # revdep-rebuild -p

Configuring search environment for revdep-rebuild

Checking reverse dependencies...

Packages containing binaries and libraries broken by a package update

will be emerged.

Collecting system binaries and libraries... done.

  (/root/.revdep-rebuild.1_files)

Collecting complete LD_LIBRARY_PATH... done.

  (/root/.revdep-rebuild.2_ldpath)

Checking dynamic linking consistency...

  broken /usr/lib/aqbanking/plugins/0/bankinfo/de.la (requires /usr/lib/libaqbanking.la)

  broken /usr/lib/aqbanking/plugins/0/bankinfo/de.la (requires /usr/lib/libgwenhywfar.la)

  broken /usr/lib/aqbanking/plugins/0/imexporters/ofx.la (requires /usr/lib/libaqbanking.la)

  broken /usr/lib/aqbanking/plugins/0/imexporters/ofx.la (requires /usr/lib/libgwenhywfar.la)

  broken /usr/lib/aqbanking/plugins/0/providers/aqofxconnect.la (requires /usr/lib/libaqbanking.la)

  broken /usr/lib/aqbanking/plugins/0/providers/aqofxconnect.la (requires /usr/lib/libgwenhywfar.la)

  broken /usr/lib/avifile-0.7/ac3pass.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/ac3pass.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/audiodec.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/audiodec.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/ffmpeg.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/ffmpeg.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/mad_audiodec.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/mad_audiodec.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/mp3lame_audioenc.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/mp3lame_audioenc.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/mp3lamebin_audioenc.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/mp3lamebin_audioenc.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/mpeg_audiodec.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/mpeg_audiodec.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/osmjpeg.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/osmjpeg.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/vorbis_audio.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/vorbis_audio.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/win32.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/win32.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/avifile-0.7/xvid4.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/avifile-0.7/xvid4.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/lib-org-w3c-dom.la (requires /usr/lib/libgcj.la)

  broken /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/lib-org-xml-sax.la (requires /usr/lib/libgcj.la)

  broken /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgcjawt.la (requires /usr/lib/lib-gnu-java-awt-peer-gtk.la)

  broken /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgij.la (requires /usr/lib/libgcj.la)

  broken /usr/lib/kde3/libk3blibsndfiledecoder.la (requires /usr/kde/3.4/lib/libkio.la)

  broken /usr/lib/kde3/libk3blibsndfiledecoder.la (requires /usr/kde/3.4/lib/libkdesu.la)

  broken /usr/lib/kde3/libk3blibsndfiledecoder.la (requires /usr/kde/3.4/lib/libkwalletclient.la)

  broken /usr/lib/kde3/libk3blibsndfiledecoder.la (requires /usr/kde/3.4/lib/libkdeui.la)

  broken /usr/lib/kde3/libk3blibsndfiledecoder.la (requires /usr/kde/3.4/lib/libkdecore.la)

  broken /usr/lib/kde3/libk3blibsndfiledecoder.la (requires /usr/kde/3.4/lib/libDCOP.la)

  broken /usr/lib/kde3/libk3blibsndfiledecoder.la (requires /usr/kde/3.4/lib/libkdefx.la)

  broken /usr/lib/libaqbankingpp.la (requires /usr/lib/libaqbanking.la)

  broken /usr/lib/libaqbankingpp.la (requires /usr/lib/libgwenhywfar.la)

  broken /usr/lib/libaqofxconnect.la (requires /usr/lib/libaqbanking.la)

  broken /usr/lib/libaqofxconnect.la (requires /usr/lib/libgwenhywfar.la)

  broken /usr/lib/libaviplay.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/libaviplay.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/libkbanking.la (requires /usr/lib/libaqbanking.la)

  broken /usr/lib/libkbanking.la (requires /usr/lib/libgwenhywfar.la)

  broken /usr/lib/libqavm.la (requires /usr/lib/libaviplayavformat.la)

  broken /usr/lib/libqavm.la (requires /usr/lib/libaviplayavcodec.la)

  broken /usr/lib/libqbanking.la (requires /usr/lib/libaqbanking.la)

  broken /usr/lib/libqbanking.la (requires /usr/lib/libgwenhywfar.la)

 done.

  (/root/.revdep-rebuild.3_rebuild)

Assigning files to ebuilds... done.

  (/root/.revdep-rebuild.4_ebuilds)

Evaluating package order... done.

  (/root/.revdep-rebuild.5_order)

All prepared. Starting rebuild...

emerge --oneshot -p =sys-devel/gcc-3.4.6-r1 =sys-devel/gcc-4.1.1

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R   ] sys-devel/gcc-3.4.6-r1

[ebuild   R   ] sys-devel/gcc-4.1.1

Now you can remove -p (or --pretend) from arguments and re-run revdep-rebuild.

root@smoker / #   
```

You have any ideas why it sends me in circles?

 :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy: 

----------

## Guenther Brunthaler

 *devsk wrote:*   

> I think it uses ldd to determine if all the executables/libraries have there dependent libs present. And that's it.

 

Which meant I needed to write a script that mimics revdep-rebuild's job, only that is also uses nm on every SO in order to verify all of its entry points.

In order to be able to do that, I need to do a second pass over all SOs and executables, collecting all references to other SOs.

And all that for only a very few cases where this is actually needed.

Hmmm. Sounds not exactly fun to implement... Perhaps, if I should be very bored at some time...  :Wink: 

----------

## Guenther Brunthaler

 *dalek wrote:*   

> You have any ideas why it sends me in circles?

 

Hard to say, as it certainly depends on the set of packages you have installed.

But I can acknowledge the problem per se, although I experienced it with some different package. (Don't remember which one.)

However, I still remember that I found the origin of the problem: It was the consequence of a mixture of normal Gentoo packages installed from source code, and prebuilt binary packages.

It was either mozilla-firefox-bin, mozilla-thunderbird-bin or openoffice-bin. Don't remember which one.

After uninstalling the guilty bin package, the circles were gone.

I think the problem were the USE flags.

The faulty bin-package was compiled without some optional feature, which was required because of USE-flag settings by another package.

Because of this missing feature, some library entry point in some shared library from the bin-package was undefined, although the other package required it (as a consequence of the USE-flags).

revdep-rebuild detected the missing library-entry, and thus scheduled the faulty bin-package for re-emerging.

But as re-emerging does not change anything for a bin-package, the problem continued to exist after the re-emerge, and so the next run of revdep-rebuild detected the problem again, emerging the package again... and again... and again...

You get the picture.

----------

## dalek

Thanks for the info.  As I said I ran into this before and I usually let it do it twice then I figure the point is pretty much mute.  I don't think I have to much binary stuff installed on here though.  I even compile OOo on this thing.  I may start a seperate thread, as much as I hate to, and see if it is a bug that needs to be dealt with before it does break something for someone else.

I did get your script and I stopped the emerge -e world that I had running.  I then ran it and it seems to be working fine.  I noticed that it compiles the kernel stuff first and that it said to compile glibc again during the install.  I thought hmmm, never saw that before.  Then it started emergeing glibc just like it said too.  That was sort of funny in a way.  It's like the script was reading the things mind.    :Shocked: 

Anyway, I'll try revdep-rebuild again after your script gets done.  If I get the same thing I'll start a thread.

Thanks

 :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy: 

----------

## vicaya

 *Guenther Brunthaler wrote:*   

>  *vicaya wrote:*   emerge -e system and then world. It's probably about 30min longer than the "optimal" script. But you'll be 100% sure, everything is done correctly. 
> 
> I that only were so easy ... I would never needed to write my script.

 

Sorry, what I really meant is that just follow the official gcc upgrade instruction. emerge gcc etc... see http://www.gentoo.org/doc/en/gcc-upgrading.xml code listing 2.1 and then emerge glibc && emerge -e system && emerge -e world. With ccache, it won't be that much slower (considering the total emerging time) than any "optimal" script.

----------

## Bad Penguin

 *Guenther Brunthaler wrote:*   

> 
> 
> (can a Penguin be bad at all? Not for me. I like penguins!)
> 
> 

 

What, you didn't know that linux users are thieves, liars, terrorists, and long haired smellies?  Where have you been  :Wink: 

The Bad Penguin is a barb at the litigious assholes known as the SCO Group, in particular Darl McBride, who has falsely accused the entire open-source community of being thieves...

 *Guenther Brunthaler wrote:*   

> 
> 
> But you are not the first one to find a flaw in my guide. Hey, I'm only human! I'm not perfect, everyone can make as mistake and I'm no exception. That's why we discuss ideas like this in the forums, don't we?
> 
> For instance, devsk has already pinned down a potential problem related to circular dependencies, although it seems that (by coincidence I must admit) my script will not be affected by it. See his posting for more.

 

The flaw is not so much in the guide, but in making definitive, authoritative sounding statements that have not been tested or proven.  I am just anal that way  :Wink: 

 *Guenther Brunthaler wrote:*   

> I have not even an idea how to prove the correctness of a computer program with as far-reaching consequences as the application of my script. Perhaps if I wrote it in ADA instead as a BASH script... 

 

Perhaps by building it via your method and running the test suites of every package as you go (FEATURES="test")?  The only way to be certain is to actually use your method in some different environments and see how many bugs are produced as a result.

 *Guenther Brunthaler wrote:*   

>  *Bad Penguin wrote:*   As far as your second assertion, that your method is somehow faster, wrong again. 
> 
> Here we obviously have a simple misunderstanding: When I wrote  *Quote:*   Compared to other guides which suggest recompiling the entire system up to 6 times  I didn't mean I my guide was faster than all the guides which may exist, only faster than those which recommend excessive recompilation of the system over and over.
> 
> Which, by the way, was the primary motivation for writing my guide: 6 times was too much!

 

Heh heh, I have never seen a guide that suggests to do it 6 times.  The Rob Moss method suggests to "emerge -e system && emerge -e system && emerge -e world && emerge -e world", which compiles the system packages four times and the remaining packages (world) twice.  This method is not very useful for toolchain updates though, which need to be done in the correct order.

 *Guenther Brunthaler wrote:*   

> As you have read my argumentation and how my script works, you might have noticed that it builds a list of all packages to be recompiled when the generated build script is run.
> 
> This list countains all the packages which will be rebuilt in the correct order, including any dependencies.

 

An incorrect assumption.  If you are merely recompiling every package on your box for fun, or a change in USE flags/CFLAGS, you would be correct.  The only reason to ever recompile every package would be a toolchain update, which needs to be done in a specific order, which is not handled by portage correctly.

 *Guenther Brunthaler wrote:*   

> But an update like the 2006.1 switch is mere overkill for the power of your script, where my simple guide is obviously sufficient for such minor cases.

 

That depends on which profile and versions of everything you are upgrading from.  Not everyone is going to be switching to the 2006.1 profile from a 2006.0 profile.  Your script might work in a very small set of cases, in other cases it might turn out to be disastrous.  You are leaving one very important factor out of your method - the human factor  :Wink:   Bugs happen, people don't always know to go to the gcc/glibc migration guide, which gives out bad advice anyway.  Use revdep-rebuild on your system and you will soon find out that you have old .la files laying around that will not be removed by emerge because the files have been modified by an outside process.  Who the hell knows what code is actually being statically compiled into your packages as a result.

 *Guenther Brunthaler wrote:*   

> But why should such a compiler actually be faster than the compiler emerged at the beginning of my guide, which fully honors all MMX/SSE flags and also uses the full optimization settings as defined in /etc/make.conf?
> 
> And from then on, my compiler has to recompile all remaining packages in the system exactly once, according to my guide.

 

During a bootstrap the USE flags are only restricted during the bootstrap phase, when the actual toolchain packages are built.  The subsequent emerge -e system uses any USE flags in make.conf.  One example of why this is useful is the solitaire game pysol - which requires tk.  If all use flags were allowed in the bootstrap phase the compile of python will pull in X to build tk, then you would have to build X all over again during the subsequent emerge -e system.

 *Guenther Brunthaler wrote:*   

> How can your script, starting with a slower compiler, do the same in a faster way?
> 
> So your script/guide has to recompile at least exactly the same number of packages as my guide does.

 

It does not actually compile anything faster, it does it more efficiently.  The important thing is not that your packages compile fast, it is that they compile correctly so you don't have to go back and do it all over again later.  But no, my "method" actually only ends up compiling the bootstrap packages twice along with a small set of the bloated needs of portage (python).  You start with a cleanly bootstrapped system, the remaining packages are all compiled on a clean toolchain.

 *Guenther Brunthaler wrote:*   

> The "overhead" of my guide is the initial "update world", and the overhead of your guide is the overhead of a stage1 reinstall.

 

The "overhead" of your guide is that chances are it will really screw up peoples systems  :Wink:   Sadly, there is no way to guarantee that every package on the system is built against current libraries that are built against current libraries... yadda yadda yadda... without a brute force, recompile it multiple times approach.  There are just too many circular dependencies and too many different factors when you take USE flags into account.  The safest, and fastest way, for the large majority of people, is to start from stage1.  This is unique to gentoo because it is a source based distribution...  Too bad gentoo has abandoned the build from source approach, instead encouraging people to hack up their boxes...  I digress.

Anyway, if you are interested, I will write up some scripts that take a box from a specific state, say portage-20060301, to portage-20060905.  One script that does it your way, one that does it my way, and we can add FEATURES="test" to see how it turns out  :Wink: 

----------

## Sculler

It might be a good idea to reorder the prerequisites to emerge gentoolkit is before revdep-rebuild, because (I think) that command is part of gentoolkit.

----------

## dalek

Well, the script just finished.  I have the same problem that I had before the gcc upgrade just with the newer version so I guess all is well.  It seems revdep-rebuild just likes to emerge gcc in a circle.

Oh well.  It all works at least.    :Rolling Eyes: 

 :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy:   :Very Happy: 

----------

## Guenther Brunthaler

 *Bad Penguin wrote:*   

> Anyway, if you are interested, I will write up some scripts that take a box from a specific state, say portage-20060301, to portage-20060905.  One script that does it your way, one that does it my way, and we can add FEATURES="test" to see how it turns out 

 

Interested, yes, of course.

But I'm afraid my knowledge about Portage and the internals of ebuilds is not yet mature enough to write any "test" scripts myself. Those internals are mostly still black magic for me.

In fact I have not even finished studying the autoconf guide - and I'm even farther from writing my own ebuilds.

After all, I'm relatively a Linux newbie and there are still many aspects of the system which I don't fully understand (or understand at all). Although I am trying hard to learn. But it takes time.

So just give me a little more time please ...

----------

## Guenther Brunthaler

 *Sculler wrote:*   

> It might be a good idea to reorder the prerequisites to emerge gentoolkit is before revdep-rebuild, because (I think) that command is part of gentoolkit.

 

Thanks for pointing that out! I'll check to which package revdep-rebuild actually belongs and will reorder the prerequisites accordingly.

----------

## ANewFishMonkey

I have read through this thread, I think it will solve my problem but Im not entirely sure,.

I have recently migrated to X Server 7.0,  which I sort of got working, (but so much is broken its not useful!).

Some of the new X packages (glibc I think) failed to compile, due to an insistence of using gcc 3.4.6 not 3.3.6, so I switched and tried again.  The problem was when I recompiled my kernel,  all of the modules refused to load, complaining of magic version different.  (the complier version namely), but I did recompile all the modules with the same complier as the kernel. (or at least I think I did )

What I want to know is :

	Is there some other setting I need to change other than gcc-config to ensure modules and kernel are compiled using the same version?

Does 2.6.12-gentoo-r4 HAVE to be compiled with 3.3.6?

I got round this issue, by compiling the kernel and modules with 3.3.6 everything else with 3.4.6, but now I have messed up because the  ATI fglrx module complains about missing symbols. I pretty sure your script is what I need, but I need to know what else I need to do first,  not mad about wasting electricity and all that. 

 :Very Happy: 

BTW,  I'm currently using 2006.0 profile,  shall I switch to 2006.1 and just use gcc 4.1 saving time later?

----------

## Guenther Brunthaler

 *ANewFishMonkey wrote:*   

> Is there some other setting I need to change other than gcc-config to ensure modules and kernel are compiled using the same version?

 

No, that should be sufficient.

However, you also need to recompile all the packages compiled with the old compiler, because mixing packages compiled with different major compiler versions can cause troubles.

Unfortunately, portage is not aware about those problems, and will not automatically recommend recompiling all the packages. You have to do this on your own.

Common sense is that all packages should be recompiled at least once, which is what my script does.

There are different points of view whether this is enough; I say "yes". (See my argumentation link in the article.)

Recent major GCC updates which require recompiling the system are: From pre-3.4 to 3.4, and from pre-4.0 to 4.x.

As you are updating from 3.3.6 to 3.4.6 or 4.1 you are affected anyway.

BTW: Just switching back to 3.3.6 won't help if you already compiled some packages with 3.4.6: Either you recompile all those packages again with 3.3.6, or you will end up in a system with mixed versions, which will cause troubles sooner or later for sure.

I thus suggest completely updating your system to either 3.4.6 or 4.1 to have a stable system again. You may use my script for this, or some of the other guides (which take a longer time, but their authors believe it's better. I don't believe this, but you decide! The most pragmatic version is probably using my guide first, because it's fastest under normal circumstances, and if you should be so unlucky to be the first one having problems with it, then you still can use one of the other guides.)

 *ANewFishMonkey wrote:*   

> Does 2.6.12-gentoo-r4 HAVE to be compiled with 3.3.6?

 

No. The only known limitation I am aware of is that GCC4 cannot be used to compile a kernel 2.4 or older.

3.3.6 as well as 3.4.6 or 4.1 should not have any problems compiling a 2.5+ kernel.

 *ANewFishMonkey wrote:*   

> ATI fglrx module complains about missing symbols.

 

I also had that problem, although with the NVIDIA drivers.

It was not a problem of the compiler, but of the kernel version: They had changed something in the new kernel version, which also required a corresponding change in the NVIDIA code.

When an update was available to the NVIDIA binary drivers, all the problems went away.

In meantime, I was using the open source drivers instead. Although they had inferior 3D speed (= unusable for games), there was no difference for normal X applications.

So I did not play 3D games for a couple of weeks, but otherwise there was no disadvantage using the open source X11 drivers until the new NVIDIA binary drivers were finally released.

I'm pretty sure things will be the same for ATI.

 *ANewFishMonkey wrote:*   

> BTW,  I'm currently using 2006.0 profile,  shall I switch to 2006.1 and just use gcc 4.1 saving time later?

 

Yes, you can. I did it also. Just look into the official Gentoo GCC update guide (but don't follow it if you want to use my guide). They have a paragraph there descibing how to mask GCC4 in the 2006.1 profile, so that GCC 3.4.6 will be used instead.

But - is there a specific reason why you won't be using GCC4? If you recompile your entire system anyway, why not using the best compiler if you can, which certainly is 4.x?

Sure, 4.x works slower than 3.x, which means it takes more time to recompile your system.

But then, 4.x also generates better code, which means the programs it compiles will run (slightly) faster then. GCC4 is slower because it does a better job, which takes more time to complete.

The only problem of GCC4 is its hunger for RAM. If you have a box with less than 512 megs, it may be better to stay with 3.4.6.

But otherwise, I would suggest using 4.x.Last edited by Guenther Brunthaler on Sat Sep 09, 2006 10:36 pm; edited 1 time in total

----------

## Lloeki

```
$ free

             total       used       free     shared    buffers     cached

Mem:       1554960    1244864     310096          0     137388     872464

-/+ buffers/cache:     235012    1319948

Swap:      3502128      71036    3431092

```

While compiling with kde, firefox & al running, I wouldn't call that exactly hunger. it would fit in 256M. (btw, currently compiling OOo203)

----------

## Guenther Brunthaler

 *Lloeki wrote:*   

> I wouldn't call that exactly hunger. it would fit in 256M. (btw, currently compiling OOo203)

 

Looks good; yes.

However, this is a snapshot from one instance in time. The current source file obviously does not need much, but who can say the next source file will also have the same decent requirements.

In order to analyze the memory requirements, it would be necessary to collect the output in a loop, taking a sample all couple of seconds and write it to a log file.

Afterwards, we can then search for the sample with the maximum and minimum memory requirements, and also calculate the average requirements.

A loop similar to

while true; do free; sleep 10; done > memory.log &

should do the job (it must be manually killed when the recompilation is done).

Of course, now it may be too late, if you're almost done recompiling. But perhaps the next time.

----------

## Guenther Brunthaler

 *Lloeki wrote:*   

> 
> 
> ```
> $ free
> ```
> ...

 

I just looked at the man page for free; perhaps it's not necessary to run free in a loop and analyze the log contents afterwards.

free has a -l option which reports low and high memory usages, and the -s <N> option repeats the output all <N> seconds.

However, I don't know whether the -l values are cummulative, or just per-sample. That needs further investigation...

----------

## mirek

I have got an error 

```
# sh genscript.sh

Decoding attachment - please standby.

genscript.sh: line 7: syntax error near unexpected token `||'

genscript.sh: line 7: `|| die "I need openssl for base64 decoding!" '
```

when I run sh genscript.sh script

I have openssl installed

```
# emerge -p openssl

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R   ] dev-libs/openssl-0.9.8c
```

and test for it

```
# echo 'dGVzdGluZwo=' | openssl base64 -d

testing

```

Last edited by mirek on Sat Sep 09, 2006 8:28 am; edited 1 time in total

----------

## blackcell

Great Guide and script   :Very Happy: 

I had no problems first time through all the way to completion, 40 hours later. Only failure was mod_gzip fails to compile.

Gzip bug

----------

## agrypa1

Hi, thank you for the script. My system is being compiled right now, as we speak  :Smile: . I have an issue though which I resolved in two ways. I am not sure the first solution hasn't spoiled my system, though.

Namely, during the compilation process the script stopped with an error at a package from an overlay tree. No thinking much I unmerged the package and ran the script again. Of course it didn't work that way, because the package was still on the list of the script. So I removed the files the script generated (script.ar, remaining-packages, entire -system). I recreated the script and started anew. At this point I do not know whether gcc and linux headers were compiled again or not?

the script failed again at qemu-user and qemu-softmmu. This time I thought it would better be that I find a way to skip the compilation of the packages instead of removing them from my ssystem ans start from the beginning. So I found out that I can comment out the troublesome packages in the recompile-remaining-packages script. It worked.

Is this the only and correct way of recovering form a failure of a uncompilable package by commenting it out?

Thank you

Agryppa

----------

## s_wilk

Hi

Guenther Brunthaler wrote:

 *Quote:*   

> However, if someone reading this has some bandwith left to waste, he/she is invited to host the decoded script at their web site and send me the download URL

 

So I put the script and a copy of the guide on my website (perserving information about the author).

You can find it here:

http://knip.pol.lublin.pl/~sw/gentoo/

BTW. I have a question to Guenther:

If I understand correctly Your script recompiles the packages in the exact versions as installed at the moment whith the same use flags (cause you say to do #emerge --ask --update --deep --newuse world before running your script)

So I assume I shouldn't use this script to switch to hardened profile from 2006.0?

Do you think it does make sense to do emerge -avu --deep --newuse world and then run your script?

I am not sure how much packages are affected with use flags change (hardened impicates at least  -nptl hardened flags)

----------

## juantxorena

First of all, I don't want to offend anybody. I haven't tried this script, so I will say my opinions a priori:

1.- If this script works better than the emerge -e system && emerge -e world stuff that we can see in the official guide, why don't try to contact a developer to talk about this? Maybe the script can become an official part of Gentoo in the future. Have you contact any developer? What he/they say about this? Messing with the system like this is a dangerous thing, I don't want to break anything.

2.- Don't mean to offend you, but reading you post, the way they are writen and the way you explain things, I think that you are acting like a televangelist or an infomercial vendor. This undermines you credibility. If this thing works, talk with some devs in a chat, or something. But trying to "sell your product" with "Because the amazing truth is: It is in fact UNNECESSARY to recompile any package bla bla bla" or other similar quotes is not the best idea.

Personally I would wait to see what devs think about this.

----------

## Guenther Brunthaler

 *mirek wrote:*   

> I have got an error

 

I have re-checked it on my box; it still runs fine for me either using bash or ash.

It seems most likely to me there went something wrong with copy/paste.

Please verify the MD5 checksum:

```
cat > genscript.sh.md5 << END

07d0667e0893d99d4ddca0be0f2c2449 *genscript.sh

END

md5sum -c genscript.sh.md5

```

----------

## Guenther Brunthaler

 *agrypa1 wrote:*   

> No thinking much I unmerged ...  So I removed the files the script generated (script.ar, remaining-packages, entire -system). I recreated the script and started anew.

 

You did well. This should fix any potential problems.

 *agrypa1 wrote:*   

> At this point I do not know whether gcc and linux headers were compiled again or not?

 

"linux-headers" is really just that - a bunch of header files. They will only be downloaded and installed, but nothing will ever be compiled by emerging that package.

Besides that, re-emerging "linux-headers" is the very first thing my script does - so you certainly need not be worried about it.

GCC is another story: It won't be re-emerged by my script, because it is not necessary: If you were following my guide, you did this yourself already.

And as you can see in my related argumentation article ("myth 1 - GCC gets better each time it is re-emerged"), there is no point in re-emerging the same version of GCC ever again. It cannot get any better than it already is, no matter which profile or glibc was current at the time it was emerged.

So you also need not worry about re-emerging GCC too.

(The only case when re-emerging the same version of GCC makes sense is if the CFLAGS in /etc/make.conf or the USE flags for GCC have been changed by you.)

 *agrypa1 wrote:*   

> Is this the only and correct way of recovering form a failure of a uncompilable package by commenting it out?

 

No, the lines could be deleted as well. But I preferred instructing the users of my guide to comment the lines out, because that way it's easy to un-comment the lines again in case they did comment out the wrong lines mistakenly.

Also, just commenting out allows looking at the script file contents after the system rebuild is complete, and see which packages were skipped.

Then the users could decide to try re-emerging those missing packages again, hoping it will then work as anything else will already be up-to-date.

----------

## Guenther Brunthaler

 *s_wilk wrote:*   

> So I put the script and a copy of the guide on my website (perserving information about the author).
> 
> You can find it here:
> 
> http://knip.pol.lublin.pl/~sw/gentoo/

 

Thanks very much for doing this!

I'll be to happy to update my guide and include your link.

 *s_wilk wrote:*   

> If I understand correctly Your script recompiles the packages in the exact versions as installed at the moment whith the same use flags

 

That's correct. I wanted to use a stable version snapshot at some instance in time, or otherwise an imprudent "emerge --sync" could easily make an ongoing system rebuild operation break.

Using a stable version snapshot of all packages, the chances for an intermittent "emerge --sync" to break things are much lower.

 *s_wilk wrote:*   

> So I assume I shouldn't use this script to switch to hardened profile from 2006.0?

 

There should not be a problem if you switch the profile before starting my script, because then the version snapshot will be stable again.

My script generates the package list based on the current system profile at the time the generator script is called.

And as my script will recompile everything anyway, all packages will be compiled according to the settings of the new profile.

With one exception: My script will never recompile GCC, because this will already have been done manually when users were following my guide.

This assumes however, that GCC has been emerged with the correct USE-flags and CFLAGS already set in /etc/make.conf (i. e. after the profile has been changed.)

 *s_wilk wrote:*   

> Do you think it does make sense to do emerge -avu --deep --newuse world and then run your script?

 

Well, my guide actually requires the user to do so before invoking my script.

However, in your case this might actually be an overkill, because if the USE-flags changed by the hardended profile were all that changed, you can save some time by using the following approach:

Do the emerge -auvDN world while still being attached to the old profile.

Switch to the hardened profile. But don't do any emerge --update yet.

Re-emerge GCC as

```
emerge --ask --update --oneshot --newuse --deep gcc
```

This will use the new USE-flag for the new GCC, if there are any. Look at the outpout of emerge whether any USE-flags did actually change. If there are none, and CFLAGS in /etc/make.conf also didn't change, there is no point in re-emerging GCC, and you can skip this step.

Run my script. It will recompile anything else, according to the hardened profile.

Update: I have also updated by guide (as of version 1.11) to support your scenario directly.Last edited by Guenther Brunthaler on Sat Sep 09, 2006 7:14 pm; edited 1 time in total

----------

## Guenther Brunthaler

 *juantxorena wrote:*   

> why don't try to contact a developer to talk about this? Maybe the script can become an official part of Gentoo in the future.

 

While this may be an option for the future, I want to see much more testing before I will actually consider doing this.

While the script itself has not changed since this thread has been started, the accompanying guide serveral times has.

As more and more users were testing my guide, a couple of them have also suggested various improvements, which in turn led to several revisions of the guide to take those improvements into account.

I want this to get on for some time before I eventually might dare to contact any Gentoo developers.

 *juantxorena wrote:*   

> Have you contact any developer? What he/they say about this?

 

I am not related to the Gentoo project in any way except for my commitment to the project. I'm just a normal user like most here in the forums.

 *juantxorena wrote:*   

> Messing with the system like this is a dangerous thing,

 

Yes it is. That's also why I consider more testing to be essential.

Although it seems the guide actually worked fine with the people posting here, that says nothing about how well it will work in unusual cases of system configurations.

 *juantxorena wrote:*   

> I don't want to break anything.

 

I can understand this.

But there is actually little reason to be afraid of: The script is actually a wrapper around a set of emerge statements, each one emerging another package. The script will not try to interfere with the internals of Portage and won't in fact do anything you were not also able to do manually.

The worst thing that could happen to your system is that the script stops before being finished due to a build error of some package.

If that happens, simply skip that package and let my script continue. Then, after all else has been rebuild, re-emerge the failing package again, hoping the emerge will then work better.

And if it turns out that my script can't continue, because that failing script is an essential dependency for other important packages, you can still revert to the usual

```
emerge -e system && emerge -e world

emerge -e system && emerge -e world

emerge -e system && emerge -e world

emerge -e system && emerge -e world

emerge -e system && emerge -e world

emerge -e system && emerge -e world

```

which some people consider to be a better solution. (I doubt it will, but I could be wrong. The only difference one will see for sure, is when looking at the next electricity bill.)

And if that standard method considered "safe" by most will also fail, you still have the last-resort option of doing a complete stage1 system reinstall, just as badpenguin suggests in his guide.

So - no matter whether my guide actually works for you or not, it will never damage your system to a degree that would not allow you to revert to any of the other guides around.

Such a case has not been reported so far, though.

 *juantxorena wrote:*   

> the way you explain things, I think that you are acting like a televangelist or an infomercial vendor

 

Since when do televangelists or infomercial vendors explain anything?  :Wink: 

They usually want you to believe something, which is definitively not my intention. If it were, I would only post my guide, but not any accompanying guides where I explain the ideas and concepts which eventually led to the creation of my guide.

So, if you felt like listening to an televangelists when reading my guide, I have to apologize: This has not been my intention in any way.

Also, as I am not a native English speaker, I will certainly not always be able to express my thoughts in the same natural way a native speaker would be able to do it in the same situation.

In other words: Please excuse my rather bad English!

 *juantxorena wrote:*   

> This undermines you credibility.

 

To be honest: Being still a Linux newbie, I do not have any credibility yet. So there is hardly anything to be undermined.  :Wink: 

So I'm doing the best I can to read man pages, HOWTOs, source files, etc and figure out how to do things in the most optimal way.

And when I find a solution to a problem that finally pleases me, but is different from the solutions I have found being suggested by others so far, I will share it with the community by posting an article.

Of course this will be done in the hope it may also be useful for others.

But the other reason is that is allows others to comment on and suggest improvements, which has shown to be a quite effective way for improving the quality of any guide.

 *juantxorena wrote:*   

> It is in fact UNNECESSARY to recompile any package bla bla bla" or other similar quotes is not the best idea.

 

Well, perhaps you are right. It may not have been the most polite way to put that statement.

But it truly reflects my personal opinion about the issue, so it's at least not a lie.

Also, except for unusual scenarios where the usual "emerge -e system && emerge -e world" would fail as well, there have not been any serious problems reported with the most current version of my guide so far. (Which does not mean there were no problems reported with older versions of the guide. But I updated the guide in such cases in order to fix it.)

But up to date, my guide is still the fastest way to rebuild an entire system (assuming users have been updating their systems on a regular basis even before considering using my script; my guide will exploit this).

There are known situations in which it will not work, such as updating glibc from version 2.x to 3.x. But I won't expect such a major glibc update to happen within the current decade.

Also, in such a situation, all the other guides based on "emerge -e world" will also fail. You have to do a complete stage1 reinstall in such cases.

But otherwise, the guide seems just to work.

 *juantxorena wrote:*   

> Personally I would wait to see what devs think about this.

 

I assume the Gentoo devs have more serious things on their do-to-lists than caring about such minor issues.

After all, my script is a mere optimization. It is nothing new or revolutionary on its own, it will only save you some bucks on your next electricity bill. That's all.

----------

## Drysh

I don't think it's this script fault, but I had some problems while using it. Please, take a look: https://forums.gentoo.org/viewtopic-t-496817.html

BTW: I nice side effect of your script: It gave me a list of packages that I emerged to look for the evil one.

----------

## COMKEEN

 *mirek wrote:*   

> I have got an error 
> 
> ```
> # sh genscript.sh
> 
> ...

 

I encountered the same problem. I found out that the script always dies when a line ends with an "escaped line feed" (don't know how it is called). Remove the line feed and put all the commands in one line.

Example:

Change

```

emerge --oneshot --update openssl \ 

 || die "I need openssl for base64 decoding!" 

```

to

```

emerge --oneshot --update openssl || die "I need openssl for base64 decoding!" 

```

HTH,

 Christian

----------

## Guenther Brunthaler

 *COMKEEN wrote:*   

> I encountered the same problem. I found out that the script always dies when a line ends with an "escaped line feed" (don't know how it is called).

 

Thanks for that more accurate problem description!

I therefore conclude the origin of the problem must be the following: When copying/pasting the script, whitespace has somehow been added after the backslash which was not there in the original script.

In order to fix things, that white space must be manually removed: The backslash must be the very last character in the line! No spaces/tabs are allowed to follow.

Seems your editor somehow screwed up the pasted text by inserting some whitespace after the backslashes.

 *COMKEEN wrote:*   

> Remove the line feed and put all the commands in one line.

 

Originally, this was exactly as the line looked! But as I never allow any line in one of my scripts to get longer than 79 characters, I inserted escaped line feeds in order to break long lines.

For the future, I will try to find a way to avoid the escaped backslashes, in order to make the script even work if an editor unintentionally adds whitespace at the line end.

UPDATE: I have updated the script in the article by an updated version (1.12) of the script which avoids all escaped backslashes. (As an unintended side effect, it looks even more cryptic now. But who cares.)

Please try again with the updated script!

----------

## COMKEEN

 *Guenther Brunthaler wrote:*   

>  *COMKEEN wrote:*   I encountered the same problem. I found out that the script always dies when a line ends with an "escaped line feed" (don't know how it is called). 
> 
> Thanks for that more accurate problem description!
> 
> I therefore conclude the origin of the problem must be the following: When copying/pasting the script, whitespace has somehow been added after the backslash which was not there in the original script.
> ...

 

Actually it is not the text editor's fault, but the browser's fault. I am using Konqueror and when I am copying/pasting your code snippet into a text editor (no matter whether I use a GUI editor like KWrite or a command line editor running in Konsole), there are additional whitespaces inserted at the beginning as well as at the end of each line. When I am copying the script from Firefox, everything seems alright.

To remove the leading and trailing whitespaces, one could use this command:

```

$ cat genscript_with_spaces.sh | sed -e "s/^\ //g" | sed -e "s/\ $//g" > genscript.sh

```

(I am sure that the multiple use of sed could be avoided - but since I don't use it very often, I don't know how.)

----------

## AssociateX

Nice!

Hey, what about when two versions of the same package are installed? I have dev-libs/lzo-1.08-r1 and dev-libs/lzo-2.02-r1 installed. I did a "emerge --sync && emerge --update --deep --newuse world && revdep-rebuild" and everything is good there. lzo-2.02-r1 is what is installed with an "emerge -u lzo" and kino needs liblzo.so.1 from lzo-1.08-r1. You can see that they are both installed below.

```
athlon ~ # emerge -pv lzo

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R   ] dev-libs/lzo-2.02-r1  USE="examples" 0 kB

Total size of downloads: 0 kB

athlon ~ # emerge -pv =lzo-1.08-r1

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R   ] dev-libs/lzo-1.08-r1  0 kB

Total size of downloads: 0 kB
```

But! 

```
athlon ~ # grep lzo recompile-remaining-packages

item 537 =dev-libs/lzo-2.02-r1

athlon ~ # 
```

It looks like lzo-1.08-r1 is installed but will not get recompiled with the script. Am I right? If so, can this be fixed?

Guenther Brunthaler, I just wanted to say that I always enjoy reading your threads and other posts. Thank you for the good reading.

----------

## AssociateX

Strange, I was just hunting for other cases like that last one that I posted and found another case.

```

athlon ~ # ls /var/db/pkg/app-crypt/ | grep gnupg

gnupg-1.4.5/

gnupg-1.9.20-r3/

athlon ~ # cat recompile-remaining-packages | grep gnupg

item 140 =app-crypt/gnupg-1.4.5

item 141 =app-crypt/gnupg-1.9.20-r3

```

But in this case both installed versions were found by your script. I wonder what I might have done wrong to mess up the other case.

#edit starts here.

I found this:

```
athlon ~ # emerge -ep world | grep -e lzo -e gnupg

[ebuild  N    ] app-crypt/gnupg-1.4.5

[ebuild  N    ] app-crypt/gnupg-1.9.20-r3

[ebuild  N    ] dev-libs/lzo-2.02-r1
```

This must be the method that the script uses to find installed packages although it doesn't really list them all other wise it would have found lzo-1.08-r1 also. Maybe something like the following could be used to generate your list of installed apps, minus the sys-devel/gcc and old sys-kernel/* or what ever other stuff you know doesn't need to be installed again :

```
find /var/db/pkg -type d | sed "s/\/var\/db\/pkg\//=/" | grep / | sed "s/\/var\/db\/pkg//"|grep =
```

By the way I have not read the whole thread if this has already been brought up.

----------

## count_zero

Guenther, great job!

Check out my script and how I've made it interface with yours for interruption-free compiling of the entire system (in the correct order).

----------

## AssociateX

...

----------

## Guenther Brunthaler

 *AssociateX wrote:*   

> But in this case both installed versions were found by your script. I wonder what I might have done wrong to mess up the other case.

 

My script is based on the output of "emerge --emptytree". It re-orders that output and removes duplicate and unnecessary emerges, but will not add anything to that list.

In your case it seems that Portage thinks both GPG-versions are necessary, but not both of the LZO versions.

This might be possible if you left out the optional "emerge -a --depclean"-step of my guide, which will remove unnecessary packages from your system.

But as emerge --depclean is "known to be broken" (as it's own output says), I did not want to make it a non-optional step of my guide.

So, what can you do?

I suggest the following approach:

Run 

```
emerge -avuDN world
```

 to be sure everything is in a consistent state from emerge's point of view.

Run 

```
emerge --ask --depclean
```

. Make sure no packages are suggested for removal which you obviously don't want to be removed. Accept the removal of suggested packages only if you agree. This should remove any unnecessary packages.

Run 

```
revdep-rebuild
```

 repeatedly until no more packages can been found containing broken library dependencies. (revdep-rebuild is part of gentoolkit.)

If you are lucky, your old LZO package should have gone by then.

If it's still there, you can use equery depends to check whether any other package actually depends on the old LZO version.

If you don't find any dependent packages, just remove the old version of the LZO package manually using emerge --unmerge.

In order to be sure, use qpkg before the removal and create a binary archive from the LZO version to be removed. If it later turns out the old version is actually required by anyone, you can easily re-install it.

As a general precaution when removing shared libraries, I would also suggest emerging and running sash, the "Stand-Alone Shell", in a different window before removing the library. (A statically-linked busybox can alternatively be used for the same purpose.) sash is a statically-linked shell which will even continue to run if most other tools and shells won't because of shared library problems. In such cases you can use the already-running sash to fix things even if bash won't work any longer.

----------

## Guenther Brunthaler

 *count_zero wrote:*   

> Guenther, great job!
> 
> Check out my script and how I've made it interface with yours for interruption-free compiling of the entire system (in the correct order).

 

I like it!

And I totally agree with your idea of an interruption-free rebuild.

I even think it would be the best to incorporate your great idea on next occasion directly into my script, so people won't have to mess around with different scripts for basically the same purpose.

Of course, your contribution will be honored accordingly in the updated version of the guide!

Until then, I recommend using your script for the purpose of interruption-free rebuilding the entire system in combination with my script.

I am also happy to see that my original guide has undergone several revision since then, adding significant improvements suggested by various people, making it even better.

It should be also clear from this, that it's no longer my guide - it's our guide! I'm just the original author and primary maintainer.

----------

## count_zero

 *Guenther Brunthaler wrote:*   

>  *count_zero wrote:*   Guenther, great job!
> 
> Check out my script and how I've made it interface with yours for interruption-free compiling of the entire system (in the correct order). 
> 
> I like it!
> ...

 

Sounds good.

If I can help, let me know.  :Cool: 

----------

## AssociateX

lzo is a depend but for some reason it was in the world file. That stinks, my fault. Sorry about that.

----------

## Guenther Brunthaler

 *count_zero wrote:*   

> Sounds good.

 

I have just updated the script and guide as of version 1.15. The script now features interruption-free operation, just as you suggested. Your contribution has been mentioned in the guide.

After the package list has been processed, the script automatically tries to recompile the packages again which failed earlier, hoping it will work then. This process will continue recursively, but care has been taken to avoid an infinite loop.

I hope this new approach will also defeat any issues that might arise from circular dependencies in ebuilds.

Furthermore, I have added a log file which captures screen output.

----------

## AssociateX

Ok, my problem with lzo is a problem with the ebuild. I filed a bug report https://bugs.gentoo.org/show_bug.cgi?id=148266 but it was closed as a duplicate of https://bugs.gentoo.org/show_bug.cgi?id=143939. That bug too was closed but I reopened it, read to see my reason why.

This all brings me back here with some things that I have learned that someone may want to take into consideration with this howto. cd to the dir with the recompile-remaining-packages script and do the following:

To make a sorted list of all packages that are installed:

```
find /var/db/pkg -type d|sed "s/\/var\/db\/pkg\//=/"|grep /|sed "s/\/var\/db\/pkg//"|grep =|sort>db.tmp
```

To make a sorted list of all packages that are recompiled by "emerge -e world":

```
emerge -ep world|perl -ne 'if (/]/) { s{^\[[^\]]+\]\s+}{=}; print $_ }'|sort>emerge.tmp
```

To make a sorted list of all packages that are recompiled by the recompile-remaining-packages script:

```
perl -ne 'if (/item [1-9]/) { s{item \d{1,4} }{}; print $_ }' recompile-remaining-packages|sort>recompile.tmp
```

Now to see the differences.

To list the packages that the recompile-remaining-packages script will not recompile but emerge -e world would:

```
diff -bn emerge.tmp recompile.tmp|grep =
```

For me this gave back:

=sys-devel/gcc-4.1.1

=sys-devel/gcc-config-1.3.13-r3

Pretty much novelty information.

What packages are installed but will NOT be recompiled by "emerge -e world":

```
diff -bn emerge.tmp db.tmp|grep =
```

I got 4 packages listed here that would NOT be recompiled:

=dev-libs/lzo-1.08-r1

=sys-devel/automake-1.4_p6

=sys-devel/automake-1.8.5-r3

=sys-kernel/gentoo-sources-2.6.17-r4

This information is important because these packages are also not recompiled by the recompile-remaining-packages script. Btw, lzo is in that list because of the lack of it's DEPEND in the kino ebuild so I --oneshot it in to keep it out of the world file(as stated above, I already filed a bug about this). I don't know why the other three are not recompiled by emerge -e world. I have 6 versions of automake but only 2 are not recompiled by emerge -e world, and those 2 are not removed by --depclean.

Now I'm off to figure out a one-liner to tell which packages in the world file are dependencies and so may be removed from the world file. 

Also, my world file is located at /var/lib/portage/world, not /var/portage/world as this howto has it. Is there a default or even an option to change where the world file is located?

Thank you... and I hope that I'm not being too annoying here.

----------

## paulj

 *Guenther Brunthaler wrote:*   

> Hi all,
> 
> (You are reading version 1.15 of this guide.)
> 
> ABSTRACT: How to recompile your entire system / toolchain with MINIMUM PROCESSING TIME EFFORT.
> ...

 

Guenther, you are a star   :Very Happy:  . I have had several trying periods over the last few years with GCC upgrades, and system recompilation, and your script has just made this process much smoother. I am now in the final stages of a full recompilation using your script, and I am sure it has saved me much heartache. Due to my misunderstanding of the changes to the profiles (specifically the need to move from 2006.0 to 2006.1/desktop, not 2006.1) I have had to carry out the compilation operation twice. First time I used the time honoured method, and this time with your script.

THANKYOU!

----------

## Guenther Brunthaler

 *AssociateX wrote:*   

> Also, my world file is located at /var/lib/portage/world, not /var/portage/world as this howto has it.

 

You are absolutely right. It's /var/lib/portage/world, not /var/portage/world.

What you have found was clearly a typo in my guide!

Thank you for reporting it!

I have just fixed it as of version 1.16.

----------

## Guenther Brunthaler

Fixed a rather embarassing bug in version 1.16: The log file was created with the 'minute' number at the position of the 'month' number.  :Wink: 

----------

## steveL

Hi

  Firstly, thanks for the script!

  I'm using the latest version that you uploaded today. However I get an error (unexpected token >) at line 62 of the recompile-remaining-packages script, which is `exec > >('

Any ideas?

Edit:

Also, is it really necessary to emerge world newuse etc before running the script? Surely these are all going to be compiled anyway?

----------

## Guenther Brunthaler

 *steveL wrote:*   

> I get an error (unexpected token >) at line 62 of the recompile-remaining-packages script, which is `exec > >('

 

Perhaps you might have been using a very old version of bash? My script assumes a moderately modern version of bash.

The >(...) expression is a process substitition, which will be replaced by the filename of an ephemeral named pipe, which will be connected to the statements in the parentheses. Those statements will be run as a background process, and will read from the pipe; implementing the log file feature.

That is, the

```
exec > >(...)
```

is actually an

```
exec > anonymous_pipe
```

where anonymous_pipe has been constructed on the fly by bash.

However, process substitution is not a feature that all shells provide. For instance, ash does not.

That's why my script starts with

```
#! /bin/bash
```

and does not contain the usual

```
#! /bin/sh
```

 *steveL wrote:*   

> Also, is it really necessary to emerge world newuse etc before running the script? Surely these are all going to be compiled anyway?

 

Yes and no. The problem is, my script assumes a working compiler and tools to be available when it starts its job.

That is, tools like gcc, ld, glibc, make, sed, bison, flex, cp, ar, as, libtool and any other tools involved in the process of an "emerge" must be in a consistent state.

But certainly it will be unnecessary to bring packages like openoffice up to date just for rebuilding the whole system.

However, most people run

```
emerge -avuDN world
```

on a regular basis, which means their systems are close to being up-to-date anyway, and the additional emerge won't hurt.

If you know what you are doing, it is certainly possible to just do a

```
emerge --sync
```

and omit the the emerge -avuDN world step.

But I will not take any responsibility for problems arising from an improperly functioning toolchain, in case the tool chain was in an inconsistent state.

----------

## steveL

Thank you for the lucid explanation, on both points.

It really is cool to learn something new! I didn't know about anonymous pipes.

I am running bash-3.1_p17; while I don't know if this is old, I doubt it since I only installed this system a month ago. Is there anything else that could cause this?

On the second point, how does the emerge world guarantee a consistent toolchain; surely that's the point of the whole recompile everything metaphor?

Also (minor point) the version given in the generated script is "1.13".

----------

## Guenther Brunthaler

 *steveL wrote:*   

> I didn't know about anonymous pipes.

 

Well, actually they are not really anonymous from the viewpoint of the kernel, but rather temporary: Bash uses mkfifo or something similar internally to create a pipe with a temporary name in /tmp (I guess), then runs the process substitution, and eventually removes the pipe automatically.

Which means the pipe will actually have some name, but from the viewpoint of the shell script it works like an anonymous pipe. It's also fun and easy to use!

...as long as it works. What seems to be a problem with your system configuration.

 *steveL wrote:*   

> I am running bash-3.1_p17; while I don't know if this is old

 

My bash --version gives "3.1.16(1)-release (i686-pc-linux-gnu)", from which I conclude your bash should be good enough.

So the problem you encountered must be triggered by something else.

 *steveL wrote:*   

> Is there anything else that could cause this?

 

Perhaps it might be easiest to run a few tests manually before going into details how my script works.

First, let's check whether process substitution runs at all in your environment. Try this:

```
cat <( echo hello )
```

This should just print "hello". Note that there must not be a space between the "<" and the "(".

 *steveL wrote:*   

> On the second point, how does the emerge world guarantee a consistent toolchain; surely that's the point of the whole recompile everything metaphor?

 

The idea is very simple: emerge -avuDN world is intended to bring the system up-to-date, including any build tools.

It it doesn't, then Portage has a problem ... and my script will have one also.

 *steveL wrote:*   

> Also (minor point) the version given in the generated script is "1.13".

 

That's OK, the script and the guide have different version numbers. I am referring always to the guide version.

So far, the guide has been much more often updated than the script, and I'm incrementing the version number on each change for both of them independently.

But I realize this might be a cause for confusion; perhaps I'll change this in the future.

BTW: I'm going offline now (it's midnight where I live), so I'm not going to answer any more postings before tomorrow.

----------

## steveL

 *Guenther Brunthaler wrote:*   

> 
> 
> My bash --version gives "3.1.16(1)-release (i686-pc-linux-gnu)", from which I conclude your bash should be good enough.
> 
> ..
> ...

 

Yes it does.

 *Quote:*   

> .. the script and the guide have different version numbers. I am referring always to the guide version.
> 
> So far, the guide has been much more often updated than the script, and I'm incrementing the version number on each change for both of them independently.
> 
> But I realize this might be a cause for confusion; perhaps I'll change this in the future.
> ...

 

Ok, no biggy.

 *Quote:*   

> 
> 
> BTW: I'm going offline now (it's midnight where I live), so I'm not going to answer any more postings before tomorrow.

 

Understood. Thanks for responding!

----------

## Guenther Brunthaler

 *steveL wrote:*   

> Yes it does.

 

OK, so input redirection works. Now let's check output redirection.

The following script

```

#! /bin/bash

exec > >(

   echo "starting output"

   while read LINE; do

      echo "OUTPUT: $LINE"

   done

   echo "stopping output"

)

echo "Hello, world!"

echo "Hello, universe!"

```

should generate this output:

```

starting output

OUTPUT: Hello, world!

OUTPUT: Hello, universe!

stopping output

```

However, as the output is generated by a background process, it is possible that the shell prompt for the next command might be messed up by the output. This is normal under such conditions.

----------

## steveL

No, this gives me the same error:

test.sh: line 2: syntax error near unexpected token `>'

test.sh: line 2: ` exec > >( '

Guess we've found the problem (and it's not your script.) Just wish I knew what the sol'n was!

----------

## Guenther Brunthaler

 *steveL wrote:*   

> Guess we've found the problem (and it's not your script.) Just wish I knew what the sol'n was!

 

I think now it must have something to do with you bash's settings.

You know, the operation of bash is influenced by several environment variables like SHELLOPTS and POSIXLY_CORRECT as well as by builtins such as shopt and set.

I therefore suggest that you check those settings.

For instance, if your bash runs in POSIX mode, it will disable most extensions, including process substitution. Read section 6.11 of the bash info manual for more details on POSIX mode.

----------

## ck42

One request....

my / partition is fairly small (512M) and the log file that is created and kept in /root is getting VERY close to filling that partition.  Not sure what happens at the point that the partition is full, but I'd rather not find out.  I have another ~300 packages left and only about 90Megs left.

Can you provide an option in the script to prompt for the log file location?

----------

## Guenther Brunthaler

 *ck42 wrote:*   

> Can you provide an option in the script to prompt for the log file location?

 

A good point. Having a small root partition is actually a preferable system layout, especially when some logical volume manager is used for the other partitions.

I will add a command line option for this in the next version, allowing to freely specify the base directory where log files will be created.

I'm also considering to change the default base directory to somewhere below /var/tmp, as there is typically plenty of space there in Gentoo installations.

Or perhaps just using the system logger? What are services like syslog-ng and cron/logrotate good for, if not for such tasks? But that's just an idea yet.

In the meantime, please manually edit the generated script and change the $HOME part of the line

```
LOG_FILE="$HOME/${BASENAME}_$(date '+%Y-%m-%dT%T').log"
```

to a directory of your choice.

----------

## ck42

Thanks.

Looks like ~60 packages left....and about 60Megs left.  I think I'll make it this time.  For other systems I'll go in edit $HOME though...until the option is made available.

----------

## steveL

 *Guenther Brunthaler wrote:*   

> I think now it must have something to do with you bash's settings.
> 
> You know, the operation of bash is influenced by several environment variables like SHELLOPTS and POSIXLY_CORRECT as well as by builtins such as shopt and set.
> 
> I therefore suggest that you check those settings.
> ...

 

I don't think it's in POSIX mode; POSIXLY_CORRECT is not set.

echo $SHELLOPTS gives:

braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor

(no posix)

shopt gives:

cdable_vars     off

cdspell         off

checkhash       off

checkwinsize    on

cmdhist         on

dotglob         off

execfail        off

expand_aliases  on

extdebug        off

extglob         off

extquote        on

failglob        off

force_fignore   on

gnu_errfmt      off

histappend      on

histreedit      off

histverify      off

hostcomplete    on

huponexit       off

interactive_comments    on

lithist         off

login_shell     off

mailwarn        off

no_empty_cmd_completion off

nocaseglob      off

nocasematch     off

nullglob        off

progcomp        on

promptvars      on

restricted_shell        off

shift_verbose   off

sourcepath      on

xpg_echo        off

Could it be one of those?

----------

## Guenther Brunthaler

 *steveL wrote:*   

> echo $SHELLOPTS gives:
> 
> braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor

 

Looks good.

 *steveL wrote:*   

> shopt gives:

 

I diff'ed your settings and mine:

```

$ diff -bu yours mine

--- yours       2006-10-01 06:30:01.000000000 +0200

+++ mine        2006-10-01 06:30:12.000000000 +0200

@@ -7,15 +7,15 @@

 execfail off

 expand_aliases on

 extdebug off

-extglob off

+extglob                on

 extquote on

 failglob off

 force_fignore on

 gnu_errfmt off

 histappend on

-histreedit off

-histverify off

-hostcomplete on

+histreedit             on

+histverify             on

+hostcomplete           off

 huponexit off

 interactive_comments on

 lithist off

```

The only flag that seems interesting here is the extglob flag.

But unfortunately, setting extglob to off did not change anything on my box.

Which means there must be some other reason for your shell to not support process substitution.

Perhaps it's not a shell issue, but rather a system issue?

Bash supports process substitution if it is available - but what if it's not avaiable for some reason?

Do you perhaps use a special/customized version of the Kernel oder glibc? Or is there anything else unusual / especially tailored in your system that you are aware about?

Perhaps you are running your Gentoo installation on a special hardware platform (PDA or something)?

Obviously there must be some presumably small but essential difference in our basic system configurations; we only have to find it.

I have an up-to-date system using the default-linux/x86/2006.1/desktop profile; standard PC-platform. Nothing special regarding bash or the typical Gentoo execution environment.

----------

## steveL

Firstly, can I just say thanks for trying to help me with this.

 *Guenther Brunthaler wrote:*   

> The only flag that seems interesting here is the extglob flag.
> 
> But unfortunately, setting extglob to off did not change anything on my box.
> 
> Which means there must be some other reason for your shell to not support process substitution.

 

Agreed. according to man bash:

extglob If set, the extended pattern matching features described above under Pathname Expansion are enabled.

 *Quote:*   

> 
> 
> Perhaps it's not a shell issue, but rather a system issue?
> 
> Bash supports process substitution if it is available - but what if it's not avaiable for some reason?
> ...

 

I'm running gentoo on a desktop, same profile as you. The only possible difference I can think of is that I originally had -mfpmath=sse (it's an Athlon-XP) in my make.conf, and then I read that it actually works slower under current gcc, due to problems in glibc.

I installed my system (from stage 3) with gcc 4.1.1 (first thing i emerged from 2006.0 disk) and have never actually had any problems (apart from this one, of course!) I've reemerged glibc recently. I certainly haven't put any special options for bash or anything else I can think of. Are there kernel options I might have set? (Kernel is self-compiled.) It couldn't be anything to do with having linux-headers-2.11 originally when the kernel is 2.6.17-r7?

Although I'm sure bash has been updated since the kernel headers were..

I guess I could try recompiling the kernel for the newer version that came out recently (2.6.17-r8,) although that feels like a cop out.

I was going to try using your script to generate the list of packages and then this script to compile (there are instructions in that topic to do exactly that.) I'd still, however, like to solve this problem, tho hopefully it'd go away after a complete recompile.

----------

## Guenther Brunthaler

 *steveL wrote:*   

> 
> 
> I'm running gentoo on a desktop, same profile as you. The only possible difference I can think of is that I originally had -mfpmath=sse

 

I doubt this problem has something to do with FP math or special compiler options. This would render your whole system unstable, and not just that one specific bash feature.

Most likely it's the fault of some non-standard option in some configuration or initialization file related to bash operation.

If you haven't especially customized the bash initialization files in /etc, it might be easiest to re-emerge bash

```
emerge --oneshot bash
```

and then run etc-update or dispatch-conf and let those tools replace your old copies of the initialization files by up-to-date copies.

 *steveL wrote:*   

> I was going to try using your script to generate the list of packages and then this script to compile

 

That should no longer be necessary, as the current version of my script has already incorporated  count_zero's extensions.

Also note that the process substitution feature is not necessary for the primary operation of the script. It's only used for the newly added logfile feature.

Just delete the following lines

```

    exec > >(

         echo "Note: Logging to file '$LOG_FILE'."

         tee "$LOG_FILE"

         echo

         echo "Note: A log has been written to '$LOG_FILE'."

      )

```

from the generated script. The script will then run without generating a log file.

But you can do this yourself by running

```
./recompile-remaining-packages 2>&1 | tee mylogfile
```

Of course this will not solve your process substitution problem, but at least you can use my script to recompile your system.

If re-emerging bash does not help in re-activating the process substitution feature, then I'm afraid I have no more clues what to do next.

But I'll remember the issue and will tell you if any better ideas should happen to come to my mind.

----------

## steveL

 *Guenther Brunthaler wrote:*   

> I doubt this problem has something to do with FP math or special compiler options. This would render your whole system unstable, and not just that one specific bash feature.

 

Nah, didn't think it would be the problem. One thing that gets me with this whole thing is the issue of binary compatibility- after all gcc should be compiling to the same ABI (unless one has changed eg -malign-double) so personally I can't see the need for all this. (And yes I have read all the explanations as to why it's necessary; I just don't buy 'em.) I'm doing this more because I changed my NLS USE flag (to turn it on.)

 *Quote:*   

> Most likely it's the fault of some non-standard option in some configuration or initialization file related to bash operation.
> 
> If you haven't especially customized the bash initialization files in /etc, it might be easiest to re-emerge bash
> 
> ```
> ...

 

Does your script delete dependencies of failed packages from the list as his does? (Only I can't find that in the recompile script.)

 *Quote:*   

> 
> 
> Also note that the process substitution feature is not necessary for the primary operation of the script. It's only used for the newly added logfile feature.
> 
> Just delete the following lines
> ...

 

Thanks that's excellent.

 *Quote:*   

> Of course this will not solve your process substitution problem, but at least you can use my script to recompile your system.
> 
> If re-emerging bash does not help in re-activating the process substitution feature, then I'm afraid I have no more clues what to do next.
> 
> But I'll remember the issue and will tell you if any better ideas should happen to come to my mind.

 

Thank you for your excellent support.

----------

## Guenther Brunthaler

 *steveL wrote:*   

> 
> 
> Does your script delete dependencies of failed packages from the list as his does? (Only I can't find that in the recompile script.)
> 
> 

 

No, but that should not be too bad: The script tries to recompile the failing packages again at the end of the recompilation anway. And if the dependencies of the failing packages are already compiled at that time, they need not be compiled then.

This means the dependendies will be compiled sooner than necessary in this case, but they will not be compiled unnecessarily.

 *steveL wrote:*   

> Thank you for your excellent support.

 

I'm glad if I can help.

----------

## steveL

 *Guenther Brunthaler wrote:*   

> No, but that should not be too bad: The script tries to recompile the failing packages again at the end of the recompilation anway. And if the dependencies of the failing packages are already compiled at that time, they need not be compiled then.
> 
> This means the dependendies will be compiled sooner than necessary in this case, but they will not be compiled unnecessarily.

 

(Just being pedantic) can I suggest you incorporate that in your next update? I'm thinking of a situation where a library changes a header.

Again, vielen dank.

----------

## Guenther Brunthaler

 *steveL wrote:*   

> can I suggest you incorporate that in your next update? I'm thinking of a situation where a library changes a header.

 

Incorporate what?

If you meant the recompilation feature: This is already part of the current script!

 *steveL wrote:*   

> 
> 
> Again, vielen dank.
> 
> 

 

Gerne geschehen!  :Smile: 

----------

## steveL

I don't mean recompilation, I mean deferring dependencies until after the failed package.

So if a lib changed its header files, and the lib failed to emerge, then its dependencies would also be marked failed and consequently compiled after it.

Just trying to avoid dependency hell/ gentoo borks in the longer term.

----------

## Guenther Brunthaler

 *steveL wrote:*   

> I don't mean recompilation, I mean deferring dependencies until after the failed package.

 

Oh, I see.

 *steveL wrote:*   

> So if a lib changed its header files, and the lib failed to emerge, then its dependencies would also be marked failed and consequently compiled after it.

 

This sounds great, indeed.

But I'm afraid I have no idea how to implement this: My script uses the combined output of emerge -ep system and emerge -ep world as input for its operation.

This is a long list of packages to be compiled. But there is no indication which of the packages in the list are dependencies.

So my script simply can't tell which of those packages are dependencies of what other packages ... it all compiles them in order, skipping failing packages (and retrying them again at the end).

Certainly the dependency information is dug somewhere inside Portage, but like most of us I have only very limited knowledge about the internals of Portage: I'm not a Gentoo dev guru.

So I'm afraid I cannot implement that feature with my current level of knowledge.  :Wink: 

Perhaps I'll retry some time later after I have gained more Gentoo experience.

----------

## steveL

Well count_zero's script uses:

```

       failedpkgdeps=`echo $failedpkg | sed 's/-[0-9].*//'` 

       deps=`equery depends $failedpkgdeps | sed '/^\[/d' | sed 's/-[0-9].*//'` 

       for each in `echo $deps` 

       do 

          if [[ -n `cat $emergelist | grep "$each"` ]] 

             then 

             if [[ -n `emerge -p $each | grep "$failedpkg"` ]] 

                then 

                each2=`cat $emergelist | grep $each | sed 's/\=//'` 

                echo "$each2 (depends on $failedpkg)" >> $failedlist 

                eachsed=`echo $each | sed 's|\/|\\\/|'` 

                cat $emergelist | sed "/$eachsed/d" > $emergetemp 

                mv $emergetemp $emergelist 

                echo "*** $each depends on $failedpkg, skipping." 

                else : 

             fi 

             else : 

          fi 

       done 

       echo "*** Continuing with emerge world"

```

That should give you a start?

I'm off now, but I'll check on this later.

PS: Why doesn't (or how does) what this topic work? This is the 2nd topic I've tried to watch, but I never get any indication that something's been added. (And I'm a bit tired of seeing `egosearch'  :Wink: )

----------

## Guenther Brunthaler

 *steveL wrote:*   

> That should give you a start?

 

I'll check it out.

Anyway, one has to be very cautious when dropping dependencies: What if a dependent package is also a direct or an indirect dependency of some other package?

If it skipped a package in such a situation, the other packages which depend on the skipped dependencies would fail building as well, potentially leading to an unwanted avalance-effect.

In order to track dependencies, my script also needed to run "equery depends" for each and every package, and record the results somewhere. This would require a major rewrite of my script.

Also, skipping dependencies is a rather futile effort anyway:

By definition, dependencies must be compiled before the package which uses them.

If a package using the dependencies fails to compile, it will be too late: The dependencies have already been compiled.

Of course, it makes sense to skip packages which depend on dependencies which are known to not have been built successfully.

Which brings us back to the problem of how to get, store and retrieve that list from within a simple shell script (I'm not using Perl or something in the generated script for stability reasons: Perl & friends depend on too many components which could result in problems when those programming envrironments are recompiled themselves during the system rebuild.)

And there is also the "statistical" argument: If the emerge -uDN world was performed successfully before running my script, chances for packages failing the subsequent system rebuild are rather small.

Which means, only a very small number of packages will be affected by those problems at all.

 *steveL wrote:*   

> PS: Why doesn't (or how does) what this topic work?

 

Via e-mail. Each time a followup-article is posted, an e-mail containing a link to the new entry will be sent to you.

As far as I can remember, it's all part of the forum user profile. At least I didn't change anything since I registered with the forum, and it has worked fine so far.

Perhaps there is a simple typing error in your registered e-mail-address? Be sure to check it.

----------

## steveL

 *Guenther Brunthaler wrote:*   

> Anyway, one has to be very cautious when dropping dependencies: What if a dependent package is also a direct or an indirect dependency of some other package?
> 
> If it skipped a package in such a situation, the other packages which depend on the skipped dependencies would fail building as well, potentially leading to an unwanted avalance-effect.

 

I think we're using the same word to opposite effect; I'm using dependency to mean a package which need this one. So a lot of things are dependencies of glibc for instance. But agreed dependencies of the dependencies (eg if glibc failed, and a lib using it which is then used by another package) do also need to be failed.

 *Quote:*   

> In order to track dependencies, my script also needed to run "equery depends" for each and every package, and record the results somewhere. This would require a major rewrite of my script.

 

No, you just need to check when a package fails as count_zero's does. (That bit would go in the main loop of recompile_remaining_packages.) Granted for every dependency that you moved to the fail list, you'd also have to check packages that require it, but that's still a lot less than `each and every package.' And as you said, you get to the fail list at the end. If a package fails again, you can just mark it (and it's dependencies) as permanently failed.

 *Quote:*   

> Also, skipping dependencies is a rather futile effort anyway:
> 
> By definition, dependencies must be compiled before the package which uses them.
> 
> If a package using the dependencies fails to compile, it will be too late: The dependencies have already been compiled.
> ...

 

This is where we're getting our words mixed up as outlined above. Sorry if I'm using dependency in an invert to the normal usage.

 *Quote:*   

> Which brings us back to the problem of how to get, store and retrieve that list from within a simple shell script (I'm not using Perl or something in the generated script for stability reasons: Perl & friends depend on too many components which could result in problems when those programming envrironments are recompiled themselves during the system rebuild.)

 

Well I reckon we can easily slot that bit into your bash script. I'll have a look in a little while.

 *Quote:*   

> And there is also the "statistical" argument: If the emerge -uDN world was performed successfully before running my script, chances for packages failing the subsequent system rebuild are rather small.
> 
> Which means, only a very small number of packages will be affected by those problems at all.

 

Yes, but it' be even better if we could only run an emerge --sync and then your script.

 *Quote:*   

> Via e-mail. Each time a followup-article is posted, an e-mail containing a link to the new entry will be sent to you.
> 
> As far as I can remember, it's all part of the forum user profile. At least I didn't change anything since I registered with the forum, and it has worked fine so far.

 

Thanks, I've finally been notified of a couple of updates. (Dunno why it wasn't doing them over the last few days, but wtf.)

----------

## count_zero

I think that you would run into problems trying to implement this dependency feature--you may as well just do it the old way, and allow the emerge world to die if a package fails.  

Suppose a package fails.  For example, I just picked out a random package from xorg, x11-misc/xbitmaps.  The following packages on (my) system "depend" on xbitmaps:

x11-base/xorg-server-1.1.1

x11-apps/xsetroot-1.0.1

x11-libs/openmotif-2.2.3-r9

Now these packages have to be considered "failed" packages as well, and here are the packages that depend on them:

x11-drivers/xf86-input-keyboard-1.1.0

x11-drivers/xf86-video-nv-1.2.0

x11-drivers/xf86-video-fbdev-0.3.0

x11-drivers/xf86-video-vesa-1.2.1

x11-drivers/xf86-input-mouse-1.1.1

x11-drivers/synaptics-0.14.5-r1

x11-drivers/nvidia-drivers-1.0.8774

gnome-base/libbonoboui-2.14.0

gnome-base/libgnomecanvas-2.14.0

x11-base/xorg-x11-7.1

x11-libs/gtk+-2.8.20-r1

media-video/kaffeine-0.8.1-r1

virtual/x11-7.0-r2

kde-base/kdebase-startkde-3.5.4

This is as far as I went, but you would have to do the same for *each* of these packages as well--you would delay building any packages that depend on them, and those that depend on them, et cetera--as Guenther put it, an "avalanche" effect.  Is it really best to delay building all of these packages for xbitmaps?  Do I care if kaffeine is built upon an xorg-server that is built with the latest xbitmaps?  Maybe, but probably not.  I already have an older version of xbitmaps that will allow xorg to compile.   Skipping xbitmaps and allowing the rest of the system to continue building is probably good enough.

Just my 2 cents.  :Smile: 

----------

## steveL

 *count_zero wrote:*   

> This is as far as I went, but you would have to do the same for *each* of these packages as well--you would delay building any packages that depend on them, and those that depend on them, et cetera--as Guenther put it, an "avalanche" effect.  Is it really best to delay building all of these packages for xbitmaps?  Do I care if kaffeine is built upon an xorg-server that is built with the latest xbitmaps?  Maybe, but probably not.  I already have an older version of xbitmaps that will allow xorg to compile.   Skipping xbitmaps and allowing the rest of the system to continue building is probably good enough.
> 
> Just my 2 cents. 

 

Blimey that's a lot. Fair enough. So what does your script do (one layer of depends?)

----------

## count_zero

My script looks for packages that require the failed package to be built before proceeding.  For instance, continuing with my example above:

if xbitmaps were to fail, the script would perform "equery depends x11-misc/xbitmaps", producing these packages as dependents:

x11-base/xorg-server-1.1.1 

x11-apps/xsetroot-1.0.1 

x11-libs/openmotif-2.2.3-r9

These aren't automatically removed from the emerge queue.  Rather, the output of emerge -p is performed on each of them.  If xbitmaps were really needed to proceed, portage would include xbitmaps in the output of emerge -p.  However, since I already have an old version of xbitmaps, portage doesn't require that the new version be installed before proceeding with these emerges.  Thus, the script only removes packages which must be built before proceeding.  

I hope I explained this okay.

----------

## gerard27

Hi Günther,

Man this is some script you wrote!

My box is churning away now.

I am having a problem with php not wanting to compile for all kinds of reasons.

When I did an emerge -uD world it would stop and I'd have to issue emerge --resume --skipfirst to

continue.Now I can go to bed and it will simply wait till all the rest is compiled.

My box runs alright without it even though some programs depend on it.

I don't do any php coding.

Thanks a lot man.

Gerard

----------

## Guenther Brunthaler

 *steveL wrote:*   

> I think we're using the same word to opposite effect

 

I see; you are right. Then it's also clear to me now what you actually wanted to suggest in the first place.

 *steveL wrote:*   

> No, you just need to check when a package fails as count_zero's does.

 

Initially, I was not sure whether this could be dangerous: My script uses static version numbers; that is it uses the version numbers (and dependencies) at the time when the script was generated.

I thus had concerns whether an equery depends, run at some later time, would actually return the same dependency version numbers, or different ones (after another emerge --sync).

But as I see it now, there are two possibilities in this case:

The version numbers of the dependencies are still the same. In this case it would not be a problem to move the matching packages to the list of failing packages.

The version numbers are different. In this case the match will not be detected and the dependent packages will still be emerged, resulting in a potential failure. But this is actually the same as it happens now for each package, and thus not worse than it is now.

However, there is still another problem with the suggested approach: Circular dependencies.

Unfortunately, some ebuilds contain circular dependencies.

That is, package A depends on package B, and package B depends on package A.

Although such a relationship is clearly illegal from a logical point of view, it nevertheless is a fact that the Portage tree contains such dependencies.

When emerge --emptytree world is run, it selects an arbitrary package order (such as by lexicographical order of the package name), and uses that order in the package list it generates.

So, let's say the real dependency were: First package B, then package A. But emerge --emptytree shows the opposite order.

In this case, the current version of my script will first compile package A, which will fail (because package B is not available), and put package A into the list of failing packages.

Then it will try to compile package B (because it was not aware of the fact that package B seems to be dependent on package A), which will succeed (because package B is not really dependent on package A).

Finally, after all the other packages have been recompiled, my script will recompile package A again. And this time it will succeed, because package A is already there.

Which means, my script will not be severely affected by incorrect dependency entries in Portage: Due to its incremental nature, any package order will eventually work.

The script would even succeed if no package dependency information was available at all and if the packages are displayed by emerge --emptytree in arbitrary random order.

In other words: The script takes a safe approach which allows to succeed even in case of conflicting or even completely wrong dependency ordering as reported by equery depend.

If it would honor dependency information, it could be sped up by skipping dependent packages, but it would lose its capability to reliably resolve circular dependencies.

 *steveL wrote:*   

> If a package fails again, you can just mark it (and it's dependencies) as permanently failed.

 

As outline above, this could be dangerous in the case of circular dependencies, especially in cases where indirect dependency cycles are used: Package A depends on B depends on C depends on A.

The current approach allows to resolve all such dependencies correctly (if they can be resolved at all), being tolerant to incorrect dependency information.

The simple algorithm used also guarantees that the process will terminate eventually, never resulting in an infinite loop.

----------

## hielvc

 *Quote:*   

> When emerge --emptytree world is run, it selects an arbitrary package order (such as by lexicographical order of the package name), and uses that order in the package list it generates. 

 

Wrong. If you check out "man 5 ebuild " You have DEPEND, RDEPEND, and  PDEPEND. Portage goes thourgh the ebuilds checking the depends such that  kde isnt built before X11. The major problem for scripts like ours is we are getting list from "'emerge blah -p > file" but portage doesnt run the ebuilds in --pretend and will miss some of the DEPEND's. Heck it miss's some ocassionally when running for real. In the last week 1 or 2 patchs have been rreleashed improving this. 

In  emwrap.sh one of the options is to build "world -e" minus "system -e" where for a small gain in speed I sort the system and world outputs "comm" them to remove the system files and then useing the orignal world sort the "world - minus system" files back into the same order for building.

As for what to do with failed builds I stick them into file called 'faild' list them at the end and the next time emwrap.sh is run offer to build them. In the case of "emwrap.sh world -uDN it isnt really important just re-run emwrap.sh -wuDn and anything needing updateing show up again. In the case of a system/world --emptytree it can but rarely does cause problems. Its only when your useing it to rebuild a stage3 or stage1 form scratch that you could end up wtih a really long failed list   :Wink: 

----------

## Guenther Brunthaler

 *hielvc wrote:*   

>  *Quote:*   When emerge --emptytree world is run, it selects an arbitrary package order (such as by lexicographical order of the package name), and uses that order in the package list it generates.  
> 
> Wrong. If you check out "man 5 ebuild " You have DEPEND, RDEPEND

 

Sorry for being not specific enough. I meant the above in the context of circular dependencies only, where there seems not to be any "right order" to do things.

In this context only I assumed Portage will select an arbitrary order, such as lexicographical ordering (because I have no idea how it actually determines the package order in such cases).

----------

## Guenther Brunthaler

Hi all,

Due to multiple misunderstandings how my script resolves "circular" (i. e. incorrectly reported) dependencies, I have written a simulation in order to demonstrate by example how the algorithm works.

Here is the output of the simulation run:

```

"recompile-entire-system" Algorithm Simulation

Number of simulated packages: 200

Number of simulated dependencies: 600

Packages in simulated "emerge --emtpytree" order: DC, E, ES, GL, BF, EH, GG, 

    FJ, FM, AA, CH, CB, GQ, P, BE, BR, BU, GS, GA, AF, FC, EQ, CJ, R, F, K, GO,

    AW, BP, FY, EE, CS, CW, BK, DQ, CL, Y, DK, EJ, CV, GF, DL, FR, EW, DM, CC, 

    DW, CY, AJ, DT, DJ, GM, ET, EM, AO, DP, C, CU, I, AD, FZ, DE, BH, GE, CA, 

    CI, B, EV, BW, AY, ED, FD, EA, Z, FS, G, EU, BV, GR, AM, L, CX, GP, AP, FU,

    CF, AB, FG, GI, BN, BY, FV, FL, W, FH, FP, EK, DZ, CK, J, CM, BC, T, DH, 

    BD, GK, FE, AN, BQ, X, AX, EX, DN, EL, V, EY, FN, EN, AU, H, CN, DY, CR, 

    DX, DO, BI, GH, AE, EG, FX, M, AV, EP, GB, CE, DU, D, BB, BX, EI, AT, AS, 

    EF, O, GC, DV, BM, BT, AK, BL, BZ, DA, DF, AC, GD, AZ, DB, DI, S, AR, AQ, 

    FF, ER, DS, EO, BG, FB, EB, CQ, BJ, U, FI, FA, FO, FQ, GN, N, DR, BS, GJ, 

    BA, AI, FK, CZ, FW, EC, EZ, DD, BO, CP, CO, CG, AG, AH, FT, AL, CD, DG, Q, 

    CT.

Dependencies (Format: "package(dependencies)"): AA(EI), AB(BL, BR, D), AC(Z), 

    AD(J), AE(BO, CO), AG(AB), AH(AI, AJ, Z), AK(DY, Z), AL(CV, DL, R), AM(AJ, 

    AR, BL, M, Z), AN(AJ, AR, BY, CY, F, FE, FQ), AO(DF), AP(AJ, FE, J, Z), 

    AQ(AF, AI, FE), AR(AF, FJ), AS(AI, CZ, EJ, FJ, FR), AT(AO, CR, DT, DY, GQ),

    AU(AF, CV, FE), AV(BT, ED, F, GH), AW(AJ, CM), AX(CM, CV, DT), AY(DF, DT, 

    FQ, J, X), AZ(BD, CB, DF, DS, EA), B(AC, BD, BM, BS, CP, EF, GG, GK), 

    BA(AJ, BS, CM, D, FM), BB(BL, Z), BC(AF, AH, AK, CO, EG, X), BD(BL, BT), 

    BE(AM, F, X), BF(EK, FG, GQ), BG(BS, CH, G, GQ, J, X), BH(AJ, BS, D, FJ, 

    GR), BI(EW), BJ(BL, F, FG, J, R, T), BK(BD, CX, FJ, X), BM(FE), BN(BL, CY, 

    EW), BO(EK, FD), BP(BS, CO, ES), BQ(AR, DC, EW, FE), BR(CV, ED), BS(DF), 

    BT(Z), BU(BI, CO, DF, F, GL), BV(BL, FT), BW(AR, BM), BX(AB, DT, GK, J), 

    BY(AF), BZ(BY), C(AF, AJ), CA(EP, FG, GK), CB(AQ, BL, DS, EX), CC(AR, BZ, 

    DH, ET), CD(CO, DV, EA, FG), CE(EQ), CF(BY, D, EQ, J), CG(FJ, GM, GO, X), 

    CH(AV, ER), CI(BF, BS, DT, EW, F), CJ(DF, DH, Q), CK(AI, EP, FG, GM, J), 

    CL(AQ, D), CM(FJ), CN(AC, AO, BM, CO, CZ, R), CO(ED), CP(CV, DF, DL, FJ, 

    FT), CQ(AI, BZ, CM), CR(CV, D, DF, DL, J), CS(BL, BS, GP), CT(CM, DL, F, 

    Z), CU(BN), CV(DF), CW(AJ, BT, CZ, D, DV, F), CX(BR, DI), CY(BL, CM, EW), 

    D(AR, BL, CO, T), DA(CV, DF, GB), DB(BS, CZ, EI), DC(FQ, GQ), DD(AJ, GM), 

    DE(AB, BI, FE, GK), DG(C, CO, GF), DH(BD), DI(EW, FO, GK), DJ(AU, CV, DL, 

    EW), DK(ER), DM(CO, FR), DN(AF, BM, DV, EC), DO(AI, AR, BV, DT, EK, EW), 

    DP(AD, BT, FG, FJ), DQ(CQ, T), DR(CM, CV, ED, FT), DS(BI, EM), DU(AF, DR, 

    DT, FT), DV(FG), DW(DV, EI, GG), DX(AR, DS, FE, FG, GQ), DY(AI, DL, EW), 

    DZ(AH, CM), E(AC, AJ, CH, FE, FG, S), EA(AF, DL), EB(AC, BD, BI, BQ), 

    EC(AR, DL, FO, GK), EE(AF, X, Z), EF(DF, EW, FE), EG(DF, EW, FJ, GF), 

    EH(CO, FG, GK, GQ, X), EI(AF, D, GQ, J), EJ(ED, FG, FT, Y), EK(AR, F), 

    EL(EK, EX, FE, FG), EM(G), EN(AF, AU, FK), EO(AQ, BY, CU), EP(CV, F, GK, 

    J), EQ(CP, DF), ER(AR, BY, EF, GK), ES(AK, CM), ET(BL, BV, FU), EU(AI, BF, 

    CM, CO, CR, DL), EV(BI, BM), EW(F, GK), EX(BT, DR), EY(BT, FE, FT, GK, T), 

    EZ(BT, CN, E, EP, FG), FA(CO, FJ, FU), FB(C, EW, F, FG, GJ), FC(AR, BD, DD,

    DV), FD(BB, FG, GK), FE(ED), FF(AS, CZ, EA, EF), FH(BM, DT, FY), FI(AR, F, 

    FE, GP), FJ(AJ), FK(AI, DN, DT, FO), FL(AC, CZ, EA, FJ, J), FM(AO, CM, FJ, 

    J), FN(AF, AH, AI, CO, FE, GA, J), FO(CV, FE, FG, FT, J, T), FP(AK, T), 

    FQ(CY, DV), FR(DV), FS(AI, AO, FM), FU(J, O), FV(AQ, CV, EI), FW(GK, Z), 

    FX(BR, GB), FY(AJ, BN, CV, EK, FE), FZ(CV, EE, FT, T), G(BM, CO), GA(AI, 

    DT, FE), GB(BW, DF, DL, FJ), GC(AI, BN, FJ), GD(CY, FJ), GE(FA, GK, X), 

    GF(BL, DF, FO), GG(AC, DV, ED, FE, GK), GH(CV, DF, DT), GI(BQ, CV, DL), 

    GJ(DL, GF), GL(BW, DV, ED, F, FJ, FT, X), GM(AR, BM, EP, FM, Z), GN(CV, DU,

    GJ), GO(AD, DV), GP(AO, BQ, DI, DY, FE, GK), GQ(BS), GR(DF, DX, GK, GQ), 

    GS(CO, DT, F, T), H(DF, DZ, EW), I(CZ, DM), J(BM, F), K(AH, AM, CV, DY, FT,

    GF), L(BL, BT, CO, DF, G, GK), M(AI, AR, BT), N(DF), O(EA, GQ), P(AC, BD, 

    BM, J), Q(EI), R(DS, FS), S(EA, J), T(BM), U(BD, CZ, FT, M), V(BY, FJ), 

    W(AF, BL, DM, DS, FJ), X(AJ, DV), Y(BC, DF, ED, ER, FG, GG, GK), Z(DF).

Phase 1:

Packages left to be recompiled in order: DC, E, ES, GL, BF, EH, GG, FJ, FM, AA,

    CH, CB, GQ, P, BE, BR, BU, GS, GA, AF, FC, EQ, CJ, R, F, K, GO, AW, BP, FY,

    EE, CS, CW, BK, DQ, CL, Y, DK, EJ, CV, GF, DL, FR, EW, DM, CC, DW, CY, AJ, 

    DT, DJ, GM, ET, EM, AO, DP, C, CU, I, AD, FZ, DE, BH, GE, CA, CI, B, EV, 

    BW, AY, ED, FD, EA, Z, FS, G, EU, BV, GR, AM, L, CX, GP, AP, FU, CF, AB, 

    FG, GI, BN, BY, FV, FL, W, FH, FP, EK, DZ, CK, J, CM, BC, T, DH, BD, GK, 

    FE, AN, BQ, X, AX, EX, DN, EL, V, EY, FN, EN, AU, H, CN, DY, CR, DX, DO, 

    BI, GH, AE, EG, FX, M, AV, EP, GB, CE, DU, D, BB, BX, EI, AT, AS, EF, O, 

    GC, DV, BM, BT, AK, BL, BZ, DA, DF, AC, GD, AZ, DB, DI, S, AR, AQ, FF, ER, 

    DS, EO, BG, FB, EB, CQ, BJ, U, FI, FA, FO, FQ, GN, N, DR, BS, GJ, BA, AI, 

    FK, CZ, FW, EC, EZ, DD, BO, CP, CO, CG, AG, AH, FT, AL, CD, DG, Q, CT.

Packages successfully compiled: AF, F, DL, AJ, DT, C, ED, EA, FG, BY, GK, FE, 

    DV, BM, BL, BZ, DF, N, BS, AI, CZ, CO, FT, CD.

Failed packages: DC, E, ES, GL, BF, EH, GG, FJ, FM, AA, CH, CB, GQ, P, BE, BR, 

    BU, GS, GA, FC, EQ, CJ, R, K, GO, AW, BP, FY, EE, CS, CW, BK, DQ, CL, Y, 

    DK, EJ, CV, GF, FR, EW, DM, CC, DW, CY, DJ, GM, ET, EM, AO, DP, CU, I, AD, 

    FZ, DE, BH, GE, CA, CI, B, EV, BW, AY, FD, Z, FS, G, EU, BV, GR, AM, L, CX,

    GP, AP, FU, CF, AB, GI, BN, FV, FL, W, FH, FP, EK, DZ, CK, J, CM, BC, T, 

    DH, BD, AN, BQ, X, AX, EX, DN, EL, V, EY, FN, EN, AU, H, CN, DY, CR, DX, 

    DO, BI, GH, AE, EG, FX, M, AV, EP, GB, CE, DU, D, BB, BX, EI, AT, AS, EF, 

    O, GC, BT, AK, DA, AC, GD, AZ, DB, DI, S, AR, AQ, FF, ER, DS, EO, BG, FB, 

    EB, CQ, BJ, U, FI, FA, FO, FQ, GN, DR, GJ, BA, FK, FW, EC, EZ, DD, BO, CP, 

    CG, AG, AH, AL, DG, Q, CT.

Phase 2:

Packages left to be recompiled in order: DC, E, ES, GL, BF, EH, GG, FJ, FM, AA,

    CH, CB, GQ, P, BE, BR, BU, GS, GA, FC, EQ, CJ, R, K, GO, AW, BP, FY, EE, 

    CS, CW, BK, DQ, CL, Y, DK, EJ, CV, GF, FR, EW, DM, CC, DW, CY, DJ, GM, ET, 

    EM, AO, DP, CU, I, AD, FZ, DE, BH, GE, CA, CI, B, EV, BW, AY, FD, Z, FS, G,

    EU, BV, GR, AM, L, CX, GP, AP, FU, CF, AB, GI, BN, FV, FL, W, FH, FP, EK, 

    DZ, CK, J, CM, BC, T, DH, BD, AN, BQ, X, AX, EX, DN, EL, V, EY, FN, EN, AU,

    H, CN, DY, CR, DX, DO, BI, GH, AE, EG, FX, M, AV, EP, GB, CE, DU, D, BB, 

    BX, EI, AT, AS, EF, O, GC, BT, AK, DA, AC, GD, AZ, DB, DI, S, AR, AQ, FF, 

    ER, DS, EO, BG, FB, EB, CQ, BJ, U, FI, FA, FO, FQ, GN, DR, GJ, BA, FK, FW, 

    EC, EZ, DD, BO, CP, CG, AG, AH, AL, DG, Q, CT.

Packages successfully compiled: FJ, GQ, GA, CV, FR, EW, DM, AO, I, Z, G, BV, J,

    CM, T, X, AX, V, AU, DY, BI, GH, EP, BB, EF, O, BT, AK, AC, S, AR, AQ, ER, 

    CQ, FO, DR, FW, EC, CP, AH, CT.

Failed packages: DC, E, ES, GL, BF, EH, GG, FM, AA, CH, CB, P, BE, BR, BU, GS, 

    FC, EQ, CJ, R, K, GO, AW, BP, FY, EE, CS, CW, BK, DQ, CL, Y, DK, EJ, GF, 

    CC, DW, CY, DJ, GM, ET, EM, DP, CU, AD, FZ, DE, BH, GE, CA, CI, B, EV, BW, 

    AY, FD, FS, EU, GR, AM, L, CX, GP, AP, FU, CF, AB, GI, BN, FV, FL, W, FH, 

    FP, EK, DZ, CK, BC, DH, BD, AN, BQ, EX, DN, EL, EY, FN, EN, H, CN, CR, DX, 

    DO, AE, EG, FX, M, AV, GB, CE, DU, D, BX, EI, AT, AS, GC, DA, GD, AZ, DB, 

    DI, FF, DS, EO, BG, FB, EB, BJ, U, FI, FA, FQ, GN, GJ, BA, FK, EZ, DD, BO, 

    CG, AG, AL, DG, Q.

Phase 3:

Packages left to be recompiled in order: DC, E, ES, GL, BF, EH, GG, FM, AA, CH,

    CB, P, BE, BR, BU, GS, FC, EQ, CJ, R, K, GO, AW, BP, FY, EE, CS, CW, BK, 

    DQ, CL, Y, DK, EJ, GF, CC, DW, CY, DJ, GM, ET, EM, DP, CU, AD, FZ, DE, BH, 

    GE, CA, CI, B, EV, BW, AY, FD, FS, EU, GR, AM, L, CX, GP, AP, FU, CF, AB, 

    GI, BN, FV, FL, W, FH, FP, EK, DZ, CK, BC, DH, BD, AN, BQ, EX, DN, EL, EY, 

    FN, EN, H, CN, CR, DX, DO, AE, EG, FX, M, AV, GB, CE, DU, D, BX, EI, AT, 

    AS, GC, DA, GD, AZ, DB, DI, FF, DS, EO, BG, FB, EB, BJ, U, FI, FA, FQ, GN, 

    GJ, BA, FK, EZ, DD, BO, CG, AG, AL, DG, Q.

Packages successfully compiled: ES, EH, GG, FM, BR, GS, EQ, AW, BP, EE, DQ, DK,

    GF, CY, DJ, GM, EM, AD, FZ, CA, EV, BW, FD, FS, L, AP, FU, BN, FL, FP, EK, 

    DZ, CK, BD, EX, DN, EL, EY, FN, H, DO, EG, M, AV, GB, CE, DU, D, EI, GC, 

    DA, GD, DB, DI, DS, U, FA, FQ, GJ, BA, FK, DD, BO, DG, Q.

Failed packages: DC, E, GL, BF, AA, CH, CB, P, BE, BU, FC, CJ, R, K, GO, FY, 

    CS, CW, BK, CL, Y, EJ, CC, DW, ET, DP, CU, DE, BH, GE, CI, B, AY, EU, GR, 

    AM, CX, GP, CF, AB, GI, FV, W, FH, BC, DH, AN, BQ, EN, CN, CR, DX, AE, FX, 

    BX, AT, AS, AZ, FF, EO, BG, FB, EB, BJ, FI, GN, EZ, CG, AG, AL.

Phase 4:

Packages left to be recompiled in order: DC, E, GL, BF, AA, CH, CB, P, BE, BU, 

    FC, CJ, R, K, GO, FY, CS, CW, BK, CL, Y, EJ, CC, DW, ET, DP, CU, DE, BH, 

    GE, CI, B, AY, EU, GR, AM, CX, GP, CF, AB, GI, FV, W, FH, BC, DH, AN, BQ, 

    EN, CN, CR, DX, AE, FX, BX, AT, AS, AZ, FF, EO, BG, FB, EB, BJ, FI, GN, EZ,

    CG, AG, AL.

Packages successfully compiled: DC, GL, BF, AA, CH, CB, P, BU, FC, R, GO, FY, 

    CW, CL, DW, ET, DP, CU, GE, CI, B, AY, AM, CX, CF, AB, FV, W, FH, BC, DH, 

    AN, BQ, EN, CN, CR, DX, AE, FX, BX, AT, AZ, EO, BG, FB, EB, BJ, GN, CG, AG,

    AL.

Failed packages: E, BE, CJ, K, CS, BK, Y, EJ, CC, DE, BH, EU, GR, GP, GI, AS, 

    FF, FI, EZ.

Phase 5:

Packages left to be recompiled in order: E, BE, CJ, K, CS, BK, Y, EJ, CC, DE, 

    BH, EU, GR, GP, GI, AS, FF, FI, EZ.

Packages successfully compiled: E, BE, CJ, K, BK, Y, EJ, CC, DE, EU, GR, GP, 

    GI, AS, FF, FI, EZ.

Failed packages: CS, BH.

Phase 6:

Packages left to be recompiled in order: CS, BH.

Packages successfully compiled: CS, BH.

Failed packages: <none>.

Success - all packages have been recompiled!

```

In the above example, the algorithm required 6 phases (passes) in order to reach its ultimate goal "no more packages left to be recompiled".

The simulator assumes a package will compile successfully if all its dependencies are already compiled at the time the package is due to be compiled.

The order in which compilations are attempted is the order as shown in the lists.

Note that the example uses random ordering of the packages, which is equivalent to having no dependency information available at all to the algorithm.

The real script uses the dependency information from Portage, which means multiple passes will only result from incorrectly reported dependencies (or from other temporary factors, such as an ebuild failing because the machine ran out of memory during the compilation when other processes ate up to much memory at that time).Last edited by Guenther Brunthaler on Tue Oct 03, 2006 3:19 am; edited 4 times in total

----------

## Guenther Brunthaler

For those who like to toy around with the simulation, here is the simulator script:

```

#! /usr/bin/perl -w

use strict;

srand 42;

my $NUM_PACKAGES= 200;

my $MAX_NUM_DEPS= 600;

my $MAX_TRIES= 1e5;

#

my($n, $i, $i2, @f, @s, %h, $k, $a, $t, @p);

# List of packages.

$k= 'A';

for ($n= $NUM_PACKAGES; $n--; ) {

   push @p, ++$k;

}

# Reorder as displayed by simulated "emerge -ep world".

for ($i= @p; $i--;) {

   $i2= int rand @p;

   @p[$i, $i2]= @p[$i2, $i];

}

# Generate random dependencies.

my %d;

sub dep {

   my($p, $on)= @_;

   return 1 if $d{$p} && $d{$p}->{$on};

   foreach (keys %{$d{$p} || {}}) {

      return 1 if &dep($p, $_);

   }

   return undef;

}

$t= 0;

for ($n= $MAX_NUM_DEPS; $n--; ) {

   ($i, $i2)= @p[int(rand @p), int rand @p];

   if ($i eq $i2 || exists $d{$i} && exists $d{$i}->{$i2} || dep $i2, $i) {

      last if ++$t > $MAX_TRIES;

      redo;

   }

   $d{$i}= {} unless exists $d{$i};

   $d{$i}->{$i2}= 1;

   $t= 0;

}

sub pwrap {

   our $c= 0; our $width= 79, our $oi; our $indent= " " x 4;

   my @a= map {

      $_ ne "\n" && /^\s+$/ ? " " : $_;

   } split /

      (?<= \n) | (?= \n)

      | (?<= \s) (?= \S)

      | (?<= \S) (?= \s)

   /xs, join '', @_;

   my $ol;

   for (my $i= 0; $i < @a; ) {

      if ($a[$i] eq "\n") {

         if ($c) {print "\n"; $c= 0}

         undef $oi;

         while ($i + 1 < @a && $a[$i + 1] eq "\n") {

            print "\n"; ++$i;

         }

      } else {

         $ol= length $a[$i];

         if ($c + $ol > $width) {

            print "\n"; $c= 0; $oi= 1;

            while ($i < @a && $a[$i] eq " ") {

               ++$i;

            }

            next;

         }

         if ($oi) {

            print $indent;

            $oi= 0; $c= length $indent;

         }

         print $a[$i]; $c+= $ol;

      }

      ++$i;

   }

}

sub sum {

   my $s= 0;

   for (my $i= @_; $i--; ) {

      $s+= $_[$i];

   }

   return $s;

}

pwrap

   "\"recompile-entire-system\" Algorithm Simulation\n\n"

   , "Number of simulated packages: ", scalar(@p), "\n"

   , "Number of simulated dependencies: "

   , sum(map scalar keys %{$d{$_}}, keys %d), "\n"

   , "Packages in simulated \"emerge --emtpytree\" order: "

   , join(", ", @p), ".\n"

   , "Dependencies (Format: \"package(dependencies)\"): "

   , join(

      ", ", map "$_(" . join(", ", sort keys %{$d{$_}}) . ")", sort keys %d

   ), ".\n\n"

;

for (my $p= 1; @p; ++$p) {

   pwrap

      "Phase $p:\nPackages left to be recompiled in order: "

      , join(", ", @p), ".\n"

   ;

   undef $a;

   PKG: foreach $k (@p) {

      foreach (keys %{$d{$k} || {}}) {

         if (!exists $h{$_}) {

            push @f, $k;

            next PKG;

         }

      }

      $a= $h{$k}= 1;

      push @s, $k;

   }

   pwrap

      "Packages successfully compiled: "

      , join(", ", @s, @s ? () : "<none>"), ".\n"

      , "Failed packages: ", join(", ", @f, @f ? () : "<none>"), ".\n\n"

   ;

   last unless $a;

   @p= @f; @s= (); @f=();

}

pwrap

   $a

   ? "Success - all packages have been recompiled!\n"

   : "Giving up - the remaining packages cannot be recompiled!\n"

;

```

Remove the 

```
srand 42;
```

 line if you want different results for each repeated run.

Change the assigned values for $NUM_PACKAGES, $MAX_NUM_DEPS and $MAX_TRIES to any values you like.

$NUM_PACKAGES is the total number of simulated packages.

$MAX_NUM_DEPS is the maximum number of dependencies to be generated. The script will try to create that number of dependencies, but will give up creating dependencies if it is unable to find a possible new dependency within $MAX_TRIES tries of random selection.

----------

## steveL

 *count_zero wrote:*   

> My script looks for packages that require the failed package to be built before proceeding.  For instance, continuing with my example above:
> 
> if xbitmaps were to fail, the script would perform "equery depends x11-misc/xbitmaps", producing these packages as dependents:
> 
> x11-base/xorg-server-1.1.1 
> ...

 

Makes perfect sense, thanks!

It seems your approach is stronger than just what depends on this, ie by using the -p flag. Why can't we just use that approach in the bash part of Guenther's script?

@Guenther - thanks for all the work! The simulator results were interesting.

Would we be able to use the algorithm count_zero outlines (ie if APPB depends on FAILED_APP, do an emerge --pretend of APPB to see if it would actually require FAILED_APP to be emerged first, and only if so move APPB to the failed list.)

----------

## Guenther Brunthaler

 *steveL wrote:*   

> ie if APPB depends on FAILED_APP, do an emerge --pretend of APPB to see if it would actually require FAILED_APP to be emerged first, and only if so move APPB to the failed list.

 

We could - if only Portage dependency information was to be trusted!

It's still the same problem: emerge --pretend also uses Portage's dependency information, which might be wrong. It's not only emerge --emptytree which is affected.

To be fair: Actually, it's not Portage's fault at all!

Not portage makes a mistake in such cases, but the ebuilds simply provide incorrect dependency information.

That is, they specify more dependencies than they really need.

Or - even worse - they do not specify dependencies which they actually do require.

In the first case, nothing disastrous happens (more packages are compiled than necessary), unless that incorrect dependency leads to a circular dependency graph.

In the second case, the missing dependency will go undiscovered as long as the actually required package happens already to be installed by coincidence (or rather due to dependency requirements of other packages which are already installed).

The algorithm in my script will handle both cases, because it is not really dependent on Portage's dependency information. While Portage's information will be used, it actually only serves as a speed-up for the algorithm.

So the problem is clearly in the ebuilds.

Unforunately, incorrectly specified dependencies are not necessarily easy to detect.

It's not always just so that a package says: "I depend on this and that package, period."

Rather there are typically conditional dependencies involved: "If this USE-flag is set, and that USE flag is not set, then I'll require that first dependency. Otherwise, if that third USE flag is set, and ..."

Furthermore, dependencies cannot only be triggered by USE-flags, but also on other types of conditions like package version number ranges. Actually, the full power of the shell programming language can be used to evaluate conditions, which means the complexity of dependency specification is potentially unlimited.

And if that was not complicated enough, the complexity is further increased by the fact that ebuilds can be inherited from other ebuilds, including their dependencies.

Summing up, dependency specifications can get pretty complex and error-prone in ebuilds, and one should not be too surprised if some ebuilds contains incorrect dependency information as a consequence.

----------

## Lloeki

just to be clear, dependency terminology is as follows:

dependencies are packages that a package needs

reverse dependencies are packages that need this package

out of the blue, here's a basic recursive dep' resolution algorithm, which I bet emerge (and lots of other package managers) uses, or something alike (maybe not recursive, but principle is the same):

```
function install(list)

foreach pkg in list

 insert(pkg)

build buildlist
```

```
function listofdeps(pkg)

return deps of pkg
```

```
function insert(pkg,revdep)

if pkg is not in buildlist,

 if pkg has deps,

   foreach listofdeps(pkg) as dep

     if dep not in revdep

      insert(dep,revdep+pkg)

 add pkg to buildlist
```

example:

let a, b, c, d some packages

a depends on b

b depends on d 

c depends on d

d depends on a

so we have a cycle a>b>d>a, and an innocent package c.

we run install(a c), here's the running pass, done by hand:

```
insert(a) is called

 a not in buildlist

 pkg has dep b

 b not in revdep

 insert(b,a) is called

  b not in buildlist

  pkg has dep d

  d not in revdep

  insert(d,a b) is called

   d not in buildlist

   pkg has dep a

   a is in revdep, a ignored

   add d to buildlist

   buildlist is d

   return

  add b to buildlist

  buildlist is d b

  return 

 add a to buildlist

 buildlist is d b a

 return

insert(c)

 pkg has dep d

 d in buildlist, ignored

 add c to buildlist

 build list is d b a c

 return

build d b a c

```

now run install(c a), you will end up with a different order of a, b and d (it's: b a d c). which one is right? hell if I know. they may even be both right or both wrong. all the fault of a package seemingly not related to the cyclic dependency of a b d. so when you do emerge world, the order of the content of any package of the world file influences the order of cyclic dependencies. now, it's insane to try to twist around a thousand unrelated packages just to build things in the right order.

what the algorithm does inside, is building an acyclic oriented graph, and move recursively inside it, building the buildlist along the path.

with graph theory, it is provable that:

1. each package not in a dep cycle will have all of its deps in the right order

2. it is impossible to solve circular deps with the given information, as it results in an infinite loop (thus the passage of redep, which eventually breaks the n-cycle at n-1 steps)

the only way to solve cyclicness is to have more information. this specific information is the extra bit of contextual knowledge we have over the machine:

- linux-headers do not compile, so they can be installed before gcc

- you already have some other gcc that can build glibc, so glibc depends on gcc in a less important manner gcc depends on glibc 

this will result in the instinctive order of linux-headers glibc gcc. so how did we solve cyclicness inn the end? we didn't. cyclicness is unsolvable. what we added is more information to the graph, rendering it somehow acyclic. what we did is prevailing one path over another on the graph. we said one was dependent in a 'less important manner', so what we did is that we weighted the graph. by assigning a weight, a value, a priority to each dependency of a package, we could sum them while walking the graph, and take the path with the bigger (smaller, whatever distinctive) priority, thus favoring an order.

this way, install(a c) and install(c a) will yield the same, and right, order of a b and d.

so, "if only Portage dependency information was to be trusted" is a bit harsh  :Wink: 

in the end, emerge does a great job dealing with dependencies. it just lacks information.

----------

## steveL

Well, this is all really informative. In terms of implementation, it comes down to doing the best we can. In this case I reckon that comes down to using emerge --pretend, as that's the best info we have. After all it takes into account the user's current USE flag set, and does give a list of what we'll need to pull in as against what we might.

I prefer the algorithm in count_zero's script so I reckon I'll try building the list (for ordering) with Guenther's script and then doing the actual install with count_zero's.

One question: what does portage do if it gets a circular dependency like you both outline? Just that I've never seen it happen. It's not hard to detect such a situation in any case.

----------

## Lloeki

I don't know, seeing how emerge -pve gcc and emerge -pve glibc give out the same result. Anyway, emerge has no other choice than to break the cycle, whether it does it haphazardly or willingly at some place is beyond my knowledge: a look at the source would be much helpful to understand the black magic, but I'm fairly new (albeit learning) to python.

Guenther, have you considered python as a language for your script? this is really a great language (even if I miss curly braces: they really help in readability, all the more with pair highlighting in IDEs), and one certainly has to worry if an upgrade breaks python, since it will break portage too... so yes, relying on perl is not the thing to do, but python may be a good choice.

----------

## Guenther Brunthaler

 *Lloeki wrote:*   

> Guenther, have you considered python as a language for your script?

 

Well, yes and no: I have no experience with python, while I have much experience with Perl.

Another language to learn... where learning a new language is usually the smallest obstacle: Learning the associated libraries/APIs and run-time-environment is the real challenge.

For instance JAVA: The core language is very small and can easily be learned in a couple of hours (especially with experience in C++). But the wealth of runtime libraries available surely need considerable time to master.

And I'm afraid, the same will hold for python, which also provides numerous runtime-libraries for the programmer to use.

Being a Perl guy, I admit python has a smaller, easier to learn core language, a more consise syntax, and generally looks more elegant than Perl.

I especially like its generators and the fact that all of its runtime libraries are based on exception handling.

But there are also drawbacks.

When first trying python on a Windows host, I encountered many serious problems in its UNICODE support, rendering it almost useless for internationalized applications. (Perhaps they have fixed that since then?)

I also dislike its habit of prefixing/postfixing everything with underscores (such as constructor names) for special-purpose functions. To me, it's rather a hint what features the language designers forgot to think about in the first place, than a well-designed syntactic construct.

Another issue are its rather idiosyncratic string-quotation mechanisms which I also consider not the most elegant solutions to the problem.

Finally, I like to make heavy use of anonymous functions in Perl.

And the lambda functions or python are so crippled that they are not really a replacement for that.

Summing up, I consider python to be good, but not excellent.

BTW, I think JavaScript is a largely underestimated language (regarding its potential).

It has amicably stolen many of the best features of Perl, but avoided to also inherit Perl's bloat. It has a cleaner syntax than Perl which is closer to C or JAVA, and allows easier integration into host applications than most other script language.

And in comparison to other popular scripting languages like LUA, JavaScript provides much more expressive power.

I would really be happy to see JavaScript as a standard shell language instead of the Bourne shell.

But reality is different...  :Wink: 

 *Lloeki wrote:*   

> so yes, relying on perl is not the thing to do, but python may be a good choice.

 

Agreed.

But to be honest: I'm just too lazy to learn a new scripting language, only to improve a script which already works and does it's job.

Also, this whole problem only arised after another improvement has been added to the script: Interruption-free operation.

Before that, the script simply stopped as soon as it encountered the first failing package, and the user was free to decide whether to edit the script file and remove any packages (such as dependent packages, a.k.a . "reverse dependencies") from it.

And finally, we still have count zero's excellent script!

I therefore suggest the following approach:

For the common cases, where the emerge -auDN world at the beginning of my guide worked without any problems, the current version of my script will do fine. This will usually be the case for installations where administrators run "emerge --sync && emerge -auDN world" in regular intervals anyway in order to keep their systems up-to-date.

For more specific cases, such a partially inconsistent systems or if rebuilding some packages can be expected to fail in advance, use count zero's excellent script which uses my script only as a dependency list generator backend.

For the hardest cases where nothing else works, use BadPenguin's method of doing a complete stage-1 reinstall.

Always remember: In Gentoo it's all about choices!

----------

## gerard27

Hi all,

I ran the program (script) after having gone through the necessary preliminaries according to the 

guide.

The list to be recompiled contained some 800 packages.

I know this is a lot,but sometimes you try something once and leave it in.Pure lazyness on my part

and having 250 GB of which only 8% is used.

Some people wondered how long it would take:

from oct.2  21:17  to oct.5  02:48 hr.

My box: athlon-xp @1447 Mhz,1G RAM IDE HD.

I had a problem after reboot:

The kernel was compiled with gcc 3.4 , the rest with gcc 4.1. so X didn't work.

I recompiled the kernel and then did 

```
module-rebuild rebuild
```

When the nvidia-driver finished compiling there was a message on the screen:

```
nvidia: Unknown symbol remap_page_range

nvidia: Unknown symbol pci_find_class
```

and X still couldn't start.

I emerge synced and a new nvidia driver was available: 1.0-8774.

I emerged that one,same error message.

Apparently something like this has happened before to people and they always solved it by emergeing a newer driver.

For me this didn't work as you can see.

I desperation I downloaded the same driver from the nvidia site as a file which is selfextracting

and ran it.No error messages and lo and behold it worked!

The kernel module is now located at different folder.

I would like to know if I did anything wrong,if so what?

Gerard.

----------

## lorebett

 *Quote:*   

> By the way, the generated script will do this in a recoverable way:
> 
> It can be aborted at any time by you, and will continue where it left off
> 
> when you re-run it. (The package where the script was interrupted will
> ...

 

Isn't there a way to avoid this (The package where the script was interrupted will

have to be compiled again from its beginning)?

This would be quite useful for huge packages, e.g., OpenOffice...

----------

## gerard27

One more remark:

Now when it boots X doesn't start until eth0 is finished.

It used to start before the starting of eth0 was visible on VT1.

Gerard

----------

## Guenther Brunthaler

 *Gerard van Vuuren wrote:*   

> I had a problem after reboot:
> 
> The kernel was compiled with gcc 3.4 , the rest with gcc 4.1. so X didn't work.

 

This should normally not be a problem, because the difference between gcc 3.4 and 4.1 is only significant for C++ programs and the whole kernel is written in C only (not C++).

Hoewever, who knows what sort of object code binary drivers contain... perhaps the guys at NVIDIA actually did use C++, and require linkage against C++ runtime libraries.

This will then result in a requirement to re-emerge the NVIDIA drivers.

Same is true if the kernel is recompiled and the option has been enabled for the kernel build process to store version information within the modules.

 *Gerard van Vuuren wrote:*   

> When the nvidia-driver finished compiling there was a message on the screen:
> 
> ```
> nvidia: Unknown symbol remap_page_range
> 
> ...

 

I remember having had the same problem a couple of months ago.

The reason for the error message was quite simple: The NVIDIA-drivers I was using at that time were outdated, and so a driver update was required in order for the NVIDIA drivers to work with the new kernel. (The "unknown symbol" problems arise from the fact that the NVIDIA drivers wants to use old kernel functions which have been removed from newer versions of the kernel.)

So it's neither a problem of Portage nor my script; you just need to update your NVIDIA drivers.

 *Gerard van Vuuren wrote:*   

> I emerge synced and a new nvidia driver was available: 1.0-8774.
> 
> I emerged that one,same error message.

 

I'm not using binary NVIDIA drivers on the box where I'm writing this posting, so I can't check. But I remember there was an important change in the NVIDIA driver packages a couple of months ago: Before the change, there were two different ebuilds, one for OpenGL/GLX and one for the NVIDIA kernel driver. And since the change, a new ebuild has been provided (with a different package name) which combined both ebuilds into a single one.

Or was it exactly the opposite of what I've written here? Can't remember.

Anyway, look for a those new ebuilds. The NVIDIA Hardware Acceleration Guide has also been updated accordingly, I presume.

 *Gerard van Vuuren wrote:*   

> For me this didn't work as you can see.

 

Because you only updated your old driver; but you need to unmerge it and install the completely new driver.

 *Gerard van Vuuren wrote:*   

> No error messages and lo and behold it worked!
> 
> The kernel module is now located at different folder.

 

The new ebuild will also eliminate that problem.

----------

## Guenther Brunthaler

 *Gerard van Vuuren wrote:*   

> One more remark:
> 
> Now when it boots X doesn't start until eth0 is finished.
> 
> It used to start before the starting of eth0 was visible on VT1.
> ...

 

That could be the result of some bug fix in the service startup scripts.

Unless you have modified your X startup files to pass the -nolisten tcp option to the X server, the X server will listen on port 6000 for inbound connection requests from remote applications which want to use your display. (The xauth cookies should normally ensure that only authenticated remote applications can get access to your display.)

But in order to listen to network interfaces, the network drivers must have brought up before. Which means network has to start before X.

BTW, I always use -nolisten tcp in my X startup scripts, as I am not using X via network connections other than through SSH tunnels (which will bypass the port 6000 connection method anyway; i. e. that port 6000 listening is not required at all for SSH X11 forwarding).

Note that X connections via port 6000 are also totally insecure, because they are not encrypted and thus only make sense (if at all) within a completely trusted LAN.

----------

## Guenther Brunthaler

 *lorebett wrote:*   

> Isn't there a way to avoid this (The package where the script was interrupted will
> 
> have to be compiled again from its beginning)?

 

Unfortunately, not when using emerge.

The whole emerge/ebuild stuff is essentially nothing else than a bunch of scripts which run the Makefile of a package with the right arguments and environment variable settings.

However, it is sometimes possible to restart a Makefile's actions from "within the middle" if the emerge's actions are done manually.

That is, in order to emerge some package xxx, instead of doing a

```
emerge xxx
```

you can do the following:

```
ebuild $(equery which xxx) compile
```

If there is an error during the compile, you can go into the ebuild's working directory (usually /var/tmp/xxx/work) and run make again there (after fixing the problem), which should continue to compile the remaining source files.

As soon as the compilation finished successfully, you need to run the following commands to complete the job:

```
ebuild $(equery which xxx) qmerge
```

which will install the compiled binaries into your live filesystem

```
ebuild $(equery which xxx) clean
```

which will remove the build directory

```
emerge --noreplace xxx
```

which will register the package in the world list of installed packages

```
emerge --ask --clean xxx
```

which will remove old versions of the package

Unfortunately, this assumes the Makefile of the package has been written in a clean way, which will usually be the case. But there might also be some packages where the above approach will not work; in such cases it is not possible to restart a make "from the middle".

----------

## Lloeki

 *Quote:*   

> That could be the result of some bug fix in the service startup scripts. 

 

you might just be right, there's that: 

```

$ cat /etc/init.d/xdm

(snip)

depend() {

        need localmount

        # this should start as early as possible

        # we can't do 'before *' as that breaks it

        # (#139824) Start after ypbind and autofs for network authentication

        after bootmisc readahead-list ypbind autofs openvpn gpm netmount

        before alsasound net.lo

        # Start before X

        use acpid hald xfs

}

(snip)

```

so, if you have netmount, which evidently depends on net, x will start after netmount, thus after net.

you can adjust some things in /etc/conf.d/rc, namely RC_NET_STRICT_CHECKING, but htis might badly affect some things.

or remove netmount altogether from startup, or if x really doesn't need netmount but you mount some remote folder, remove netmount from the after line.

----------

## cyclohexan

Thanks for your guide, Guenther. But one question:

You don't mention /sbin/fix_libtool_files.sh (described here)

Is this tool not necessary, while using your guide?

----------

## Guenther Brunthaler

 *cyclohexan wrote:*   

> You don't mention /sbin/fix_libtool_files.sh (described here)
> 
> Is this tool not necessary, while using your guide?

 

It's not necessary to run it with my guide. This script searches for outdated libraries, but as my guide will recompile the entire system, no outdated libraries will be left to be found.

----------

## Doogman

Last weekend, I used the script to update my system to the new GCC and everything seems to be working fine!  I especially like the hands-free aspect and it's definately a time saver.  No more baby-sitting portage with "--skip-first" to get around the troublesome packages.

Although I did manage to find one package that would trip-up the script: wargus.  When you emerge it, the ebuild halts while waiting for the game CD.  Luckily it was at the end of the compilation run and I noticed it.  I didn't have the game CD handy, however, so I altered the state file of your script to skip ahead a package.  Perhaps a "--skip-first" option would be handy after all?

----------

## Guenther Brunthaler

 *Doogman wrote:*   

> the ebuild halts while waiting for the game CD.

 

Your encountered an interactive ebuild... Hmm.

I'm not sure how to handle this from within a batch script. If there were a --batch switch for emerge or ebuild, I would use it.

I could redirect the output of yes as the standard input for the ebuilds, or just send newline characters.

But that won't help in such cases, because operator action (such as inserting CDs) is actually required; not just pressing Enter.

 *Doogman wrote:*   

> so I altered the state file of your script to skip ahead a package.

 

Which is the right thing to do in such cases.

 *Doogman wrote:*   

> Perhaps a "--skip-first" option would be handy after all?

 

That's a good point - I'll consider this. But skipping is not exactly what is wanted in this scenario: If a package is skipped by simply incrementing the index from the state file, it is considered as being "compiled and up to date". This means the admin has to keep track about such skipped packages, and manually re-emerge them later.

Perhaps I should create another package list, where manually skipped packages will be recorded as a reference for the administrator later?

However, until I implement something, incrementing the state file entry manually is the easiest way to skip a package.

I'll also post a description of the state file contents to this forum.

----------

## Guenther Brunthaler

Hi all,

Motivated by a posting from Doogman I decided to publish a description of the state file which is maintained automatically by my script for the reference of those who want to hack the script or modify its behaviour.

The state file is based on the same basename and filesystem location as the generated script, but has the string ".state" appended to the basename, and a dot (".") prepended to the basename.

The state file is a normal UNIX format text file.

The format is simple:

The first line contains the state file version number. This version number must match the internal version number of the script which is in control of the state file. The script will consider the state file to be outdated if the version numbers do not match, and will ignore its contents.

The remaining lines of the state file have only a specific meaning in the context of a specific version number in the first line.

In the following, there is a description of the state file contents for state file version number 1.13:

Line 1: The version number as stated above.

Line 2: The GCC version which is currently used for compilation. If this does not match the actual default compiler version, the state file contents will be ignored and recompilation of all packaged will restart from the beginning. This is a safeguard for situations where a compiler version update was made before the script finishes its execution in the first place, which should normally not be the case.

Line 3: A line containing 3 unsigned integer values, separated by a single space character. The values have the following meaning:

Progress counter. This is the item index number of the last package emerged successfully. When re-running the script after an interruption, it will skip all packages with item index numbers less than or equal to this value. This means, an effect similar to emerge's --skipfirst can be emulated by just manually incrementing this state file value before re-running the script.

The number of packages recompiled successfully in the current pass of the script.

The number of failing packages in the current pass of the script. If this number is nonzero when the current pass of the script is done, another pass will be run for the failing packages. But this will only be done if a least one package has been rebuilt successfully in the last pass. This constraint has been added to eliminate the possibility for infinite loops.

The package index numbers mentioned above refer to a specific package name and version each.

The associations are directly listed in the generated script file in lines starting with the word item.

The format of those lines is simple:

item index package_name_and_version

where index is the item index number, and package_name_and_version is the full package name and package version.

----------

## fxtl

(Sorry, this might be in a wrong thread, but I was reinstalling gcc and after that using Guenther's script..)

Ok, I have a really weird problem.. emerge --pretend --update --deep --newuse world wanted to recompile gcc 4.1.1 with "GTK" flag, so I did that  :Razz:  My system is built with 2006.1/desktop profile, so there was no gcc upgrade, just a reinstall. After rebooting, I noticed that ntfs-3g mounts are no longer working, I get an error message:

"Error: Volume name could not be converted to current locale: Invalid or incomplete multibyte or wide character"

It still mounts the volume, but "ls -l -a" doesn't show anything, not even the dot-directories. The volume is not corrupted, I checked it in Windows. Also kernel's read-only ntfs mount works fine.

And here are two other symptoms that are perhaps related (they appeared at the same time):

While booting:

---

Starting Automounter..

Glib: Cannot convert message: Cannot convert fallback '?' to codeset 'ISO-8859-15'

---

Also when I run emerge --sync, I get partly this kind of output:

---

MOTD brought to you by motd-o-matic, version 0.3

\#162\#145\#143\#145\#151\#166\#151\#156\#147\#040\#146\#151\#154\#145\#040\#154\#151\#163\#164\#040\#056\#056\#056\#040\#144\#157\#156\#145

\#164\#151\#155\#145\#163\#164\#141\#155\#160\#056\#143\#150\#153

...

---

Seems to be some kind of character set problem/Glib problem, whatever 'Glib' is. Emerge and the rest of the system still works fine, these are the only problems I've found so far.

I decided to recompile everything using Guenther's instructions and script, that went without problems, but no help, nothing changed  :Sad: 

I also checked the Gentoo localization guide and checked my locale settings, they were ok and unchanged. I also ran locale-gen, no help.

A newbie like me can't fix problems like these.. Linux/GCC Gods, please help  :Smile: 

----------

## fxtl

Yes, I got it working by switching to UTF8 and changing the default NLS option in kernel  :Smile:  (http://www.gentoo.org/doc/en/utf-8.xml)

----------

## Doogman

Looks like a new setting for interactive ebuilds is being proposed.

http://thread.gmane.org/gmane.linux.gentoo.devel/43195

Maybe this would help with the problem I had with the script being stuck at a game CD ebuild.

----------

## drescherjm

Thanks for this script. Its working quite well to upgrade my dual processor amd64 box to gcc-4.1.1.

----------

## allengwinn

If you're going to run this script with cyrus-sasl, you might want to read this first and patch.  Somebody fix this!

https://bugs.gentoo.org/show_bug.cgi?id=152544

----------

## drescherjm

It was not easy but I got it done. I am not sure exactly why but gcc-config-1.3.14 did not fully activate gcc-4.1.  Although it did say gcc-4.1 was default. The problem was parts of kde  were still looking in the /usr/lib/gcc/x86_64-pc-linux-gnu/3.4.6 folder for libraries and complaining that the ABI was wrong. After installing eselect-compiler-2.0.0_rc2-r1 (said that gcc-3.4.6 was default) and using it to select gcc-4.1 everything worked. I did have this script rebuild my packages again just to be sure and all but 5 packages emerged:

item 894 =sys-apps/net-tools-1.60-r12

item 895 =sys-fs/reiserfsprogs-3.6.19

item 896 =net-fs/nfs-utils-1.0.6-r6

item 897 =dev-libs/cyrus-sasl-2.1.22

item 898 =dev-cpp/gnome-vfsmm-2.12.0

I believe the first 3 are because I am using linux-headers-2.6.18 and I am not sure of the other 2.

----------

## [n00b@localhost]

I have just used Guenther's script for a second time to recompile my entire system (once for gcc 4.1/gentoo 2006.1 and another time for glibc 2.5). Whilst it was compiling I developed the following script to find out how long it would take to finish.

```
#!/bin/bash

# Script to find out how long it will take to finish recompiling

# all the packages in Guenther Brunthaler's recompile-remaining-packages

# script (http://forums.gentoo.org/viewtopic-t-494331.html)

#

# Gary Macindoe

# 29-Oct-2006

export recompile_script="/root/recompile-remaining-packages"

export genlop_c="$(mktemp)" || exit 1

genlop -nc > "${genlop_c}"

# Functions to add dates together

# parse_date <string>

# Takes a lingual date as the first argument and parses

# it into environment variables.

# The variables set by this function are:

# SECS:  the number of seconds

# MINS:  the number of minutes

# HOURS: the number of hours

# DAYS:  the number of days

# WEEKS: the number of weeks

# YEARS: the number of years

# The function returns 0 on success, 1 on failure.

function parse_date () {

   local number token

   export YEARS=0

   export WEEKS=0

   export DAYS=0

   export HOURS=0

   export MINS=0

   export SECS=0

   for token in $1;

      do

      case $token in

         [0-9]*)

            number=$token

            ;;

         year|years)

            export YEARS=$number

            ;;

         week|weeks)

            export WEEKS=$number

            ;;

         day|days)

            export DAYS=$number

            ;;

         hour|hours)

            export HOURS=$number

            ;;

         minute|minutes)

            export MINS=$number

            ;;

         second|seconds)

            export SECS=$number

            ;;

         and)

            ;;

         *)

            echo "Error: Unrecognised token $token"

            return 1

            ;;

      esac

   done

   return 0

}

# add_dates <days1> <hours1> <minutes1> <seconds1> <days2> <hours2> <minutes2> <seconds2>

# Takes two dates and adds them. Carries over seconds, minutes and  hours to minutes, hours and days.

# The variables set by this function are:

# TOTAL_DAYS:  the total number of days

# TOTAL_HOURS: the total number of hours

# TOTAL_MINS:  the total number of minutes

# TOTAL_SECS:  the total number of seconds

# Returns 0 on success, 1 on failure.

function add_dates () {

   if [ $# -ne 8 ];

      then

      return 1

   fi

   export TOTAL_SECS=$[ $[ $4 + $8 ] % 60 ]

   export TOTAL_MINS=$[ $[ $3 + $7 + $[ $[ $4 + $8 ] / 60 ] ] % 60 ]

   export TOTAL_HOURS=$[ $[ $2 + $6 + $[ $[ $3 + $7 + $[ $[ $4 + $8 ] / 60 ] ] / 60 ] ] % 24 ]

   export TOTAL_DAYS=$[ $1 + $5 + $[ $[ $2 + $6 + $[ $[ $3 + $7 + $[ $[ $4 + $8 ] / 60 ] ] / 60 ] ] / 24 ] ]

   return 0;

}

# format_date <days> <hours> <minutes> <seconds>

# Takes a date and formats it. Fields that are zero are excluded.

# Returns 0 on success, 1 on failure.

function format_date () {

   local output

   if [ $# -ne 4 ];

      then

      return 1

   fi

   output=""

   if [ $4 -ne 0 ];

      then

      if [ $4 -eq 1 ];

         then

         output=" [1;32;40m$4  [0;37;40msecond"

      else

         output=" [1;32;40m$4  [0;37;40mseconds"

      fi

   fi

   if [ $3 -ne 0 ];

      then

      if [ $3 -eq 1 ];

         then

         if [ "$output" ];

            then

            output=" [1;32;40m$3  [0;37;40mminute and $output"

         else

            output=" [1;32;40m$3  [0;37;40mminute"

         fi

      else

         if [ "$output" ];

            then

            output=" [1;32;40m$3  [0;37;40mminutes and $output"

         else

            output=" [1;32;40m$3  [0;37;40mminutes"

         fi

      fi

   fi

   if [ $2 -ne 0 ];

      then

      if [ $2 -eq 1 ];

         then

         if echo "$output" | grep "and" &>/dev/null;

            then

            output=" [1;32;40m$2  [0;37;40mhour, $output"

         elif [ "$output" ];

            then

            output=" [1;32;40m$2  [0;37;40mhour and $output"

         else

            output=" [1;32;40m$2  [0;37;40mhour"

         fi

      else

         if echo "$output" | grep "and" &>/dev/null;

            then

            output=" [1;32;40m$2  [0;37;40mhours, $output"

         elif [ "$output" ];

            then

            output=" [1;32;40m$2  [0;37;40mhours and $output"

         else

            output=" [1;32;40m$2  [0;37;40mhours"

         fi

      fi

   fi

   if [ $1 -ne 0 ];

      then

      if [ $1 -eq 1 ];

         then

         if echo "$output" | grep "and" &>/dev/null;

            then

            output=" [1;32;40m$1  [0;37;40mday, $output"

         elif [ $output ];

            then

            output=" [1;32;40m$1  [0;37;40mday and $output"

         else

            output=" [1;32;40m$1  [0;37;40mday"

         fi

      else

         if echo "$output" | grep "and" &>/dev/null;

            then

            output=" [1;32;40m$1  [0;37;40mdays, $output"

         elif [ $output ];

            then

            output=" [1;32;40m$1  [0;37;40mdays and $output"

         else

            output=" [1;32;40m$1  [0;37;40mdays"

         fi

      fi

   fi

   echo "$output."

   return 0

}

# date-add "<date1>" "[date2]"

# Adds two lingual dates and prints the result. If date2 is omitted it is read from stdin

# Returns 0 on success, 1 on failure.

function date-add () {

   local input input1 input2

   local secs1 mins1 hours1 days1

   local secs2 mins2 hours2 days2

   if [ $# -lt 1 -o $# -gt 2 ];

      then

      echo -e "Usage: $0 <date1> [date2]\nIf date2 is omitted it is read from stdin"

      return 1

   fi

   input1=$( echo $1 | tr -d "[:punct:]" )

   if [ $# -eq 2 ];

      then

      input2=$( echo $2 | tr -d "[:punct:]" )

   else

      input2=$( read $input && echo $input | tr -d "[:punct:]" )

   fi

   if parse_date "$input1";

      then

      secs1=$SECS

      mins1=$MINS

      hours1=$HOURS

      days1=$DAYS

      if parse_date "$input2";

         then

         secs2=$SECS

         mins2=$MINS

         hours2=$HOURS

         days2=$DAYS

         add_dates $days1 $hours1 $mins1 $secs1 $days2 $hours2 $mins2 $secs2

         format_date $TOTAL_DAYS $TOTAL_HOURS $TOTAL_MINS $TOTAL_SECS

      fi

   fi

   return 0

}

# Genlop -c functions

# current_package_name

# Parses the genlop -c output for the name of the package being compiled

function current_package_name () {

   egrep --colour=never "^ \* [a-z]+-[a-z]+/[-_+0-9a-z]+-[.0-9]+[a-z]?(_alpha[0-9]*|_beta[0-9]*|_pre[0-9]*|_rc[0-9]*|_p[0-9]*)?(-r[0-9]+)?" "${genlop_c}" | cut -c4-

}

# current_package_number

# Finds the number of the package currently being compiled from the list of packages to be compiled

function current_package_number () {

   egrep --colour=never "^item [0-9]+ ="$(current_package_name) "${recompile_script}" | gawk '{ print $2 }'

}

# current_package_time_left

# Parses the genlop -c output to find the time left for the package being compiled

function current_package_time_left () {

   egrep --colour=never "^       ETA: " "${genlop_c}" | cut -c13-

}

# total_packages

# Finds the total number of packages to be compiled from the package list

function total_packages () {

   egrep --colour=never "^item [0-9]+ =[a-z]+-[a-z]+/[-_+0-9a-z]+-[.0-9]+[a-z]?(_alpha[0-9]*|_beta[0-9]*|_pre[0-9]*|_rc[0-9]*|_p[0-9]*)?(-r[0-9]+)?" "${recompile_script}" | tail -n1 | gawk '{ print $2 }'

}

# time_left

# Fakes an emerge -ep output into genlop -p to find the time left to compile the rest of the packages in the list

function time_left () {

   local packages_left=$[$(total_packages) - $(current_package_number)]

   (echo -e "\nThese are the packages that would be merged, in order:" &&

   echo -e "\nCalculating dependencies   ... done!\n" &&

   for x in $(egrep --colour=never "^item [0-9]+ =[a-z]+-[a-z]+/[-_+0-9a-z]+-[.0-9]+[a-z]?(_alpha[0-9]*|_beta[0-9]*|_pre[0-9]*|_rc[0-9]*|_p[0-9]*)?" "${recompile_script}" | tail -n$[ ${packages_left} - 2 ] | cut -d' ' -f3 | cut -c2-);

      do

      echo "[ebuild   R   ] $x"

   done) | genlop -np | egrep --colour=never "^Estimated update time: " | cut -d':' -f2 | cut -c2-

}

# if the recompile script cannot be found, exit

if [ ! -r "${recompile_script}" ];

   then

   echo "Cannot read ${recompile_script}"

   rm -rf "${genlop_c}"

   exit 1

fi

# current_package_name returns nothing if there is no emerge in progress

if [ -z "$(current_package_name)" ];

   then

   echo "No emerge in progress"

   rm -rf "${genlop_c}"

   exit 1

fi

echo " [0;37;40mCurrently compiling  [0;31;40m$(current_package_number) [0;37;40m of  [0;31;40m$(total_packages) [0;37;40m:  [0;34;40m$(current_package_name) [0;37;40m"

current_package_time_left="$(current_package_time_left)"

# Replace words with numbers for adding

if [ "${current_package_time_left}" = "less than a minute." ];

   then

   current_package_time_left="1 minute"

elif [ "${current_package_time_left}" = "any time now." ];

   then

   current_package_time_left="0 minutes"

fi

# Add the time left for the current package to the time left for the rest of the package list

time_left=$( date-add "$(time_left)" "${current_package_time_left}" )

echo "Time left: ${time_left}"

# Clean up

rm -rf "${genlop_c}"

```

(The ^[ characters are generated by ctrl+v esc in vi. I have posted a bzipped version in case copy and pasting doesn't work.)

Example output:

```

garyslaptop ~ # recompile-status

Currently compiling 805 of 811: sys-libs/db-4.2.52_p4-r2

Time left: 32 minutes and 47 seconds.

```

Hope somebody else will find this useful!

Gary

----------

## Gentree

 *Guenther wrote:*   

> Because in my opinion the amazing truth is: It is in fact UNNECESSARY to recompile any package, including the new GCC, more than exactly once when rebuilding your entire system! (For those who cannot believe this, see my argumentation in the related article https://forums.gentoo.org/viewtopic-p-3548628.html#3548628.)

 

Well if that were true it would be a great time saver. So, having read your very clear and well argued explaination I thought I would give it a try.

Since you admitted somewhere that there were things like binutils that you were wrong about I took a couple of extra steps before taking your "only once" approach.

To be precise:

I already have gcc-4.1.1 which had been compiled twice before I swapped. It seems this is now taken care of in the ebuild anyway.

I then cloned the entire system to another partition , rebooted to the new system and started the gcc-4 rebuild .

First I did 

```
emerge binutils glibc 
```

, then a full 

```
emerge -e world
```

.

Since I wanted to follow exactly what was happening this was all done by hand not using your script. I had to deal with a few packages that failed but after a couple of updates and a bit of rebuilding I got the end of the world rebuild.

Now when I try to run kdevelop I get this:

bash-3.1#kdevelop

```
kdevelop: /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/libstdc++.so.6: version `CXXABI_1.3.1' not found (required by /usr/kde/3.5/lib/libkhtml.so.4)

```

I have previously used the tiresome multi-emerge approach and not had any of this sort of issue. 

What do you think is the cause of this breakage?

 :Cool: 

----------

## drescherjm

 *Quote:*   

> What do you think is the cause of this breakage?

 

I had the same problem with other parts of kde. For me it was caused by gcc-config-1.3.14 not properly setting the default compiler fully to gcc-4.1.1. To fix this I upgraded gcc-config and eselect-compiler-2.0.0_rc2-r1 and used eselect compiler to set the compiler to gcc-4.1 (  when I ran this it reported the gcc--3.4.6 was default even though I had set 4.1 to default a few days back with gcc-config-1.3.14) 

After that step the broken kde packages were fixed with no emerging necessary.

----------

## Gentree

Many thanks, that fixed it. These fora are a gold mine of info.

Now what was I trying to do 2 days ago before I found out kdevelop was broken....

 :Cool: 

----------

## steveL

 *Gentree wrote:*   

> These fora are a gold mine of info.

 

Yes, if only there were some way to automate the mining and extraction  :Wink: 

----------

## Gentree

Dont bother trying, if that could be done Gentoo would have been bought up by a large mining conglomorate and we'd have to pay to use it.   :Wink: 

----------

## Bones McCracker

Guenther,

I have found several of your posts to be very useful.  Thanks for taking the time to document things so nicely.  

Aside from your excruciatingly annoying avatar, it is a pleasure to read your material.

----------

## moshiach

Incredibly helpful and well written.  I dup'd my machine to another one I have handy and ran your guide on one and the stardard rebuild, rebuild, rebuild stuff on the other.  Yours just finished and so far I see no problems at all other than libidn which forced me to learn that Java now has generations.  The other one is still going and three times now I've had to manually intervene to fix a problem. 

I think your guide should go up for posterity.

----------

## Guenther Brunthaler

When following my guide, sometimes the problem arises that "emerge --update --newuse --deep world" or "revdep-rebuild" fails, and the question arises whether my guide can still be followed in those cases.

Well, it depends on the packages which are failing.

The main reason why I want the "--update" is that I want stable, up-to-date build tools at the time my script will be started.

The reason for the "--deep" is that I also want those build tools to be based on up-to-date libraries, i. e. glibc and other important system libraries.

"--newuse" is there for the case that the new profile which the user might have selected enables a different set of build options for the compiler or libraries.

So, "--update --deep --newuse" together will take care of all that and make sure we start our rebuild in a sane environment and with an up-to-date and working set of tools.

The "revdep-rebuild" is there in order to fix things the "emerge --update --newuse --deep world" might have "forgotten".

So, each package which fails in either "emerge --update --newuse --deep world" or "revdep-rebuild" is a package which cannot be trusted to operate correctly during the initial phases of the system rebuild.

Whether it is wise to continue using my script in such cases depends on the relative importance of the failing packages: Will it be required by the build system in order to rebuild the basic build tools themselves?

The basic build tools are: portage, python, GCC, binutils, autoconf, automake, m4, awk, flex, bison. (I hope I didn't forget one.) As well as all libraries they depend on. And updating "baselayout" is certainly also wise.

At least those packages must be up-to-date before a system rebuild is attempted.

So, if "emerge --update --newuse --deep world" or "revdep-rebuild" fail, you can try to update the above packages explicitly, and then hope it will suffice to rebuild the system.

At the very minimum, I'll suggest to do an

```
emerge --ask --update --deep --newuse portage linux-headers glibc gcc binutils
```

It might be that this is indeed enough to make my script work.

And if it does, this will save the day: As my script rebuilds everything (except for the C compiler which already has be up-to-date when the script starts) from scratch, it does not matter whether any of the remaining packages are also up-to-date when the script starts. They will be rebuilt by my script anyway. Only the build tools themselves and everything they depend on for successful operation *must* be up-to-date.

And if that minimal approach does not work that way: Don't blame me!

I have never tested rebuilding the system without a successful "emerge --update --newuse --deep world" and "revdep-rebuild", so it may work or it may not.

You have been warned! I'll take no responsibility for what might happen.

So make backups. Hope for the best. Pray! Or trust on your Live-CD if all your prayers won't help...

If someone is willing to risk trying out the minimal approach suggested above instead, please post here whether it worked for others to share your experiences.

----------

## lonegd

Might be worth noting you need to update CFLAGS in /etc/make.conf due to GCC 4.x dropping support for -mcpu

```

       -mcpu=cpu-type

           A deprecated synonym for -mtune.

       -march=cpu-type

           Generate instructions for the machine type cpu-type.  The choices for cpu-type are the same as for -mtune.

           Moreover, specifying -march=cpu-type implies -mtune=cpu-type.

       -mtune=cpu-type

           Tune to cpu-type everything applicable about the generated code, except for the ABI and the set of avail-

           able instructions.  The choices for cpu-type are: <SNIP - see gcc 4.x man page>

```

I had originally set my CFLAGS based on http://www.freehackers.org/gentoo/gccflags/flag_gcc3.html which is obviously now out of date.

The warning messages you see during emerge's is:-

```

`-mcpu=' is deprecated. Use `-mtune=' or '-march=' instead 

```

HTH

----------

## Karsten1973

 *Guenther Brunthaler wrote:*   

>  *vicaya wrote:*   Interesting. But I'd recommend install ccache (make sure it's enabled in FEATURES in make.conf) and do it the usual way 
> 
> Without question, ccache is a great tool. I'm using it all the time.
> 
> And it works perfectly with my script. (At least I did not encounter any problems. However, I manually cleared the cache of ccache after using gcc-config to set the new compiler. Just to be sure there are no cached object files left originating from the old compiler.)

 

Maybe you should add this step to your how-to. I was several hours into the rebuild when I read this. Now I'm restarting...    :Evil or Very Mad: 

----------

## drescherjm

Are you worried about having to clear ccache? I rebuilt my system without clearing and did not have a problem.

----------

## Guenther Brunthaler

 *drescherjm wrote:*   

> Are you worried about having to clear ccache?

 

Yes.

But it is not as if I was certain that ccache will actually cause troubles.

I just don't know how it works internally, and that worries me, because I cannot rule out the possibility that ccache might return compiled files generated by an outdated GCC.

The question is: How is it implemented?

I only can say: If I was about to implement ccache's functionality, then it would work as follows:

I would run the preprocessor unaltered, passing through any command line options which influence the preprocessor. This will be done for every source file specified on the command line.

The preprocessor will return me a list of C/C++ source files with all preprocessor macros replaced by their contents. Conditional compilation (#ifdef etc) or #include files will also have been taken care of already.

Now a loop follows over each already-preprocessed source file:

The complete contents of the commandline will be concatenated to the string representation of the exact version number of the compiler, linker and assember, and finally to the contents of the already-preprocessed current source file.

An MD5-hash (or some other hash) is calculated over the contents of that concatenated text stream.

Then the resulting hash value has to be looked up in the cache database: Is it already there?

If yes, then return the already-compiled object file represented by the cache entry, bypassing an actual invocation of the compiler. Also touch the cache entry to make it the newest one.

If no, then run the compiler normally first. But prior to returning the compiled object file, make a copy from it and store it in the cache database with the previously calculated hash value as the access key.

If the maximum cache size has been exceeded by the new entry, delete the oldest cache entry file in order to keep the cache size within its defined limits.

This continues until all source files have been processed.

If the real ccache actually works like that, there is indeed no need to worry: The changed version number of the compiler will result into a different hash value for each source file, and so object files compiled with the old compiler will never be returned.

But it ccache doesn't include the version numbers of the build tools into the hash value it calculates, then ccache will not detect the fact that an object file has been compiled by a different GCC than the current one,  and an outdated object file might be returned.

So, it depends.

If you clear the cache, you are on the safe side.

 *drescherjm wrote:*   

> I rebuilt my system without clearing and did not have a problem.

 

But have you also tested every single executable since then?  :Wink: 

The problem is, ccache maintains a cache. Which means, it will not store every object file ever compiled there, but just the latest 2 gigs or so of them. Older ones will be purged, to make room for newer ones.

Which essentially means: Whether or not an outdated object file has a chance to "survive" a full system rebuild depends on the build order.

If enough source files will be compiled first which only have cache misses, the resulting object files will replace all the existing (outdated) object files as the new onles are saved into the cache.

But if a package is recompiled which produces cache hits for outdated object files in the cache, those outdated object files will be returned.

So it might be very well that everything went OK for you - but you might just have been lucky...

If anyone is reading this who knows how ccache acually works in this regard - please share your knowledge with us!

----------

## drescherjm

 *Quote:*   

> If the real ccache actually works like that, there is indeed no need to worry: The changed version number of the compiler will result into a different hash value for each source file, and so object files compiled with the old compiler will never be returned. 

 

I remember reading the docs for ccache it does mention that it works like this. I mean the new gcc version will generate a new hash.

John

----------

## Guenther Brunthaler

 *drescherjm wrote:*   

> I remember reading the docs for ccache it does mention that it works like this.

 

Thanks for that hint! I RTFM'd and finally found:

 *Quote:*   

> HOW IT WORKS
> 
>        The basic idea is to detect when you are compiling exactly the same code a 2nd time  and  use
> 
>        the previously compiled output. You detect that it is the same code by forming a hash of:
> ...

 

This looks very good indeed, because it includes a hash of the whole compiler, and not just the version number of it.

However, the question remains what the author of the man page had in mind when he was referring to "the compiler": Only the driver program? Or all the many executables the GNU compiler collection consists of as well?

And what about changes to the assembler?

For instance, assemblers typically can perform "jump optimizations" of various types which cannot reliably be detected at the assembler source level (where the GCC code generator backend operates).

Some CPU architectures feature "long" and "short" jump instructions. Depending on the jump distance from a given assembler source line to some other instruction specified by a symbolic label, the assembler can use either the longer or the shorter version of the jump instruction. And the jump distance depends on the actual encoding size of the machine code statements emitted by the assembler which the compiler cannot know, but only the assembler.

On the other hand, "shorter" means not always "better", because changes to the code alignment in memory can also trigger changes to the CPU cache usage.

Also, the superscalar architecture of today's CPUs with its pipelines and jump prediction heuristics can make an optimal decision "what is the best way to encode, potentially reorder and align an assember statement" a really complicated problem to be solved.

Of course, the compiler will take care of most of those optimization matters.

But the GCC backend (as well as the backends of most traditional UNIX C compilers) is restricted by the fact that it generates assembler source code, not binary machine code.

That means, although GCC might output the optimal assember statements, it does not know how the assembler will transform those assembler statements into actual machine instructions.

This also means GCC cannot really optimize for CPU cache usage, because it cannot predict the alignment.

If any such optimization is performed at all, it can only be done by the assembler itself (unless GCC will be completely rewritten to directly emit machine code).

And if a new version of the assembler changes the way how it optimizes those things - a different object file will be the result; even though GCC might emit exactly the same assembler statements.

If ccache only checksums the compiler executables, and not the assembler, it cannot detect such changes.

On the other hand: Who cares...  :Smile: 

But even if we ignore those (possible) differences: What about environment variables?

There are a couple of environment variables which can change the way the compiler works.

Just think about CFLAGS!

And I cannot remember having read that ccache also checksums the contents of such variables. Which could lead to totally different object files for different compiler runs, even though all the command line options as well as all the source files are identical.

Only the fact that most packages use automake protect from this, because automake-generated makefiles will set up their own CFLAGS.

But... what will happen for packages that do not use automake? There are some of those in the portage tree.

What if the CFLAGS in /etc/make.conf have been changed manually by the user? Will ccache detect that?

----------

## Karsten1973

Another thing you might want to consider:

When I was half way through the recompile, I noticed that modules would no longer work. It took me a while to figure out that the modules had been recompiled (nvidia-drivers) with the new gcc, while the kernel was still compiled with the old one.

You might want to add to your guide, that a kernel-recompile is needed.

----------

## Guenther Brunthaler

 *Karsten1973 wrote:*   

> I noticed that modules would no longer work.

 

You are right. Actually, I have a GeForce4 in one of my own boxes, and already encountered the same problems.

But I was already so used to permanently encounter problems with those d*mned closed source drivers, that I did not feel it was worth mentioning it explicitly...  :Wink: 

 *Karsten1973 wrote:*   

> You might want to add to your guide, that a kernel-recompile is needed.

 

OK, I'll add a general advice for recompiling the kernel when ebuilds containing kernel modules are among the packages in the world list.

Because not only nvidia drivers will have those problems, but any externally built kernel modules potentially could have similar problems.

Thanks for the hint.

----------

## Karsten1973

My last note: For some reasons I could not discern, after the script was almost finished, GRUB would no longer work. I had to boot from CD and reinstall grub on the mbr. After that everything seems to be fine. (I can't compile app-misc/beagle anymore - that stopped your script from finishing).

Thanks for your great script and guide. My system is now up to date and compiled with the gcc 4.1.1

GREAT WORK!

----------

## recordista

 *Guenther Brunthaler wrote:*   

> 
> 
> Of course, {count zero's} contribution will be honored accordingly in the updated version of the guide!
> 
> Until then, I recommend using your script for the purpose of interruption-free rebuilding the entire system in combination with my script.
> ...

 

So is recompile-entire-system-20060926 the latest version, or did I miss something (or is there a new home for this cool tool?)

----------

## recordista

Seems Guenther's post has been updated since that 20060926 date, so I went ahead and used that version.  It's been running for awhile now, and all seems well -- but v.e.r.y s.l.o.w.  These emerge commands are taking several times longer than they would with a normal emerge command run on this system.  Is this typical?  Since this is such a small system resource-wise, I'm wondering if the script is attempting too much parallelism for the architecture?

Anyway, thanks again for automating and simplifying a complex process.

----------

## drescherjm

 *Quote:*   

>  Is this typical?

 

I did not notice any slowness at all, however I ran it on a dual processor opteron 850 system with 4GB of memory and ccache running...

----------

## recordista

More importantly, things seem to be breaking here

```

# grep Emerg recompile-remaining-packages_2006-12-22T03\:47\:22.log 

Emerging package # 1 ('=sys-kernel/linux-headers-2.6.17-r2')...

>>> Emerging (1 of 1) sys-kernel/linux-headers-2.6.17-r2 to /

Emerge failed return code 1 (will be retried later).

Emerging package # 2 ('=sys-libs/glibc-2.4-r4')...

>>> Emerging (1 of 1) sys-libs/glibc-2.4-r4 to /

Emerge failed return code 1 (will be retried later).

Emerging package # 3 ('=sys-devel/binutils-2.16.1-r3')...

>>> Emerging (1 of 1) sys-devel/binutils-2.16.1-r3 to /

Emerge failed return code 1 (will be retried later).

Emerging package # 4 ('=sys-libs/zlib-1.2.3-r1')...

>>> Emerging (1 of 1) sys-libs/zlib-1.2.3-r1 to /

Emerge failed return code 1 (will be retried later).

...

Emerging package # 39 ('=sys-apps/help2man-1.36.4')...

>>> Emerging (1 of 1) sys-apps/help2man-1.36.4 to /

Emerge failed return code 1 (will be retried later).

Emerging package # 40 ('=sys-devel/automake-wrapper-2-r1')...

>>> Emerging (1 of 1) sys-devel/automake-wrapper-2-r1 to /

Emerge failed return code 1 (will be retried later).

Emerging package # 41 ('=sys-devel/automake-1.9.6-r2')...

>>> Emerging (1 of 1) sys-devel/automake-1.9.6-r2 to /

Emerge failed return code 1 (will be retried later).

Emerging package # 42 ('=sys-devel/libtool-1.5.22')...

>>> Emerging (1 of 1) sys-devel/libtool-1.5.22 to /

Emerge failed return code 1 (will be retried later).

Emerging package # 43 ('=sys-apps/coreutils-6.4')...

```

----------

## drescherjm

Are you out of space?

I would stop there and take a look at the end of /var/log/emerge.log to see what the error is.

```
tail -n 100 /var/log/emerge.log 
```

If you can not tell by the log try emerging one of the packages directly

```
emerge --nodeps -a =sys-kernel/linux-headers-2.6.17-r2
```

----------

## recordista

Already killed the script and started a manual rebuild.  kernel-headers completed normally, glibc is still cooking.

emerge.log snippets for this package, first from my original mass upgrade (using gcc 3.3.6):

 *Quote:*   

> 
> 
> 1166662520:  >>> emerge (13 of 95) sys-kernel/linux-headers-2.6.17-r2 to /
> 
> 1166662520:  === (13 of 95) Cleaning (sys-kernel/linux-headers-2.6.17-r2::/usr/portage/sys-kernel/linux-headers/linux-headers
> ...

 

Now from the scripted rebuild

 *Quote:*   

> 
> 
> 1166777251:  *** emerge --oneshot --nodeps =sys-kernel/linux-headers-2.6.17-r2
> 
> 1166777253:  >>> emerge (1 of 1) sys-kernel/linux-headers-2.6.17-r2 to /
> ...

 

and now from my manual rebuild:

 *Quote:*   

> 
> 
> 1166820926: Started emerge on: Dec 22, 2006 15:55:26
> 
> 1166820926:  *** emerge --deep linux-headers
> ...

 

I was running this as a background task, and then disowned it & logged out.  Can't see how, but could that be the issue?  Otherwise, it seems there might be something in the script that is breaking these builds right before they complete.  This is a small, slow system.

----------

## sonix

 *Guenther Brunthaler wrote:*   

>  *outspoken wrote:*   Your guide should read more like this: 
> 
> OK, so you are suggesting to do the gcc-config before the emerge --update then.
> 
> It should certainly not a be problem changing this in my guide, but first I have to understand what exactly the problem was.
> ...

 

I tried to do emerge --ask --update --deep --newuse world and when it got to compiling glibc 2.4, i got the error These critical programs are missing or too old: gcc and it got kicked out of the emerge. 

emerge --ask --update --deep --newuse portage linux-headers glibc gcc binutils results in the same problem. not sure how to work around this.

----------

## drescherjm

 *Quote:*   

> I tried to do emerge --ask --update --deep --newuse world and when it got to compiling glibc 2.4, i got the error These critical programs are missing or too old: gcc and it got kicked out of the emerge. 
> 
> emerge --ask --update --deep --newuse portage linux-headers glibc gcc binutils results in the same problem. not sure how to work around this.

 

Although you can ask here I think you will be better served if you ask that as a new thread and post a few bits of info 

emerge --info 

and then the exact error you are getting.

----------

## Guenther Brunthaler

 *sonix wrote:*   

> I tried to do emerge --ask --update --deep --newuse world and when it got to compiling glibc 2.4, i got the error These critical programs are missing or too old: gcc and it got kicked out of the emerge. 
> 
> emerge --ask --update --deep --newuse portage linux-headers glibc gcc binutils results in the same problem. not sure how to work around this.

 

Hmm, that's really bad: Even the GCC toolchain, or your Portage/Python installation - or both! - are too old to work currently with your system.

If only one of GCC/toolchain or Portage/Python is hopelessly outdated, there are still chances to bring back things to normal by emerging the other component group first.

Which means in your case: Can you at least re-emerge/update sys-apps/portage?

If that won't be possible, more desperate measures are required...

My guide has been written for the casual case where systems are updated regularly, i. e. at least once each couple of months: At least for reasons of security updates.

If a system is running in a shielded secure or totally offline environment, this assumption can of course be false then. In such scenarios it's perfectly ok to let installations run as long as they are able to do their jobs, without ever updating.

Another case is installations which have been shut down for a very long time, because there is no obligation to regularly update a down system of course.

So, what to do in such cases?

In a nutshell, those "desperate measures" are: Download a current version of a current Gentoo Live-CD and attempt to fix things with it.

That is, boot from the live-cd and update at least "system" using the live-cd's Portage/GCC toolchain.

This can be done pretty much in the same way as you installed the first optional packages of your system when following the official Gentoo installation handbook. (Of course you should skip all steps which format or set up the disk partitions and take care not to accidentally overwrite your manually-tuned configuration files.)

If you are lucky, you can reboot then and your system will again be able to do emerges by itself; then you can follow my guide again to update the rest of "world".

If you are not so lucky, then it might be necessary updating everything when being booted off the live-cd, which is essentially a full stage 2 reinstall.

Badpenguin has written a guide for those worst-case scenarios (you can find the link in my guide), and it can always be used as a last resort.

----------

## smono927

You could try setting your profile to /usr/portage/profiles/obsolete/x86:

```
# rm /etc/make.profile

# ln -s /usr/portage/profiles/obsolete/x86 /etc/make.profile
```

```
# cat /usr/portage/profiles/obsolete/README

These are profiles to let people upgrade from non-cascading to cascading.  

They contain the bare minimum for old portages to upgrade to the latest 

version.
```

After emerging portage and gcc, switch your profile back to the latest you want.

I did this to upgrade from an old 2005 profile.

I'm not sure if it is the proper thing to do.

----------

## lorebett

Hi

I'm using this script for the second time to rebuild the entire system (I changed the processor and wanted to rebuild the entire system with -march=pentium4).

I noticed that some kde packages and gcc itself are not scheduled for rebuilding.

For instance, I have kdemultimedia and kdenetwork

```

*  kde-base/kdemultimedia

      Latest version available: 3.5.5

      Latest version installed: 3.5.5

```

but they do not appear in the packages that will be emerged:

```

grep kde recompile-remaining-packages

item 416 =kde-base/kdebase-pam-6

item 458 =kde-base/arts-3.5.5

item 481 =kde-base/kdelibs-3.5.5-r9

item 495 =kde-base/kdebase-3.5.5-r3

item 503 =dev-util/kdevelop-3.3.5

item 515 =kde-base/kdesdk-3.5.5

```

am I missing something?

----------

## palsyboy

In response to this thread, I'm trying to run the script.  I ran it once, and it didn't fix my video problem, but at least I had a fully update machined.  :Smile: 

I then switched from kernel 2.6.18-gentoo-r4 to kernel 2.6.20-gentoo-r8, and got slightly better video response.  I tried running this script again around the new kernel (if it makes a difference; I don't understand the system well enough), and packages stopped emerging due to a full root partition.  I restarted in case it would clear out some of /var/, which I suspected might have been the problem, but that only freed about 50 MB.  I also ran an emerge --sync to see if that would clear some space from /usr/portage.  Instead, /usr/ has remained at about 5 GB, thus filling my 10 GB root partition.

Here are the potential problem folders, and I'm not sure how to begin cleaning them:

/usr/src/: 1.4 GB

/usr/share/: 456 MB

/usr/portage/: 2.7 GB

/usr/lib/: 1 GB

Any help would be appreciated!  :Smile: 

----------

## red-wolf76

Depending on whether you are planning to revert to older versions that are no longer in the portage tree, you may try weeding out /usr/portage/distfiles, which I believe makes up the bulk of /usr/portage, as the downloaded source tarballs are stored there.

When my system is relatively stable (no flipping around between package versions), I usually erase the contents of /usr/portage/distfiles completely and perform an emerge -ef world to get back the sources for what packages I have installed.

----------

## lorebett

I usually erase /usr/portage/distfiles after every emerge --update world

----------

## red-wolf76

Which, depending on your setup, may well be sensible. I have a rather piquant mixture of stable and ~arch packages, however, and sometimes the packages in ~arch get flung from the tree without being committed to stable, resulting in toothaches when the previous ~arch version worked, but is gone. Call me a paranoid old-fogey, but I sleep much easier knowing that at least I'm able to remerge every package I have installed, even if I manage to wreck my internet.

----------

## lorebett

you're not paranoid, in your scenario, you're right!   :Smile: 

----------

## palsyboy

red-wolf76 and lorebett: Can I safely run "rm -R /usr/portage/distfiles/", or do you erase them in a more specific way?  That is indeed the swollen folder.

I really appreciate the help.

----------

## lorebett

I simply run rm -f /usr/portage/distfiles/* 

this will not erase the cvs dir in there

----------

## red-wolf76

Yep. Better just stick to ditching the actual files in there, not subdirectories or the directory itself. Personally, I don't have CVS installed (at least not consciously!  :Laughing:  ) but it doesn't hurt to be safe.

----------

## palsyboy

Thanks!

----------

## lorebett

 *red-wolf76 wrote:*   

> Yep. Better just stick to ditching the actual files in there, not subdirectories or the directory itself. Personally, I don't have CVS installed (at least not consciously!  ) but it doesn't hurt to be safe.

 

I think that the cvs dir is used by portage itself

----------

## red-wolf76

 *lorebett wrote:*   

> I think that the cvs dir is used by portage itself

 O RLY?  :Shocked:  Learning something new every day... Thanks for the info. I'll see what's in there on my box then...

----------

## bguan

After I attempted to upgrade to GCC4 a few months ago, following those steps that took days to recompile tool chain and system...

There are consistently a few things that are still bothering my system.  

For example: when emerge hits

  esound 

I always get

...

make  all-recursive

make[1]: Entering directory `/var/tmp/portage/media-sound/esound-0.2.38/work/esound-0.2.38'

Making all in docs

make[2]: Entering directory `/var/tmp/portage/media-sound/esound-0.2.38/work/esound-0.2.38/docs'

jw -f docbook -b html -o html ./esound.sgml

Using catalogs: /etc/sgml/sgml-docbook-3.0.cat

Using stylesheet: /usr/share/sgml/docbook/utils-0.6.14/docbook-utils.dsl#html

Working on: /var/tmp/portage/media-sound/esound-0.2.38/work/esound-0.2.38/docs/./esound.sgml

jade: /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/libstdc++.so.6: version `CXXABI_1.3.1' not found (required by /usr/lib/libosp.so.5)

make[2]: *** [html/index.html] Error 8

make[2]: Leaving directory `/var/tmp/portage/media-sound/esound-0.2.38/work/esound-0.2.38/docs'

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory `/var/tmp/portage/media-sound/esound-0.2.38/work/esound-0.2.38'

make: *** [all] Error 2

...

I have been doing emerge --resume --skipfirst to get over esound emerge failure everytime after a emerge --sync and emerge -DuN world.

In a fit of madness, I unmerged esound and realized too late that I can't re-emerge it due to the same failure and now more packages are broken due to missing dependency on esound.

Becuase of esound I can't satisfy Guenther's script preconditions:

* Your system is up-to-date (emerge --ask --update --deep --newuse world) 

*  Run "revdep-rebuild". If there are problems found and fixed, repeat running "revdep-rebuild" until no more problems are reported. 

I am tempted to just run the script anyway... Will Guenther's script fix my gentoo box?  Or will it break it even more?

If this fails, I think I should just switch to Ubuntu and admit that Gentoo is too hard core for me.

----------

## gsoe

It seems that your old gcc version was 3.4.6. Have you tried to run

```
fix_libtool_files.sh 3.4.6
```

First of course make sure that you have set the new gcc version:

```
gcc-config -l
```

----------

## bguan

Thanks gsoe for the tip...

I did run that step as part of the whole GCC4 migration but just to make sure, I tried it again:

  fix_libtool_files.sh 3.4.6

All seems well so I proceeded to emerge esound.

Same problem.

What else can I try?

Thanks.

FYI, my gcc-config -l is indeed showing GCC4 as my current gcc i.e.

[1] i686-pc-linux-gnu-3.3.6

[2] i686-pc-linux-gnu-3.4.6

[3] i686-pc-linux-gnu-3.4.6-hardened

...

[8] i686-pc-linux-gnu-4.1.2 *

----------

## gsoe

I'm not really sure if it will solve your problem, but as the older gcc-versions are to no use with the new glibc, you should delete them:

```
emerge -Pav gcc
```

and then for compatibility install libstdc++:

```
emerge virtual/libstdcc++
```

That should bring with it libstdc++-v3-3.3.6, if it doesn't then emerge it separately. Then try again to emerge esound and do revdep-rebuild (and maybe fix_libtool_files).

Good luck!

----------

## bguan

Thanks!  

I think removing the old gcc versions is not only nice-to-have, but necessary - otherwise emerge esound seems to want to use the older version even though gcc-config is pointing to the new one.

After removing the old gcc versions, I also unmerge and emerge libstdc++, env-update, finally esound can be emerged successfully.

Now I am on the way to try to satisfy the pre-conditions of Guenther's script.

Thanks again

----------

## billydv

Guenther ,  the  nice  thing  that  I  like  about  your  script  is  that  it  is  restartable  and  doesnt  stop  because  a  package  failed  to  compile,  Great  addition,  Thanks.

----------

## billydv

I  am  curious,  although  I  can  see  from  the  logs  how  many  packages  have  been  compiles  so  far,  is  there  a  way  to  tell  how  many  are  left,  also  I  am  assuming  that  failed  packages are  in  .recompile-remaining-packages.failures,  Do  they  show  up  in  that  log  immediately?

And  what  happens  if  its  compiling  a  game  or  something  that  requires  a  yes  and  an  enter?  How  long  will  it  wait  until  it  moves  on  to  the  next  package?

----------

## PaulBredbury

Remove games from the list:

```
sed -i -e "/^.*games-/d" recompile-remaining-packages
```

----------

## billydv

Okay,  I  found  out  some  answers  already,  apparently,  if  an  answer  is  required  its  gonna  sit  and  wait,

to  find  out  the  packages  in  order  run  grep item recompile-remaining-packages

Heres  my  list,  How  long  do  you  think  this  will  take?

item 1 =sys-kernel/linux-headers-2.6.21

item 2 =sys-libs/glibc-2.5-r4

item 3 =sys-devel/binutils-2.17

item 4 =sys-apps/portage-2.1.3_rc6

item 5 =dev-util/pkgconfig-0.21-r1

item 6 =virtual/libintl-0

item 7 =sys-devel/gnuconfig-20070118

item 8 =app-arch/unzip-5.52-r1

item 9 =sys-devel/automake-wrapper-3-r1

item 10 =app-arch/bzip2-1.0.4

item 11 =virtual/libiconv-0

item 12 =app-text/build-docbook-catalog-1.2

item 13 =media-libs/audiofile-0.2.6-r3

item 14 =sys-apps/tcp-wrappers-7.6-r8

item 15 =media-libs/libogg-1.1.3

item 16 =sys-devel/autoconf-wrapper-4-r3

item 17 =app-misc/pax-utils-0.1.15

item 18 =dev-libs/gmp-4.2.1-r1

item 19 =sys-apps/eject-2.1.5-r1

item 20 =media-sound/alsa-headers-1.0.14

item 21 =app-portage/portage-utils-0.1.28

item 22 =app-crypt/hashalot-0.3-r2

item 23 =dev-util/unifdef-1.20

item 24 =sys-libs/libutempter-1.1.5

item 25 =sys-libs/pwdb-0.62

item 26 =dev-db/mysql-init-scripts-1.2

item 27 =app-text/libpaper-1.1.21

item 28 =dev-libs/lzo-2.02-r1

item 29 =app-text/rman-3.2

item 30 =sys-fs/sysfsutils-2.1.0

item 31 =sys-libs/timezone-data-2007f

item 32 =sys-fs/device-mapper-1.02.19-r1

item 33 =sys-apps/pciutils-2.2.6-r1

item 34 =sys-apps/dmidecode-2.9

item 35 =app-arch/cabextract-1.2

item 36 =media-fonts/gnu-gs-fonts-std-8.11

item 37 =app-portage/portage-manpages-20070122

item 38 =sys-apps/sandbox-1.2.18.1

item 39 =sys-apps/debianutils-2.21

item 40 =sys-apps/hdparm-7.6

item 41 =sys-devel/patch-2.5.9-r1

item 42 =sys-apps/busybox-1.6.0

item 43 =app-arch/cpio-2.9

item 44 =sys-apps/setarch-2.0

item 45 =dev-java/java-config-wrapper-0.13

item 46 =dev-java/java-sdk-docs-1.6.0-r1

item 47 =dev-libs/mpfr-2.2.1_p5

item 48 =sys-apps/mktemp-1.5

item 49 =dev-libs/libpcre-7.2

item 50 =sys-apps/sysvinit-2.86-r8

item 51 =net-misc/iputils-20070202

item 52 =virtual/init-0

item 53 =sys-libs/zlib-1.2.3-r1

item 54 =media-libs/libpng-1.2.18

item 55 =media-libs/libid3tag-0.15.1b

item 56 =dev-libs/expat-2.0.0

item 57 =sys-devel/gettext-0.16.1-r1

item 58 =sys-apps/sed-4.1.5

item 59 =dev-libs/popt-1.10.7

item 60 =sys-devel/m4-1.4.9-r1

item 61 =dev-libs/libgpg-error-1.5

item 62 =sys-libs/com_err-1.40

item 63 =sys-devel/make-3.81

item 64 =sys-apps/findutils-4.3.8

item 65 =sys-apps/gawk-3.1.5-r3

item 66 =sys-devel/flex-2.5.33-r2

item 67 =app-arch/gzip-1.3.12

item 68 =app-arch/tar-1.18-r1

item 69 =sys-apps/grep-2.5.1a-r1

item 70 =sys-apps/kbd-1.13

item 71 =sys-apps/net-tools-1.60-r13

item 72 =sys-devel/bison-2.3

item 73 =media-libs/libart_lgpl-2.3.19-r1

item 74 =sys-libs/ss-1.40

item 75 =x11-misc/emacs-desktop-0.2-r1

item 76 =sys-devel/binutils-config-1.9-r4

item 77 =media-libs/libvorbis-1.1.2

item 78 =gnome-base/gnome-common-2.12.0

item 79 =media-video/mpeg2vidcodec-12-r1

item 80 =x11-themes/hicolor-icon-theme-0.10

item 81 =dev-python/python-docs-2.4.4

item 82 =app-admin/python-updater-0.2

item 83 =net-print/foomatic-db-ppds-20070508

item 84 =virtual/opengl-7.0

item 85 =virtual/glu-7.0

item 86 =virtual/emacs-22

item 87 =virtual/mysql-5.0

item 88 =virtual/jdk-1.6.0

item 89 =app-admin/eselect-opengl-1.0.5

item 90 =virtual/xft-7.0

item 91 =sys-kernel/gentoo-sources-2.6.21-r3

item 92 =virtual/ghostscript-0

item 93 =sys-fs/udev-113

item 94 =sys-apps/baselayout-1.12.10-r4

item 95 =sys-apps/module-init-tools-3.2.2-r3

item 96 =sys-apps/man-pages-2.61

item 97 =sys-apps/diffutils-2.8.7-r2

item 98 =virtual/perl-Storable-2.16

item 99 =virtual/glut-1.0

item 100 =sys-process/procps-3.2.7

item 101 =virtual/perl-Sys-Syslog-0.18

item 102 =app-admin/eselect-esd-20060719

item 103 =virtual/perl-Test-Simple-0.70

item 104 =x11-misc/xdg-utils-1.0.1

item 105 =virtual/jre-1.6.0

item 106 =app-admin/eselect-emacs-1.1

item 107 =app-misc/ca-certificates-20070303-r1

item 108 =net-print/foomatic-filters-ppds-20070501

item 109 =sys-devel/autoconf-2.61-r1

item 110 =sys-devel/libtool-1.5.24

item 111 =media-libs/jpeg-6b-r8

item 112 =media-libs/tiff-3.8.2-r2

item 113 =sys-devel/automake-1.10

item 114 =dev-libs/libgcrypt-1.2.4

item 115 =dev-lang/tcl-8.4.14

item 116 =sys-apps/attr-2.4.38

item 117 =dev-libs/libusb-0.1.12-r1

item 118 =media-libs/libmad-0.15.1b-r3

item 119 =sys-apps/acl-2.2.44

item 120 =dev-db/sqlite-3.3.17

item 121 =sys-apps/usbutils-0.72-r4

item 122 =sys-fs/cryptsetup-luks-1.0.4-r3

item 123 =dev-db/sqlite-2.8.16-r4

item 124 =net-misc/rsync-2.6.9-r2

item 125 =x11-misc/util-macros-1.1.5

item 126 =x11-proto/xproto-7.0.10

item 127 =x11-proto/xextproto-7.0.2

item 128 =x11-proto/randrproto-1.2.1

item 129 =x11-proto/renderproto-0.9.2

item 130 =x11-proto/fontsproto-2.0.2

item 131 =x11-proto/videoproto-2.2.2

item 132 =x11-proto/inputproto-1.4.2

item 133 =x11-proto/xf86driproto-2.0.3

item 134 =x11-libs/libdrm-2.3.0

item 135 =x11-proto/xf86dgaproto-2.0.2

item 136 =x11-proto/kbproto-1.0.3

item 137 =x11-proto/glproto-1.4.8

item 138 =x11-libs/libXau-1.0.3

item 139 =x11-libs/xtrans-1.0.3

item 140 =x11-libs/libICE-1.0.3

item 141 =x11-libs/libSM-1.0.2

item 142 =x11-proto/xf86miscproto-0.9.2

item 143 =x11-proto/xineramaproto-1.1.2

item 144 =x11-proto/xf86vidmodeproto-2.2.2

item 145 =x11-libs/libfontenc-1.0.4

item 146 =x11-proto/bigreqsproto-1.0.2

item 147 =x11-libs/libXdmcp-1.0.2

item 148 =x11-proto/fixesproto-4.0

item 149 =x11-proto/trapproto-3.4.3

item 150 =x11-proto/recordproto-1.13.2

item 151 =x11-proto/resourceproto-1.0.2

item 152 =media-fonts/font-util-1.0.1

item 153 =x11-proto/xf86rushproto-1.1.2

item 154 =x11-misc/xbitmaps-1.0.1

item 155 =x11-proto/xcmiscproto-1.1.2

item 156 =x11-proto/xf86bigfontproto-1.1.2

item 157 =x11-libs/libX11-1.1.2-r1

item 158 =media-libs/freetype-2.3.5

item 159 =x11-libs/libXext-1.0.3

item 160 =x11-libs/libXt-1.0.5

item 161 =media-libs/giflib-4.1.4

item 162 =x11-libs/startup-notification-0.8

item 163 =dev-lang/tk-8.4.14

item 164 =x11-libs/libXmu-1.0.3

item 165 =x11-libs/libXrender-0.9.2

item 166 =x11-libs/libXpm-3.5.6

item 167 =media-libs/gd-2.0.35

item 168 =x11-libs/libXi-1.1.0

item 169 =x11-apps/mkfontscale-1.0.3

item 170 =x11-libs/libxkbfile-1.0.4

item 171 =x11-apps/mkfontdir-1.0.3

item 172 =x11-libs/libXtst-1.0.1

item 173 =media-fonts/encodings-1.0.2

item 174 =x11-apps/xauth-1.0.2

item 175 =x11-apps/xinit-1.0.3-r4

item 176 =x11-libs/libXres-1.0.3

item 177 =x11-libs/libXrandr-1.2.1

item 178 =x11-libs/libXxf86vm-1.0.1

item 179 =x11-apps/xrdb-1.0.3

item 180 =x11-misc/xorg-cf-files-1.0.2

item 181 =x11-misc/imake-1.0.2

item 182 =app-doc/opengl-manpages-20001215

item 183 =x11-libs/libXfixes-4.0.3

item 184 =x11-libs/libXcursor-1.1.8

item 185 =x11-proto/printproto-1.0.3

item 186 =x11-libs/libXp-1.0.0

item 187 =x11-libs/libXxf86misc-1.0.1

item 188 =x11-libs/liblbxutil-1.0.1

item 189 =x11-libs/libxkbui-1.0.2

item 190 =x11-apps/rgb-1.0.1

item 191 =x11-apps/iceauth-1.0.1

item 192 =x11-proto/fontcacheproto-0.1.2

item 193 =x11-libs/libXfont-1.3.0

item 194 =x11-apps/bdftopcf-1.0.0

item 195 =x11-apps/xkbcomp-1.0.3

item 196 =x11-proto/damageproto-1.1.0

item 197 =x11-proto/evieext-1.0.2

item 198 =x11-apps/luit-1.0.2

item 199 =x11-proto/compositeproto-0.3.1

item 200 =x11-misc/makedepend-1.0.1

item 201 =media-libs/mesa-6.5.2-r1

item 202 =media-libs/freeglut-2.4.0-r1

item 203 =media-libs/glitz-0.5.6

item 204 =x11-proto/scrnsaverproto-1.1.0

item 205 =x11-libs/libXv-1.0.3

item 206 =x11-libs/libXvMC-1.0.4

item 207 =x11-libs/libXTrap-1.0.0

item 208 =x11-misc/gccmakedep-1.0.2

item 209 =x11-apps/xprop-1.0.2

item 210 =media-fonts/font-alias-1.0.1

item 211 =x11-wm/twm-1.0.3

item 212 =sys-apps/man-1.6e-r3

item 213 =dev-lang/perl-5.8.8-r2

item 214 =dev-perl/XML-Parser-2.34

item 215 =perl-core/Storable-2.16

item 216 =sys-devel/automake-1.8.5-r3

item 217 =perl-core/Sys-Syslog-0.18

item 218 =perl-core/Test-Simple-0.70

item 219 =dev-perl/Locale-gettext-1.05

item 220 =sys-devel/automake-1.5

item 221 =dev-perl/SGMLSpm-1.03-r5

item 222 =dev-perl/Net-Daemon-0.43

item 223 =app-crypt/opencdk-0.5.7

item 224 =dev-libs/libtasn1-0.3.9

item 225 =sys-devel/automake-1.7.9-r1

item 226 =sys-devel/automake-1.6.3

item 227 =perl-core/Test-Harness-2.64

item 228 =perl-core/PodParser-1.35

item 229 =sys-devel/automake-1.4_p6

item 230 =dev-util/intltool-0.35.5

item 231 =sys-apps/help2man-1.36.4

item 232 =net-libs/openslp-1.2.1

item 233 =x11-misc/xkeyboard-config-0.9

item 234 =dev-perl/PlRPC-0.2020-r1

item 235 =dev-perl/DBI-1.58

item 236 =gnome-base/gnome-mime-data-2.18.0

item 237 =dev-libs/openssl-0.9.8e-r1

item 238 =net-misc/wget-1.10.2

item 239 =x11-base/xorg-server-1.3.0.0

item 240 =x11-drivers/xf86-video-ark-0.6.0

item 241 =x11-drivers/xf86-video-tseng-1.1.1

item 242 =x11-drivers/xf86-video-vga-4.1.0

item 243 =x11-drivers/xf86-video-v4l-0.1.1

item 244 =x11-drivers/xf86-video-s3virge-1.9.1

item 245 =x11-drivers/xf86-video-vmware-10.15.0

item 246 =x11-drivers/xf86-video-nv-2.0.2

item 247 =x11-drivers/xf86-video-neomagic-1.1.1

item 248 =x11-drivers/xf86-video-i128-1.2.1

item 249 =x11-drivers/xf86-video-siliconmotion-1.5.1

item 250 =x11-drivers/xf86-video-dummy-0.2.0

item 251 =x11-drivers/xf86-input-evdev-1.1.5-r1

item 252 =x11-drivers/xf86-input-mouse-1.2.1

item 253 =x11-drivers/xf86-video-apm-1.1.1

item 254 =x11-drivers/xf86-video-i810-2.0.0

item 255 =x11-drivers/xf86-video-trident-1.2.3

item 256 =x11-drivers/xf86-input-keyboard-1.1.1

item 257 =x11-drivers/xf86-video-cirrus-1.1.0

item 258 =x11-drivers/xf86-video-mga-1.4.6.1

item 259 =x11-drivers/xf86-video-sisusb-0.8.1

item 260 =x11-drivers/xf86-video-savage-2.1.2-r1

item 261 =x11-drivers/xf86-video-chips-1.1.1

item 262 =x11-drivers/xf86-video-via-0.2.2

item 263 =x11-drivers/xf86-video-voodoo-1.1.1

item 264 =x11-drivers/xf86-video-rendition-4.1.3

item 265 =x11-drivers/xf86-video-tga-1.1.0

item 266 =x11-drivers/xf86-video-s3-0.5.0

item 267 =x11-drivers/xf86-video-cyrix-1.1.0

item 268 =x11-drivers/xf86-video-tdfx-1.3.0

item 269 =x11-drivers/xf86-video-vesa-1.3.0

item 270 =x11-drivers/xf86-video-sis-0.9.3

item 271 =x11-drivers/xf86-video-fbdev-0.3.1

item 272 =x11-drivers/xf86-video-glint-1.1.1

item 273 =x11-drivers/xf86-video-ati-6.6.3

item 274 =dev-lang/python-2.4.4-r4

item 275 =dev-java/java-config-2.0.33-r1

item 276 =dev-python/docutils-0.4-r2

item 277 =sys-libs/cracklib-2.8.10

item 278 =dev-python/python-fchksum-1.7.1

item 279 =dev-java/java-config-1.3.7

item 280 =sys-apps/file-4.21

item 281 =dev-python/pyrex-0.9.5.1a

item 282 =dev-python/pyopengl-2.0.1.09-r1

item 283 =dev-python/numeric-24.2-r4

item 284 =app-text/iso-codes-0.58

item 285 =dev-python/pycrypto-2.0.1-r5

item 286 =sys-libs/pam-0.99.8.0

item 287 =sys-apps/shadow-4.0.18.1-r1

item 288 =dev-java/sun-jdk-1.6.0.01

item 289 =sys-libs/db-4.5.20_p2

item 290 =dev-lang/swig-1.3.31

item 291 =sys-libs/gdbm-1.8.3-r3

item 292 =media-libs/lcms-1.16

item 293 =sys-libs/libcap-1.10-r10

item 294 =sys-devel/libperl-5.8.8-r1

item 295 =dev-libs/glib-2.12.12

item 296 =app-admin/gamin-0.1.8

item 297 =dev-libs/libIDL-0.8.8

item 298 =dev-util/desktop-file-utils-0.12

item 299 =dev-libs/libxml2-2.6.28

item 300 =app-text/docbook-xsl-stylesheets-1.72.0

item 301 =dev-libs/libxslt-1.1.20

item 302 =dev-perl/XML-NamespaceSupport-1.09

item 303 =x11-misc/shared-mime-info-0.20

item 304 =net-misc/neon-0.26.3

item 305 =dev-libs/libcroco-0.6.1

item 306 =dev-perl/XML-LibXML-Common-0.13

item 307 =dev-perl/XML-SAX-0.16

item 308 =dev-python/pygobject-2.12.3

item 309 =dev-perl/XML-LibXML-1.63

item 310 =dev-perl/XML-Simple-2.16

item 311 =x11-misc/icon-naming-utils-0.8.2

item 312 =x11-themes/gnome-icon-theme-2.18.0

item 313 =sys-libs/ncurses-5.6-r1

item 314 =sys-apps/texinfo-4.8-r5

item 315 =app-shells/bash-3.2_p17

item 316 =sys-libs/gpm-1.20.1-r6

item 317 =sys-apps/coreutils-6.9-r1

item 318 =www-client/lynx-2.8.6-r2

item 319 =net-misc/netkit-rsh-0.17-r8

item 320 =sys-process/psmisc-22.5

item 321 =sys-apps/less-406

item 322 =app-editors/nano-2.0.6

item 323 =sys-libs/readline-5.2_p4

item 324 =sys-devel/automake-1.9.6-r2

item 325 =sys-apps/ed-0.5

item 326 =app-admin/eselect-1.0.10

item 327 =sys-apps/groff-1.19.2-r1

item 328 =sys-devel/autoconf-2.13

item 329 =sys-fs/e2fsprogs-1.40

item 330 =sys-apps/which-2.16

item 331 =dev-util/dialog-1.1.20070704

item 332 =media-libs/aalib-1.4_rc5

item 333 =app-admin/perl-cleaner-1.04.3

item 334 =app-text/sgml-common-0.6.3-r5

item 335 =x11-libs/libXaw-1.0.3

item 336 =dev-db/mysql-5.0.42

item 337 =sys-apps/util-linux-2.12r-r7

item 338 =media-libs/libmng-1.0.9-r1

item 339 =app-text/aspell-0.60.5

item 340 =app-text/docbook-xml-dtd-4.1.2-r6

item 341 =media-libs/nas-1.8b

item 342 =app-text/docbook-xml-dtd-4.4-r1

item 343 =app-text/docbook-xml-dtd-4.2-r2

item 344 =app-text/docbook-dsssl-stylesheets-1.79

item 345 =app-text/docbook-sgml-dtd-3.0-r3

item 346 =app-text/docbook-sgml-dtd-4.1-r3

item 347 =app-text/docbook-sgml-dtd-4.0-r3

item 348 =app-text/docbook-sgml-dtd-3.1-r3

item 349 =app-text/docbook-xml-simple-dtd-4.1.2.4-r2

item 350 =app-text/docbook-xml-simple-dtd-1.0-r1

item 351 =net-libs/libwww-5.4.0-r7

item 352 =x11-apps/xsm-1.0.1

item 353 =dev-perl/DBD-mysql-4.00.5

item 354 =app-dicts/aspell-en-6.0.0

item 355 =app-text/xmlto-0.0.18

item 356 =app-text/tetex-3.0_p1-r3

item 357 =app-text/scrollkeeper-0.3.14-r2

item 358 =app-text/gnome-doc-utils-0.10.3

item 359 =app-crypt/mit-krb5-1.5.3

item 360 =app-text/opensp-1.5.2-r1

item 361 =app-text/openjade-1.3.2-r1

item 362 =dev-db/libpq-8.2.4

item 363 =net-misc/openssh-4.6_p1-r2

item 364 =app-text/jadetex-3.13-r1

item 365 =app-text/docbook-sgml-utils-0.6.14

item 366 =media-libs/fontconfig-2.4.2

item 367 =x11-libs/libXft-2.1.12

item 368 =media-fonts/font-adobe-75dpi-1.0.0

item 369 =media-fonts/font-misc-misc-1.0.0

item 370 =media-fonts/font-cursor-misc-1.0.0

item 371 =media-fonts/corefonts-1-r3

item 372 =app-text/poppler-0.5.4-r1

item 373 =x11-terms/xterm-227

item 374 =x11-apps/xclock-1.0.2

item 375 =media-libs/libsndfile-1.0.17

item 376 =x11-libs/gtk+-2.10.13

item 377 =app-editors/emacs-22.1

item 378 =dev-util/gtk-doc-1.8

item 379 =app-emacs/po-mode-0.16.1

item 380 =app-emacs/rst-0.4

item 381 =app-emacs/autoconf-mode-2.61

item 382 =dev-libs/atk-1.18.0

item 383 =gnome-base/orbit-2.14.7

item 384 =x11-libs/libwnck-2.18.2

item 385 =gnome-base/gconf-2.18.0.1

item 386 =gnome-base/libglade-2.6.1

item 387 =gnome-base/libbonobo-2.18.0

item 388 =app-doc/doxygen-1.5.2

item 389 =media-libs/alsa-lib-1.0.14a-r1

item 390 =sys-apps/dbus-1.0.2-r2

item 391 =media-libs/imlib2-1.4.0

item 392 =media-libs/flac-1.1.2-r8

item 393 =dev-libs/libdaemon-0.11-r1

item 394 =media-libs/libexif-0.6.16

item 395 =dev-libs/dbus-glib-0.73

item 396 =media-sound/esound-0.2.38

item 397 =gnome-base/gnome-keyring-0.8.1

item 398 =media-libs/libcaca-0.99_beta11

item 399 =media-sound/jack-audio-connection-kit-0.103.0

item 400 =sys-apps/hal-0.5.9.1

item 401 =dev-python/dbus-python-0.81.0

item 402 =app-misc/hal-info-20070618

item 403 =gnome-base/gnome-vfs-2.18.1

item 404 =gnome-base/libgnome-2.18.0

item 405 =net-print/cups-1.2.11

item 406 =x11-libs/qt-3.3.8-r2

item 407 =kde-base/arts-3.5.5

item 408 =x11-libs/pango-1.16.4

item 409 =gnome-base/libgnomecanvas-2.14.0

item 410 =x11-libs/libsexy-0.1.11

item 411 =gnome-base/libbonoboui-2.18.0

item 412 =gnome-base/gail-1.18.0

item 413 =x11-misc/notification-daemon-0.3.7

item 414 =gnome-base/libgnomeui-2.18.1

item 415 =app-text/ghostscript-gpl-8.57

item 416 =net-libs/gnutls-1.6.2

item 417 =x11-libs/gtkglarea-1.99.0

item 418 =x11-libs/libnotify-0.4.4

item 419 =dev-db/unixODBC-2.2.12

item 420 =gnome-base/gnome-desktop-2.18.2

item 421 =media-libs/libwmf-0.2.8.4

item 422 =net-print/foomatic-filters-3.0.20070501

item 423 =media-gfx/graphviz-2.12

item 424 =net-nds/openldap-2.3.36

item 425 =x11-libs/qt-4.3.0

item 426 =media-gfx/imagemagick-6.3.4-r1

item 427 =dev-db/qt-unixODBC-3.3.8

item 428 =net-fs/samba-3.0.24-r3

item 429 =x11-libs/cairo-1.4.10

item 430 =dev-python/pycairo-1.4.0

item 431 =dev-python/pygtk-2.10.4

item 432 =net-dns/avahi-0.6.20

item 433 =gnome-base/gnome-menus-2.18.2

item 434 =gnome-extra/libgsf-1.14.3

item 435 =gnome-base/eel-2.18.0.1

item 436 =gnome-base/librsvg-2.16.1-r2

item 437 =gnome-base/nautilus-2.18.1

item 438 =gnome-base/gnome-mount-0.6

item 439 =dev-libs/DirectFB-1.0.0

item 440 =media-libs/libsdl-1.2.11-r2

item 441 =dev-java/javatoolkit-0.2.0-r1

item 442 =app-emulation/emul-linux-x86-compat-1.0-r3

item 443 =dev-libs/nspr-4.6.7

item 444 =sys-libs/libraw1394-1.2.1

item 445 =app-arch/zip-2.32

item 446 =sys-apps/miscfiles-1.4.2

item 447 =dev-libs/fribidi-0.10.7

item 448 =app-admin/hddtemp-0.3_beta15-r1

item 449 =media-libs/amd64codecs-20061203

item 450 =media-libs/libmp4v2-1.5.0.1

item 451 =dev-util/yacc-1.9.1-r3

item 452 =dev-libs/libsigc++-2.0.17

item 453 =app-crypt/libgssapi-0.11

item 454 =dev-libs/STLport-5.1.2

item 455 =dev-ruby/ruby-config-0.3.2

item 456 =sys-libs/db-1.85-r3

item 457 =app-admin/php-toolkit-1.0-r2

item 458 =app-misc/mime-types-7

item 459 =dev-libs/eventlog-0.2.5

item 460 =dev-libs/icu-3.6

item 461 =net-dns/c-ares-1.4.0

item 462 =media-libs/cal3d-0.11.0-r1

item 463 =media-libs/dumb-0.9.3-r1

item 464 =dev-util/gperf-3.0.3

item 465 =dev-util/boost-build-1.34.0

item 466 =media-libs/libuninameslist-20060907

item 467 =app-doc/quanta-docs-20051201

item 468 =net-libs/adns-1.4

item 469 =sys-process/cronbase-0.3.2

item 470 =sys-libs/libaal-1.0.5

item 471 =net-libs/enet-1.0

item 472 =dev-libs/libevent-1.3a

item 473 =dev-libs/apr-0.9.12

item 474 =dev-libs/libmcrypt-2.5.8

item 475 =sys-fs/hfsutils-3.2.6-r5

item 476 =sys-fs/dosfstools-2.11-r3

item 477 =sys-fs/reiserfsprogs-3.6.19-r1

item 478 =net-analyzer/traceroute-1.4_p12-r5

item 479 =net-misc/netkit-fingerd-0.17-r3

item 480 =sys-apps/hotplug-base-20040401

item 481 =dev-libs/libbulletml-0.0.6

item 482 =app-arch/unrar-3.7.5

item 483 =app-cdr/bin2iso-19b-r2

item 484 =x11-themes/silver-xcursors-0.4

item 485 =games-board/phalanx-22

item 486 =dev-libs/libol-0.3.18

item 487 =x11-themes/korilla-1.3.5

item 488 =games-rpg/twclone-0.14

item 489 =app-benchmarks/bonnie-2.0.6

item 490 =x11-themes/gentoo-xcursors-0.3.1

item 491 =games-puzzle/drod-bin-1.6.6

item 492 =x11-themes/blueglass-xcursors-0.4

item 493 =x11-themes/golden-xcursors-0.8

item 494 =x11-themes/flatsvg-1.0

item 495 =x11-themes/lush-0.1.0-r2

item 496 =x11-themes/gentoo-artwork-livecd-2007.0

item 497 =dev-cpp/libbinio-1.4

item 498 =games-arcade/cavezofphear-0.5

item 499 =x11-themes/gentoo-artwork-0.4.2

item 500 =net-misc/dhcpcd-3.0.18

item 501 =sys-apps/sdparm-1.01

item 502 =games-mud/ytin-1.83.5-r1

item 503 =x11-themes/nuvola-1.0-r1

item 504 =x11-themes/noia-1.0-r1

item 505 =app-benchmarks/bonnie++-1.93c

item 506 =dev-libs/nss-3.11.7

item 507 =dev-java/java-sdk-docs-1.4.2

item 508 =dev-java/java-sdk-docs-1.5.0-r1

item 509 =net-nds/portmap-6.0

item 510 =media-libs/portaudio-18.1-r6

item 511 =sys-libs/libavc1394-0.5.3

item 512 =sys-power/cpufrequtils-002-r3

item 513 =net-libs/librpcsecgss-0.14-r1

item 514 =games-rpg/eternal-lands-data-1.3.3

item 515 =games-board/fruit-2.1

item 516 =games-fps/ut2004-redorchestra-3.3-r2

item 517 =games-fps/freedoom-0.5

item 518 =games-emulation/emutos-0.8.1

item 519 =games-server/tetrix-1.13.16.1.40c-r2

item 520 =games-emulation/caps-20060612

item 521 =sys-libs/lib-compat-loki-0.2

item 522 =games-fps/ut2004-airbuccaneers-1.6-r1

item 523 =games-util/fteqcc-2501

item 524 =games-board/crafty-20.14

item 525 =app-dicts/myspell-en-20060316

item 526 =media-tv/linuxtv-dvb-headers-3.1

item 527 =media-libs/pablio-18.1

item 528 =media-libs/taglib-1.4-r1

item 529 =dev-games/physfs-1.0.1

item 530 =games-util/uz2unpack-0.1

item 531 =games-util/ucon64-2.0.0

item 532 =media-gfx/fbgrab-1.0

item 533 =media-gfx/pngtoico-1.0.1

item 534 =media-libs/musicbrainz-2.1.4

item 535 =dev-libs/libcdio-0.78.2

item 536 =app-text/recode-3.6-r2

item 537 =dev-lang/yasm-0.6.0

item 538 =games-misc/fortune-mod-1.99.1-r2

item 539 =app-emulation/emul-linux-x86-baselibs-10.2

item 540 =xfce-extra/xfce4-dev-tools-4.4.0

item 541 =dev-libs/pth-2.0.7

item 542 =sys-power/powermgmt-base-1.22

item 543 =gnome-extra/gnome-games-extra-data-2.14.0

item 544 =net-misc/dhcp-3.0.6

item 545 =app-admin/logrotate-3.7.2

item 546 =dev-games/hawknl-1.68-r1

item 547 =gnome-extra/gnome-audio-2.0.0

item 548 =sys-power/acpid-1.0.4-r5

item 549 =app-doc/gimp-user-manual-2.0

item 550 =dev-util/jam-2.5-r3

item 551 =games-misc/fortune-mod-powerpuff-0.3

item 552 =games-misc/fortune-mod-dubya-20050118

item 553 =games-misc/fortune-mod-humorixfortunes-1.4-r1

item 554 =games-misc/fortune-mod-tao-1

item 555 =games-misc/fortune-mod-chucknorris-0.1

item 556 =games-misc/fortune-mod-starwars-0.1

item 557 =games-misc/fortune-mod-dune-2.0.1

item 558 =games-misc/fortune-mod-hitchhiker-0.1

item 559 =games-misc/fortune-mod-familyguy-0.2

item 560 =games-misc/fortune-mod-sp-fortunes-0.2

item 561 =games-misc/fortune-mod-kernelcookies-9

item 562 =games-misc/fortune-mod-homer-0.1

item 563 =games-misc/fortune-mod-firefly-2.0

item 564 =games-misc/fortune-mod-pqf-6.0

item 565 =games-misc/fortune-mod-calvin-0.1.1

item 566 =games-misc/fortune-mod-smac-0.1

item 567 =games-misc/fortune-mod-gentoo-forums-20041207

item 568 =games-misc/fortune-mod-osfortune-1

item 569 =games-misc/fortune-mod-strangelove-20041203

item 570 =games-misc/fortune-mod-futurama-0.2

item 571 =games-misc/fortune-mod-bofh-excuses-1.2

item 572 =games-misc/fortune-mod-zx-error-1.0

item 573 =games-misc/fortune-mod-slackware-1.13

item 574 =games-misc/fortune-mod-simpsons-chalkboard-0.1

item 575 =games-misc/fortune-mod-scriptures-1.1.0

item 576 =app-text/enscript-1.6.4-r3

item 577 =games-misc/fortune-mod-it-1.99

item 578 =sys-power/hibernate-script-1.95-r3

item 579 =games-misc/fortune-mod-rss-20030120

item 580 =games-misc/fortune-mod-thomas-ogrisegg-20030120

item 581 =games-misc/fortune-mod-at-linux-20030120

item 582 =games-misc/fortune-mod-debilneho-0.1

item 583 =games-misc/fortune-mod-fvl-20030120

item 584 =games-misc/fortune-mod-martin-piskernig-20030120

item 585 =games-misc/fortune-mod-cs-1.6.9

item 586 =games-misc/fortune-mod-norbert-tretkowski-20030120

item 587 =games-misc/fortune-mod-mormon-1.1.0

item 588 =app-doc/php-docs-20070202-r1

item 589 =app-emulation/emul-linux-x86-xlibs-10.0

item 590 =app-emulation/emul-linux-x86-qtlibs-10.0-r1

item 591 =app-emulation/emul-linux-x86-soundlibs-10.0-r1

item 592 =app-emulation/emul-linux-x86-gtklibs-10.0-r1

item 593 =app-emulation/emul-linux-x86-sdl-10.1

item 594 =net-www/netscape-flash-9.0.31.0

item 595 =app-emulation/emul-linux-x86-medialibs-10.2

item 596 =app-text/acroread-7.0.9-r1

item 597 =games-arcade/barbarian-bin-1.01

item 598 =games-fps/ut2004-3369-r4

item 599 =games-fps/ut2004-bonuspack-cbp1-1-r1

item 600 =games-fps/ut2004-deathball-2.3-r1

item 601 =games-fps/ut2004-alienswarm-1.32

item 602 =games-fps/ut2004-action-1

item 603 =games-fps/ut2004-strikeforce-3.01-r1

item 604 =games-fps/ut2004-da2-1.6_beta

item 605 =games-fps/ut2004-troopers-5.0

item 606 =games-fps/ut2004-ultraduel-2.1

item 607 =games-fps/ut2004-damnation-2.0

item 608 =games-fps/ut2004-crossfire-1.7

item 609 =games-fps/ut2004-muralis-1.15

item 610 =games-fps/ut2004-cor-1.01

item 611 =games-fps/ut2004-fragops-2.16-r1

item 612 =games-fps/ut2004-bonuspack-cbp2-1-r1

item 613 =dev-libs/libnl-1.0_pre6

item 614 =net-firewall/iptables-1.3.8-r1

item 615 =virtual/perl-MIME-Base64-3.07

item 616 =www-client/mozilla-launcher-1.56

item 617 =virtual/jdk-1.4.2

item 618 =dev-util/ctags-5.6-r3

item 619 =virtual/perl-Digest-MD5-2.36

item 620 =sys-fs/fuse-2.7.0

item 621 =sys-apps/pcmciautils-014-r1

item 622 =sys-fs/ntfs3g-1.616-r2

item 623 =virtual/perl-Scalar-List-Utils-1.19

item 624 =games-fps/ut2004-data-3186-r3

item 625 =games-fps/ut2004-bonuspack-ece-1-r3

item 626 =games-fps/ut2004-bonuspack-mega-1-r2

item 627 =virtual/perl-libnet-1.21

item 628 =virtual/perl-digest-base-1.15

item 629 =net-print/foomatic-db-engine-3.0.20070508

item 630 =net-print/foomatic-db-20070508

item 631 =games-fps/enemy-territory-2.60b

item 632 =games-fps/enemy-territory-fortress-1.6-r2

item 633 =games-fps/enemy-territory-etpro-3.2.6-r1

item 634 =games-fps/enemy-territory-truecombat-0.49b

item 635 =virtual/jdk-1.5.0

item 636 =virtual/perl-Test-Harness-2.64

item 637 =net-mail/mailbase-1

item 638 =app-admin/eselect-timidity-20061203

item 639 =media-sound/timidity-eawpatches-12-r5

item 640 =virtual/libstdc++-3.3

item 641 =virtual/perl-File-Spec-3.25

item 642 =net-wireless/wireless-tools-29_pre22

item 643 =net-wireless/wpa_supplicant-0.5.8

item 644 =sys-apps/lm_sensors-2.10.3

item 645 =app-admin/eselect-vi-1.1.5

item 646 =virtual/perl-Time-HiRes-1.97.07

item 647 =virtual/perl-DB_File-1.815

item 648 =virtual/perl-PodParser-1.35

item 649 =kde-base/kdepim-meta-3.5.7

item 650 =media-gfx/ebdftopcf-2

item 651 =dev-libs/libmcs-0.4.1

item 652 =app-admin/eselect-oodict-20061117

item 653 =app-arch/rar-3.7.0

item 654 =kde-base/kdemultimedia-meta-3.5.7

item 655 =net-libs/opal-2.2.8

item 656 =app-shells/bash-completion-20060301

item 657 =app-shells/gentoo-bashcomp-20050516

item 658 =virtual/jre-1.4.2

item 659 =x11-misc/sux-1.0-r3

item 660 =kde-base/kdenetwork-meta-3.5.7

item 661 =kde-base/kdeaccessibility-meta-3.5.7

item 662 =kde-base/kdeadmin-meta-3.5.7

item 663 =kde-base/kdeutils-meta-3.5.7

item 664 =kde-base/kdegraphics-meta-3.5.7

item 665 =kde-base/kdetoys-meta-3.5.7

item 666 =xfce-extra/xfwm4-themes-4.4.1

item 667 =kde-base/kdebase-meta-3.5.7

item 668 =kde-base/kdeedu-meta-3.5.7

item 669 =kde-base/kdeaddons-meta-3.5.7

item 670 =kde-base/kdewebdev-meta-3.5.7

item 671 =kde-base/kdeartwork-meta-3.5.7

item 672 =kde-base/kdegames-meta-3.5.7

item 673 =games-misc/fortune-mod-all-1

item 674 =games-puzzle/galaxis-1.7

item 675 =games-arcade/pydance-songs-20040410

item 676 =x11-wm/beryl-0.2.1

item 677 =games-puzzle/tetrinet-0.11

item 678 =www-client/mozilla-firefox-bin-2.0.0.4

item 679 =games-misc/funny-manpages-1.3_rc5-r1

item 680 =games-rpg/nwn-1.68-r1

item 681 =games-arcade/triplexinvaders-1.08

item 682 =games-misc/cowsay-3.03-r1

item 683 =games-fps/americas-army-250

item 684 =app-portage/cfg-update-1.8.2-r1

item 685 =games-util/biounzip-1.1a

item 686 =games-arcade/funnyboat-1.5

item 687 =games-arcade/solarwolf-1.5

item 688 =games-arcade/mtp-target-bin-1.2.0

item 689 =games-arcade/watermelons-1.1.1

item 690 =games-strategy/outerspace-0.5.64

item 691 =games-strategy/castle-combat-0.8.1

item 692 =x11-themes/mplayer-skins-0.2-r5

item 693 =net-nds/smbldap-tools-0.9.2a

item 694 =games-misc/asr-manpages-1.3_rc5

item 695 =kde-base/kdesdk-meta-3.5.7

item 696 =media-sound/mp3info-0.8.5a

item 697 =games-strategy/darwinia-1.4.0_beta9

item 698 =games-rpg/galaxymage-0.3.0

item 699 =media-libs/win32codecs-20061022-r1

item 700 =games-board/cgoban2-2.6.12

item 701 =games-arcade/jardinains-2.0

item 702 =virtual/jre-1.5.0

item 703 =app-office/koffice-meta-1.6.3

item 704 =games-strategy/bos-2.0.1

item 705 =x11-misc/googleearth-4

item 706 =games-rpg/dragonhunt-3.56

item 707 =games-action/phobiaii-1.1

item 708 =games-action/astromenace-bin-1.0

item 709 =kde-base/kdebindings-meta-3.5.7

item 710 =x11-misc/genmenu-1.0.7-r1

item 711 =games-action/parsec-0197

item 712 =x11-themes/fusionx-aqua-1.1

item 713 =games-puzzle/4stattack-2.1.4

item 714 =kde-base/kde-meta-3.5.7

item 715 =games-fps/legends-0.4.1.42

item 716 =games-simulation/simutrans-0.88.10.5

item 717 =xfce-base/xfce4-extras-4.4.1-r1

item 718 =x11-themes/gdm-themes-livecd-2007.0

item 719 =gnome-base/gnome-2.18.0

item 720 =games-simulation/qct-0.7

item 721 =media-gfx/jpeginfo-1.6.0

item 722 =perl-core/Scalar-List-Utils-1.19

item 723 =dev-perl/URI-1.35

item 724 =perl-core/libnet-1.21

item 725 =perl-core/digest-base-1.15

item 726 =dev-perl/extutils-depends-0.205

item 727 =dev-perl/extutils-pkgconfig-1.07

item 728 =dev-perl/Compress-Raw-Zlib-2.005

item 729 =dev-perl/HTML-Tagset-3.10

item 730 =perl-core/Time-HiRes-1.97.07

item 731 =dev-perl/Unicode-String-2.09

item 732 =app-text/texi2html-1.76

item 733 =dev-perl/File-BaseDir-0.02

item 734 =dev-perl/IO-String-1.08

item 735 =dev-perl/yaml-0.65

item 736 =games-misc/fortune-mod-gentoo-dev-20070428

item 737 =dev-perl/Text-Iconv-1.4

item 738 =dev-perl/Convert-ASN1-0.21

item 739 =dev-perl/Text-CharWidth-0.04

item 740 =dev-perl/TermReadKey-2.30

item 741 =dev-perl/Socket6-0.19

item 742 =dev-perl/Net-IP-1.25-r1

item 743 =dev-libs/klibc-1.5

item 744 =dev-libs/libical-0.26.7

item 745 =dev-perl/XML-Writer-0.603

item 746 =dev-perl/File-DesktopEntry-0.02

item 747 =dev-perl/File-Which-0.05

item 748 =dev-perl/Exporter-Lite-0.02

item 749 =dev-perl/Digest-MD4-1.5

item 750 =dev-perl/Font-AFM-1.19

item 751 =dev-perl/Unicode-Map-0.112

item 752 =dev-perl/Jcode-2.06

item 753 =dev-perl/Archive-Rar-1.9.2

item 754 =dev-perl/HTML-Clean-0.8-r1

item 755 =dev-perl/ConfigReader-0.5

item 756 =dev-perl/Config-IniFiles-2.39

item 757 =games-misc/rfksay-0.1

item 758 =dev-perl/HTML-FillInForm-1.06

item 759 =dev-perl/Config-Simple-4.58

item 760 =dev-perl/Compress-Bzip2-2.09

item 761 =dev-perl/Config-Crontab-1.20

item 762 =dev-perl/HTML-Parser-3.56

item 763 =dev-perl/Digest-SHA1-2.11

item 764 =dev-libs/libsigc++-1.2.5

item 765 =dev-perl/IO-Compress-Base-2.005

item 766 =dev-perl/IO-Socket-INET6-2.51

item 767 =dev-perl/MailTools-1.77

item 768 =dev-perl/Text-WrapI18N-0.06

item 769 =dev-perl/Unicode-Map8-0.12

item 770 =dev-perl/XML-RegExp-0.03-r1

item 771 =dev-perl/Crypt-SmbHash-0.12

item 772 =app-text/htmltidy-5.10.26-r2

item 773 =perl-core/Digest-MD5-2.36

item 774 =dev-perl/Config-Tiny-2.10

item 775 =dev-perl/HTML-Tree-3.23

item 776 =dev-perl/Digest-HMAC-1.01-r1

item 777 =dev-perl/IO-Compress-Zlib-2.005

item 778 =x11-themes/gnome-backgrounds-2.18.3

item 779 =xfce-extra/xfce4-icon-theme-4.4.1

item 780 =dev-perl/Compress-Zlib-2.005

item 781 =dev-perl/Net-DNS-0.60

item 782 =dev-perl/DBD-SQLite-1.13

item 783 =dev-perl/HTML-Format-2.04

item 784 =dev-perl/IO-Zlib-1.05

item 785 =dev-perl/Email-Valid-0.179

item 786 =dev-perl/Archive-Tar-1.32

item 787 =dev-perl/Email-Find-0.10

item 788 =dev-perl/module-build-0.28.08

item 789 =dev-perl/HTML-FromText-2.05

item 790 =dev-perl/ExtUtils-CBuilder-0.19

item 791 =dev-perl/Test-Number-Delta-1.03

item 792 =dev-perl/Unicode-MapUTF8-1.11

item 793 =dev-perl/Error-0.17.008

item 794 =dev-perl/File-Type-0.22

item 795 =perl-core/File-Spec-3.25

item 796 =dev-perl/extutils-parsexs-2.18

item 797 =dev-perl/Archive-Zip-1.20

item 798 =dev-perl/Net-SSLeay-1.30

item 799 =net-libs/libgadu-1.7.1

item 800 =dev-perl/IO-Socket-SSL-1.07

item 801 =media-gfx/xv-3.10a-r12

item 802 =media-libs/libsamplerate-0.1.2-r1

item 803 =app-crypt/qca-1.0-r2

item 804 =dev-python/qscintilla-1.7.1

item 805 =games-util/agistudio-1.2.2

item 806 =kde-misc/kaptain-0.72

item 807 =games-puzzle/cuyo-1.8.6

item 808 =app-crypt/qca-tls-1.0-r3

item 809 =kde-base/kdelibs-3.5.7

item 810 =kde-base/kdebase-data-3.5.7

item 811 =kde-base/kdeartwork-emoticons-3.5.4

item 812 =kde-base/kdeartwork-iconthemes-3.5.7

item 813 =kde-base/kdeartwork-sounds-3.5.6

item 814 =kde-base/kdeartwork-styles-3.5.7

item 815 =kde-base/kdeartwork-wallpapers-3.5.6

item 816 =kde-misc/kload-0.9.4

item 817 =kde-base/kdeartwork-kworldclock-3.5.7

item 818 =kde-base/kdeartwork-icewm-themes-3.5.7

item 819 =games-simulation/kfreeflight-0.2.1_rc1

item 820 =kde-base/libkdegames-3.5.7

item 821 =kde-base/libkonq-3.5.7

item 822 =kde-base/libkmime-3.5.7

item 823 =kde-base/kdialog-3.5.5

item 824 =kde-base/libkdeedu-3.5.7

item 825 =kde-base/kdesu-3.5.7

item 826 =kde-misc/kxdocker-1.1.4a

item 827 =kde-base/ktnef-3.5.7

item 828 =kde-base/libkcddb-3.5.7

item 829 =kde-base/kate-3.5.7

item 830 =kde-base/libkpgp-3.5.4

item 831 =kde-base/libkholidays-3.5.7

item 832 =kde-base/mimelib-3.5.7

item 833 =kde-base/kcminit-3.5.6

item 834 =kde-base/ksplashml-3.5.7

item 835 =kde-base/smoke-3.5.6

item 836 =kde-base/librss-3.5.6

item 837 =kde-base/kdeaddons-docs-konq-plugins-3.5.7

item 838 =kde-base/kpersonalizer-3.5.7

item 839 =app-office/koffice-data-1.6.3

item 840 =kde-base/kode-3.5.6

item 841 =kde-base/kmailcvt-3.5.5

item 842 =kde-base/libksieve-3.5.7

item 843 =kde-base/khotkeys-3.5.7

item 844 =kde-base/kdegraphics-kfile-plugins-3.5.7

item 845 =kde-base/kworldwatch-3.5.7

item 846 =kde-base/kmid-3.5.7

item 847 =kde-base/ksmserver-3.5.7

item 848 =kde-base/ksysguard-3.5.7

item 849 =kde-base/kreadconfig-3.5.6

item 850 =kde-base/kwordquiz-3.5.7

item 851 =kde-base/kverbos-3.5.7

item 852 =kde-base/networkstatus-3.5.7

item 853 =kde-base/lisa-3.5.7

item 854 =kde-base/kdesdk-kfile-plugins-3.5.7

item 855 =kde-base/superkaramba-3.5.7

item 856 =kde-base/ktimer-3.5.7

item 857 =kde-base/kedit-3.5.7

item 858 =kde-base/kview-3.5.7

item 859 =kde-base/artsplugin-audiofile-3.5.4

item 860 =kde-base/ksnapshot-3.5.7

item 861 =kde-base/kdeaddons-kfile-plugins-3.5.6-r1

item 862 =kde-base/kolourpaint-3.5.7

item 863 =kde-base/kuiviewer-3.5.7

item 864 =kde-base/kruler-3.5.7

item 865 =kde-base/klettres-3.5.7

item 866 =kde-base/kde-i18n-3.5.7

item 867 =kde-base/kspy-3.5.6

item 868 =kde-base/kmenuedit-3.5.7

item 869 =kde-base/kdemultimedia-kappfinder-data-3.5.7

item 870 =kde-base/kmrml-3.5.7

item 871 =kde-base/kommander-3.5.7

item 872 =kde-base/kdebugdialog-3.5.6

item 873 =media-libs/libkipi-0.1.5

item 874 =kde-base/kgeography-3.5.7

item 875 =kde-misc/kdiff3-0.9.92

item 876 =kde-base/kbugbuster-3.5.7

item 877 =kde-base/ktip-3.5.7

item 878 =kde-base/kcoloredit-3.5.7

item 879 =kde-base/kdnssd-3.5.7

item 880 =kde-base/kodo-3.5.7

item 881 =kde-base/kweather-3.5.7

item 882 =kde-base/kgamma-3.5.7

item 883 =kde-base/kdenetwork-filesharing-3.5.7

item 884 =kde-base/kdict-3.5.7

item 885 =kde-base/kdenetwork-kfile-plugins-3.5.7

item 886 =kde-base/kget-3.5.7

item 887 =kde-base/ksirc-3.5.7

item 888 =kde-base/kpf-3.5.7

item 889 =kde-base/kvoctrain-3.5.7

item 890 =kde-base/kfloppy-3.5.7

item 891 =kde-base/khexedit-3.5.7

item 892 =kde-base/klinkstatus-3.5.7

item 893 =kde-base/kimagemapeditor-3.5.7

item 894 =kde-base/renamedlg-audio-3.5.7

item 895 =kde-base/kdesdk-misc-3.5.6

item 896 =kde-base/kmag-3.5.7

item 897 =kde-base/kdeaccessibility-iconthemes-3.5.7

item 898 =kde-base/kmousetool-3.5.7

item 899 =kde-base/kmouth-3.5.7

item 900 =kde-base/kig-3.5.7

item 901 =kde-base/kdeadmin-kfile-plugins-3.5.7

item 902 =kde-base/kdesdk-scripts-3.5.7

item 903 =kde-base/kompare-3.5.7

item 904 =kde-base/kapptemplate-3.5.6

item 905 =kde-base/umbrello-3.5.7

item 906 =kde-base/kregexpeditor-3.5.7

item 907 =kde-base/kjots-3.5.7

item 908 =kde-base/ktux-3.5.7

item 909 =kde-base/kbruch-3.5.7

item 910 =kde-base/eyesapplet-3.5.7

item 911 =kde-base/blinken-3.5.7

item 912 =kde-base/kcalc-3.5.7

item 913 =kde-base/secpolicy-3.5.6

item 914 =kde-base/lilo-config-3.5.7

item 915 =kde-base/kuser-3.5.7

item 916 =kde-base/ksim-3.5.7

item 917 =kde-base/kmilo-3.5.7

item 918 =kde-base/kwalletmanager-3.5.7

item 919 =kde-base/kdf-3.5.7

item 920 =kde-base/kcharselect-3.5.7

item 921 =kde-base/ark-3.5.7

item 922 =kde-base/dcopperl-3.5.6-r1

item 923 =kde-base/ksig-3.5.7

item 924 =kde-base/ksystraycmd-3.5.5

item 925 =kde-base/kstart-3.5.6

item 926 =kde-base/fifteenapplet-3.5.7

item 927 =kde-base/knetattach-3.5.7

item 928 =kde-base/kteatime-3.5.7

item 929 =kde-base/kiconedit-3.5.7

item 930 =kde-base/amor-3.5.7

item 931 =kde-base/kmoon-3.5.7

item 932 =kde-base/nsplugins-3.5.7

item 933 =kde-base/klipper-3.5.7

item 934 =kde-base/keduca-3.5.7

item 935 =kde-base/kdeedu-applnk-3.5.7

item 936 =kde-base/kalyptus-3.5.4

item 937 =kde-base/kiten-3.5.7

item 938 =kde-base/kturtle-3.5.7

item 939 =kde-base/kdcop-3.5.7

item 940 =kde-base/kpager-3.5.7

item 941 =kde-base/kmplot-3.5.7

item 942 =kde-base/kpercentage-3.5.7

item 943 =kde-base/kanagram-3.5.7

item 944 =kde-base/renamedlg-images-3.5.7

item 945 =kde-base/kfilereplace-3.5.7

item 946 =kde-base/kpackage-3.5.7

item 947 =kde-misc/kleds-0.8.0

item 948 =kde-misc/kexchange-1.0

item 949 =app-admin/klogview-0.6

item 950 =games-board/hearts-1.98

item 951 =kde-misc/katapult-0.3.2.1

item 952 =games-sports/kbilliards-0.8.7b

item 953 =kde-misc/kkeyled-0.8.11

item 954 =kde-base/artsplugin-mpg123-3.5.6

item 955 =kde-misc/kvdrmon-0.6-r1

item 956 =kde-misc/knetstats-1.6.1

item 957 =kde-misc/kkbswitch-1.4.3

item 958 =games-engines/kwest-1.0

item 959 =kde-misc/ksystemlog-0.3.2

item 960 =kde-misc/knetdockapp-0.67.3

item 961 =kde-misc/kooldock-0.4.6

item 962 =games-board/kwappen-1.1.5

item 963 =games-arcade/kamikaze-0.2.2

item 964 =kde-misc/kalbum-0.8.0

item 965 =games-puzzle/easysok-0.3.5

item 966 =kde-misc/mtaskbar-0.7

item 967 =kde-misc/dolphin-0.8.2-r1

item 968 =games-puzzle/quadros-0.1

item 969 =kde-misc/ksensors-0.7.3-r1

item 970 =kde-misc/krename-3.0.14

item 971 =kde-misc/metamonitor-0.4.5

item 972 =kde-misc/ksmoothdock-4.5

item 973 =kde-base/libkcal-3.5.7

item 974 =kde-base/kdepim-kioslaves-3.5.7

item 975 =kde-base/knewsticker-3.5.7

item 976 =kde-base/kfind-3.5.7

item 977 =kde-base/kdepasswd-3.5.7

item 978 =kde-base/atlantik-3.5.7

item 979 =kde-base/kalzium-3.5.7

item 980 =kde-base/kbounce-3.5.7

item 981 =kde-base/kbattleship-3.5.7

item 982 =kde-base/klatin-3.5.7

item 983 =kde-base/kbackgammon-3.5.7

item 984 =kde-base/kjumpingcube-3.5.7

item 985 =kde-base/kpat-3.5.7

item 986 =kde-base/ktron-3.5.7

item 987 =kde-base/ksnake-3.5.7

item 988 =kde-base/kmahjongg-3.5.7

item 989 =kde-base/khangman-3.5.7

item 990 =kde-base/kscd-3.5.7

item 991 =kde-base/ksmiletris-3.5.7

item 992 =kde-base/dcoprss-3.5.7

item 993 =kde-base/kasteroids-3.5.7

item 994 =kde-base/kate-plugins-3.5.7

item 995 =kde-base/ktuberling-3.5.7

item 996 =kde-base/ksame-3.5.7

item 997 =kde-base/kshisen-3.5.7

item 998 =kde-base/kmines-3.5.7

item 999 =kde-base/kblackbox-3.5.7

item 1000 =kde-base/kenolaba-3.5.7

item 1001 =kde-base/kfouleggs-3.5.7

item 1002 =kde-base/kstars-3.5.7

item 1003 =kde-base/kpoker-3.5.7

item 1004 =kde-base/kreversi-3.5.7

item 1005 =kde-base/kspaceduel-3.5.7

item 1006 =kde-base/kolf-3.5.7

item 1007 =kde-base/ksokoban-3.5.7

item 1008 =kde-base/konquest-3.5.7

item 1009 =kde-base/klines-3.5.7

item 1010 =kde-base/kwin4-3.5.7

item 1011 =kde-base/lskat-3.5.7

item 1012 =kde-base/ksirtet-3.5.7

item 1013 =kde-base/ktouch-3.5.7

item 1014 =kde-base/kgoldrunner-3.5.7

item 1015 =kde-base/klickety-3.5.7

item 1016 =kde-base/katomic-3.5.7

item 1017 =kde-misc/kate-symbolviewer-plugin-1.10.0

item 1018 =kde-misc/ksplash-engine-moodin-0.4.2

item 1019 =kde-misc/kxdocker-trayiconlogger-1.0.0-r1

item 1020 =kde-misc/kxdocker-i18n-1.0.2-r1

item 1021 =kde-misc/kxdocker-resources-1.1.0

item 1022 =kde-misc/kxdocker-configurator-1.0.2

item 1023 =kde-misc/kxdocker-dcop-1.0.0-r1

item 1024 =kde-misc/kgraphspace-0.3.0_pre1

item 1025 =kde-base/libkdepim-3.5.7

item 1026 =kde-base/libkpimexchange-3.5.7

item 1027 =kde-base/atlantikdesigner-3.5.7

item 1028 =kde-base/knewsticker-scripts-3.5.7

item 1029 =kde-base/ksync-3.5.6

item 1030 =kde-base/konsolekalendar-3.5.7

item 1031 =kde-base/kandy-3.5.7

item 1032 =kde-base/kdemultimedia-arts-3.5.7

item 1033 =kde-base/noatun-3.5.7

item 1034 =kde-base/kaboodle-3.5.7

item 1035 =kde-base/konqueror-3.5.7

item 1036 =kde-base/konq-plugins-3.5.7

item 1037 =kde-base/kghostview-3.5.7

item 1038 =kde-base/kviewshell-3.5.7

item 1039 =kde-base/kfax-3.5.7

item 1040 =kde-base/khelpcenter-3.5.7

item 1041 =kde-misc/filelight-1.0

item 1042 =kde-misc/filelight-i18n-1.0

item 1043 =kde-base/kappfinder-3.5.7

item 1044 =kde-base/korn-3.5.7

item 1045 =kde-base/kppp-3.5.7

item 1046 =kde-base/ktalkd-3.5.7

item 1047 =kde-base/drkonqi-3.5.7

item 1048 =kde-base/cervisia-3.5.7

item 1049 =kde-base/kcron-3.5.7

item 1050 =kde-base/kgpg-3.5.7

item 1051 =kde-base/kdebase-startkde-3.5.7

item 1052 =kde-base/kxkb-3.5.7

item 1053 =kde-base/konqueror-akregator-3.5.7

item 1054 =games-board/knights-0.6

item 1055 =app-portage/kuroo-0.80.2-r1

item 1056 =app-office/oooqs-2.1.0.1

item 1057 =kde-misc/kshutdown-1.0

item 1058 =games-emulation/dboxfe-0.0.5

item 1059 =media-libs/speex-1.1.12

item 1060 =media-libs/a52dec-0.7.4-r5

item 1061 =media-libs/faad2-2.0-r13

item 1062 =dev-scheme/guile-1.8.1-r3

item 1063 =media-libs/faac-1.25

item 1064 =media-libs/xvid-1.1.3

item 1065 =media-libs/libdvdcss-1.2.9-r1

item 1066 =media-libs/libdts-0.0.2-r5

item 1067 =dev-libs/glib-1.2.10-r5

item 1068 =net-libs/libpcap-0.9.6

item 1069 =sci-libs/fftw-3.1.2

item 1070 =www-misc/htdig-3.2.0_beta6-r2

item 1071 =net-libs/libnet-1.1.2.1-r1

item 1072 =media-libs/libmodplug-0.8.4-r2

item 1073 =dev-util/valgrind-3.2.3

item 1074 =dev-libs/apr-1.2.8

item 1075 =x11-misc/read-edid-1.4.1-r1

item 1076 =dev-libs/lzo-1.08-r1

item 1077 =x11-themes/wm-icons-0.4.0

item 1078 =media-libs/libdvdread-0.9.7

item 1079 =dev-libs/libksba-1.0.2

item 1080 =dev-libs/libmix-2.05

item 1081 =dev-scheme/slib-3.1.1-r1

item 1082 =media-sound/normalize-0.7.7

item 1083 =dev-util/xdelta-1.1.3-r2

item 1084 =dev-tcltk/tclx-8.4-r1

item 1085 =net-analyzer/netcat-110-r8

item 1086 =dev-db/sqlitebrowser-1.3

item 1087 =sys-apps/hotplug-20040923-r2

item 1088 =media-libs/freealut-1.1.0

item 1089 =media-libs/libao-0.8.6-r3

item 1090 =media-sound/oggtst-0.0

item 1091 =dev-games/KXL-1.1.7-r1

item 1092 =app-misc/lirc-0.8.2

item 1093 =games-arcade/xbubble-0.5.8

item 1094 =games-puzzle/hoh-bin-1.01

item 1095 =games-action/xbomber-101

item 1096 =games-board/xmahjongg-3.7

item 1097 =x11-misc/xbindkeys-1.8.0

item 1098 =media-gfx/xloadimage-4.1-r5

item 1099 =dev-perl/perl-tk-804.027

item 1100 =games-action/xblast-2.10.2

item 1101 =sys-fs/mtools-3.9.10

item 1102 =games-puzzle/xtris-1.15

item 1103 =x11-apps/ttmkfdir-3.0.9-r3

item 1104 =games-action/geki2-KXL-2.0.3-r1

item 1105 =games-arcade/grande-KXL-0.6

item 1106 =games-action/geki3-KXL-1.0.3-r1

item 1107 =net-misc/rdesktop-1.5.0-r2

item 1108 =games-puzzle/xblockout-1.1.5

item 1109 =games-board/xfreecell-1.0.5b

item 1110 =games-simulation/lincity-1.12.1

item 1111 =x11-libs/gtk+-1.2.10-r12

item 1112 =media-libs/imlib-1.9.15-r1

item 1113 =media-libs/libjsw-1.5.6

item 1114 =games-arcade/xbill-2.1-r1

item 1115 =games-simulation/corewars-0.9.13-r1

item 1116 =games-board/mah-jong-1.7

item 1117 =games-puzzle/glickomania-1.0

item 1118 =games-puzzle/codebreaker-1.2.1-r1

item 1119 =games-mud/gMOO-0.4.8-r1

item 1120 =games-kids/gtans-1.2

item 1121 =kde-base/kuickshow-3.5.7

item 1122 =games-kids/stickers-0.1.3-r2

item 1123 =games-kids/lletters-0.1.95-r1

item 1124 =games-puzzle/xphotohunter-1.4-r1

item 1125 =games-action/nighthawk-2.2

item 1126 =games-action/0verkill-0.16-r3

item 1127 =games-puzzle/quadra-1.1.8

item 1128 =games-arcade/xgalaga-2.0.34-r6

item 1129 =games-misc/xpenguins-2.2-r1

item 1130 =games-misc/gnurobots-1.0d

item 1131 =games-puzzle/construo-0.2.2

item 1132 =kde-base/krdc-3.5.7

item 1133 =kde-base/krfb-3.5.7

item 1134 =x11-misc/numlockx-1.1

item 1135 =x11-misc/imwheel-1.0.0_pre12

item 1136 =media-libs/libdc1394-1.2.1

item 1137 =media-libs/libdc1394-2.0.0_rc3

item 1138 =x11-libs/libXxf86dga-1.0.1

item 1139 =media-libs/libgii-1.0.2

item 1140 =x11-libs/libXScrnSaver-1.1.2

item 1141 =kde-base/klaptopdaemon-3.5.7

item 1142 =kde-misc/rsibreak-0.8.0

item 1143 =x11-libs/libXcomposite-0.3.2

item 1144 =kde-base/kicker-3.5.7

item 1145 =kde-base/kicker-applets-3.5.7

item 1146 =x11-libs/libXdamage-1.1.1

item 1147 =kde-base/kwin-3.5.7

item 1148 =kde-base/kdeartwork-kwin-styles-3.5.7

item 1149 =kde-base/kjsembed-3.5.7

item 1150 =x11-themes/crystal-1.0.2

item 1151 =x11-libs/libXinerama-1.0.2

item 1152 =x11-apps/xdpyinfo-1.0.2

item 1153 =x11-apps/xhost-1.0.1

item 1154 =x11-libs/Xaw3d-1.5-r1

item 1155 =games-arcade/xscavenger-1.4.4

item 1156 =media-gfx/xli-1.17.0-r3

item 1157 =games-strategy/xbattle-5.4.1

item 1158 =games-board/xskat-4.0

item 1159 =x11-misc/xmountains-2.7

item 1160 =games-board/xmille-2.0-r1

item 1161 =x11-misc/xsnow-1.42

item 1162 =games-action/xpilot-4.5.4

item 1163 =dev-games/ode-0.8

item 1164 =kde-base/kscreensaver-3.5.7

item 1165 =media-libs/glew-1.3.5

item 1166 =x11-libs/xforms-1.0.90-r1

item 1167 =media-libs/ftgl-2.1.2-r1

item 1168 =x11-apps/mesa-progs-6.5.2

item 1169 =media-libs/gle-3.1.0-r1

item 1170 =games-action/poopmup-1.2

item 1171 =games-action/transcend-0.3

item 1172 =games-simulation/cultivation-7

item 1173 =games-sports/billardgl-1.75-r1

item 1174 =media-libs/libprojectm-0.99-r1

item 1175 =x11-apps/xmodmap-1.0.2

item 1176 =x11-proto/dmxproto-2.2.2

item 1177 =x11-apps/xrandr-1.2.1

item 1178 =x11-apps/setxkbmap-1.0.2

item 1179 =sys-apps/dmapi-2.2.8

item 1180 =x11-apps/xlsclients-1.0.1

item 1181 =x11-apps/xvinfo-1.0.1

item 1182 =x11-libs/libdmx-1.0.2

item 1183 =x11-libs/libXevie-1.0.2

item 1184 =x11-apps/sessreg-1.0.2

item 1185 =x11-libs/libFS-1.0.0

item 1186 =app-doc/xorg-docs-1.4

item 1187 =x11-apps/xwininfo-1.0.2

item 1188 =x11-apps/appres-1.0.1

item 1189 =x11-libs/libXfontcache-1.0.4

item 1190 =x11-apps/xcursorgen-1.0.1

item 1191 =kde-base/kcachegrind-3.5.7

item 1192 =x11-apps/xset-1.0.2

item 1193 =x11-apps/xsetroot-1.0.1

item 1194 =kde-base/konsole-3.5.7

item 1195 =x11-apps/xlsfonts-1.0.2

item 1196 =x11-apps/xfs-1.0.4

item 1197 =app-misc/gentoo-0.11.56

item 1198 =games-board/scid-3.6.1-r1

item 1199 =x11-apps/xev-1.0.2

item 1200 =games-roguelike/tome-2.3.4

item 1201 =x11-themes/xcursor-themes-1.0.1

item 1202 =kde-misc/krecipes-1.0_beta1

item 1203 =x11-themes/gtk-engines-2.10.2

item 1204 =x11-misc/beryl-manager-0.2.1

item 1205 =games-misc/gtklife-5.1

item 1206 =kde-misc/kgtk-0.8

item 1207 =games-board/eboard-1.0.4

item 1208 =sys-apps/lshw-02.10b

item 1209 =games-puzzle/gtkballs-3.1.5-r1

item 1210 =net-www/nspluginwrapper-0.9.91.4

item 1211 =games-misc/gBhed-0.17

item 1212 =games-sports/bygfoot-2.2.0

item 1213 =games-puzzle/groundhog-1.4

item 1214 =x11-misc/e16keyedit-0.5

item 1215 =games-puzzle/arrows-0.6

item 1216 =media-libs/sdl-image-1.2.5-r1

item 1217 =media-libs/sdl-ttf-2.0.8

item 1218 =media-libs/sdl-net-1.2.6

item 1219 =media-libs/smpeg-0.4.4-r9

item 1220 =media-libs/plib-1.8.4-r1

item 1221 =media-libs/sdl-gfx-2.0.16

item 1222 =media-libs/libdv-1.0.0-r2

item 1223 =media-libs/libmpeg2-0.4.1

item 1224 =games-rpg/nwn-data-1.29-r1

item 1225 =games-simulation/pmars-sdl-0.9.2e

item 1226 =games-rpg/egoboo-2.22-r1

item 1227 =games-roguelike/wrogue-0.7.7b

item 1228 =games-emulation/fbzx-1.8

item 1229 =games-action/snipes-1.0.1

item 1230 =games-arcade/pengupop-2.2.2

item 1231 =games-arcade/tuxpuck-0.8.2-r1

item 1232 =games-arcade/cob-0.9

item 1233 =x11-misc/fireflies-2.07

item 1234 =games-action/deathchase3d-0.9

item 1235 =games-arcade/yarsrevenge-0.99

item 1236 =games-sports/foosball-0.92-r1

item 1237 =games-puzzle/lpairs-1.0.1

item 1238 =games-rpg/aklabeth-1.0

item 1239 =games-arcade/trailblazer-0.9

item 1240 =games-arcade/lbreakout-010315

item 1241 =games-arcade/digger-20020314

item 1242 =games-fps/alienarena-20070613

item 1243 =games-arcade/balloonchase-0.9.6

item 1244 =games-arcade/betna-0.9.7

item 1245 =games-util/joystick-20060731

item 1246 =games-board/atakks-1.0

item 1247 =dev-games/guichan-0.6.1

item 1248 =media-libs/paragui-1.1.8

item 1249 =games-puzzle/mures-0.5

item 1250 =games-action/supertuxkart-0.2

item 1251 =games-rpg/gwiz-0.8

item 1252 =games-action/maelstrom-3.0.6-r1

item 1253 =games-arcade/snake3d-0.9

item 1254 =games-arcade/late-0.1.0

item 1255 =games-arcade/kobodeluxe-0.4_pre10

item 1256 =games-action/tuxkart-0.4.0

item 1257 =games-simulation/senken-0.3.0

item 1258 =games-sports/gracer-0.1.5

item 1259 =games-arcade/gav-0.9.0

item 1260 =games-arcade/openbubbles-1.2

item 1261 =games-arcade/skystreets-0.2.4

item 1262 =games-arcade/tuxdash-0.8

item 1263 =games-rpg/tux_aqfh-1.0.14

item 1264 =games-fps/darkplaces-20060616_beta1

item 1265 =dev-libs/liboil-0.3.12

item 1266 =dev-perl/glib-perl-1.144

item 1267 =xfce-extra/xarchiver-0.4.6

item 1268 =dev-libs/g-wrap-1.9.6-r3

item 1269 =app-admin/syslog-ng-2.0.4

item 1270 =sys-apps/irqbalance-0.55

item 1271 =dev-util/scons-0.97

item 1272 =dev-libs/boost-1.34.0

item 1273 =net-zope/zopeinterface-3.0.1-r1

item 1274 =dev-python/sip-4.6

item 1275 =dev-python/pyxml-0.8.4-r1

item 1276 =dev-util/xxdiff-3.2-r1

item 1277 =dev-python/pysqlite-2.3.4-r1

item 1278 =dev-python/egenix-mx-base-2.0.6

item 1279 =dev-python/dnspython-1.5.0

item 1280 =dev-libs/zziplib-0.13.49

item 1281 =app-portage/gentoolkit-0.2.4_pre5

item 1282 =app-admin/webapp-config-1.50.16

item 1283 =kde-base/dcoppython-3.5.7

item 1284 =games-board/pysol-sound-server-3.01

item 1285 =games-sports/ski-6.5

item 1286 =dev-java/blackdown-jdk-1.4.2.03-r16

item 1287 =dev-python/PyQt-3.17.2

item 1288 =dev-python/pyode-1.2.0

item 1289 =app-emulation/emul-linux-x86-java-1.6.0.01

item 1290 =app-emulation/emul-linux-x86-java-1.5.0.12

item 1291 =games-board/pouetchess-0.2.0-r1

item 1292 =games-strategy/glob2-0.8.23

item 1293 =dev-java/blackdown-jre-1.4.2.03-r14

item 1294 =kde-base/kdebase-pam-7

item 1295 =net-dialup/ppp-2.4.4-r8

item 1296 =kde-base/kcheckpass-3.5.6

item 1297 =sys-apps/slocate-3.1-r1

item 1298 =kde-misc/kio-locate-0.4.5

item 1299 =dev-java/ant-core-1.7.0

item 1300 =kde-base/qtjava-3.5.7

item 1301 =dev-java/antlr-2.7.7

item 1302 =dev-java/higlayout-1.0-r1

item 1303 =dev-java/gnu-crypto-2.0.1-r2

item 1304 =dev-java/sun-jms-1.1-r2

item 1305 =sys-libs/db-4.3.29-r2

item 1306 =sys-libs/db-4.2.52_p4-r2

item 1307 =dev-java/junit-3.8.2-r1

item 1308 =dev-java/xml-commons-external-1.3.04

item 1309 =dev-java/commons-logging-1.1-r2

item 1310 =dev-java/servletapi-2.4-r5

item 1311 =dev-java/log4j-1.2.14-r1

item 1312 =dev-java/ant-nodeps-1.7.0

item 1313 =dev-java/relaxng-datatype-1.0-r1

item 1314 =perl-core/DB_File-1.815

item 1315 =dev-java/xml-commons-resolver-1.2

item 1316 =dev-java/jakarta-oro-2.0.8-r2

item 1317 =sys-apps/iproute2-2.6.20.20070313

item 1318 =dev-java/commons-collections-3.2-r1

item 1319 =dev-java/gjdoc-0.7.8

item 1320 =dev-java/xjavac-20041208-r5

item 1321 =dev-java/xpp3-1.1.4c

item 1322 =dev-java/icu4j-3.0-r1

item 1323 =dev-java/jakarta-regexp-1.3-r4

item 1324 =dev-java/bcel-5.2

item 1325 =dev-java/javacup-0.11a_beta20060608

item 1326 =dev-java/bcprov-1.36-r1

item 1327 =dev-java/servletapi-2.3-r3

item 1328 =dev-java/iso-relax-20050331-r1

item 1329 =dev-java/sun-jaf-1.1

item 1330 =kde-base/kbabel-3.5.7

item 1331 =dev-java/commons-lang-2.0-r2

item 1332 =kde-base/noatun-plugins-3.5.7

item 1333 =dev-java/jzlib-1.0.7-r1

item 1334 =dev-java/swing-layout-1.0.1-r1

item 1335 =dev-java/jgoodies-looks-2.0.4-r2

item 1336 =app-portage/portagemaster-0.2.1-r1

item 1337 =dev-java/browserlauncher2-1.0

item 1338 =dev-perl/BerkeleyDB-0.31

item 1339 =dev-java/commons-lang-2.1-r1

item 1340 =games-board/jrisk-1.0.9.0

item 1341 =dev-java/jakarta-regexp-1.4-r1

item 1342 =dev-java/eclipse-ecj-3.2.2

item 1343 =dev-java/rhino-1.5.5-r4

item 1344 =dev-java/jdepend-2.9-r4

item 1345 =gnome-base/libgtop-2.14.9

item 1346 =net-dns/libidn-0.6.9-r1

item 1347 =dev-java/xalan-serializer-2.7.0

item 1348 =dev-java/commons-cli-1.0-r5

item 1349 =dev-java/ant-junit-1.7.0

item 1350 =dev-java/sun-javamail-1.4

item 1351 =dev-java/javacc-4.0-r4

item 1352 =dev-java/javahelp-2.0.02_p46

item 1353 =dev-java/commons-net-1.4.1-r1

item 1354 =dev-db/hsqldb-1.7.3.1-r3

item 1355 =games-strategy/freecol-0.6.1

item 1356 =dev-java/commons-beanutils-1.7.0-r2

item 1357 =dev-java/commons-beanutils-1.6.1-r3

item 1358 =dev-java/jsch-0.1.33

item 1359 =dev-java/xerces-2.9.0

item 1360 =media-gfx/splashutils-1.4.3

item 1361 =net-misc/whois-4.7.21

item 1362 =media-libs/devil-1.6.7-r1

item 1363 =dev-java/lucene-2.1.0

item 1364 =app-benchmarks/bootchart-0.9-r1

item 1365 =dev-java/xalan-2.7.0-r4

item 1366 =dev-java/xsdlib-20050627-r1

item 1367 =dev-java/xpp2-2.1.10-r1

item 1368 =media-gfx/splash-themes-gentoo-20050429

item 1369 =media-gfx/splash-themes-livecd-2007.0

item 1370 =media-gfx/splash-themes-livecd-2006.1

item 1371 =dev-java/ant-trax-1.7.0

item 1372 =dev-java/xmldb-20011111-r1

item 1373 =dev-java/msv-20050627-r1

item 1374 =dev-java/jaxme-0.3.1-r4

item 1375 =dev-java/saxpath-1.0-r2

item 1376 =dev-java/tagsoup-1.1

item 1377 =dev-java/jsr173-1.0-r1

item 1378 =dev-java/jdom-1.0-r2

item 1379 =dev-java/xom-1.0-r2

item 1380 =dev-java/dom4j-1.6.1-r2

item 1381 =dev-java/jaxen-1.1.1

item 1382 =dev-java/xml-xmlbeans-1.0.4_pre20041217

item 1383 =dev-java/rhino-1.6.5

item 1384 =app-arch/rpm2targz-9.0-r6

item 1385 =media-video/realplayer-10.0.8-r2

item 1386 =app-cdr/nero-3.0.0.0

item 1387 =dev-java/sun-jdk-1.5.0.12

item 1388 =dev-java/avalon-logkit-1.2-r2

item 1389 =dev-java/avalon-logkit-2.1-r1

item 1390 =dev-java/commons-codec-1.3-r1

item 1391 =sys-process/vixie-cron-4.1-r10

item 1392 =games-arcade/bub-n-bros-1.5

item 1393 =dev-java/sun-jre-bin-1.6.0.01-r1

item 1394 =x11-themes/ximian-artwork-0.2.32.1

item 1395 =media-libs/sdl-mixer-1.2.7

item 1396 =dev-python/pygame-1.7.1

item 1397 =dev-games/flatzebra-0.1.1

item 1398 =dev-perl/sdl-perl-2.1.3-r3

item 1399 =games-action/powermanga-0.80

item 1400 =games-arcade/rocksndiamonds-3.2.3

item 1401 =games-arcade/penguin-command-1.6.11

item 1402 =games-arcade/cdogs-sdl-0.3

item 1403 =games-arcade/missile-1.0.1

item 1404 =games-arcade/blobwars-1.07

item 1405 =games-arcade/project-starfighter-1.1-r1

item 1406 =games-kids/tuxtype-1.0.3-r1

item 1407 =games-arcade/komi-1.04

item 1408 =games-arcade/criticalmass-1.0.0

item 1409 =games-arcade/viruskiller-1.0

item 1410 =games-arcade/rockdodger-0.6.0a-r1

item 1411 =games-arcade/defendguin-0.0.11

item 1412 =games-arcade/sdljump-1.0.0

item 1413 =games-puzzle/lmarbles-1.0.7

item 1414 =games-arcade/holotz-castle-1.3.8

item 1415 =games-simulation/cannonsmash-0.6.6

item 1416 =games-puzzle/shaaft-0.5.0

item 1417 =games-action/blobandconquer-0.91

item 1418 =games-puzzle/concentration-1.2

item 1419 =games-puzzle/neverball-1.4.0

item 1420 =games-arcade/conveysdl-1.3

item 1421 =games-arcade/wop-0.4.3-r1

item 1422 =games-misc/OilWar-1.2.1-r1

item 1423 =games-puzzle/mirrormagic-2.0.2

item 1424 =games-puzzle/icebreaker-1.9.5

item 1425 =games-puzzle/einstein-2.0

item 1426 =games-kids/tuxtype2-1.5.3-r1

item 1427 =games-strategy/netpanzer-0.8.1

item 1428 =games-arcade/briquolo-0.5.6

item 1429 =games-arcade/tomatoes-1.55-r2

item 1430 =games-arcade/bomns-0.99.1

item 1431 =games-strategy/asc-1.16.4.0

item 1432 =games-rpg/freedroid-1.0.2

item 1433 =games-arcade/jumpnbump-1.50-r1

item 1434 =games-sports/race-0.5

item 1435 =games-arcade/pachi-1.0

item 1436 =games-arcade/emilia-pinball-0.3.1

item 1437 =games-arcade/abe-1.1

item 1438 =games-rpg/freedroidrpg-0.10.2

item 1439 =games-puzzle/tong-1.0

item 1440 =games-arcade/diameter-0.4.0.1

item 1441 =games-arcade/pacmanarena-0.15

item 1442 =games-simulation/gl117-1.3.2

item 1443 =games-arcade/openmortal-0.7

item 1444 =games-arcade/primateplunge-1.1-r1

item 1445 =games-roguelike/scourge-0.18

item 1446 =games-puzzle/gemdropx-0.9-r1

item 1447 =games-action/rrootage-0.23a

item 1448 =games-arcade/supertux-0.1.3

item 1449 =games-puzzle/twindistress-1.1.0

item 1450 =games-action/barrage-1.0.2-r1

item 1451 =games-arcade/crack-attack-1.1.14-r1

item 1452 =games-arcade/bumprace-1.5.3

item 1453 =games-arcade/ceferino-0.97.5

item 1454 =games-kids/tuxmath-20010907

item 1455 =games-action/formido-1.0.1

item 1456 =games-arcade/sable-1.0

item 1457 =games-arcade/sdlroids-1.3.4-r3

item 1458 =games-puzzle/flobopuyo-0.20-r1

item 1459 =games-strategy/widelands-0.0.10

item 1460 =games-arcade/ri-li-2.0.0

item 1461 =games-strategy/freelords-0.3.8

item 1462 =games-puzzle/ltris-1.0.11

item 1463 =games-arcade/circuslinux-1.0.3

item 1464 =games-arcade/sdb-1.0.2

item 1465 =games-engines/exult-1.2

item 1466 =games-puzzle/hangman-0.9.2

item 1467 =games-arcade/insaneodyssey-000311

item 1468 =games-action/bomberclone-0.11.7

item 1469 =games-puzzle/anagramarama-0.2

item 1470 =games-simulation/dangerdeep-0.3.0

item 1471 =games-puzzle/torrent-0.8.2

item 1472 =games-arcade/lbreakout2-2.5.2

item 1473 =games-strategy/galaxyhack-1.69

item 1474 =games-puzzle/toppler-1.1.2a

item 1475 =games-strategy/lgeneral-1.2_beta12-r1

item 1476 =games-action/trackballs-1.1.2

item 1477 =games-strategy/wesnoth-1.2.5

item 1478 =games-rpg/openglad-0.98

item 1479 =games-action/glaxium-0.5

item 1480 =games-puzzle/scramble-0.9.5

item 1481 =games-action/luola-1.3.0-r1

item 1482 =games-arcade/pydance-1.0.3

item 1483 =games-kids/childsplay-0.85.1

item 1484 =games-puzzle/angrydd-1.0.1

item 1485 =games-simulation/singularity-0.25

item 1486 =games-puzzle/monsterz-0.7.0

item 1487 =games-arcade/afternoonstalker-1.1.1

item 1488 =games-arcade/burgerspace-1.8.1

item 1489 =games-action/accelerator3d-0.1.1

item 1490 =games-kids/pytraffic-2.5.4

item 1491 =media-libs/gstreamer-0.10.12

item 1492 =media-libs/netpbm-10.39.0

item 1493 =media-video/vcdimager-0.7.23

item 1494 =media-libs/libsvg-0.1.4

item 1495 =media-gfx/fontforge-20070607

item 1496 =games-rpg/xu4-0.9

item 1497 =games-puzzle/krystaldrop-0.7.2

item 1498 =games-simulation/lincity-ng-1.1.0

item 1499 =games-board/gtkatlantic-0.4.2

item 1500 =media-libs/gst-plugins-base-0.10.12

item 1501 =dev-perl/File-MimeInfo-0.14

item 1502 =kde-base/quanta-3.5.7

item 1503 =media-gfx/tuxpaint-0.9.16-r1

item 1504 =kde-base/kxsldbg-3.5.7

item 1505 =games-strategy/crimson-0.5.1

item 1506 =app-text/asciidoc-8.1.0

item 1507 =games-puzzle/pathological-1.1.3-r1

item 1508 =media-plugins/gst-plugins-ogg-0.10.12

item 1509 =media-plugins/gst-plugins-mad-0.10.6

item 1510 =media-plugins/gst-plugins-vorbis-0.10.12

item 1511 =media-libs/gst-plugins-good-0.10.5

item 1512 =media-plugins/gst-plugins-xvideo-0.10.12

item 1513 =dev-perl/XML-Filter-BufferText-1.01

item 1514 =media-libs/gst-plugins-ugly-0.10.6

item 1515 =media-plugins/gst-plugins-dvdread-0.10.6

item 1516 =media-plugins/gst-plugins-ffmpeg-0.10.2

item 1517 =media-plugins/gst-plugins-a52dec-0.10.6

item 1518 =media-plugins/gst-plugins-mpeg2dec-0.10.6

item 1519 =media-plugins/gst-plugins-oss-0.10.5

item 1520 =media-plugins/gst-plugins-x-0.10.12

item 1521 =media-plugins/gst-plugins-faad-0.10.4

item 1522 =dev-perl/XML-SAX-Writer-0.50

item 1523 =dev-perl/perl-ldap-0.34

item 1524 =x11-themes/gnome-themes-2.18.1

item 1525 =kde-base/kopete-3.5.7

item 1526 =kde-base/juk-3.5.7

item 1527 =media-plugins/gst-plugins-alsa-0.10.12

item 1528 =media-sound/lame-3.97

item 1529 =app-crypt/pinentry-0.7.3

item 1530 =dev-java/libreadline-java-0.8.0-r2

item 1531 =sys-devel/gdb-6.6-r2

item 1532 =sys-libs/slang-1.4.9-r2

item 1533 =net-misc/netkit-talk-0.17-r4

item 1534 =sys-boot/grub-0.97-r3

item 1535 =games-misc/bsd-games-non-free-2.17

item 1536 =games-misc/robotfindskitten-1.4142135.349

item 1537 =games-arcade/ascii-invaders-0.1b

item 1538 =games-misc/bsd-games-2.17-r3

item 1539 =games-arcade/aop-0.6

item 1540 =games-arcade/alienwave-0.3.0

item 1541 =games-arcade/ninvaders-0.1.1

item 1542 =app-shells/tcsh-6.15-r1

item 1543 =app-misc/screen-4.0.3

item 1544 =games-emulation/gcube-0.4

item 1545 =games-puzzle/tint-0.03a

item 1546 =games-puzzle/bastet-0.41

item 1547 =games-misc/typespeed-0.6.2

item 1548 =dev-lang/ocaml-3.09.3-r1

item 1549 =games-roguelike/adom-1.1.1-r1

item 1550 =app-portage/ufed-0.40-r6

item 1551 =games-puzzle/braincurses-0.5b

item 1552 =games-arcade/netris-0.52

item 1553 =games-strategy/tornado-1.3

item 1554 =games-roguelike/moria-5.5.2

item 1555 =dev-java/bsh-2.0_beta4-r3

item 1556 =app-arch/sharutils-4.6.3

item 1557 =kde-base/krec-3.5.7

item 1558 =dev-java/jython-2.1-r11

item 1559 =games-misc/wumpus-1.4

item 1560 =dev-libs/newt-0.52.2

item 1561 =app-shells/bash-completion-config-0.8-r2

item 1562 =dev-lang/ruby-1.8.6_p36-r3

item 1563 =media-sound/cdparanoia-3.9.8-r5

item 1564 =dev-lang/lua-5.1.2-r1

item 1565 =sys-fs/xfsprogs-2.8.21

item 1566 =app-editors/vim-core-7.1.002

item 1567 =games-board/gnuchess-5.07

item 1568 =app-text/hunspell-1.1.4-r2

item 1569 =sys-devel/bc-1.06.95

item 1570 =media-libs/libggi-2.2.2

item 1571 =net-mail/metamail-2.7.53.3

item 1572 =sys-fs/reiser4progs-1.0.6

item 1573 =sys-apps/parted-1.8.6

item 1574 =x11-misc/xaos-3.2

item 1575 =app-portage/gentoolkit-dev-0.2.6.6

item 1576 =games-arcade/kajaani-kombat-0.7

item 1577 =games-board/gnugo-3.7.10

item 1578 =dev-java/bsf-2.4.0-r1

item 1579 =sys-kernel/genkernel-3.4.8

item 1580 =dev-ruby/ruby-glib2-0.16.0

item 1581 =app-text/enchant-1.2.5

item 1582 =app-text/docbook-xml-dtd-4.3-r1

item 1583 =app-editors/vim-7.1.002

item 1584 =x11-apps/xmessage-1.0.1

item 1585 =app-text/docbook-sgml-dtd-4.4

item 1586 =kde-base/qtruby-3.5.7

item 1587 =media-plugins/gst-plugins-cdparanoia-0.10.12

item 1588 =sys-apps/apmd-3.2.2_p5

item 1589 =dev-util/cmake-2.4.6-r1

item 1590 =sys-fs/xfsdump-2.2.45

item 1591 =kde-base/mpeglib-3.5.6

item 1592 =dev-ruby/rubygems-0.9.4

item 1593 =games-board/gnushogi-1.3

item 1594 =games-misc/xcruiser-0.30

item 1595 =games-arcade/xjump-2.7.5

item 1596 =games-board/xfrisk-1.2

item 1597 =games-roguelike/angband-3.0.6

item 1598 =games-board/gnuchess-book-1.01

item 1599 =games-board/xboard-4.2.7-r1

item 1600 =games-puzzle/xbomb-2.1-r1

item 1601 =games-puzzle/xwelltris-1.0.1

item 1602 =kde-misc/tellico-1.2.11

item 1603 =app-text/docbook-sgml-dtd-4.2-r2

item 1604 =games-board/xgammon-0.98

item 1605 =games-sports/foobillard-3.0a

item 1606 =x11-apps/xfontsel-1.0.2

item 1607 =games-action/koth-0.8.0

item 1608 =games-puzzle/fish-fillets-0.7.4

item 1609 =dev-ruby/ruby-gdkpixbuf2-0.16.0

item 1610 =dev-ruby/ruby-libart2-0.16.0

item 1611 =app-doc/gimp-help-0.12

item 1612 =kde-base/korundum-3.5.7

item 1613 =app-vim/gentoo-syntax-20070506

item 1614 =app-vim/vim-spell-en-20060123

item 1615 =kde-base/artsplugin-mpeglib-3.5.7

item 1616 =games-puzzle/ksudoku-0.4

item 1617 =dev-ruby/sqlite-ruby-2.2.3-r1

item 1618 =gnome-extra/gnome2-user-docs-2.16.1

item 1619 =app-text/po4a-0.30

item 1620 =xfce-base/libxfce4util-4.4.1

item 1621 =app-text/gtkspell-2.0.11-r1

item 1622 =media-libs/gstreamer-0.8.12

item 1623 =app-arch/dpkg-1.13.25

item 1624 =net-libs/gnet-2.0.7

item 1625 =xfce-base/libxfcegui4-4.4.1

item 1626 =xfce-base/libxfce4mcs-4.4.1

item 1627 =dev-python/pyorbit-2.14.3

item 1628 =dev-ruby/ruby-atk-0.16.0

item 1629 =media-libs/gst-plugins-0.8.12

item 1630 =xfce-base/xfce-mcs-manager-4.4.1

item 1631 =x11-misc/xscreensaver-5.02-r2

item 1632 =media-plugins/gst-plugins-gconf-0.10.3

item 1633 =dev-ruby/ruby-gconf2-0.16.0

item 1634 =xfce-extra/xfce4-appfinder-4.4.1

item 1635 =media-sound/audacious-1.3.2

item 1636 =xfce-extra/xfce4-taskmanager-0.3.2-r1

item 1637 =xfce-extra/mousepad-0.2.12

item 1638 =xfce-base/xfce-mcs-plugins-4.4.1-r1

item 1639 =kde-base/kdeartwork-kscreensaver-3.5.7

item 1640 =media-plugins/gst-plugins-oss-0.8.12

item 1641 =media-plugins/gst-plugins-ogg-0.8.12

item 1642 =media-plugins/gst-plugins-xvideo-0.8.12

item 1643 =media-plugins/gst-plugins-vorbis-0.8.12

item 1644 =media-plugins/gst-plugins-faad-0.8.12

item 1645 =media-plugins/gst-plugins-mad-0.8.12

item 1646 =app-accessibility/gnome-speech-0.4.11

item 1647 =dev-ruby/ruby-gtk2-0.16.0

item 1648 =dev-ruby/ruby-libglade2-0.16.0-r1

item 1649 =kde-base/kaudiocreator-3.5.7

item 1650 =kde-base/kdvi-3.5.7

item 1651 =net-analyzer/gnome-nettool-2.18.0

item 1652 =app-vim/svncommand-1.67.3

item 1653 =gnome-extra/gnome-art-0.2-r2

item 1654 =games-board/gnono-1.9.1

item 1655 =games-board/gtkboard-0.11_pre0

item 1656 =games-puzzle/skoosh-2.5.0

item 1657 =x11-wm/beryl-core-0.2.1

item 1658 =x11-drivers/nvidia-drivers-100.14.11

item 1659 =x11-drivers/xf86-video-i740-1.1.0

item 1660 =x11-drivers/xf86-video-nsc-2.8.2

item 1661 =x11-drivers/xf86-input-joystick-1.2.1

item 1662 =x11-drivers/xf86-video-imstt-1.1.0

item 1663 =x11-wm/emerald-0.2.1

item 1664 =media-libs/t1lib-5.0.2

item 1665 =x11-misc/beryl-settings-bindings-0.2.1

item 1666 =dev-tex/latex-unicode-20041017

item 1667 =x11-wm/aquamarine-0.2.1

item 1668 =net-dialup/mgetty-1.1.35-r2

item 1669 =sci-electronics/gnucap-0.35.20070221

item 1670 =dev-util/cvs-1.12.12-r4

item 1671 =x11-themes/emerald-themes-0.2.1

item 1672 =app-office/kletterwizard-0.9.8

item 1673 =dev-db/postgresql-8.2.4-r1

item 1674 =app-admin/system-tools-backends-1.4.2-r1

item 1675 =gnome-extra/libgda-1.2.4

item 1676 =dev-libs/cyrus-sasl-2.1.22-r2

item 1677 =dev-perl/DBD-Pg-1.49

item 1678 =dev-libs/libpqxx-2.6.8

item 1679 =dev-libs/gmime-2.2.3

item 1680 =sys-libs/libieee1284-0.2.10

item 1681 =dev-python/python-ldap-2.3

item 1682 =dev-libs/apr-util-1.2.8-r1

item 1683 =net-libs/libnfsidmap-0.19

item 1684 =dev-libs/a

----------

## billydv

Another  question,  If  you  have  changed  the  profile  from  206 desktop  to  2007  desktop,  does  the  kernel  need  to  be  recompiled?  I  noticed  after  doing  that  that  several  packages  needed  to  be  remerged  with  the  acl  use  flag.

Another  thing,  my  computer  has  locked  up  twice  in  this  process,  will  that  effect  the  compiling (or  what  has  been  compiled?)

----------

## billydv

Guenther,  it  would  be  great  if  you  could  modify  the  script  to  answer  yes  whenever  something  required  input

----------

## billydv

Okay  Guenther,  let  me  understand  something,  I  see  that  the  script, recompile-remaining-packages  actually  has  the  list  in  it,  what  if  I  was  to  delete  a  few  of  the  entries,  for  a  few  games  that  require  a  yes  or  something,  Will  it  mess  up  the  script  as  the  numbers  will  not  be  in  order,  or  to  more  accurately  say  there  would  be  missing  numbers?

----------

## Moonlight-Flower

I started using this script on someone's suggestion, since I was having trouble after the libexpat upgrade. The script runs fine until it tries to recompile xterm, whereupon it stops at a line running /bin/sh ./minstall.sh for a file called xterm.man. It just stops. No CPU usage, hard disk doesn't do anything, the rest of the computer is completely unaffected (I can open other programs, play games, listen to music).

If I cancel the script and run "emerge xterm" on its own, xterm emerges perfectly, without any problems.

Did I make some mistake running the script, or is there some strange glitch involved?

----------

## billydv

Open  the  script  and  delete  the  line  with  xterm.  Then  running  the  script  wont  call  xterm  to  emerge,  you  can  also  do  this  to  delete  anything  you  dont  want  reemerged  like  games  that  require  a  yes  and  an  enter  or  other  packages  emerged  with  special  instructions.

----------

## Guenther Brunthaler

Hi billydv,

 *billydv wrote:*   

> if  I  was  to  delete  a  few  of  the  entries,  for  a  few  games  that  require  a  yes  or  something,  Will  it  mess  up  the  script  as  the  numbers  will  not  be  in  order,  or  to  more  accurately  say  there  would  be  missing  numbers?

 

No, there should not be a problem if some numbers are missing.

The script just remembers the number of the last package which has successfully been compiled, and skips all entries with lower numbers when re-examining the list for the next time.

This means the only thing important is that the numbers are increasing, but the specific values are not important.

----------

## Guenther Brunthaler

 *billydv wrote:*   

> Guenther,  it  would  be  great  if  you  could  modify  the  script  to  answer  yes  whenever  something  required  input

 

Well, perhaps. But it's somewhat dangerous answering "yes" to unknown prompts.

Just imagine some module prompting

```

Do you want to initialize a device for database extents now? yes

Shall I auto-detect a device of sufficient size? yes

Checking... /dev/hda is sufficient! Use it? yes

Are you sure (All Data on /dev/hda will be destroyed)? yes

Are you really sure? yes

This is your last chance to quit - are you REALLY really sure? yes

You wanted it so!

Initializing ... 10 % ... 20 % ... 30 % ...

Kernel Panic: Corrupted superblock on root device!

```

Besides, I have no idea how to detect whether an emerge requires input to be provided via stdin. At least not from a shell script like this. I suppose I would have to use something like expect for this - but that's TCL stuff (which I'm not experienced with).

----------

## Guenther Brunthaler

 *billydv wrote:*   

> Heres  my  list,  How  long  do  you  think  this  will  take?

 

It may take days or week, depending how fast your system is.

In general, it is impossible to say. There are just too many factors involved: What processor do you have? How much free RAM? How fast is your disk? Are your filesystems heavily fragmented? Are you running other applications while recompiling?

I have about 1300 total packages installed, and it took about a week for my script to finish on my Athlon 1.3 GHz machine.

----------

## billydv

After  having  used  your  script  several  times,  not  only  for  reinstalling  everything  but  if  I  have  alot  of  packages  that  I  want  to  reinstall  I  can  be  sure  that  a  failed  emerge  wont  stop  it  all,  I  simply  add  in  the  packages  to  generated  script  of  yours,  and  its  certainly  simple  enough  to  delete  any  package  lines  you  dont  want  rememerged.  All  in  all,  great  script,

BTW  my  athlon  x2 4200  does  approximately  2200  packages  in  3  days  or  less.

----------

## Guenther Brunthaler

Hi billydv,

 *billydv wrote:*   

> I  simply  add  in  the  packages  to  generated  script  of  yours

 

Glad to see it is useful to you even beyond its primary purpose!

Actually, I have a modified version of the script which has additional command line options which allow to specify a list of packages, and the script uses them for the package list instead of obtaining it from "emerge -pe world".

Unfortunately the help text is not yet up to date, so I did not post it yet.

But I guess I'll fix the help text in the next couple of days and will then post a new version.

 *billydv wrote:*   

> BTW  my  athlon  x2 4200  does  approximately  2200  packages  in  3  days  or  less.

 

Seems it's about time for me to consider purchasing a new box  :Wink: 

But I'm still waiting for the quad-core prices to drop.

And I would also like to get a CPU with more than the usual 48 Bit virtual address space. After all, it's a 64 bit CPU and not a 48 bit CPU!

----------

## billydv

Although  I  have  numerous  boxes  which  I  use  to  test  various  linux  distros,  the  x2 4200's  of  which  I  have  two  are  the  best  I  have,  they  blow  away  even  an  fx57  that  I  also  have.  My  next  build  will  be  an  intel  quad  core  and  Im  waiting  for  prices  to  drop  on  that  too!!!

----------

## thekk

 *Guenther Brunthaler wrote:*   

> If you just do a "emerge -e world", you will eventually find yourself with a recompiled system where about 75 packages have been linked against the old version of glibc ... see https://forums.gentoo.org/viewtopic.php?p=3555009#3552919 for more of this.
> 
> So, if you want to be "100 % sure", "emerge -e world" won't provide you with what you want.
> 
> My script does the same as an "emerge -e world", only in the correct order for recompiling a system.
> ...

 

I don't know if this has been suggested before in this thread, but I think it would be awesome if the ordering of the packages would end up in the default portage/emerge toolkit. Using your analogy before, it would probably require a new switch. Though I'm puzzled whether it would hurt to order the packages the right way when using the -e switch.

----------

## Phoenix591

why'd this 2 year old (ok, the last post was 4 months ago, but still) thread get dug up? (no resurrecting/posting in old, long silent threads) plz)

----------

## billydv

Gunther,  I  love  your  script  and  have  used  it  every  few  months  to  keep  my  system  up  to  date  but  unfortunately  your  script  no  longer  works.  It  says  that  it  dies  at  line  352.  Any  chance  for  an  update?

----------

## dalek

 *billydv wrote:*   

> Gunther,  I  love  your  script  and  have  used  it  every  few  months  to  keep  my  system  up  to  date  but  unfortunately  your  script  no  longer  works.  It  says  that  it  dies  at  line  352.  Any  chance  for  an  update?

 

I to would like a update.  Here is a work around, sort of.  Run the script and when it generates this list of packages, contained in recompile-remaining-packages, then copy the list and create a package set.  You know, the @set thingy.  You will have to remove all the weird parts on each line which can take some time but it should work.  Just emerge @<set name> and let it run.  

I do wish this script could be updated tho.  It comes in handy on occasion. I just wonder why portage doesn't use the same method?  

 :Very Happy:   :Very Happy: 

----------

## billydv

For  whatever  reason  it  once  again  works!!  I  love  this  script!!

----------

## dalek

It is working here as well, so far anyway.

```
root@smoker / # cat /root/.recompile-remaining-packages.state

1.13

i686-pc-linux-gnu-4.1.2

599 597 2

root@smoker / #
```

Maybe it doesn't need upgrading?  There are about 800 packages on here. 

 :Very Happy:   :Very Happy: 

----------

## billydv

I have 2500 packages!!

----------

## dalek

 *billydv wrote:*   

> I have 2500 packages!!

 

 HOLY CRAP !!!   

----------

## billydv

Just wanted to say that I used this script again this week and it still works great!!!!

----------

## Simba7

Holy crap, Batman! Digging up a 4 year old thread?

Not to mention, who the hell uses the old 2006.1 profile anyway? I ditched that 3 1/2 years ago. Is there *ANY* benefit from using it vs. the latest ones? I don't see any on my end (besides willingly giving yourself a massive migraine).

----------

## dalek

 *Simba7 wrote:*   

> Holy crap, Batman! Digging up a 4 year old thread?
> 
> Not to mention, who the hell uses the old 2006.1 profile anyway? I ditched that 3 1/2 years ago. Is there *ANY* benefit from using it vs. the latest ones? I don't see any on my end (besides willingly giving yourself a massive migraine).

 

I'm using the latest profile and the script still works.  It's not about the profile, it's about the script.  The person posted because they used the script and it worked so they posted that it worked. 

 :Very Happy:   :Very Happy: 

----------

## billydv

Yes, Indeed the script still works. I just used it last week when I upgraded my box to a quad core amd with new c flags.

----------

## anonybosh

I have used this script quite a few times to date, thanks Guenther!

However, I just noticed a bug:

When using color output (specifically when set globally in the make.conf: EMERGE_DEFAULT_OPTS=" --color y") the recompile-remaining-packages script that is generated includes the color codes around each package name. For some reason, the end of the color tag is interpreted as an 's', and so all package names become invalid when emerge is run.

An easy workaround is of course to just disable color output during the use of the base script.

----------

