# Clean out unused packages from Portage

## n0n

Hello, all.  I've already brought this up a couple of times, but I finally got around to writing something to take care of the problem and figured I'd share it here for anyone else who cares.  Read the comments at the head of the file to get a feel for what it's all about.  If you'd rather not cut-and-paste, I've made the file available at http://apocalyptech.com/software/gentoo/gentoo-slotcheck.py.  Have fun, use at your own risk, comments welcome!

(Update: Made a rather minor change to the code, doesn't affect functionality at all, just consolidated a couple of loops into one.)

```
#!/usr/bin/env python

#

# gentoo-slotcheck.py

# Author: CJ Kucera <pez@apocalyptech.com>

# URL: http://apocalyptech.com/software/gentoo

#

# This script is designed to help clean up your Gentoo package

# installation database.  Portage has a concept of SLOTs that

# enable different versions of packages to exist side-by-side.

# More information on SLOTs can be found at:

#       http://gentoo.org/doc/portage-manual.html

#

# The early "skeleton" ebuild file, upon which many ebuilds were

# written, did not have a default SLOT defined, or it was blank.

# As ebuilds were revised, SLOT values were put in, but because

# the original versions had a blank SLOT, the newer versions

# existed side-by-side with the previous versions.  For instance,

# on my machine right now, I've got app-admin/chkrootkit version

# 0.35 and version 0.36.  0.35 has a blank SLOT, whereas 0.36

# has a SLOT of zero.  There's no reason to keep around the

# older version, but there it will sit until manually removed.

# Doing an "emerge clean" has no effect on SLOT-less packages.

#

# It should be noted that these extra versions really aren't

# that big of a deal, because "side-by-side" in this case

# is fairly meaningless; the new version is installed basically

# on top of the old version.  Still, old libraries can be kept

# around, stray files CAN cause confusion, and if you're a

# system-organizing freak like me, it'll bug you.  But if you

# want to make sure that you don't break things, and the

# lingering versions don't bother you, then you can pretty

# much ignore this script.

#

# If you feel like cleaning up your Portage tree by hand, you

# can do so by glancing around inside /var/db/pkg, but this

# script is designed to make it a bit easier.  It will scan

# through that directory and notice any program that's

# installed twice.  If one of those installations has a blank

# SLOT defined, it'll ask you if you'd like to remove that

# version.  If you say yes, it will first show you everything

# that depends on the program you're about to destroy (just

# so you're extra careful here), and also the output of

# "emerge --pretend unmerge <package>", so that you know what's

# about to happen to your system.  If you're okay with what

# was displayed, say Yes again, and it'll do the work for you.

#

# Note that while this program SHOULDN'T do anything nasty to

# your system, there's a chance that it'll erase your hard

# drive, fry your CPU, poison your cat, default your mortgage,

# promote racial hatred, and cause our sun to implode in on

# itself.  If you're uncomfortable about removing some package,

# then don't.  Again, it's typically not that big of a deal.

# Use with care and caution, and I'M NOT RESPONSIBLE!

#

# I'll hereby release this thing under the GPL, in case anyone

# cares.  See http://www.gnu.org/licenses/gpl.txt for details.

#

# Lastly, if anyone deems this script worthy of improvement,

# go for it.  It could certainly use some.  :)

#

# Version History:

#    2002.09.13 - CJ Kucera <pez@apocalyptech.com>

#                 Initial release

#    2002.09.13 - CJ Kucera <pez@apocalyptech.com>

#                 Consolidated a couple of loops (minor)

#

# TODO:

#  * Seeing as "emerge" is written in Python, it would probably

#    make sense to just call its functions in here, instead of

#    spawning off a new Python process.

#  * I'm aware that system() is far less secure than exec*()

#    or spawn*() or any other function that more rigidly defines

#    argument input.

#

import os,sys,portage

installcache = portage.db["/"]["vartree"]

for package in installcache.getallnodes():

  versions = installcache.getnode(package)

  if (len(versions) > 1):

    blankslots=[]

    totalversions=[]

    for version in versions:

      slot = installcache.getslot(version[0])

      version.append(slot)

      if (slot == ""):

        blankslots.append(version[0])

    if (len(blankslots) > 0):

      print "------------------------"

      print

      print "Package " + package + " has a SLOT mismatch:"

      print

      for version in versions:

        print " * " + version[0] + " has a SLOT of '" + version[2] + "'"

      print

      print "The following versions will be unmerged:"

      print

      unmergelist = ""

      for blank in blankslots:

        print " * " + blank

        unmergelist = unmergelist + blank + " "

      print

      sys.stdout.write("Do the unmerge [N/y/q]? ")

      dounmerge = sys.stdin.readline()

      print

      if (dounmerge[0].lower() == "y"):

        print "First we'll check qpkg and --pretend to make sure things are kosher . . ."

        print

        os.system("qpkg -q " + unmergelist)

        print

        os.system("emerge --pretend unmerge " + unmergelist)

        print

        sys.stdout.write("Still do the unmerge [N/y/q]? ")

        dounmerge = sys.stdin.readline()

        print

        if (dounmerge[0].lower() == "y"):

          print "Okay, here we go."

          os.system("emerge unmerge " + unmergelist)

        elif (dounmerge[0].lower() == "q"):

          sys.exit()

        else:

          print "Maybe next time, then."

      elif (dounmerge[0].lower() == "q"):

        sys.exit()

      else:

        print "We'll skip that one for now, then."

      print

# The End
```

----------

## Schnacki

Cool script. Did you ever find out, if this slot-less baselayout is needed or not? As I am such a "system-organizing freak" like you, I really wonder if I can safely remove it.  :Smile: 

ciao, Schnacki

----------

## senectus

this is a real n00b question.. but how do you run it?

----------

## El_Presidente_Pufferfish

copy it into a text file(i did nano -w slotclean.py)

chmod +x it

then

./slotclean.py

----------

## n0n

 *Schnacki wrote:*   

> Cool script. Did you ever find out, if this slot-less baselayout is needed or not? As I am such a "system-organizing freak" like you, I really wonder if I can safely remove it. :)

 

Yeah, I eventually just unmerged the thing and everything was fine, so go ahead and do it yourself.

If you're feeling apprehensive about it, you could always use "quickpkg" to create a binary package out of the slotless version; that way if it does screw anything up you can just re-emerge the old version.  But I don't think you'd need to do that.

----------

## n0n

Actually, for a long time now I've been using a much better version of the script.  I just uploaded it to http://apocalyptech.com/linux/gentoo/gentoo-checkdupes.py, and I'll paste it here as well.  There are a couple of rather hokey things about it, but I like it.  It's mostly self-explanatory...  It goes through all the packages with more than one version installed and asks you if you want to unmerge any of them.  It will automatically add any slot-less packages to the list of packages to unmerge...  You can modify the list by hitting "m."  There are also options to run a qpkg dependency check, and also I wrote a section which will show you which files will probably be removed by doing the unmerge (which can be useful sometimes).  I dig it.  Anyway, here it is, the usual disclaimers apply:

```
#!/usr/bin/env python

import os,sys,portage,string

#

# This function should be smart enough to figure out what's going

# to be deleted if we unmerge a package.  This duplicates a lot

# of things that are going on inside portage.py, and should probably

# be contained there, but I'm not feeling up to hacking it right

# now.

#

def checkdeletions(packageVersion):

  # We should really just use portage.dblink, but for now I'm just

  # going to do this instead.

  fileName = "/var/db/pkg/" + packageVersion + "/CONTENTS"

  try:

    df = open(fileName, "r")

    fileinfo = df.readlines()

    df.close()

  except IOError:

    print "That package doesn't exist."

    sys.exit()

  #packageArray = string.split(packageVersion, "/")

  #mydblink = portage.dblink(packageArray[0], packageArray[1], "/")

  #pkgfiles=mydblink.getcontents()

  #mykeys=pkgfiles.keys()

  #mykeys.sort()

  #mykeys.reverse()

  #print str(mykeys)

  #return

  deletedfiles=[]

  dirs=[]

  # Make a list of files that would get deleted, and a list of all the dirs.

  for line in fileinfo:

    line = line[:-1]

    lineinfo = string.split(line, " ")

    if (lineinfo[0] == "dir"):

      if (os.path.isdir(lineinfo[1])):

        dirs.append(lineinfo[1])

    elif (lineinfo[0] == "obj"):

      if (os.path.exists(lineinfo[1])):

        try:

          statInfo = os.stat(lineinfo[1])

        except OSError:

          print "Something's wrong with file " + lineinfo[1]

          continue

        if (str(statInfo.st_mtime) == lineinfo[3]):

          md5stream = os.popen("/usr/bin/md5sum " + lineinfo[1] + " 2>/dev/null", "r")

          md5line = md5stream.readline()

          md5stream.close()

          md5val = string.split(md5line, " ")[0]

          if (md5val == lineinfo[2]):

            print "File " + lineinfo[1] + " will get hosed."

            deletedfiles.append(lineinfo[1])

    elif (lineinfo[0] == "sym"):

      if (os.path.islink(lineinfo[1])):

        if (os.readlink(lineinfo[1]) == lineinfo[3]):

          try:

            statInfo = os.lstat(lineinfo[1])

          except OSError:

            print "Something's wrong with symlink " + lineinfo[1]

            continue

          if (str(statInfo.st_mtime) == lineinfo[4]):

              print "Symlink " + lineinfo[1] + " will get hosed."

              deletedfiles.append(lineinfo[1])

    else:

      print "Unsupported type: " + lineinfo[0]

  # We should probably display directories here, too.

# Now the meat of everything

installcache = portage.db["/"]["vartree"]

for package in installcache.getallnodes():

  versions = installcache.getnode(package)

  if (len(versions) > 1):

    versions.sort()

    totalversions=[]

    totalblank=0

    counter=0

    for version in versions:

      counter = counter + 1

      slot = installcache.getslot(version[0])

      version.append(slot)

      version.append(counter)

      if (slot == ""):

        totalblank = totalblank + 1

    print "------------------------"

    print

    print "Package " + package + " has multiple versions installed:"

    print

    donewithloop = 0

    unmergelist = []

    while (not donewithloop):

      for version in versions:

        print " " + str(version[3]) + ") " + version[0] + " has a SLOT of '" + version[2] + "'"

      print

      for version in versions:

        if (version[2] == ""):

          unmergelist.append(version)

      if (len(unmergelist) > 0):

        print "Here's what we'd like to unmerge:"

        print

        for unmerge in unmergelist:

          print " * " + unmerge[0]

        print

      else:

        print "Currently nothing is marked for unmerging."

        print

      sys.stdout.write("Do the unmerge [(N)o/(y)es/(m)odify/(d)ependencies/(f)iles/(q)uit]? ")

      dounmerge = sys.stdin.readline()

      response = dounmerge[0].lower()

      print

      if (response == "m"):

        numlist = " "

        for unmerge in unmergelist:

          numlist = numlist + str(unmerge[3]) + " "

        numlist = numlist[1:-1]

        print "Type the numbers of the versions you want to unmerge, seperated"

        sys.stdout.write("by spaces: [" + numlist + "]: ")

        newnumlist = sys.stdin.readline()

        newnumlist = newnumlist[:-1]

        if (len(newnumlist) > 0):

          newnums = string.split(newnumlist)

          unmergelist = []

          for version in versions:

            if (str(version[3]) in newnums):

              unmergelist.append(version)

        print 

      elif (response == "y"):

        if (len(unmergelist) == 0):

          print "First select some versions to unmerge, by pressing 'm'"

          print

        else:

          print "First we'll check emerge --pretend to make sure things are kosher . . ."

          print

          tounmerge = ""

          for unmerge in unmergelist:

            tounmerge = tounmerge + unmerge[0] + " "

          os.system("emerge --pretend unmerge " + tounmerge)

          print

          sys.stdout.write("Still do the unmerge [N/y/q]? ")

          dounmerge = sys.stdin.readline()

          print

          if (dounmerge[0].lower() == "y"):

            print "Okay, here we go."

            os.system("emerge unmerge " + tounmerge)

            donewithloop = 1

          elif (dounmerge[0].lower() == "q"):

            sys.exit()

          else:

            print "Maybe next time, then."

      elif (response == "d"):

        if (len(unmergelist) == 0):

          print "First select some versions to unmerge, by pressing 'm'"

          print

        else:

          for unmerge in unmergelist:

            os.system("qpkg -q " + unmerge[0])

            print

      elif (response == "f"):

        if (len(unmergelist) == 0):

          print "First select some versions to unmerge, by pressing 'm'"

          print

        else:

          for unmerge in unmergelist:

            print "Package " + unmerge[0] + ":"

            checkdeletions(unmerge[0])

            print

      elif (response == "q"):

        sys.exit()

      else:

        print "We'll skip that one for now, then."

        donewithloop = 1

      print

# The End
```

----------

## tetrahydroc

Nice one, thanks a lot!

----------

## hgomersall

nice script. good work.

Is the [d]ependencies option going to do something?

cheers

hen

----------

## n0n

 *hgomersall wrote:*   

> nice script. good work.
> 
> Is the [d]ependencies option going to do something?

 

Yeah, it already does.  It calls "qpkg -d" on the file.  If

you don't have gentoolkit installed, it'll probably just

drop you back out to the menu.  qpkg -d has varying levels

of usefulness, but with gentoolkit installed it should be all

right.

----------

## rcxAsh

I know that it's been a couple months since the last post in this thread, but I just came across it recently and have a question regarding n0n's script.  I noticed that so far, most of the duplicate packages that have been listed by the script are listed as depended upon when I run a

```
 qpkg --installed --query-deps packagename 
```

.

For example, gentoo-checkdupes.py says that there are multiple versions of atk installed, 1.2.3 and 1.4.0.  However, qpkg says this:

```
lostech root # qpkg --installed --query-deps atk

dev-libs/atk-1.2.3 *

DEPENDED ON BY:

        gnome-base/libglade-2.0.1

        net-ftp/gproftpd-8.1.0

        x11-libs/gtk+-2.2.4-r1

dev-libs/atk-1.4.0 *

DEPENDED ON BY:

        gnome-base/libglade-2.0.1

        net-ftp/gproftpd-8.1.0

        x11-libs/gtk+-2.2.4-r1
```

As you can see, qpkg reports that the exact same packages depend on both atk 1.2.3 and 1.4.0... is this just a quirk of qpkg?  So can I still unmerge the duplicate older package?  

Also, what about glib?  This is qpkg's output:

```
lostech root # qpkg --installed --query-deps glib            

sys-libs/glibc-2.3.2-r1 *

DEPENDED ON BY:

        net-misc/netkit-fingerd-0.17-r2

dev-libs/glib-1.2.10-r5 *

DEPENDED ON BY:

        app-cdr/xcdroast-0.98_alpha13

        app-text/dgs-0.5.10-r1

        dev-libs/atk-1.2.3

        dev-libs/atk-1.4.0

        dev-libs/libIDL-0.8.2

        dev-python/wxPython-2.4.0.7

        gnome-base/ORBit-0.5.17

        gnome-base/ORBit2-2.8.2

        gnome-base/gconf-1.0.8-r5

        gnome-base/gconf-2.4.0.1

        gnome-base/gnome-common-1.2.4-r3

        gnome-base/gnome-vfs-2.4.0

        gnome-base/libbonobo-2.4.0

        gnome-base/libglade-2.0.1

        gnome-base/librsvg-2.4.0

        gnome-extra/libgsf-1.8.2

        kde-base/arts-1.1.2

        kde-base/kdemultimedia-3.1.2

        media-libs/libmovtar-0.1.3-r1

        media-libs/openquicktime-1.0-r1

        media-video/kino-0.6.4

        media-video/mjpegtools-1.6.1.90-r1

        media-video/mplayer-0.92

        net-ftp/gproftpd-8.1.0

        net-im/gaim-0.71-r2

        net-irc/xchat-2.0.3-r1

        net-www/mozilla-1.3-r2

        x11-libs/gtk+-2.2.4-r1

        x11-libs/pango-1.2.1-r1

        x11-libs/pango-1.2.5

dev-libs/glib-2.2.3 *

DEPENDED ON BY:

        app-cdr/xcdroast-0.98_alpha13

        app-text/dgs-0.5.10-r1

        dev-libs/atk-1.2.3

        dev-libs/atk-1.4.0

        dev-libs/libIDL-0.8.2

        dev-python/wxPython-2.4.0.7

        gnome-base/ORBit-0.5.17

        gnome-base/ORBit2-2.8.2

        gnome-base/gconf-1.0.8-r5

        gnome-base/gconf-2.4.0.1

        gnome-base/gnome-common-1.2.4-r3

        gnome-base/gnome-vfs-2.4.0

        gnome-base/libbonobo-2.4.0

        gnome-base/libglade-2.0.1

        gnome-base/librsvg-2.4.0

        gnome-extra/libgsf-1.8.2

        kde-base/arts-1.1.2

        kde-base/kdemultimedia-3.1.2

        media-libs/libmovtar-0.1.3-r1

        media-libs/openquicktime-1.0-r1

        media-video/kino-0.6.4

        media-video/mjpegtools-1.6.1.90-r1

        media-video/mplayer-0.92

        net-ftp/gproftpd-8.1.0

        net-im/gaim-0.71-r2

        net-irc/xchat-2.0.3-r1

        net-www/mozilla-1.3-r2

        x11-libs/gtk+-2.2.4-r1

        x11-libs/pango-1.2.1-r1

        x11-libs/pango-1.2.5
```

It seems that 1.2.10 and 2.2.3 are both depended by the same packages, and the only package needing 2.3.2 is fingerd...

What do I do?

----------

## n0n

 *rcxAsh wrote:*   

> As you can see, qpkg reports that the exact same packages depend on both atk 1.2.3 and 1.4.0... is this just a quirk of qpkg?  So can I still unmerge the duplicate older package?

 

Yeah, qpkg output for dependencies can be a bit quirky.  I imagine that 1.2.3 could probably safely go...

 *rcxAsh wrote:*   

> It seems that 1.2.10 and 2.2.3 are both depended by the same packages, and the only package needing 2.3.2 is fingerd...

 

Well, the glib 1.2.10 is in SLOT 1, and 2.2.3 is in SLOT 2, which means that they're designed to coexist with each other.  My script should, by default, not elect to unmerge anything that's got separate slots in it...  I'd recommend keeping both of those around...

----------

## Mandos

Nice script, any chance of one that can scan the packages database and /usr/portage/distfiles 

and list files in /usr/portage/distfiles that are for old versions in the package list, old versions 

no longer in the package list or which appear to be a partial download?

It would be nice to be able to easily keep /usr/portage/distfiles clean, but for some things 

such as automake/autoconfig multiple versions need to be kept.

----------

## gargan

I wrote a script that takes care of duplicate packages. I found out there are quite a few versions of some of my packages... deleted some by hand -> messed up my system -> had to reinstall most packages... I searched through the forums do find out how the depend system works. So I wrote this script to help you find the packages that reside on your system with more than one version, and also check if the older versions are still needed. It displays the dependency or the unmerge options. I didnt try it out too long, but it seems to work pretty good.

I also put it on http://www.gargan.org/linux

After a basic cleanup of stuff I knew I didnt need anymore (like old kernels) - I saved about 1 gig of space... and until now everything seems to work

```

#!/bin/sh

# finddupes.sh

# Find duplicate packages and make sure their depndencies are taken into account

# by Niko Berger

# you can find updated versions under www.gargan.org/linux

#

findem()

{

local numdeps

local numdups

local hasdep

numdeps=0

numdups=0

# for pkg in `qpkg -d -v -nc` - not used since pqkg is deprecated

for pkg in `equery l '*' | sort | awk -f finddupes.awk`

do

        hasdep=0

        numdups=`expr $numdups + 1`

        for dep in `grep $pkg /var/db/pkg/*/*/*DEPEND`

        do

                hasdep=1

                if [ "$d" -eq "1" ]; then

                        echo $pkg depends on \"$dep\"

                else

                        break;

                fi

        done

        if [ "$hasdep" -eq "0" ]; then

                if [ "$d" -eq "1" ]; then

                        echo $pkg

                fi

                if [ "$u" -eq "1" ]; then

                        echo emerge unmerge \"=$pkg\"

                fi

                if [ "$l" -eq "1" ]; then

                        echo $pkg

                fi

        else

                let numdeps+=1

        fi

done

if [ "$s" -eq "0" ]; then

        echo Found $numdups duplicate packages, $numdeps have dependencies

        echo use \"$0 -v\" for display of all dependencies

fi

}

Usage (){

echo $0 - find and print duplicate packages including their dependencies

echo "usage: $0 [-u] [-d] [-o] [-l]"

echo " without parameter the program will only display a summary"

echo " -u Print the unmerge command to STDOUT"

echo " -d Print all packages with their dependencies (if any)"

echo " -o Print only packages with dependencies"

echo " -l List only packages without dependencies in a list"

echo " -s Skip summary report"

echo ""

echo "To get all packages that are duplicate, without dependencies and therefore"

echo "safe to unmerge in the actual unmerge command style use: "

echo " \"$0 -u -s\""

exit 1

}

u=0

d=0

l=0

s=0

while [ $# -gt 0 ]

do

    case "$1" in

        -u)     u=1;;

        -d)     d=1;;

        -o)     o=1;;

        -l)     l=1;;

        -s)     s=1;;

        -h)     Usage;;

        -?)     Usage;;

    esac

    shift

done

findem $u $d $o $l $s

```

finddupes.awk

```

# This code is from rubik-wuerfel

# see: http://forums.gentoo.org/viewtopic.php?t=274448&highlight=duplicate+equery

BEGIN{

  package_old=" ";

}

{

  new=$0;

  match(new,/-([1234567890.rp\-_]|pre|rc)+$/);

# version number starts with a "-", then at least one

# of 1-0,.,r,p,-,_,pre,rc

  package_new = substr(new,0,RSTART-1);

# print only the old package

  if (package_new==package_old){

    print old;

  }

  old=new;

  package_old=package_new;

}

```

Edit: updated the code so it does not use qpkg any more 

Thanks to xchris for his mentioning of thisLast edited by gargan on Mon Feb 07, 2005 5:05 pm; edited 3 times in total

----------

## xchris

be carefull!

qpkg is deprecated!

It doesn't work well!

It doesn't check useflags.... and so on.

bye

----------

## gargan

 *xchris wrote:*   

> be carefull!
> 
> qpkg is deprecated!
> 
> It doesn't work well!
> ...

 

Yea, this and what I wrote on the homepage... Since I forgot mentioning, that it displays all versions (even the installed ones).... I am currently tweaking it so it will display only the older ones. 

And I use equery now...

----------

## xchris

to be honest i don't think equery is very accurate. (although it is newer than qpkg)

I noticed mistakes on mplayer pkg for example.

bye

----------

## gargan

 *xchris wrote:*   

> to be honest i don't think equery is very accurate. (although it is newer than qpkg)
> 
> I noticed mistakes on mplayer pkg for example.
> 
> 

 

Ok I updated the script with equery. Also it doesnt display ALL packages anymore, only the "older" ones. I am not sure, if the "not very accurate" of equery matters that much for this script. As long as it does not show a current package as deleteable. If it finds packages that were already removed, it wont display them anyways, and if it does not show packages which could be removed its not that big of a problem (since it only eats up space, but doesnt crash your system...)

I edited my post so it matches the changes...[/code]

----------

## xchris

ok ok  :Smile: 

just wanted to inform you  :Smile: 

Bye

----------

## gargan

 *xchris wrote:*   

> ok ok 
> 
> just wanted to inform you 
> 
> Bye

 

Thanks...   :Laughing: 

----------

