Gentoo: emerge broken

The other day I was trying to update one of my servers which was also my local portage mirror when I received the following surprising output:

# emerge portage -pv

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild U ] sys-apps/portage-2.1.2-r9 [2.1.2] USE="-build -doc -epydoc (-selinux)" LINGUAS="-pl" 20 kB

Total: 1 package (1 upgrade), Size of downloads: 20 kB
# emerge portage -v
Calculating dependencies... done!

>>> Emerging (1 of 1) sys-apps/portage-2.1.2-r9 to /
#

So the server was reading the local portage but was unable to install any applications (or upgrade). This is an unusual issue and I have not encountered an issue like this in the 3 years that I have been using Gentoo. To resolve this issue, I am going to attempt to re-install portage from the latest snapshot.

The first step of restoring a corrupt portage install is to download the latest release from distfiles.gentoo.org.

# cd /tmp/ && wget http://distfiles.gentoo.org/distfiles/portage-2.1.2.tar.bz2

Un-tar the package to a safe location out of the way, possibly /tmp/portage_recover (as recommended by the Gentoo Handbook).

# mkdir /tmp/portage_recover && cd /tmp/portage_recover
# mv /tmp/portage-2.1.2.tar.bz2 /tmp/portage_recover
# tar xjf portage-2.1.2.tar.bz2

You will now have a directory structure like the following:

# ll
total 151
-rw-r--r-- 1 portage portage 133926 Jan 14 15:00 ChangeLog
-rw-r--r-- 1 portage portage 1444 Jan 13 13:01 DEVELOPING
-rw-r--r-- 1 portage portage 3934 Jan 14 12:43 NEWS
-rw-r--r-- 1 portage portage 2406 Dec 23 11:05 RELEASE-NOTES
-rw-r--r-- 1 portage portage 1022 Jan 10 22:12 TEST-NOTES
drwxr-xr-x 2 portage portage 2064 Jan 14 15:00 bin
drwxr-xr-x 2 portage portage 752 Oct 16 12:01 cnf
drwxr-xr-x 2 portage portage 368 Jan 13 23:00 man
drwxr-xr-x 4 portage portage 920 Jan 14 15:00 pym
drwxr-xr-x 4 portage portage 144 Aug 1 2006 src
drwxr-xr-x 6 portage portage 224 Jan 14 14:54 tests

Now the trick is to copy the appropriate binaries and scripts from this location to your system.

# cp -R pym bin /usr/lib/portage/

Unfortunately, this does not solve my problem:

# emerge portage -v
Calculating dependencies... done!

>>> Emerging (1 of 1) sys-apps/portage-2.1.2-r9 to /
#

So I am back to the troubleshooting board.