Gentoo: emerge broken continued

Ok, so I have discovered another clue in the puzzle here, when I execute 'emerge --info', I get the following output at the top:

# emerge --info
!!! No gcc found. You probably need to 'source /etc/profile'
!!! to update the environment of this terminal and possibly
!!! other terminals also.

I also see the following in the FEATURES line:

FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"

I had no idea that all features were included if not explicitly excluded, so I added this line to my /etc/make.conf:

FEATURES="-*"

Now I need to hunt down the problem with the missing gcc. I believe that is why portage is failing to update itself. I test this theory by removing the downloaded package and trying to run the same command again:

# rm /usr/portage/distfiles/portage-2.1.2.tar.bz2
# emerge portage -v
Calculating dependencies... done!

>>> Emerging (1 of 1) sys-apps/portage-2.1.2-r9 to /
#

I am not so confident in this theory anymore, as I would have expected portage to be downloaded again, but I am going to continue to resolve the issue with the missing gcc.

Gentoo: emerge broken

The other day I was trying to update one of my servers which was also my local portage mirror when I received the following surprising output:

# emerge portage -pv

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild U ] sys-apps/portage-2.1.2-r9 [2.1.2] USE="-build -doc -epydoc (-selinux)" LINGUAS="-pl" 20 kB

Total: 1 package (1 upgrade), Size of downloads: 20 kB
# emerge portage -v
Calculating dependencies... done!

>>> Emerging (1 of 1) sys-apps/portage-2.1.2-r9 to /
#

So the server was reading the local portage but was unable to install any applications (or upgrade). This is an unusual issue and I have not encountered an issue like this in the 3 years that I have been using Gentoo. To resolve this issue, I am going to attempt to re-install portage from the latest snapshot.

The first step of restoring a corrupt portage install is to download the latest release from distfiles.gentoo.org.

# cd /tmp/ && wget http://distfiles.gentoo.org/distfiles/portage-2.1.2.tar.bz2

Un-tar the package to a safe location out of the way, possibly /tmp/portage_recover (as recommended by the Gentoo Handbook).

# mkdir /tmp/portage_recover && cd /tmp/portage_recover
# mv /tmp/portage-2.1.2.tar.bz2 /tmp/portage_recover
# tar xjf portage-2.1.2.tar.bz2

You will now have a directory structure like the following:

# ll
total 151
-rw-r--r-- 1 portage portage 133926 Jan 14 15:00 ChangeLog
-rw-r--r-- 1 portage portage 1444 Jan 13 13:01 DEVELOPING
-rw-r--r-- 1 portage portage 3934 Jan 14 12:43 NEWS
-rw-r--r-- 1 portage portage 2406 Dec 23 11:05 RELEASE-NOTES
-rw-r--r-- 1 portage portage 1022 Jan 10 22:12 TEST-NOTES
drwxr-xr-x 2 portage portage 2064 Jan 14 15:00 bin
drwxr-xr-x 2 portage portage 752 Oct 16 12:01 cnf
drwxr-xr-x 2 portage portage 368 Jan 13 23:00 man
drwxr-xr-x 4 portage portage 920 Jan 14 15:00 pym
drwxr-xr-x 4 portage portage 144 Aug 1 2006 src
drwxr-xr-x 6 portage portage 224 Jan 14 14:54 tests

Now the trick is to copy the appropriate binaries and scripts from this location to your system.

# cp -R pym bin /usr/lib/portage/

Unfortunately, this does not solve my problem:

# emerge portage -v
Calculating dependencies... done!

>>> Emerging (1 of 1) sys-apps/portage-2.1.2-r9 to /
#

So I am back to the troubleshooting board.

Creating Virtual IP Addresses on Linux

Virtual IP addresses (or VIPs) allow you to use multiple IPs on a single physical network interface. Creating virtual IP addresses is often done to allow webservers to host multiple SSL encrypted web sites on a single webserver or to allow cluster suites to communicate on a dedicated IP address. This article will cover the two primary means of creating virtual IPs on a Linux host.

ifconfig

The first and most common method employed is to use the Linux command 'ifconfig' to create a VIP in the following manner, assuming that the interface being used is eth1.

# ifconfig eth1:0 192.168.1.28

This command will create a VIP on eth0 with a name of eth1:0 and will look like the following:

eth1:0 Link encap:Ethernet HWaddr 00:14:6C:83:39:92
inet addr:192.168.1.28 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:33442 errors:0 dropped:0 overruns:0 frame:0
TX packets:38225 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20240563 (19.3 Mb) TX bytes:3483829 (3.3 Mb)
Interrupt:18


This command creates an apparent separate device from eth1 with it's own IP address, netmask, and broadcast address. This VIP can now be used to host services and servers, fielding connections to clients or other hosts.

To remove the VIP, simply execute ifconfig on the device with the down command:

# ifconfig eth1:0 down

iproute2

The iproute tool set is tremendously powerful and not often used, even by experienced administrators. The description of the ip command from the man pages describes this suite well:

ip - show / manipulate routing, devices, policy routing and tunnels

While one can easily perform complex tasks on the network stack of any Linux host with this tool, this article will restrict it's coverage to creating VIPs. The command to create a VIP using the ip command is as follows:

# ip addr add 192.168.1.28 dev eth1


Interesting enough, when the previous command is issued, ifconfig does not show anything different. To see the new VIP, one must use the ip command as follows:

ip addr show

When this command is executed, the output will appear like the following:

3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:14:6c:83:39:92 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.10/24 brd 192.168.1.255 scope global eth1
inet 192.168.1.28/32 scope global eth1

This output is similar to that of a more traditional UNIX variant when compared to the output of ifconfig. The VIP created here can now be used for any purpose deemed suitable by the administrator.

To remove the VIP, execute the ip command with the following options:

ip addr del 192.168.1.28/32 dev eth1

Note that the device must be specified when creating and deleting the VIP for it to function properly. Note also that the subnet mask was specified on the deletion command and that this is not required.

Conclusion

Creating VIPs is a very simple task and one that can benefit every system administrator. Once learned, these techniques will be a great asset for common networking tasks.

Using cdrecord --scanbus with ATAPI devices

As fewer people are required to use scsi emulation to use cdrecord in Linux, here is how you would execute a scanbus command using cdrecord on ATAPI devices:

primary Linux # cdrecord dev=ATAPI --scanbus
Cdrecord-ProDVD-Clone 2.01.01a20 (i686-pc-linux-gnu) Copyright (C) 1995-2006 Jörg Schilling
scsidev: 'ATAPI'
devname: 'ATAPI'
scsibus: -2 target: -2 lun: -2
Warning: dev=ATA: is preferred over dev=ATAPI:.
Warning: Using ATA Packet interface.
Warning: The related Linux kernel interface code seems to be unmaintained.
Warning: There is absolutely NO DMA, operations thus are slow.
Using libscg version 'schily-0.8'.
scsibus0:
0,0,0 0) 'SONY ' 'CD-RW CRX230E ' 'QYS1' Removable CD-ROM
0,1,0 1) *
0,2,0 2) *
0,3,0 3) *
0,4,0 4) *
0,5,0 5) *
0,6,0 6) *
0,7,0 7) *

RHCE: Success!

I have received the results of the RHCE exam and I have passed well! Hopefully I can begin updating my blog more frequently.

RHCE: I took the exam on 12/02/2005

After taking the exam yesterday, I should find out the results next week. I'll let you know how I did.



Note: As is common knowledge, no discussion of the exam or contents can occur after taking the exam by participants. Please refer to the website at http://redhat.com/training for any questions you may have. There is a detailed exam prep page that will outline all of the requirements.

RHCE: Run-time RAID Configuration

Here is a link to an excellent article on creating RAID arrays using the mdadm tool.

Here is an excerpt:

Creating an Array

Create (mdadm --create) mode is used to create a new array. In this example I use mdadm to create a RAID-0 at /dev/md0 made up of /dev/sdb1 and /dev/sdc1:

# mdadm --create --verbose /dev/md0 --level=0

--raid-devices=2 /dev/sdb1 /dev/sdc1

mdadm: chunk size defaults to 64K

mdadm: array /dev/md0 started.

The --level option specifies which type of RAID to create in the same way that raidtools uses the raid-level configuration line. Valid choices are 0,1,4 and 5 for RAID-0, RAID-1, RAID-4, RAID-5 respectively. Linear (--level=linear) is also a valid choice for linear mode. The --raid-devices option works the same as the nr-raid-disks option when using /etc/raidtab and raidtools.

In general, mdadm commands take the format:

mdadm [mode]  [options] 

Each of mdadm's options also has a short form that is less descriptive but shorter to type. For example, the following command uses the short form of each option but is identical to the example I showed above.

# mdadm -Cv /dev/md0 -l0 -n2 -c128 /dev/sdb1 /dev/sdc1

RHCE: Installing RPMs from an NFS share

Using an NFS share to install packages is a very convenient means of ensuring that all of your systems are running similar packages and always having those packages available. I will not cover configuring an NFS share and will assume that the reader is already familiar with that function or has the capacity to figure it out.

1. Verify that the NFS share is available and mount on the local filesystem, if not already mounted.

mount -t nfs /local/install/point /nfs/server

2. If using the command line, simple use the RPM command to install the application:

rpm -Uvh /path/to/share/application.rpm

3. If using the package manager, use the following command from the command line:

system-config-packages --tree=/path/to/nfs/share &

You will now be able to select the applications that you would like to add or remove.

RHCE: Logical Volume Manager

Red Hat Enterprise Linux 4.0 uses a logical volume manager to facilitate efficient management of disks and partitions. With the Logical Volume Manager, partitions can be created that span multiple physical volumes and partitions. In this sense a physical volume is a hard disk. With this ability, an administrator can easily expand a partition or create an efficient partition scheme. RHEL 4.0 uses LVM2 by default.

When using LVM, it may be difficult to remember all of the commands that are possible and necessary to create a Logical Volume. An easy way to get a list of all of the related commands is to enter the lvm console by typing 'lvm' on the command line. Once in the LVM console, type 'help' and all of the available commands will be listed with a short description.

CAUTION: using the logical volume manager can and probably will destroy data. Verify that you have created backups of all of your data before trying the samples below.

Using the Logical Volume Manager

1. Partition the physical hard disks that will be used as part of the Logical Volume(s)

Use 'fdisk' as appropriate. Remember to set the system type as 'Linux LVM', which is type '8e'.

2. Create physical volume(s)

From within the lvm console, use pvcreate on each partition that will participate in the logical volume group.

pvcreate 'physical partition'

Sample:

lvm> pvcreate /dev/hdd1
Incorrect metadata area header checksum
Physical volume "/dev/hdd1" successfully created

3. Create a volume group

Using the vgcreate command, create a volume group which consists of the physical volumes created previously.

vgcreate 'volume group name' 'physical volume' ['physical volume'] ...

Sample:

lvm> vgcreate test1 /dev/hdd1 /dev/hdd2
Incorrect metadata area header checksum
Volume group "test1" successfully created

Verification of the volume group creation can be done with the vgdisplay command:

lvm> vgdisplay
--- Volume group ---
VG Name test1
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 2
Act PV 2
VG Size 93.15 GB
PE Size 4.00 MB
Total PE 23846
Alloc PE / Size 0 / 0
Free PE / Size 23846 / 93.15 GB
VG UUID znEYOy-n4oJ-zmXq-QARI-cRvD-YmY5-Gq6qpd

4. Create a logical volume in an existing Logical Volume Group

Using the lvcreate command, create a logical volume:

lvcreate [-L 'size'] [-n 'logical volume name'] 'logical volume group'

Sample:

lvcreate -L 200M -n vol1 test1
Incorrect metadata area header checksum
Logical volume "vol1" created

Verify that the logical volume is as desired with the lvdisplay command:

lvm> lvdisplay
Incorrect metadata area header checksum
--- Logical volume ---
LV Name /dev/test1/vol1
VG Name test1
LV UUID HPqCiY-58NT-X1ae-5vk3-1hLw-f2no-AYe52O
LV Write Access read/write
LV Status available
# open 0
LV Size 200.00 MB
Current LE 50
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:0

5. Format the logical volume with the filesystem desired (ext3 shown)

[root@primary ~]# mke2fs -j /dev/test1/vol1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
51200 inodes, 204800 blocks
10240 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
25 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729

Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

6. In good Red Hat form, create a label for the filesystem with the e2label command

[root@primary ~]# e2label /dev/test1/vol1 test1

7. Create an entry in the /etc/fstab file for this logical volume so that it will be mounted on subsequent boots. Use a meaningful mount point:

LABEL=test1 /test1 ext3 defaults 0 0


These steps have covered how to create a logical volume and use it. You can also expand an existing logical volume and perform other maintenance tasks. The only other task covered here will be expanding an existing logical volume.

Expand a logical volume

1. Add desired disk space to your volume group, if necessary

a. use fdisk to create a new partition of type '8e'
b. use pvcreate to initialize the new partition as a physical volume

[root@primary ~]# pvcreate /dev/hdd3
Physical volume "/dev/hdd3" successfully created

c. use vgextend to add the physical volume to the volume group

[root@primary ~]# vgextend test1 /dev/hdd3
Incorrect metadata area header checksum
Incorrect metadata area header checksum
Volume group "test1" successfully extended

CAUTION: The next step will remove all data from the partitions in question. Verify that you have backups.

d. use lvextend to expand the logical volume to the desired size

[root@primary ~]# lvextend -L 300M /dev/test1/vol1
Incorrect metadata area header checksum
Extending logical volume vol1 to 300.00 MB
Logical volume vol1 successfully resized

e. re-format your logical volume, relabel it, and remount it

umount 'logical volume'
mke2fs -j 'logical volume'
e2label 'path to volume' 'label'
mount 'path to volume in /etc/fstab'


Trouble Shooting

1. 'physical partition' not identified as an existing physical volume

lvm> vgcreate 'volume group' 'physical partition' 'physical partition'
Incorrect metadata area header checksum
Incorrect metadata area header checksum
No physical volume label read from 'physical partition'
not identified as an existing physical volume
Unable to add physical volume 'physical partition' to volume group 'volume group'.

To correct this problem, use lvm to create a physical volume on each partition with the pvcreate command:

lvm> pvcreate 'physical partition'
Incorrect metadata area header checksum
Physical volume "physical partition" successfully created

Create the volume group with the vgcreate command.

RHCE: Networking and Network Configuration

One of the greatest things about Linux is the ability to easily network systems. Linux excels when using wired or wireless networking. There are several basic requirements that must be met for your Linux machine to communicate with other machines or devices on a network.

The first requirement is that your machine must have a means of communicating with other machines through hardware. This requirement is typically met through a network interface card (NIC) at the host level and a switch or router at the local area network (LAN) level. These devices are connected through ethernet cables or another suitable medium (which may also include wireless devices). This requirement is also referred to as the Physical Layer in the OSI Reference Model.

The next requirement is that each network interface must be configured properly. Each interface must be configured with an IP address, netmask, and gateway. These simple parameters allow the interface to communicate with other interfaces on the network. Each interface may be configured through the GUI tools which are provided by Red Hat, by editing configuration files with a text editor, or they can be manually configured from the command line. I will review the command line and text file configurations only. This requirement covers layers 2 and 3 of the OSI Reference Model.

Another requirement that must be met if communication with the network outside of the immediate network is desired is proper configuration of the /etc/resolv.conf file with the IP address of a valid DNS server:

     nameserver xxx.xxx.xxx.xxx
     nameserver xxx.xxx.xxx.xxx

The /etc/resolv.conf file, when configured properly, will allow the machine to obtain IP addresses for hosts which are known only by host name. It is critical that you do not allow any spaces before the nameserver directive in this file or it will not function. This file is configured automatically when using DHCP and manually when using a static IP.

There is a configuration file for each interface in the following directory:

     /etc/sysconfig/network-scripts

These files serve as parameter files for the network startup script which is:

     /etc/rc.d/rc5.d/S10network

When the network service is started, either manually or on system boot, the network startup script reads the configuration file to determine whether or not the interface should be configured to start and how to configure the interface. If the interface is configured to obtain an IP address via DHCP then it will broadcast for a DHCP server and set the IP address, netmask, default gateway, /etc/resolv.conf file, and possibly the hostname for the machine, depending on how the system is configured. If the interface is configured to use a static IP address then most of the above settings will be configured through the configuration file in:

     /etc/sysconfig/network-scripts

The hostname and /etc/resolv.conf file must be configured manually. An example of this file properly configured with a static IP address is as follows:

     [root@primary network-scripts]# cat ifcfg-eth0
     DEVICE=eth0
     BOOTPROTO=static
     BROADCAST=192.168.1.255
     HWADDR=00:E0:81:22:CC:8B
     IPADDR=192.168.1.10
     NETMASK=255.255.255.0
     NETWORK=192.168.1.0
     ONBOOT=yes
     TYPE=Ethernet

Most of the file is self-explanatory and fulfills the requirements listed above with regard to the interface configuration. If the interface is configured to use DHCP, the file will resemble the following:

     [root@primary ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
     DEVICE=eth0
     BOOTPROTO=dhcp

This simple file will cause the interface to obtain an IP address, netmask, and gateway on startup and configure the /etc/resolv.conf file with the nameserver used by the DHCP server. This is a very common configuration and makes network management very robust.

The last part of this article illustrate how to configure an interface manually on the command line. I will merely list the steps required to configure the same interface that is listed in the configuration file above with the same parameters:

     ifconfig eth0 192.168.1.10
     route add default gw 192.168.1.1
     ifconfig eth0 up

You can now verify your settings with:

     ifconfig devicename
     netstat -r

To manually obtain an IP address via DHCP, use the dhclient utility. If no interface is specified then the utility will attempt to obtain a DHCP address for each non-multicast address on the system. An example of using this utility is as follows:

     dhclient eth0

This command will configure the eth0 interface with an IP address, netmask, default gateway, and setup /etc/resolv.conf with the proper nameserver directives.

Through this article I explained what is required to network a Red Hat Enterprise Linux system. I also explained how to configure an interface so that the machine it is on will be able to communicate with other machines and devices on a network. The configuration can be done manually on the command line, through editting text files, and with the GUI tools provided by Red Hat with the operating system. The ability to configure Linux to interact with other systems is an essential skill, as a computer without the ability to network is useless.

RHEL: RAID Performance

I recently purchased 3 new 200GB Maxtor hard drives. I haven't bought a hard drive in a couple of years and the biggest one I owned before this was 80GB, with a second place runner up of 20GB. I'm doing pretty good with the new drives. They are 7200 rpm, IDE, 16MB cache type drives. I just hooked them up in my SMP athlon system and configured a few raid paritions in RAID 0 to test the performance:

root@primary ~]# hdparm -tT /dev/md1

/dev/md1:
Timing cached reads:   956 MB in  2.00 seconds = 477.36 MB/sec
Timing buffered disk reads:  264 MB in  3.01 seconds =  87.72 MB/sec
[root@primary ~]# hdparm -tT /dev/md3

/dev/md3:
Timing cached reads:   948 MB in  2.00 seconds = 473.60 MB/sec
Timing buffered disk reads:  270 MB in  3.01 seconds =  89.68 MB/sec

It's not often that you can get this kind of performance out of an IDE drive, so I'm pretty happy with my purchases. I did test them with Knoppix in a non-RAID configuration and was able to get 62 MB/sec, so this is nearly 30 MB/sec better!

RHCE: Exam Prep

All right, it's time to continue my studies for the RHCE exam. I've registered for the exam and I'll be taking it on Friday, December 2nd. I hope to be able to prepare fully by then with the book that I purchased and by practicing a lot. I just ordered some new hard disks so that I'll be able to play around with more configurations and have enough space on my main machine.

Tonight I'll be reviewing the RHCE exam prep guide.

Perl: Great scripting language...

I have always been a bash or ruby scripter and very rarely used Perl. Lately I have been using Perl quite a bit and have found that it is a joy to use! There is a lot of power behind Perl and it is very widely used in the industry. I realize that Google uses Python, but Amazon.com uses Perl. When performing job skill searches on the job boards of late, I have found quite a few that ask for Perl skills.

I am taking a short break from my studies for the RHCE because I received an un-solicited request for an interview from a large company in Seattle. I will keep you all updated.

NFS: Windows client?

In my recent studies for the RHCE I have covered the NFS file sharing system. I have used this system in the past, but not extensively. I have been told that it is faster than SMB by a great degree between Linux and Unix boxes so I am going to run some tests on it and see how it works. I am currently downloading Services for Unix from Microsoft so that I can test the Windows client that they provide for NFS. I'll post results.

RHCE: The Road to Certification

Aggregation of my posts on the RHCE:

Starting out!
RHCE Exam Prep Guide
RHCE Networking and Configuration

RHCE: The Road to Certification

Over the next couple of months I'm going to be studying for the RHCE exam. I have purchased the book, "RHCE 4th edition", by Michael Jang as a study guide since I heard it was the best one out there. I'm not terribly impressed with it so far, as I have found quite a few technical errors. I hope to be able to start posting about my learning experiences and relate that to how well I perform on the exam. One of the reasons that I am doing this is so that I can start doing some private consulting around my area and I'd like to have some certifications to back up my experience and knowledge. Stay tuned!

NX: Remote X-Server Session

I recently discovered an excellent client application for connecting to X remotely. NX is a client/server application developed by NoMachinewith some clever caching techniques which provide near real-time response for X-sessions. The NX server works with an SSH server to provide secure authentication and SSL encrypted traffic, if desired. A GPL'd server has been written and is available here, while a free client is available from Nomachine.com. The NX client will also connect to Microsoft applications which use the RDP protocol and VNC sessions. An excellent series of articles has been written on NX at Linuxjournal.com which provides some details as to the actual technical operations.

For anyone who has used VNC for remote GUI control of their systems, NX is far superior!

Speakeasy

I recently received an email from Speakeasy with an offer for broadband service in my area. I was impressed with the email as they gave me an option to test my current speed with one of their servers in Seattle. I have attached a link to this speed test in the left nav bar for all to use. I really like the way that Speakeasy does things and I do plan on switching to their service sometime in the next year. One of the good flexible options that Speakeasy has is that you can become a mini WISP if you purchase their service and they will take care of the billing and email problems with your customers. The bad part is that you have to take care of the tech support and security portions.

Responsible Disclosure: Ciscogate

For those who have not yet heard (shouldn't be anyone), Mike Flynn presented a flaw in Cisco routers at Black Hat 2005 that could bring the Internet to it's knees. There are conflicting sides to the story, but the gist is that Cisco was trying to down-play the seriousness of the flaw and keep the researcher from disclosing the vulnerability. Responsible disclosure means that after a reasonable amount of time trying to work with the vendor, the researcher must disclose the vulnerability to the security community so that the flaw may be fixed or defended against. There are rumors that the Chinese have already been exploiting this flaw, which makes it imperative that the security community know about it.

Open Standards: HTML and web technologies

As an avid supporter of open standards in all things digital, I was pleased to see this article on Slashdot wherein Paul Thurrott talks about boycotting Internet Explorer 7.0 until Microsoft comes out with a standards-compliant browser. I think that IE is a huge disappointment and a very lazy offering by MS. Any self-respecting tech company will strive to better the field that they work in and IE has made the field of web browsing and development worse for the wear. Please use Firefox.

Other reasons to not use Internet Explorer:

1. Privacy
2. Security
3. Diversity
4. Competition is better for innovation (not patents -- contrary to popular belief)