Ubuntu Server on HP Proliant - managing RAID

As part of the disk-based backup project, my Bacula Director is going on an Ubunutu 10.04 LTS server on HP Proliant hardware. All well and good, but this is my first Ubuntu Server deployment on HP hardware. Under Dell, I'm used to using the MegaRAID CLI tools to monitor and manage the RAID controller, but was stuck trying to find the equivalent for Ubuntu.

The answer is to use HP's ProLiant Support Pack which offers a bunch of CLI tools. The one I was interested in here was hpacucli

To install (as root, obviously):

1. wget http://downloads.linux.hp.com/SDR/downloads/bootstrap.sh
2. ./bootstrap.sh ProLiantSupportPack (this adds the HP repo to apt's sources)
3. wget http://downloads.linux.hp.com/SDR/downloads/ProLiantSupportPack/GPG-KEY-ProLiantSupportPack
4. apt-key add GPG-KEY-ProLiantSupportPack
5. aptitude update
6. apt-get install hpacucli

Like any good CLI program, it is finicky to learn, and intuitive thereafter :-) Here's the thing I wanted to do:

root@rm-bac-1:~# hpacucli 
HP Array Configuration Utility CLI 8.50-6.0
Detecting Controllers...Done.
Type "help" for a list of supported commands.
Type "exit" to close the console.

==> ctrl slot=0 logicaldrive all show detail

Smart Array P410i in Slot 0 (Embedded)

   array A

      Logical Drive: 1
         Size: 136.7 GB
         Fault Tolerance: RAID 1
         Heads: 255
         Sectors Per Track: 32
         Cylinders: 35132
         Stripe Size: 128 KB
         Status: OK
         Unique Identifier: 600508B1001031303434393834300300
         Disk Name: /dev/cciss/c0d0
         Mount Points: / 131.1 GB
         OS Status: LOCKED
         Logical Drive Label: A01197DF5001438010449840590B
         Mirror Group 0:
            physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 146 GB, OK)
         Mirror Group 1:
            physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 146 GB, OK)

Cool! There are other bits I'll want - I see one of the packages offers SNMP monitoring, which naturally, I'm going to use so Nagios can keep an eye on this server.

A list of the 7 packages in the Proliant Support Pack is in this file, but here's an abridged listing:

Package: hpacucli
Description: HP Command Line Array Configuration Utility
 The HP Command Line Array Configuration Utility is the disk
 array configuration program for Array Controllers.

Package: cpqacuxe
Description: HP Array Configuration Utility
 The HP Array Configuration Utility is the web-based disk array
 configuration program for Array Controllers.

Package: hpsmh
Description: HP System Management Homepage
 Provides HTTP infrastructure for HP Agent & Utility system packages.
Homepage: http://hp.com/go/proliantlinux

Package: hp-smh-templates
Description: HP System Management Homepage Templates
 This package contains the System Management Homepage Templates for all
 hp Proliant systems with ASM, ILO, & ILO2 embedded management asics.

Package: hp-snmp-agents
Description: Insight Management SNMP Agents for HP ProLiant Systems
 This package contains the SNMP server, storage, and nic agents for all
 hp Proliant systems with ASM, ILO, & ILO2 embedded management asics.

Package: hponcfg
Description: RILOE II/iLo online configuration utility
 Hponcfg is a command line utility that can be used to configure iLO/RILOE II
 from within the operating system without requiring a reboot of the server.
Homepage: http://hp.com/go/proliantlinux

Package: hp-health
Description: hp System Health Application and Command line Utility Package
 This package contains the System Health Monitor for all hp Proliant systems
 with ASM, ILO, & ILO2 embedded management asics.  Also contained are the
 command line utilities.

A disk-based backup solution: Bacula, ReadyNAS NV+

I decided to set up a new backup system. I don't like tapes, so I started looking at disk-based backups. I also settled on Bacula for handling the backups. I found I could buy a ReadyNAS NV+ for $380, then 2TB disks for $125... allowing for a cold spare, that means $625 for disks... total cost for one storage unit, about $1000 for 5.4TB of RAID-backed, hot-swappable storage. That's outstanding value!

Better yet, I discovered that some helpful people had compiled and packaged Bacula for the Sparc ReadyNAS units (of which the NV+ is one) so all I need do is install it. Then I can run the Director on my basic HP server, the Storage Daemon on the ReadyNAS, and the File Daemon on the clients to be backed up.

I also figured we'd get better redundancy by using a pair of ReadyNAS units, and writing alternate weeks' backups to each.. so weeks 1 3 and 5 go to nas-1, and weeks 2 and 4 go to nas-2.

After trying 3 different web UIs for controlling the backups, I settled on bweb, which seems to work pretty well (only a few broken bits).

I'm still part-way through the final implementation, but when I get the config all sorted out, I'll post a follow-up. So far, I have one of the ReadyNAS units running the Storage Daemon, and I've got my desktop computer backing up to it as I type. W00T!


At this stage, we're doing simple backups of user's network drives on a range of servers situated at 6 sites, to a central facility. If this scheme works out well, we may look at adding more storage, and expanding into doing bare-metal restore support for critical machines... could be exciting stuff ahead!

Findstr

Wow, so I am a bit out of the loop when it comes to Windows command-line tools. I've often cursed the uselessness of find compared to grep. But it turns out that since Windows 2000, there's been the rather handier findstr, which does regexp searches, recursive searches through files and folders... why, it sounds just like GNU grep!

Full reference details at technet.

Mummy, can we run Linux at home?

While I would generally prefer Linux as a home server, I have to hand it to the folks at Microsoft, with their promotional book Mommy, Why Is There A Server In The House? (download the 24 MB PDF here) - it's funny.

No, I'm still not going to get a Windows Home Server, but it is amusing.

Webalizer: Microsoft Internet Exploder, tee hee

Looking at some web stats for our company's website, I see the most popular UA (user agent) is... "Microsoft Internet Exploder"?? I'm no great fan of Microsoft's browser, but even I haven't called it that for a while... puts me in mind of Bud Uglly's brilliant Alvin's Internot Exploder - if you're not familiar with Bud Uglly, check 'em out. When I used to work in web development, this site would make me laugh until I was almost sick.

Anyhow, I did a bit of digging, and found our webalizer.conf - it's from 2000. I think our site has been hosted there for a while!

Squirrelmail LDAP address book filtering

I recently did some hackery to exclude email accounts that aren't "real people" (things like admin mailboxes) from our mailing lists. That was easy enough, just meant modifying the script that pulls entries out of LDAP and pokes them into the mailman's sync_members script. Cool!

Now to do the same with SquirrelMail. Ran squirrelmail_configure... hmm, no option to set up a filter. That's okay, I'm happy to edit config.php... filter, filter... nope, nothing. Hmmm... yet the doco says you can do it, with a config item called, astonishingly, filter. Reading the doco more carefully, oh noes! The filter config item wasn't added till version 1.5.1, and that's a development version. Bugger! Do I want this functionality more than I want to stay on a stable release? Hmmmm...

Taming netstat with awk

Specifically, I want to see all network connections on a Solaris host which aren't localhost or from the local LAN. Let's start with netstat:

netstat -a -n -f inet -P tcp

Which gives me *everything*. Now let's filter out localhost connections:

netstat -a -n -f inet -P tcp | awk '$2 !~ /^127\.0\./'

Sweet! Now let's get rid of stuff on our local LAN (192.168.1.0/24) and stuff that's in a LISTEN state:

bash-3.00$ netstat -a -n -f inet -P tcp | \
awk '$2 !~ /^127\.0\./ && $2 !~ /^192\.168\.1\./ && $7 !~ /LISTEN/'

TCP: IPv4
   Local Address        Remote Address    Swind Send-Q Rwind Recv-Q    State
-------------------- -------------------- ----- ------ ----- ------ -----------
      *.*                  *.*                0      0 49152      0 IDLE
      *.*                  *.*                0      0 49152      0 IDLE
      *.1017               *.*                0      0 49152      0 BOUND
      *.32832              *.*                0      0 49152      0 BOUND
192.168.1.29.22      10.88.0.90.54670     64128      0 49232      0 ESTABLISHED

Getting closer... let's also remove those idle and bound lines:

bash-3.00$ netstat -a -n -f inet -P tcp | \
awk '$2 !~ /^127\.0\./ && $2 !~ /^192\.168\.1\./ && $7 !~ /LISTEN/ && $7 !~ /BOUND/ && $7 !~ /IDLE/'

TCP: IPv4
   Local Address        Remote Address    Swind Send-Q Rwind Recv-Q    State
-------------------- -------------------- ----- ------ ----- ------ -----------
192.168.1.29.22      10.88.0.90.54670     64128      0 49232      0 ESTABLISHED

Yay. Now I can see my SSH login, and more to the point, see that there are no other connections to this host from outside our LAN, which is what I wanted to check.