Notes from installing OpenSolaris snv_74

I now have Solaris up and running and reasonably stable-looking, after only 12 hours of work. A number of things turned out to be bigger issues than I’d anticipated, largely because it’s been years since I last used Solaris and, frankly, Solaris’s disk partitioning and formatting tools suck.

  • My first problem is still unresolved: my BIOS refuses to boot from the IDE DVD drive that I installed. Once the system boots, it works just fine, so I’m not sure what’s up. Maybe a BIOS bug. Fortunately, the system’s perfectly happy booting off a USB DVD drive, and (amazingly) Solaris is happy installing from it.
  • The GC-RAMDISK card that I was looking forward to testing is a complete failure so far. I don’t know if I have a bad card or if it’s simply incompatible with both SATA chips in the system, but the BIOS completely ignores it if its plugged into the motherboard, and Solaris fails to talk to it on either bus. If it’s plugged into the MB, then I get a device failed initialization error; if it’s plugged into the PCI-X SATA card then I get device on port 5 still busy after reset. I’ve swapped cables and RAM. I’d really like to get it to work, so I’m going to try it with an older system before RMAing it.
  • Actually getting a working Solaris install took me 3 tries. The first time I installed it to the wrong drive (the first disk on the PCI-X card, not the first disk on the motherboard), and it was unable to mount the root partition after rebooting. Next, I managed to install it onto an EFI partition, and that wouldn’t boot either. Finally, I installed it onto the second drive on the right bus, and that worked.
  • Since Solaris’s installer doesn’t support ZFS yet, I had to manually copy the root filesystem onto a newly created ZFS filesystem mirrored across a pair of drives. The directions are helpful, but I kept screwing things up. First, I accidentally created the ZFS pool using the entire disk, which made ZFS re-label the drives with EFI, which makes them unbootable. Then, I missed the line in the directions that says to run format -e instead of using format; that left me with a pair of nicely partitioned drives that still used EFI. The third try worked, and the system is now booting off of ZFS via GRUB without problems. Er…
  • Well, one problem–I can’t change the GRUB menu.lst file for some reason. I don’t know where GRUB is looking for it, but it’s not in /boot/grub/menu.lst on my boot array. My changes are being completely ignored. I can live with this for the weekend.
  • OpenSolaris doesn’t ship with drivers for the ASUS P5K WS’s onboard Ethernet chips. I had to grab them from Marvell, but it was easy enough to install.
  • Creating an 8-drive ZFS filesystem is trivial. One command takes care of RAID, logical volume management, creates the filesystem, and mounts it: zpool create -f space raidz2 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c3t2d0 c3t3d0 c3t4d0 c3t5d0.
  • ZFS performance is decent. Here’s a bonnie with and without ZFS compression, using 10 GB of data on a box with 2 GB of RAM:
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU  /sec %CPU
zfs        10 105.0 55.3 163.3 27.3 121.0 30.4 119.2 88.4 287.1 36.2   169  1.8
zfs+c      10 112.9 59.7 181.5 30.3 127.8 29.1 118.1 86.0 424.9 52.2   198  2.1
  • 163 MB/sec writing and 287 MB/sec reading is good enough for me. I was expecting slightly higher numbers, but there’s nothing here to complain about. Adding compression improves writing a bit and makes a big difference reading. It’s quite a bit faster then GigE, which was my goal.

Posted by Scott Laird Sun, 21 Oct 2007 03:33:00 GMT

Read full article 7 comments


Comments

  1. Dick Davies about 9 hours later:

    The /boot/grub/menu.lst you can see is not the true /boot/grub/menu.lst.

    Last time I tried it, the ZFS root hack involved a little ufs partition with GRUB installed that ‘chainloaded’ from the zpool.

    You need to mount up the grub slice and edit the file there to make changes.

  2. Tim Foster 1 day later:

    With the current (actual) ZFS root bits, the grub menu.lst is at the top level of the pool containing your bootable filesystem.

    So, if you have tank/nv_70b as your root dataset, you’ll find the grub menu.lst file in /tank/boot/grub/menu.lst

  3. Scott Laird 1 day later:

    Ahhh. That makes perfect sense. Thanks, it seems to be working now.

  4. Charles Darke 4 months later:

    any update on how this is going? i’m thinking of zfs for storage too.

  5. tom hall 5 months later:

    Not sure bonnie is a great test though. IIRC the prefetch stuff in ZFS gives results too good to be true, we need a more realistic test.

    Tom www.thattommyhall.com

  6. tom hall 5 months later:

    zpool iostat -v 3 is how I get a view of real IO figures.

  7. peter jones 10 months later:

    how one can become superuser or root. imean in opensolaris 05/08 not in solaris 10 or other versions. this i have installed (05/08) is so draconian. it is like living in paris in 1942. with every move it asks for password and username. it logoff automatically. it locks things. it does not allow moving around or configuring things. there scores of agents with strange aliases catching you. you have certain paths and have to live in a certain home. nobody answers you as a help menu, except snapping useless and orchestrated trivial sentences, asking you to declare your consent and happiness. it is so metalic and limited. you can only use the humble firefox to go the internet. it is after that you have installed it with your own hands with ten times formatting in different schemes and guessworks and reformatting. It doesn’t allow me to edit menu.lst or removing my own password or anything. It needs a phd to know how to run it and decipher documents prepared without even the writter of the document check the very operating system under his nose. PJ