Proposal: SPF project for the sysadmin in all of us

Community
1

Hey, all!

I had an idea for a project that I thought the whole SPF community might be interested in. I was thinking that if the community contributed and was successful, and there were no legal / administrative objectives, we could even name this after SitePoint!

The Problem: Many sysadmins rely on several live / Boot CDs to help with issues on their network or to utilize tools outside of Windows on various machines. However, these CDs are read-only and not easily updatable, and their read times can be a little slower. Plus, there’s the hassle of

The Potential Solution / Project: I bought an 8 GB USB drive (a SanDisk Cruzer retractable model featured here). I will attempt to use this drive to have regular file storage in one 4GB partition, and create several smaller partitions, each of which will hold a different LiveCD operating system. These live operating systems will eventually be accessible from a main boot menu that will load when PC boots from the USB drive.

I will post links, trials, and errors along the way, and with your help, hope to turn it into a guide that might perhaps even be submitted as a SitePoint article to help sysadmins.

So, that’s the project. I’m going to get started, posting questions and links. Feel free to jump in! I’d appreciate all the help I can get.

Onwards! :smiley:

2

This drive ships with U3 Technology, but we don’t need it for what we’ll be doing.

To remove the U3 Control panel, I followed these steps:

  1. Loaded the U3 Control panel. This usually happens when the drive is inserted, or you can click on the CD-ROM drive that is created when the USB drive is inserted.
  2. Clicked the U3 icon in the system tray and clicked “settings”.
  3. In the settings screen, clicked “Control Panel settings” and clicked “Uninstall”.
  4. Selected “no” when asked if I wanted to keep the files. Warning: This will format the drive and erase all its content. (no trouble, since it’s already blank)

That’s it! Drive was formatted, U3 removed.

Next steps:

  • Figure out how to put multiple partitions on a USB drive.
  • Figure out how to manage a master boot record (MBR) and the boot records for each partition.
  • Figure out how to coordinate the startup boot list.
  • Install and configure various live CDs in the separate partitions.

This reminds me – any recommendations for which Live / Boot CDs to use? For the sake of this article, I recommend that they be legal for distribution, etc. (no need to promote piracy).

A few I’ve had in mind (please correct me if these are not legal for distribution – I’m under the impression that all of them are):

  • Knoppix
  • Ultimate Boot CD (UBCD)
  • Backtrack
  • Ubuntu
  • Trinity Rescue Kit (TRK)

Okay, so first (and easiest) steps are behind us.

Onwards…

3

Trying to partition the drive

The first step, and what (I hope) is the hardest step, is to find a way to create separate partitions on the drive that can be recognized at boot. As a bonus, it would be amazing if those partitions were recognized by windows as well. So, we have a few different things to try here:

  • Multiple partitions bootable and visible in Windows, OR
  • One main (4GB) partition visible in windows, others visible at boot.

Steps I have taken so far:

Upon reading this link, it purports that windows will not let removable media be partitioned, so we have to “fake” a permanent storage device. The link provides a default driver (since removed) that we’re supposed to edit with a new device id in order to make windows believe it is permanent media. However, the file was removed, and even when I found another one, I unfortunately am on the 64-bit version of windows, and after making the modifications, windows wouldn’t let update the driver because of its purely 32-bit nature.

I have also tried a piece of software called BootIt 1.07. This supposedly let’s you “flip the removable bit” with a button. However, my repeated attempts to use this feature were not successful in getting windows to recognize the drive as non-removable media.

So this is the first hurdle to get over. Anyone have any suggestions?

4

Another question: do I need multiple partitions, or would it be fine to make the drive bootable with its own boot loader and then put the different OSes in separate folders rather than partitions? Is this even possible?

Would appreciate any of your feedback on this.

5

Hey Sean, stick with it as its pretty good so far!
Going to whack you on the front page to see if we cant get a bit more interest :slight_smile:

6

that depends entirely on the boot CD. TRK is specifically designed to do exactly this. just don’t expect to get a WinPE based OS to do this.

7

Hi, the only Windows I have seen successfully booted of a USB is Windows XP. The hack is on Google. Put multiple partitions on the USB and boot each OS off there. You see the Windows XP hack here: http://www.winusb.de/index_en.html

Although in most cases we need Windows Server 2003 so we would have to boot it from a USB connect CD drive.

8

Hi all,

Thanks for the interest! I think some may misunderstand my task at hand:

I’m trying to create a setup so that when the USB drive boots, it presents me with a menu of which live CD / operating system I’d like to boot up from the USB drive. Upon making a selection, it will find the boot record for the partition that the chosen USB operating system is on and go from there.

I’m not worried about booting up windows, as these will all be rescue disks / linux disks to run from USB. There may be one windows-type build in there, but the focus isn’t on a windows boot.

The question here is really whether I need separate partitions in order to make that boot selection still be able to happen. I’d love any insight on that.

9

Thanks for the insight, longneck. Do you mean that TRK is design to boot multiple operating systems, or designed to partition flash drives? I’ve never seen it do either so I’d love to get into that if you have experience with it.

In terms of linux partitioning the USB device – that seems fine, except ideally I would like all partitions to show in windows so that I can add / remove files as updated versions of these OSes come out.

10

Because most of the distros I’m looking for are Linux-based in some way, I’m currently looking to set up GRUB and boot with it from my flash drive. Hopefully, this should allow me to load other images for OSes in the current file structure (no partitions required).

Will update on this latest adventure soon.

11

[B]Latest Steps: Finding a boot manager and installing it to the Master Boot Record (MBR)

[/B]In order to boot multiple operating systems from the USB drive, we clearly need to have something to manage those choices and load them.

The two I’ve heard about so far are [I]syslinux[/I] and [URL=“http://www.gnu.org/software/grub/”][I]GRUB[/I].

I’ve read good and bad things about both, but they each seem to accomplish our purpose, and so I arbitrarily picked GRUB. (if this route doesn’t work out, I’ll soon be attempting with syslinux).

[B]Formatting the USB drive

[/B]Just for the record, I formatted the USB drive in FAT32 from Vista. The USB disk must be formatted with FAT (not NTFS) for this step to work.

[B]Downloading GRUB and the utilities to install it (grubinst & wingrub)

[/B]Because I’m on a Windows Vista x64 install, I needed to snag a version of GRUB that was windows-friendly. I found grub4dos and its GUI files at http://sourceforge.net/projects/grub4dos/files/.

To make things as easy as possible, I downloaded programs with GUIs wherever I could:

  • I downloaded grubinst for windows from this SourceForge link. (link downloads the file.) Grubinst will allow us to install GRUB on the Master Boot Record (MBR) of the USB drive.
  • I downloaded wingrub from the same SourceForge site here. (link downloads the file.) WINGRUB will actually place the GRUB installation onto our USB drive.

[B]Using grubinst to place GRUB into the MBR

[/B]With the completely blank / formatted USB drive, I did the following:

  1. Extracted the grubinst ZIP file, and ran the “grubinst_gui.exe” file contained within it.
  2. In the “device name” section, I selected “disk” and from the drop-down, selected my USB device based on it’s size. WARNING: Be sure to select the proper drive; selecting your hard-drive instead of the USB could render your PC unable to boot!
  3. I selected the “grub2” checkbox, as this is the latest version of GRUB – not grub legacy.
  4. I set the timeout textbox to 30 as I like a little extra time to decide before the boot manager picks the default operating system.
  5. I hit the “Test” button to ensure there are no errors. A textbox pops up showing what the command line command would be.
  6. I then hit “Install.” A command prompt box pops up saying “The MBR / BS has been successfully installed.”

At this point, if you view the drive in windows explorer, it is still empty – we must add the GRUB files for the MBR to find them.

[B]Using WINGRUB to install GRUB to the USB stick

[/B]

  1. I extract the Wingrub ZIP file.
  2. I install the Wingrub0206.exe file (it’s a normal windows install, goes very quickly)
  3. I ran the “WinGrub GUI” application by searching for it in the start menu (program for me is located in “C:\Program Files (x86)\WinGrub\WINGRB.exe”)
  4. The program presents me with a list of drives. I select my USB drive. WARNING: Again, be sure to select the proper drive! I select “copy stage files” as well, and then hit okay.
  5. As this is a technical test, I didn’t mess with the visual elements, but you can click on the colored boxes on screen to set your GRUB screen color preferences if you prefer.
  6. Ignoring all of the other form elements, from the “Tools” menu, choose “Install Grub”.
  7. Select “Boot from MBR” and in the partitions drop-down list, choose the only one that is available – for me, it’s hd(1,0).
  8. Just to be safe, under “System Drive”, I set it to G (the current drive letter of my USB drive). I don’t think this section is used in this case, but having C selected for anything made me nervous.
  9. Click the “Install” button.
  10. The dialogue box goes away. This indicates a successful install.

In windows explorer, the USB drive now shows two files in the root – “GRLDR” and “MBR.ORG”, as well as a “grub” directory with its supporting files.

Create a “boot folder” to store your various OSes

I literally just created a folder in the root of the drive called “boot”. I plan to store each bootable section in its own folder off of this drive.

Testing it out so far

  1. I put the USB drive in a laptop that is capable of booting from a USB device.
  2. The GRUB screen comes up with one and only menu item “Windows at (0,0)”. I like this, because it is the way to load the windows OS that is currently on the local hard drive disk (assuming windows is on your first hard drive in the first partition).

OSes we will attempt to install (to see how they work, assuming some won’t):

  • BackTrack 4 pre-release
  • BackTrack 3 (usb stick version)
  • Knoppix 6.0.1
  • Ophcrack
  • Puppy Linux
  • System Rescue CD
  • Trinity Rescue Kit
  • Ubuntu 9.0.4 Desktop

Next steps!

  • Choosing our first OS (I will try to research which is the simplest so we can start there)
  • Downloading the OSes
  • Copying the OS files to the USB drive in the proper locations
  • Configuring GRUB’s menu file to display those OSes.
  • Ensuring that our target OSes start and function properly.

[B]Let me have it!

[/B]As always, this is a learning process for me, and I’d love advice / tips while going through this (particularly from linux & GRUB users).

Please respond to this post with questions / suggestions, etc. and I’ll do my best to make sure we’re all learning together.

12

TRK is designed specifically to boot from an image file on a USB stick. it uses GRUB. it is also designed to do partition management. so, the answer is both.

TRK can create FAT and FAT32 partitions.

13

I think the issue though is that I want to boot a bunch of OSes that use GRUB. So, instead of installing one that uses GRUB, I’m installing GRUB to the MBR and then modifying the GRUB menu listings for those OSes so that they’re listed universally in GRUB and accessible. That way, I can add / remove OSes much more easily.

TRK can create FAT and FAT32 partitions
Correct; there are two ways of doing it here. The first is to make a separate partition for each OS and leave the first partition as your windows-viewable one (windows only accepts / views one partition on “removable media”). However, I’d like to view all the OS contents in windows, which means one partition.

Thank you for the comments though. I appreciate the feedback; please keep it coming!

14

Resurrecting this old, old project :slight_smile: Sorry, I’d gotten busy (work, etc.)

Okay, so I’ve learned a little bit about the process since then.

My strategy is now:

  1. Install GRUB to the MBR (Master Boot Record) of the flash drive, with supplementing files in /boot/grub
  2. Install various linux kernels to /boot/kernel
  3. Install various OS images to /boot/img/[System Name]
  4. Edit menu.lst to add the various OSes & test them, referring to the kernels and images as needed (removing duplication of kernels where necessary)

So far, I’ve gotten #1 accomplished. I can see GRUB start up, and I can see it read a dummy menu.lst entry.

Next steps:

  • Need to figure out which kernels each of the above-mentioned OSes uses.
  • Need to download those various kernels
  • Need to add unique kernels to /boot/kernel

Questions:

  • Any thoughts on ways to deduce what kernels the various OSes use? usually they’re renamed to “linux” or “vmlinuz”, it seems (no version, etc.)
  • Is there a central place to download pre-compiled linux kernels, or when I figure it out, should I just take them from the various distros and rename them?

Lots of questions – hoping everyone can help me find the answers! :smiley: I’d like to turn this into the ultimate how-to for this subject when done, so that everyone can benefit.