When doing I/O intensive processing on Linux systems, I’ve found that creating a RAM based filesystem can substantially improve processing times. Of course nothing but the transitory processing data should be written to the fake filesystem to avoid data loss in the case of unintended dismount or system crash.

mount -t ramfs ramfs /tmp/ramfs -o size=4m

Soekris Engineering net5501 SBC setup with Linux

2008/09/03

net5501 is a x86 SBC that I ordered with 4 10/100 ethernet ports, 512MB memory, 500MHz Geode LX CPU

Serial console is used for setup of net5501 – BIOS writes to serial port since there is no xVGA port. <ctrl-p> to enter BIOS setup. DB9 pinout:

2 — 3

3 — 2

5 — 5

Use 19,200 bps 8 data bits, no parity, 1 stop

With the Macbook Pro, I use a Keyspan USA-19HS USB <–> DB9 RS232 serial converter (and DB9-RJ45 adapters to implement the null modem configuration and allow me to use an ethernet cable for the serial console <–> Keyspan device.

On OS X (10.5) I use “screen” to provide the serial terminal interface:

$ screen /dev/tty.USA19H1a2P1.1 19200,8

<ctrl-a><ctrl-\> to exit

On the net5501 BIOS, PXEBoot is disabled:

set PXEBoot=Disabled

I setup voyage-0.5.0 on a compact flash card then installed the card into the net5501 – works great the first boot

Default root info: root / voyage

OpenBSD setup info:

http://techblagh.blogspot.com/2008/08/installing-openbsd-43-on-soekris-5501.html

MythTV PVR HDD Mirroring 2008/07/24
Host: n43 (mythtv)
- Two SATA 500GB drives sda sdb
- current production drive is sdb

Problem: I’ve done migrations of LVM2 volumes from 320GB SATA to 500GB SATA and added
a redundant 500GB SATA. Now I want to get software RAID 1 setup to protect the
root, swap and /storage filesystems from damage if/when one of the shiny new 500GB SATA
disks bite the dust.

Followed howtoforge.com linux_lvm_p1 (start of article) to free up sda from LVM
volume group VolGroup00 .. http://www.howtoforge.com/linux_lvm_p7

0. Did a file level backup to the fileserver:
[root@n59 20080724]# ssh root@192.168.1.2This e-mail address is being protected from spambots, you need JavaScript enabled to view it “tar cf – /lib” | dd of=mythtv-lib.tar
(repeat for /boot /storage /var /etc /home)

1. Free up sda2 LVM volume. I know this volume is not used anymore,
but it still has same-disk backup of /storage from when I was tweaking
MythTV.

[root@mythtv ~]# pvmove /dev/sda2
[root@mythtv ~]# vgreduce /dev/VolGroup00 /dev/sda2
[root@mythtv ~]# pvremove /dev/sda2

- now running on sdb only -

Setup RAID 1 mirroring (md)
–
2. Partition sda for mirroring (Auto RAID label)
[root@mythtv ~]# fdisk /dev/sda
<delete partitions>
<add primary 1 whole disk>
<set flag to fd – Auto RAID>

[root@mythtv ~]# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 19 152586 83 Linux
/dev/sda2 20 60801 488231415 fd Linux raid autodetect

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 19 152586 83 Linux
/dev/sdb2 20 60801 488231415 8e Linux LVM

Notice that sdb is still using only LVM, not RAID.

3. Setup md devices – sdb is ‘missing’ – its still in use for LVM so ‘missing’
keyword is used.

[root@mythtv ~]# mdadm –create /dev/md0 –auto=yes -l 1 -n 2 /dev/sda2 missing
mdadm: array /dev/md0 started.

4. Prepare new md volume for LVM

[root@mythtv ~]# pvcreate /dev/md0
Physical volume “/dev/md0″ successfully created

5. Now extend the VolGroup00 vg to include the new md LVM volume

[root@mythtv ~]# vgextend VolGroup00 /dev/md0
Volume group “VolGroup00″ successfully extended

6. Now move data from /dev/sdb2 to LVM volume md0
[root@mythtv ~]# pvmove /dev/sdb2 /dev/md0

<long time>

7. Now we remove the sdb2 volume from LVM
[root@mythtv ~]# vgreduce VolGroup00 /dev/sdb2
[root@mythtv ~]# pvremove /dev/sdb2

8. Now we format /dev/sdb with fd (Linux RAID autodetect): (make it sdb1)
Same as sda.

9. Next add /dev/sdb2 to /dev/md0

[root@mythtv ~]# mdadm –manage /dev/md0 –add /dev/sdb2

10. Now the RAID array will be synchronized. This will take some time, you can check with

[root@mythtv ~]# cat /proc/mdstat

11. Update initrd BEFORE you reboot
If no /etc/mdadm.conf exists, you may choose to create it. It appears that I needed to
do that AND rebuild my initrd to support LVM on RAID1 during boot.

echo “DEVICE /dev/hd*[0-9] /dev/sd*[0-9]” > mdadm.conf
mdadm –detail –scan >> mdadm.conf

mkinitrd -f /boot/initrd-`uname -r`.img `uname -r`

12. Copy /boot over to sda – gets the new initrd (although /dev/sdb1 is still
labeled BOOT, so it is the partition that will be mounted as /boot)
[root@mythtv ~]# mke2fs /dev/sda1
mke2fs 1.40.2 (12-Jul-2007)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
38152 inodes, 152584 blocks
7629 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
19 block groups
8192 blocks per group, 8192 fragments per group
2008 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 22 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

[root@mythtv ~]# mkdir /tmp/newboot
[root@mythtv ~]# mount /dev/sda1 /tmp/newboot/
[root@mythtv ~]# cd /boot
[root@mythtv boot]# find . -print | cpio -pvmd /tmp/newboot
/tmp/newboot/./vmlinuz-2.6.22.4-65.fc7
/tmp/newboot/./config-2.6.22.4-65.fc7
/tmp/newboot/./initrd-2.6.21-1.3194.fc7.img
/tmp/newboot/./System.map-2.6.21-1.3194.fc7
/tmp/newboot/./initrd-2.6.22.4-65.fc7.img
/tmp/newboot/./System.map-2.6.22.4-65.fc7
/tmp/newboot/./lost+found
/tmp/newboot/./grub
/tmp/newboot/./grub/device.map.old
/tmp/newboot/./grub/jfs_stage1_5
/tmp/newboot/./grub/device.map
/tmp/newboot/./grub/ufs2_stage1_5
/tmp/newboot/./grub/e2fs_stage1_5
/tmp/newboot/./grub/xfs_stage1_5
/tmp/newboot/./grub/stage2
/tmp/newboot/./grub/splash.xpm.gz
/tmp/newboot/./grub/grub.conf
/tmp/newboot/./grub/fat_stage1_5
/tmp/newboot/./grub/vstafs_stage1_5
/tmp/newboot/./grub/grub.conf.dist
/tmp/newboot/./grub/stage1
/tmp/newboot/./grub/menu.lst
/tmp/newboot/./grub/reiserfs_stage1_5
/tmp/newboot/./grub/iso9660_stage1_5
/tmp/newboot/./grub/minix_stage1_5
/tmp/newboot/./grub/ffs_stage1_5
/tmp/newboot/./config-2.6.21-1.3194.fc7
/tmp/newboot/./vmlinuz-2.6.21-1.3194.fc7
25716 blocks
[root@mythtv boot]# cd /
[root@mythtv /]# umount /tmp/newboot

13. If you’ve mucked around with the hd ordering in grub, undo it now:
Grub configuration:
# grub
grub> device (hd0) /dev/sda
grub> device (hd1) /dev/sdb
grub> root (hd0,0)
grub> setup (hd0)

14. Reinstall grub to ensure initrd references in grub are updated
[root@mythtv ~]# /sbin/grub-install /dev/sda
[root@mythtv ~]# /sbin/grub-install /dev/sdb

NOTE: I haven’t taken this to the fall back filesystems or having the
/boot filesystem on a RAID1 volume yet, but one should be able to enable
RAID1 for /boot now (create md1 == /dev/sda1 and /dev/sdb1 and setup an ext2 fs
on md1 to hold the RAID1 protected /boot fs). Haven’t tested
that yet, but did get that to work in earlier 2.6.x kernels (Debian and Fedora).

15. Hold breath, have Fedora Core 7 Rescue disk ready, Reboot

16. Celebrate.

Add local redirection of low port to unpriv high port

Remove any existing entries:

iptables -t nat -D PREROUTING –src 0/0 -p tcp –dport 25 -j REDIRECT –to-ports 11025 2> /dev/null
iptables -t nat -D PREROUTING –src 0/0 -p tcp –dport 80 -j REDIRECT –to-ports 8080 2> /dev/null

Add new redirects:
iptables -t nat -I PREROUTING –src 0/0 -p tcp –dport 25 -j REDIRECT –to-ports 11025
iptables -t nat -I PREROUTING –src 0/0 -p tcp –dport 80 -j REDIRECT –to-ports 8080

LVM Notes

I wanted to upgrade the disks in my Linux PVR to a 1TB pair and thus had to migrate from one existing disk (/dev/sda) to the new (/dev/sdb):

1. Add new physical disk to system

2. Partition disk to have a linux LVM partition – use flag 0x8e

# fdisk /dev/sdb

3. Add to LVM

# pvcreate /dev/sdb2

4. Add physical LVM volume to a LVM volume group (VolGroup00)

# vgextend /dev/VolGroup00 /dev/sdb2

2. Move all lvm volumes off old lvm disk

# vgdisplay -v (look for old physical volume name)

# pvmove /dev/olddisk      # will move all physical extents from olddisk to any available pv in the vg

3. Remove old disk from vg

# vgreduce /dev/olddisk

4. Remove old disk from LVM

# pvremove /dev/olddisk

RAID Notes
Debian RAID setup on my PVR:
/dev/md0  /boot
/dev/hda1
/dev/hdb1
/dev/md1  /
/dev/hda2
/dev/hdb2
/dev/md2  swap
/dev/hda3
/dev/hdb3
/dev/md3  /data
/dev/hda4
/dev/hdb4

Show detail of RAID set:
# mdadm –detail /dev/md0

Detach mirror member:
- first mark member as bad (unless is really is bad, in which case it’ll already be marked faulty):
# mdadm –set-faulty /dev/md0 /dev/hdb1
- now remove it from the RAID1 set
# mdadm –remove  /dev/md0 /dev/hdb1

To reattach member (after partitioning, or if it’s the same disk):
# mdadm   /dev/md0  –add  /dev/hdb1
- to watch the progress on the resync, look at /proc/mdstat
# cat /proc/mdstat

I think now (2010/01/24) the faulty syntax is:

mdadm /dev/md0 –fail /dev/sdb1

then

mdadm /dev/md0 –remove /dev/sdb1

Crypto Filesystem Notes

Linux (2.6) crypto filesystems are supported via a loopback device. Various ciphers can be specified.  This example, default AES cipher is used and the disk partition is /dev/sdb1 – which is just setup as a normal Linux (0×83) partition.

1. Load the crypto filesystem module

modprobe cryptoloop


2. Start the crypto device (I’ll insert initialization instructions here later)

Note – you don’t need losetup, if the parameters are specified in fstab and mount does the startup. When losetup runs, it will prompt for the passphrase used to encrypt the partition. Once the crypto driver has the correct key to allow on the fly encryption/decryption, then processes that use the partition see cleartext (such as mount).

losetup -e aes /dev/loop0 /dev/sdb1 || exit 1
mount /bu