Strange Aeons

sesquipedalia verba
Simon Wells
Simon Wells

Simon Wells is an academic researching Argumentation Theory, Automated Reasoning, Intelligent Agents (IA) and MultiAgent Systems (MAS) all approached through a ludic lens.

He is particularly interested in how these tools can be usefully applied to critical literacy, security, gamification, and complex systems.

Twitter

My Other Sites

Affiliates

Tags

Archive

2011 (12)
2010 (27)
2009 (3)
 

Debian Install with RAID 1

Whilst I use LVM without RAID to manage the drives on my media server, because the media rarely changes so the offline backup is easy to keep in sync, I have started using RAID 1 to mirror the contents of my file server to guard against the failure of a single drive. This is because the data changes sufficiently often on the file server that my regular backup might get out of sync by up to a day which could lose me a days work.
  1. Using the Debian alternate install CD, at the partion disks screen, select manual
  2. Select the free space on your first drive and create 3 new partitions as follows: 2GB for /boot, 2GB for SWAP, Remainder for /
  3. Select "physical volume for RAID" at this point and ensure that the boot partition is marked as bootable.
  4. Repeat the above steps for the second drive
  5. Now select "Configure software RAID" from the main partition disks screen
  6. Create 3 Multidisk (MD) devices with each configured as RAID 1 with 2 active and 0 hotspace disks, one MD device for each of the boot swap and root partitions. Select the correct corresponding matched partitions to include in each MD device, e.g. sda1 and sdb1, sda2 and sdb2, sda3 and sdb3
  7. After returning once more to the main Partition Disks screen you should see the RAID devices that you just created listed alongside the partitions from before. Create file systems on each RAID device in the same fashion as for normal, non-RAID, partitions, e.g. RAID device #0: type ext3 mounted at /boot, RAID device #1: type SWAP, RAID device #2: type ext3 mounted at /
  8. Write the changes to disk and continue with a normal installation
After installation you must also install grub to the master boot record on the second drive in the RAID array so that the system can boot from the other drive in the case of one drive failing.
$ sudo grub-install /dev/sda $ sudo grub grub> device (hd0) /dev/sdb grub> root (hd0,0) grub> setup (hd0) grub> quit
Verify that you have the correct filesystems in the correct places, e.g.
$ grep /dev/md /etc/fstab $ df -h /
The status of RAID devices can be checked using the /proc/mdstat file. Each mdN device contains two sdXN disks and each mdN device should have "2/2" and "UU":
$ cat /proc/mdstat
The mdadm utility gives more details:
$ sudo mdadm --query --detail /dev/md0
Posted
 

The Watch Command or tail for directories

I had a long running job copying some huge media archives from external storage to a new server. I wanted to periodically keep tabs on the process and as I was using cp to copy the data I made sure to use the -v argument so that there was some feedback to the terminal to tell me how things were progressing:
cp -Rv usbhdd/ /home/media/
Whilst doing this I thought that it would be cool to have a version of tail that worked for directories rather than just for files. Well it turns out that you can, after a fashion. There is the watch command which gives similar functionality by causing a command to be executed periodically and the results displayed fullscreen:
watch -n 10 "ls -la"
This will cause the ls -la command to be executed every ten seconds and the result displayed. Whilst finding out about this I discovered that the watch command isn't on every system and that the following script can be used to provide the same functionality: 
#!/bin/sh
while (true)
do
     ls -lrt | tail -5
     sleep 5
     clear
done
Which works but causes the screen to clear each time which is distracting, and doesn't have the niceties of the watch command like using the -d differences flag to show how the output changes between successive updates.
Posted
 

Logical Volume Management on Ubuntu

An aide memoire to myself for setting up Ubuntu with LVM at install time:
  1. At the disk partitioning screen select Manual
  2. Select a drive (if they are new drives then they will require partition tables)
  3. Select free space on the drive and choose to create a new partition. First off we need a /boot partition of 100MB which is of the primary type and starts at the beginning of the disk. Select the /boot mount point for this partition and ensure that you have selected the bootable flag.
  4. Repeat the last step to create a SWAP partition of about twice the size of your RAM.
  5. The rest of the disks holding the /boot and SWAP partitions can now be used to hold an LVM containing the / filesystem for the machine.
  6. Select the free space on your first disk.
  7. Choose to make it a physical volume for LVM
  8. Repeat this step for any other drives that will be part of the LVM
  9. Select "Configure the Logical Volume Manager"
  10. Select "Create volume group"
  11. Give the group a name - this can be anything but should be easy to remember. I just used machinename_vg1
  12. Select which physical volumes will be part of the volume group
  13. Select "Create a logical volume"
  14. Select the volume group that will contain the new logical volume
  15. Name the logical volume, e.g. machinename_lv1
  16. Enter the size for the logical volume
  17. Select finish
  18. In the main menu select the volume group
  19. Select to use the volume as Ext3
  20. Select to mount the volume as the / file system
  21. Finish the regular Ubuntu install
Of course one gotcha with LVM appears to be that unless you are combining the volumes with RAID then you could lose your entire logical volume, spanned acroos all of the disks, if you lose a single physical volume to a hardware fault. I would like to be proven wrong on this, and don't want to find out the hard way so I don't currently have data solely in an LVM file system - I merely use it as a convenience for a live file system that is completely backup up elsewhere.
Posted
 

Zen & the Art of the Computer Desktop

Media_httpwwwstrangea_ddujr
I was fed up with all of the mounted network drives appearing on my desktop so finally got around to turning them off. This is my default setup on my macbook but I hadn't set the same behaviour on Ubuntu. Luckily I was spurred to fix this by an article on zenhabits. The process is straightforward:
  1. Start the Gnome Configuration Editor $ gconf-editor
  2. Navigate to /apps/nautilus/desktop/
  3. Set the volumes_visible tickbox to false to turn off the visibility of mounted volumes
You can go further and turn off all of the desktop icons which is very neat but interferes with conky which is bad as conky is my new favourite thing at the moment:
  1. Start the Gnome Configuration Editor $ gconf-editor
  2. Navigate to /apps/nautilus/preferences/
  3. Set the show_desktop tickbox false to turn off the visibility of files stored in the /home/$user/Desktop directory.
Posted
 

Converting an AVI into a DVD on Linux

Media_httpwwwstrangea_iigbj
Not everyone can watch media on their computer (apparently not everyone even has a computer -- who knew?). Anyway, I had occasion to need to convert some media stored in a good quality avi ino a DVD that would play on an average standalone DVD player. This is quite straightforward with Linux and relies on a few small and useful command line tools. You will probably find many GUI tools that do the same thing but I like to have a scriptable command line method, and anyway, the GUI tools are usually just front ends to the command line tools but with less functionality. The process is straightforward; first convert the avi into an mpg using ffmpeg:
ffmpeg -i in.avi -y -target pal-dvd -sameq out.mpg
Now create the DVD file structure:
dvdauthor --title -o dvd -f out.mpg
Build a table of contents:
dvdauthor -o dvd -T
Now convert the DVD structure into an ISO which are easier to handle and archive so that you can quickly run off a physical disk as required:
mkisofs -dvd-video -o dvd.iso dvd/
Posted
 

Apache Virtual Hosts Redux

Media_httpwwwstrangea_gsxua
This is mostly a reminder to myself and is an addendum to my earlier post on Apache VirtualHosts which seem to be my current most favourite thing in the realm of web-servery stuff. They make it much easier to migrate sites between servers, minimising downtime whilst maximising lack of link breakages. The process for adding a new virtualhost,  like I have just done for a subdomain of simonwells.org, is quite simple: 1. Create a directory in /var/www/ for your new site. 2. Add a docs subdirectory to your new directory from (1). 2a. (Optionally) Change the owner of your new directory to that of the person who will be editing the site.
$ sudo chown -R simon:simon simonwells.org
3. Create an entry in /etc/apache2/sites-available by copying an existing entry and editing it to suit your new site. 4. Create a symbolic link in sites-enabled to enable the site, e.g.
sudo ln -s ../sites-available/notes.simonwells.org notes.simonwells.org
5. Restart Apache 2.
sudo /etc/init.d/apache2 restart
You should now be able to access your new Apache 2 website or at least the space that it will occupy when you add some content.
Posted
 

Telly-On-The-Go With N810 Media Encoding

Media_httpwwwstrangea_jiijp
After my rail trip earlier this week where I had free, but so slow as to be unusable, WiFi I resolved to load some media onto my N810 to help pass the time on future journeys. Early experiments showed that the N810 can play the average xvid avi downloaded from your favourite torrent site. However it didn't necessarily play them well. Firstly, you need to install mplayer which gets better performance out of the hardware than the stock media player. Then, so long as the scene isn't high-motion you can watch your tv show, or movie, or whatever. To get good performance though really needs a re-encode, especially to play media downloaded from iplayer (maybe via my earlier technique for timeshifting British TV) which is unwatchable. The best solution that I have found is tablet-encode which provides mencoder profiles for the N810 and is command-line based so is ripe for scripting. Usage is simple, as all good software should be:
tablet-encode tablet-encode --help tablet-encode --preset list tablet-encode input.avi output.avi tablet-encode --list file1.avi file2.avi file3.avi tablet-encode --preset best input.avi output.avi tablet-encode dvd: film.avi tablet-encode --preset best dvd://1 dvd://2 dvd://3 media/ tablet-encode --gui input.avi output.avi
Tablet-encode can be downloaded from Maemo.org.
Posted
 

Forwarding X over Secure Shell

Media_httpwwwstrangea_ggdpv
I knew that I could do this but have never, until now, had a need to do it. This afternoon I needed to access documents on a secure webserver on campus that was restricted to on campus IP addresses. The easiest thing that I could come up with to access them was to shell into the server in my office then access the documents using Links. Unfortunately the webserver didn't like Links and rather than waste time finding a workaround I did the following, logged into the remote server using ssh with the -Y argument which enables trusted X11 forwarding, then I started up Firefox, e.g.
$ ssh -Y simon@192.168.0.1 which should, after passwords and other stuff, give me a shell on the remote machine, on which I can execute: $ firefox &
Which gave me a nice local GUI to the instance of Firefox that I had just started running on the remote server. Again, a simple technique that I am posting here mostly as an aide memoire for myself as it is the kind of thing that I forget about when I most need it.
Posted
 

Upgrading to Wordpress 2.7.1

Media_httpwwwstrangea_jnekg
With the move to a new server I took the opportunity to upgrade to the latest wordpress which is 2.7.1. The most visible changes appear behind the scenes, for example in the administrative and editorial interfaces which look very smooth and polished. After a few moments of wondering "where has that gotten to?", it all fits together and I am quite impressed at the moment. Additionally, I am starting to get the install of a wordpress site working swift and smooth now. In fact it is one of the reasons that I have stuck with wordpress over the last few years whilst many of my colleagues have moved to Drupal or Plone; because of the famous five minute install.

Wordpress Install Procedure

Assuming all of the prerequisites are met on your server: Download and unzip the WordPress package, e.g.
$ wget http://wordpress.org/latest.zip
Create a database for WordPress on your web server, as well as a MySQL user who has all privileges for accessing and modifying it, e.g.
> mysql -u root -p > create database wordpressDB; > grant all privileges on wordpressDB.* to "wordpressUser"@"localhost" identified by "secretpass"; > flush privileges; > exit
Rename the wp-config-sample.php file to wp-config.php and edit it to reflect the database information from step 2. Run the WordPress installation script by accessing wp-admin/install.php from a browser. A useful addendum to this, if you are installing on a minimal Debian or Ubuntu install then you might have to also install a mail transfer agent, this is as simple as doing:
$ sudo apt-get install postfix
then answering a few questions about your install. NB. These instructions are to get a basic install up and running as rapidly as possible and don't necessarily represent all of the steps necessary to get a secure production site up and running. Addendum: I am in the process of moving all of my websites into Apache virtualhosts so that it is easier to migrate them in the future once they are no longer attached quite so strongly to a physical server address. The process for moving a wordpress blog is fairly straight forward, either use the export options within Wordpress or dump your MySQL database to a file. If your site is quite large then it may exceed the 2MB default of Wordpress backups. You can fix this by editing your php.ini file to increase this size which is actually the limit that PHP puts on downloading and uploading files by default. Either way you should end up with a large text file containing the text of your site. You can then open this in a text editor and do a replace-all of the existing server URL with the URL of your new virtualhost. You can now upload the edited backup to your new wordpress installation as usual. All of your links should work correctly at this point, although you should check, and this process should not need to be repeated unless you change the URL of your site.
Posted
 

Searching a file hierarchy for strings

A colleague asked today how to search a set of directories for files containing a particular string and this is what I came up with:
find . | xargs grep gnu
where gnu was my search term. The find command with the dot alias as input recursively descends the file hierarchy from the present working directory. The output of find is piped into xargs which is a very useful little utility that takes the input from find as a long list and splits it into sub-lists, calling grep for each sublist in turn, where in this case each sublist is a filename that grep searches. Obviously he was looking for a different search term than gnu and he was only wanting to search particular file types but this was the basics of his solution.
Posted
Posterous theme by Cory Watilo.
​