Things n' Stuff Thoughts about the universe in general

The following procedure can be used to recover InnoDB database tables from a backup of a MySQL server that had the “innodb_file_per_table” setting but all the metadata (in the “ibdata*” files) was lost – for example due to a server crash.

The process involves two steps:

1. Recover the table structure from the .frm files
2. Recover the data from the .ibd files (InnoDB tablespace)

There is a lot of copying the backed up files over and over to the MySQL datadir, so its useful to have the backup available on the database server machine. In my setup the backup for the databases was copied to the directory “backup” under the database’s datadir, so – for example – for the table somedb.sometable there exist files somedb/backup/sometable.frm and somedb/backup/sometable.ibd.

Additionally the process for recovering the table structures creates a lot of superfluous metadata in the InnoDB data files, so after the first stage I’m going to destroy the InnoDB data files and let the InnoDB engine re-generate them – as a result any existing InnoDB tables will be destroyed. This is important so I’ll iterate: using the procedure detailed here will destroy any existing and working InnoDB databases! So this procedure is useful to recover a destroyed database server to a new server or as a temporary measure on a temporary server to be able to dump the data to SQL files that will later be loaded into an existing server.

There likely a way to do this which is less heavy handed – for example, check out this article from Percona’s MySQL blog, but for my purpose this is enough.

(more…)

When installing Ubuntu on a disk partitioned with a GUID partition table (GPT), the Ubuntu installer doesn’t set the boot flag correctly on the partitions, and on computers with older bioses (like my Intel ICH9 board), the BIOS will not recognize the disk as bootable and will refuse to start the boot loader, claiming that “No bootable device found”.

The solution is to set the boot flag on the boot partition (the real one in the GPT) as well as on the “protective MBR partition”:

1. Start your computer with the Ubuntu live CD again, and select “try Ubuntu before installing”
2. When the Ubuntu desktop loads, open the dash and start “gparted”
3. When gparted starts, select your boot drive and from that select your boot partition – usually the first Linux partition on the drive, and right click it and select “flags” in the menu. In the flags dialog check “boot”.
4. Open the dash again and start “terminal”
5. In the terminal type “sudo fdisk /dev/sda” (or whatever your boot drive is). You’d get a bunch of warning about the disk using a GPT scheme – ignore them. type “a” to set the bootable flag and choose partition “1”. Finally type “w” to write out your changes.

Now its safe to reboot and your old computer should have no problem starting the boot loader now.

The following script I install as a cron job on Amazon AWS virtual machines I deploy, to allow them to backup themselves automatically. The script uses the EC2 management utilities that are normally available on “Amazon Linux” installations (and can be easily installed on other Linux distributions) to create EBS snapshots of the current mounted root EBS volume¹.
(more…)

1. I don’t expect this script to work for instances that have an instance-stored root device, but I don’t expect to encounter these any more [↩]

Nvidia Optimus is a neat solution to the problem of power consumption vs. 3D performance in notebook computers – the computer comes with two graphics chips, an “integrated graphics package” – the Intel GMA embedded graphics and a “discrete graphics package” – the Nvidia chip. The setup works by running your normal windowing UI on the integrated graphics, only powering on the discrete graphics when you want to play a 3D game or something like that.

Recently I had the fortune to work on a Lenovo T420 laptop¹ that has this setup, and it works very well on MS-Windows with the Nvidia Optimus driver – you get battry life around 10 hours with the 9 cell battery.

Unfortunately I spend most of my day in Linux and because Nvidia does not support Optimus on Linux, you have the poor choice of either running everything on the Nvidia chip – decreasing battery life to around 4 hours if you’re careful, or disabling the Nvidia chip completely.

The solution comes from the Bumblebee project – a software suite to handle the switching between the Nvidia discrete graphics and the embedded Intel chip.

The setup is pretty simple to understand (though I suspect under the hood there are many problems to be solved): A service runs and waits for users to ask for 3D accelerated graphics. When a user starts a program using the special command optirun, the service loads the Nvidia driver, starts an X server using the discrete graphics (with the display disconnected from the actual screen) and runs the specified program on that “background” X server. Then it copies the visuals from the program that is rendered using the discrete graphics to a window on the main X server. When the program terminates, the service closes the secondary X server, removes the driver and powers down the graphics card – putting us back into the ~10 hours battery life.

The Bumblebee software had some problems in the past, but the current version – 3.0 – looks very good. There are a few seconds of delay when you launch the application (setting up the driver and X takes some time), but performance is about what you’d expect when running directly on the hardware. All this without any configuration – that is if you are running on the stable Ubuntu version.

As I can’t leave well enough alone, and whenever someone says “alpha”, I say – “I wants”, I’m running the current Ubuntu 12.04 alpha (which is not so bad – due to be released in a couple of months, it works very well). And of course Bumblebee doesn’t work properly here.

So this is what I had to do to get it running:

(more…)

1. with an amazing 1600×900 screen, what they call “HD+”. Truly a work of art [↩]

In my company we (still) use an Active Directory domain controller to manage central authentication¹, which is not set up very well – no SSL and the Kreberos setup is not done properly. This makes gives much trouble to modern Linuxes (e.g. not Ubuntu. yes – I’m looking at you Shuttleworth.)  such as Fedora, as Fedora have done away with NSS/PAM based LDAP authentication and instead relies on SSS – which I have yet managed to get working or even find a tutorial on how to set it up properly.

So if you still want to authentication your Fedora installation against the company’s Active Directory – and can’t/won’t rely on Winbind’s notably flaky behaviour, you can always install NSS/PAM ldap authentication manually. Unfortunately its not as easy as it sounds, and as I learned the hard way – one must pay careful attention to SELinux. So here’s the recipe:

(more…)

1. there are a lot of MS-Windows workstation, so it kinds of makes sense – but we are planning to phase it out in favour of OpenLDAP, so don’t worry about it [↩]

The purpose of this exercise is to convert a hand written note – such as your signature – to high quality digital files that can be used to embed “hand write” into documents – such as when someone asks you to fax them a “signed” copy of the PDF they emailed you. You’d be surprised how often that happens around here.

Required Ingridients:

• A computer with The GIMP installed
• Your handy smart phone with a 5MP or better camera
• A good pen and paper.

So anyway, here’s the process from top to bottom, with pictures:

1. Sit down at a proper table, and using a good black heavy-line pen¹ on a white clean high-density paper, write what you need to write – slowly and deliberately but without pauses. Try not to smear the ink so you get clean continuous lines, otherwise the quality suffers a lot.
   (more…)

1. in this shot I used a 0.7 permanent marker – which is just overdoing it – probably any 0.7 pen will work [↩]

When you want to host a Subversion repository on your CentOS (or RHEL) server, its quite annoying that the only options available to serve your repository are SSH (using the svn+ssh:// schema in the Subversion URLs – it is basically set up out of the box, not tweaking necessary) and Apache’s mod_dav_svn (using the http:// or https:// schema in the Subversion URLs – this is somewhat complex to setup but good instructions are easy to find on the web).

The main advantage of either of these methods is security: both have easy transport security (SSH by default, Apache if you set it to server over SSL) and are easy to setup authentication for (SSH authenticate against the system’s accounts using PAM and Apache authenticate against basically anything with a simple setup).

The main disadvantage of these methods are that they are slow (SSH is apparently somewhat faster then HTTP) and when supporting multiple large projects of many developers I started running into all kinds of weird connection errors when you try to manipulate many files on many projects at the same time.

Subversion itself offers another alternative using their own network service called svnserve – this is a standard unix daemon that listens on a specific port¹ and uses a native protocol to communicate with Subversion clients (using the svn:// schema in Subversion URLs). It offers very good performance, but no transport security (encryption) by default. Another major problem with using svnserve as a network service is that while CentOS ships the binary itself (it is required as part of the way that the svn+ssh:// protocol is implemented) it doesn’t ship any support files to run it as a standalone service nor to help with its configuration. Also by default svnserve can only authenticate users using its own Apache-style password database file – which makes it unsuitable to integrate in large organizations.

(more…)

1. port 3690 by default [↩]

באחד הפודקסטים שאני שומע – Stack Exchange Podcast עם ג’ואל ספולסקי¹ וג’ף אטווד – ספולסקי הזכיר כבר כמה פעמים מחקר שנעשה באוניברסיטת בן גוריון על “גורמים חיצוניים בהחלטות שיפוטיות” שבו החוקר מדד את אחוז ההצלחה בבקשות חנינה בהשוואה לכמה זמן עבר מאז ארוחת הבוקר של השופט ששומע את בקשת החנינה.

ספיציפית החוקר – שי דנצינגר – מצא ששופטים הדנים בבקשות חנינה של אסירים ישראלים מחלקים את הישיבות שלהם לשלושה חלקים המופרדים ביניהם בהפסקות אוכל, ובהנתן שאנחנו מפקטרים החוצה כל שיקול אחר (בין אם הוא לגיטימי או לא – כמו אחוז מזמן המאסר המקורי שהאסיר כבר עשה, חומרת הפשע, השתייכות עדתית וכו’) אפשר לראות בצורה מובהקת ירידה דרמטית באחוז ההצלחה של בקשות חנינה בין תחילת וסוף כל חלק, כשאחרי ההפסקה אחוזי ההצלחה חוזרים לרמתם הקודמת.

(more…)

1. “ג’ואל” זה “אינגלות” של “יואל”, אז זה משעשע אותי לפעמים להגיד “יואל ספולסקי” – אני לא בטוח מה לספולסקי יהיה להגיד על זה אז לא אכתוב כך, אבל אני בטוח שהוא משועשע מזה שחיפוש “Joel” בגוגל מביא את האתר שלו במקום הראשון בעוד ש-Billy Joel אפילו לא בעמוד הראשון [↩]

This piece of script came in handy when I wrote a utility that “recycles” space on a logging partition: before log rotation archives the current log file, we move some old log files (depending on some archive freshness policy) to a remote storage that archives older files.

The problem is that the “old archive storage” also has limited disk space and I got fed up managing the archive by hand. The solution I came up is to scan the hierarchy of  log files in the storage (logs are stored hierarchically according to origin and type) and delete old files until I have enough room to move some newer files in. That way the “old archive storage” is always kept full and keeps as much back-log as possible and does this automatically.

The piece of code that determines which files we want to delete works like this:

1. Use find to list all the files in the directory structure
2. Pipe it to perl and collect all the file names in a list
3. Use perl’s sort operator to compare the modification times of each file in the list and show them in the order (i.e. oldest first)
4. Use head to get just the first file

So it looks like this:

find /mnt/httpd_back/ -type f | perl -nle 'next unless -f; push @files, $_; END { foreach $file (sort { @a=stat($a); @b=stat($b); $a[9] <=> $b[9] } @files) { print $file; }}' | head -n1


Note: normally we use head to get some initial output and terminate the process early before it does more costly work – when head has enough data it terminates the pipe sending SIGPIPE to the upstream process and that usually terminates the process that generates the data. In this case – and in all other cases involving sort – the upstream process buffers all the data in its own memory before outputting anything, so it can sort everything, and using head here is just a filter to get what I want and does not actually save me from doing all the work. I could have easily done the same thing inside the perl script itself by replacing the block of  print $file; with print $file; last; – this has the same effect as using head, because head will send SIGPIPE to perl after getting the first print and will terminate it. Deciding which way you want to go is probably more about readability of the code and I prefer my original version because its easier to read to non-perl specialists.

I can then just remove that file, see if I have enough room to move in the newer log file and if no – repeat the process.

This would work well, I believe, but it may be inefficient if I find a bunch of small files and I want to copy in a large file. So what I did next is to take advantage of the fact that all the log files I have are named using the following simple format:

<service>-<type>_log-<year><month><day>.gz

and that allows me to easily find all the log files that record the same day and eliminate them at the same time. Subsequent moving of additional files will likely succeed because I cleared out all the log files of an entire day. If not, I can always go and clear up another day’s worth of logs.

Update: The latest update for CyanogenMod for Milestone – version 7.1.0.3 solves this problem as well.

Lately I’ve been having problems updating software on my Motorola Milestone (1, as in A853) running CyanogenMod 7.1 (thanks to Nadalbak who maintains an unofficial and unsupported port of CyanogenMod for this old device). When I start the Android Market and do an update of an existing application, I get the error “Package file is invalid” for any application. Sometimes removing an application and reinstalling it will work, but often not.

There could be several problems that cause this, apparently a problem with the file permissions on the file systems is common, and if this is your problem then it can be fixed by opening the terminal emulator, executing “su” to gain super-user permissions and then running “fix_permissions” (it will take a while to complete).

If this doesn’t solve the problem for you, then likely the problem is that you ran out of space on your “data” partition or “cache” partition. This is very common if you have a large “apps partition” on your SD card and you are in the habit of installing tons of applications – I know I am 🙂 .

(more…)