Archive for the ‘Software’ Category

Script Day: SSH to a host behind a NAT

I use SSH daily to work with different remote services, and its always a very straight-forward process… unless the remote server you want to work with is on LAN somewhere behind NAT1. When you need to access such an internal server, the only option is to SSH into the firewall2, and then SSH again to your server of choice.

But there’s a better way, and you don’t even have to fiddle with the firewall server!

(this is not actually a script, though minimal text editing is required)

The solution is actually quite simple: set up an alias in your .ssh/config file that you can use to call the remote server when you are outside the LAN (if you are inside the LAN its better to access it directly), and for that alias we will set up a ProxyCommand that will tell SSH to first access the firewall server and open a tunnel to the target LAN server.

It looks like this:

Host remote-alias
ProxyCommand ssh firewall-user@firewally-server nc lan-server 22

This set up works best if your access to the firewall-user account is without password or passphrase (using an SSH private key that is either without passphrase or already loaded in the agent), then the login is as streamlines as a direct access – but the worst is that you’d need to type in two passwords.

Enhanced by Zemanta
  1. router that does Network Address Translation so the servers address is not accessible from outside the LAN []
  2. or some other server that has legs both inside and outside the LAN – I’m using a DNATed server, what most off-the-shelf routers incorrectly call “DMZ []

Open letter to Karen Sandler and Bradley Kuhn of “Free as in Freedom”

A public response to “FaiF oggcast 0x43“, or at least to the second section (after the FOSDEM talk recording starting at 0:54:31):

Sometimes you guys just drink your own cool-aid, while being completely out of contact with reality. This is a prime example – in FaiF 0x43, at 1:12:40 Bradley says “Proprietary Javascript … is the new form of [vendor] lock-in, and People are locked in to GitHub”, which is just the worst of a large tirade where Bradley just compares people who create useful services and want to finance them – while contributing to open source and free software – to the worst proprietary software vendors, for the crime of creating their website with unlicensed Javascript code. All the while Karen just says “yes”.

(more…)

Recovering InnoDB Tables In MySQL 5

The following procedure can be used to recover InnoDB database tables from a backup of a MySQL server that had the “innodb_file_per_table” setting but all the metadata (in the “ibdata*” files) was lost – for example due to a server crash.

The process involves two steps:

  1. Recover the table structure from the .frm files
  2. Recover the data from the .ibd files (InnoDB tablespace)

There is a lot of copying the backed up files over and over to the MySQL datadir, so its useful to have the backup available on the database server machine. In my setup the backup for the databases was copied to the directory “backup” under the database’s datadir, so – for example – for the table somedb.sometable there exist files somedb/backup/sometable.frm and somedb/backup/sometable.ibd.

Additionally the process for recovering the table structures creates a lot of superfluous metadata in the InnoDB data files, so after the first stage I’m going to destroy the InnoDB data files and let the InnoDB engine re-generate them – as a result any existing InnoDB tables will be destroyed. This is important so I’ll iterate: using the procedure detailed here will destroy any existing and working InnoDB databases! So this procedure is useful to recover a destroyed database server to a new server or as a temporary measure on a temporary server to be able to dump the data to SQL files that will later be loaded into an existing server.

There likely a way to do this which is less heavy handed – for example, check out this article from Percona’s MySQL blog, but for my purpose this is enough.

(more…)

How to fix “No bootable device” after Ubuntu installation

When installing Ubuntu on a disk partitioned with a GUID partition table (GPT), the Ubuntu installer doesn’t set the boot flag correctly on the partitions, and on computers with older bioses (like my Intel ICH9 board), the BIOS will not recognize the disk as bootable and will refuse to start the boot loader, claiming that “No bootable device found”.

The solution is to set the boot flag on the boot partition (the real one in the GPT) as well as on the “protective MBR partition”:

  1. Start your computer with the Ubuntu live CD again, and select “try Ubuntu before installing”
  2. When the Ubuntu desktop loads, open the dash and start “gparted”
  3. When gparted starts, select your boot drive and from that select your boot partition – usually the first Linux partition on the drive, and right click it and select “flags” in the menu. In the flags dialog check “boot”.
  4. Open the dash again and start “terminal”
  5. In the terminal type “sudo fdisk /dev/sda” (or whatever your boot drive is). You’d get a bunch of warning about the disk using a GPT scheme – ignore them. type “a” to set the bootable flag and choose partition “1”. Finally type “w” to write out your changes.

Now its safe to reboot and your old computer should have no problem starting the boot loader now.

Powershell still sucks

Powershell is a great command line shell, if you all you know is cmd and batch. There are so many things it is missing when trying to compete with current Unix shells such as Bash, and while some of them have semi-working workarounds, many are sorely missing.

My pet peeves are:

  • A decent pager. “more” is basically at the same stage it was when I started working in MS-DOS 3.30, and it is nowhere near the functionality of “less”1.
  • Persistent history. I’ve seen some workarounds but couldn’t get any of them to work properly.

Both of these features have been available to me since I started working with Linux in 1995, and it is really difficult living without them in MS-world. A decent terminal emulator will be nice too – the Powershell box has advantage over the cmd.exe box in that it is blue – other then that they are both in the same sorry state that the “dos box” of Windows 3 fame was at. I’m using “Console 2” to get some useful work done, but it too leaves much to be desired.

Also, startup is so.. fscking.. slow.. Starting Powershell on a brand new machine (with no per-session user scripts) can take as much as 3 seconds. Those are minutes of my life everyday that I would never get back.

  1. and I’m not talking about the built-in editor, just being able to “page up” would have been nice []

Canonical announced a new display server – Mir, and it is good for the consumer

Canonical have last week announced that they are developing their own display server to replace the ubiquitous X display server, a project called Mir, and the shit storm has begun anew(as what happened after Unity, Ubuntu Touch and other Canonical announcements). Contrary to popular belief, I think that this happening is a good thing for the Linux community in whole.

There are many reasons why I think this is good, most are not really concrete technical things, but I can list a few:

  • X11 is showing its age. There were some internal efforts to modernize it (e.g. kdrive which have mostly merged into the existing code) and some external efforts to replace it (Fresco and Wayland to name a few), but none have made much of an impact on the current state of Linux display.
  • From first look, Mir is a modern code base written in C++11 and Boost, which I like.
  • Diversity is generally a good thing.

If we go over the last point in a bit more depth, I think we can see why Mir would generally be a good thing for Linux developers and users and why people should stop being negative.

(more…)

What is Windows XP for you?

I just “love” this quote from this PC-Magazine article:

… the amazingly enduring Windows XP—easily Microsoft’s most successful enterprise product ever…

Which is a really cock-eyed way to look at the operating system market, which completely ignores purchasing decisions by millions of users world-wide. A better description of Windows XP might be:

The last reasonably well-made operating system that Microsoft made

Users aren’t migrating from Windows XP because its so good1 – they keep using Windows XP because every later OS is really bad.

  1. hint: it isn’t []

TurkTrust CA certificate breach and what does it mean for you

A pseudo analysis of the issue that was brought to my attention by Eric Klien’s post here. The following text is slightly edited version of my comment on the post, reproduced here because I think its important for my readers to be aware of the issue.

A new CA kerfuffle has broken out yesterday, and here are some of the headlines:

The information for the discussion below was sourced from these, more technical, articles1:

To summarize, the problem was a botched test process in TurkTrust CA (as part of an external security audit) that caused a CA profile to be set up to generate “sub CA” certificates, and following that the profile was copied to the production system and subsequently used to generate two certificates before the problem was discovered and fixed (I assume the test profile was removed from the production system), but only 1 of those certificate was revoked. (more…)

  1. I applaud BBC for trying to present a complex security issue in “layman terms”, but as someone who is familiar with the technology in question, it gave me quite a headache, trying to “reverse translate” the text []

Script Day: Automatically backup your EC2 instance using snapshots

The following script I install as a cron job on Amazon AWS virtual machines I deploy, to allow them to backup themselves automatically. The script uses the EC2 management utilities that are normally available on “Amazon Linux” installations (and can be easily installed on other Linux distributions) to create EBS snapshots of the current mounted root EBS volume1.
(more…)

  1. I don’t expect this script to work for instances that have an instance-stored root device, but I don’t expect to encounter these any more []

The most complex Google Doodle

Please go to Google’s Home page right now to look at the Google Doodle for Robert Moog‘s1 78th Birthday – its a fully functional synthesizer that you can play on and record yourself!

It works slightly better in Chrome although the latest Firefoxes can also handle the moogdoodle. I wouldn’t count on anything else being able to play this demo that uses the experimental Web Audio API for HTML 5 that looks to aiming to bring to the web, all the capabilities of a Commodore 64 SID ;-).

Things you can do (The knobs are unfortunately unmarked, so I had to guess what each knob does):

(more…)

  1. Inventor of the analog synthesizer []