Archive for the ‘Script Day’ Category

Script Day: automatically locate the next valid transaction in MySQL binlog

Wednesday, July 27th, 2011

Sometimes the MySQL replication breaks due to some corruption in the binary log files(1). When your binary log files are corrupted, the only option (other then trying to rebuild a database of hundreds of gigabytes) is to try to skip over the corrupted region and get the slave to pick up from where the transactions are valid.

Locating the correct position in the binary log from which the server can carry on is difficult but can be made easier by the mysqlbinlog utility that can scan the binary log files and show you which position is valid using the --start-position to try random positions in the binary log file and see which position will let you read from the file(2).

(more…)


  1. I have yet to find a good explanation to why it happens and how to prevent that []
  2. because in the binary log transactions can have any size, so they can end and start at any point []

Script day: output the tail of a log based on time

Thursday, April 8th, 2010

As system administrators we often want to list the last few lines from a log file in order to track problems and see system reports. The UNIX command tail is very useful for that purpose and lets you display an arbitrary number of lines from the bottom of any file.

But often this is not really what you want – an administrator might want to see what happens in the last X minutes and the common practice to do this is to run tail with a guessed number of lines, see if you get what you want and if its not enough increase the number and try again.

Here’s another approach that works well if the log file you want to trace has time stamps for its lines (more…)

Script day: grep in jar (or zip) files

Monday, September 14th, 2009

Here is another script I wrote for work and I thought it will be interesting enough to share:

Say you want to check which JAR files (or ZIP files for that matter, as Java ARchive files are just ZIP files with a different extension) contain files that contain some text. grep is the obvious answer, but how to grep files in JARs?

(more…)

Script day – Shutting down multiple servers at once

Monday, July 20th, 2009

A system administrator in my company recently approached me with a problem – how to shutdown multiple Linux servers at the same time from a central location. Apparently this is something that people in the MS-Windows world use all kinds of applications, like the Remote Shutdown Tool from Microsoft (though I don’t understand how they handle the authentication – this tools doesn’t seem to require any authentication so it appears that any person with network access can shutdown any computer).

Anyway, apparently searching the web for “Linux remote shutdown” yields no useful results (or so I’ve been told), but frankly – when you have standard UN*X tools at your fingertips, a remote shutdown tool is simply typing ssh root@server shutdown -r now at your local console. But still, for people who want a “tool” – read on.

(more…)

Script day – randomly rotate GNOME desktop backgrounds

Tuesday, July 1st, 2008

I kind of collect desktop wallpapers – I have a lot of those, several thousands(1). It is a bit ridiculous as I mostly use maximized windows all the time so if not for the fact that in work I live on the console and I have a transparent terminal, I would rarely see my desktop wallpaper.

That being said, with a wallpaper collection, you want a software to manage it and cycle your desktop through the wallpapers. KDE has this function built it – just go to configure your wallpaper and select a directory of wallpapers, choose whether you want to cycle through the images sequentially or randomly, the delay and your done.

Not so in GNOME – simplicity for simplicity’s sake (more…)


  1. mostly anime and video games fan made as well as promotional walls, a lot of hobby photographs – mine and other people’s, and a few more professionally made art []

Script day – read configuration files

Wednesday, April 23rd, 2008

This is not really a script – more of a snippet. I don’t have a lot of spare time these days, so I can justify posting a snippet and calling it “script day” 😉 .

A lot of unix configuration files use the # sign to add comments to configuration files, and a lot of software comes with very well documented files – i.e. has lots of comments. So much that if you just want a quick glimpse at the configuration that is active (not commented out) its very difficult to wade through all the documentation.

Here’s a simple grep that will filter out all the junk and leave you with just the active configuration settings:

egrep -v '^(#|\s*$)' <config file>

and on the standard output you’d get only lines that are not commented out or empty.

Do note that some configuration files can also use ; as a comment character, but modifying the grep to support this is trivial.

Script day – find Java jar files that contain a Java class

Monday, November 12th, 2007

From time to time I need to work with a Java program or library that requires some import which I’m not familiar with. Its often very easy to just copy the fully qualified class name and search for it on Google which usually helps identify the product that contains this class.

But if you know that you have this class on your system somewhere, and you are just not sure which jar file you need to add to your project for it to compile – this script will come in handy:

(more…)

Script day – simple log graphing toolּ

Wednesday, November 7th, 2007

I wrote similar versions of this script over the years to analyze all kinds of logs, but here’s one for posterity:

This script is useful if you have a log for which you want to analyze load over time – transactions per second or whatnot (the version below does this for Apache httpd logs, but it can be easily modified to analyze anything). For apache (and most other HTTP servers) there are many readily available log analysis software packages that do a much better job then what one can do in a simple script, but you might not have such software pre-configured or it can’t filter what you need or you just want to analyze something else – in which case this script will come in handy.

The script receives time stamped log events – each event on a line – and collects the temporal information for each line. Then it will dump a simple vertical graph (i.e. time is on the Y axis) of load over time in the resolution that you want. Its output looks something like this:

Oct 30 14:40:00 2007 |#############                                    | 3.8 x/sec
Oct 30 14:50:00 2007 |##########################################       | 6.3 x/sec
Oct 30 15:00:00 2007 |###########################################      | 6.5 x/sec
Oct 30 15:10:00 2007 |#############################################    | 6.6 x/sec
Oct 30 15:20:00 2007 |###############################                  | 5.4 x/sec


(more…)

Script day – manually find a thumbnail for a video

Friday, November 2nd, 2007

In a vain effort to fuel my blog with some content ( 😉 ) I’ll try to get into the habit of posting some throw-away scripts I’m writing for my personal use (at work and at home). Hopefully some people would find them interesting, or maybe – god forbid – useful.

Today we have a script that will help you pick out a nice thumbnail for a movie file – this is useful when setting up movie galleries and stuff. Of course it can be done automatically (most gallery software that support video files has automatic thumbnail extraction) but they rarely choose the best shot. The script here below would extract 100 frames from the first 30 seconds of a movie file and display them using an image browser. The script would also open a file browser window – the user will then pick the desired thumbnail image and drop it into the file browser, and then close the image browser. The script would then complete the process by naming the selected thumbnail image to be the same name as the original movie (with a different extension of course).

(more…)

JavaScript Date Format

Tuesday, July 26th, 2005

I took the time to implement some sorely missed functionality in the JavaScript core objects – Date is missing simple date formatting to text string. While the Date object can parse arbitrary textual descriptions of dates and times as input, it can only output textual description with a fixed “locale” template.

I’ve looked up on the web and there are some utilities that achieve this simple operation, but most of them are overly complex and some people even have the nerve to charge for this very simple code !
(more…)


Spam prevention powered by Akismet