Tag Archives: Software

Blessed is the Tool Maker

In the fifth part of Douglas Adams’ Hitchhiker trilogy, Arthur Dent makes his living among a group of stone-age settlers by utilising the one skill he had that was relevant to that world: sandwich making.

I guess we all have a special skill. But my point in this article is that if you’re a software professional, your special skill (unless you’re stranded on a stone-age planet) should be making software tools.

This realisation dawned on me over a long time. It’s well known that developers range in ability by at least an order of magnitude but less clear why. I’m certainly not qualified to answer that question in its entirety, however I’m convinced that developers can punch well over their weight simply by learning to write software that automates dull tasks, such as writing other programs.

Now a lot of people reading this will be thinking, “What’s the big deal?” Is this not obvious? Does every developer not do this? Originally I thought they did but recently became clear to me that that is not the case.

As is my habit, I had written a small utility program. A colleague was going through the same problem so I shared it with him. Even if we didn’t work for the same company, the script had only taken me a couple of hours to write. I made no claims that it was perfect but I knew that it did most of the heavy lifting required.

Next came the surprise.

A couple days later I conversationally asked how he’d got on with my script. It turns out that it didn’t work straight away; there was a minor bug. As far as I can tell he immediately abandoned my code and spent five hours doing the same thing by hand. He’d not mentioned the bug to me before and, for the record, it took about ten minutes for me to fix the problem that he’d encountered.

So, let’s make things very clear here. This otherwise smart person had done five hours work rather than spend a few minutes looking through a program or, even more bafflingly, ask me a simple question ((There’s an argument that the problem here is his communication skills. I’m not convinced that is the case in this circumstance. He’s normally fine. And I think I’m fairly approachable!)).

What’s the difference? Why do some people go out of their way to write simple programs while others go out of their way to do repetitive, mind-numbing tasks?

I think it’s simple. Writing a program to write another program just does not occur to many people, they have difficulty thinking in abstract terms and, in general, can’t go “meta.”

Looking back to my university days I remember a number of parallels. A number of people had problems with the assembler course, but the most telling in many ways was on our compilers course. A compiler is a literally a program that writes another program, normally from a high level language such as C++ to assembler or machine code, and to do this you need to keep two separate worlds in your head simultaneously. Firstly you need to consider the program that you’re writing, that is you need to remember all the lexical tokens and the grammar as well as the memory you’re allocating and using. Secondly, you need to keep track of what’s going on in the program that the compiler will be writing when it runs, stuff like the stack, the memory that it will need to allocate at run-time. Since both are just computer programs, many, if not most, of the people on the course had some difficulty getting their head around either the concept or at some level of the implementation.

So far you might think that I’m bragging, but really I’m not. There’s another key part of this. It’s not just a matter of building software tools, it’s knowing when it’s not worth the effort. Had my colleague known that it would take me five hours to fix the problem then he would have made the right move (if we discount the possibility that the script could help other people if fixed).

Similarly, on a number of occasions I have spent far longer building a tool than it would have taken to do it by hand. Sometimes the desire to solve a programming challenge means that I lack the perspective to see when it’s not such a good idea to script the task. Tool building is not without risk. Those truly great developers, no doubt, have both the tool building ability and the ability to recognise when it makes sense to do so.

But overall I still think that one very significant factor in making some developers more productive than others is their ability to write software tools, particularly programs that write other programs.

Photopress/Lightbox Patch

Ever since I moved over to using the WordPress content management system to host this website I have been using a relatively small number of plugins. One of my most used is Photopress which you can see in use almost everywhere you see a picture.

However, late last year I realised how much one, small part of the functionality irritated me. You could either view a full size picture in a page on its own, but I’d never managed to create a template that worked well for both portrait and landscape images. Or you could have each image pop up in another window. I wasn’t keen on that either.

I eventually decided to “scratch an itch” and implement option number three. I’d found and liked the “zoom” effect provided by the Lightbox JavaScript library, and so decided to use that.

I did offer the patch to the author of Photopress, but I have heard nothing from him so find my changes here.

You’ll need to follow the following few steps to do the same with your own site:

  1. Disable the Photopress plugin if you already have it installed
  2. Download the Lightbox Javascript library and install it on your webserver. I put it in wp-content/lightbox ((You’ll need to keep the same directory structure as in the LightBox ZIP file. For example, you should see three directories “css”, “images”
    and “js” Inside the “lightbox” directory on the webserver.)). Remember where you put it as you’ll need it again in a couple of steps
  3. Download and install the plugin. If you don’t have Photopress, try this full version. If you already have Photopress 0.9.5, you can download either the only file that’s changed or this patch file
  4. Activate the plugin
  5. Go to the Photopress Options screen. Look at the bottom of the screen. Here you can enable/disable the Lightbox effect and point Photopress to the Lightbox Javascript routines
  6. Also make sure that you have switched off the “Link to Album” feature ((Thanks to Roman Seibel for figuring this out.))
  7. You’re done

Please let me know how you get on. Hope you like it!

Mirror

This text is taken from the README and explains what mirror does and why I wrote it:

I think that I must have been looking for the wrong thing. When I restructured my web-site it became difficult to upload changes onto the server. What I needed was a program that copied files to the server. While I could find many programs that mirrored a web-site — copied them from the server — I couldn’t find any to do what I wanted.

Being lazy I started to look at other mirror programs with the intention of modifying them. The best candidate, when I tested it, didn’t actually work (no names!) and others had major creeping featurism. All had restrictive licenses.

So I pulled out ‘Programming Perl’ and started coding my own…

It’s not very big or complex, but it does as it says on the box. And as an added bonus it also downloads, just like all the others.

mirror-1.4.tar.gz

If you find another program that does the same job (better), let me know, I’d be interested to learn whether or not I wasted my time!

GIndent

One thing that really bugs me is badly formatted code. I’ve nearly written a PL/SQL indentation program a number of times, but have never actually? completed it.

But this time it’s different. I figured that most of the pretty printers out there are very poorly written and work only on one particular programming language. However, most languages are very similar to one another. They all have comments, blocks and ‘if’ statements.

Mine may not be much better written — you be the judge of that — but it is generic, having the capability of working with multiple languages.

It’s currently nowhere near complete, so I have uploaded my current development version to Sourceforge. You can check out the latest version here,
but don’t expect it to actually work for you just yet. There’s currently a nearly working HTML formatter and a very incomplete PL/SQL module.

Random Changes

When it first happened I was irritated. A few days later I was irritated that I was still irritated. It didn’t make any sense, it wasn’t a big thing and it shouldn’t have bugged me at all, much less still a few days later.

After a while I realised that my irritation was more rational than I initially thought so I started to write them down as a way of structuring them. And here they are.

So what went wrong? Well, first we’ll need a little background, although I’m sure you will appreciate that I can’t go into all the details.

My day-job involves writing code, enhancing and fixing a sprawling, ten year old application and this day was no different. For the previous few weeks I had been developing a small improvement to one of the reports. I had satisfied the requirements by modifying an existing report, including the occasional conditional where appropriate. Not my best piece of work by any means but I had spent some time on it, had some of myself invested in it in some small way. I checked in the change to our source control system and moved on to other things. A short while later someone reported a bug against it. Not an unusual occurrence with the very vague requirements I was working to and the general standard of my code, however one thing did surprise me — I was sure that I’d tested for and added code for that particular scenario.

It turns out that I was right. The problem was that somebody had completely rewritten my code. The new implementation was very elegant, more aesthetically pleasing that my hack. Remember that I never believed it was the best code I’ve ever written, so if the code actually worked correctly I wouldn’t have cared. But it didn’t. It had fixed one small problem with my code but had introduced at least one new problem, one that was far more difficult to fix than the problem it was trying to solve.

There’s a technical, software engineering, side to this and there’s the effect is had on me. If I was being grand I suppose I’d call that the psychological side. I would like to talk about both of these aspects.

First, let’s discuss the technical side.

When modifying a large, established code base I have always believed that the correct approach is to make as few, targeted changes as possible to get the job done. The problem with a non-trivial code-base is that embedded inside it is dozens of small tweaks that you would never think of making but the software has accumulated in the School of Hard-Knocks. Real life has a tendency to throw up scenarios that, theoretically, are not possible. No matter how good you are at writing code these things happen.

This changed violated this principle. That alone annoyed me.

Of course the counter-argument is that we should be continually refactoring the code. This is the Extreme Programming approach and is one that I have talked about previously. I do have some sympathy for this argument, except that in this case it t really apply. While the new code was undoubtedly cleverer than the code it replaced, it’s difficult to say that it’s better. When you’re writing code that may well be around in a number of years time it needs to be understandable. Writing clever code can be a disadvantage. A few extra lines of code, if they’re simple and easily understood, are often a good thing. It’s easy to forget that it’s not just the computer that needs to read the code.

Naturally the reason that people do this is that it’s much easier than reading and understanding the code, and this is probably more true in the language that we use than most (it’s a proprietary one that you won’t have heard of). So the author of the new code will have saved himself a few hours of work but by creating a clever yet difficult to understand alternative he has generated more work and confusion for legions of support analysts in the future. This costs the company time and money, although the author of the new code appears to be a super-hero as he checked in hundreds of lines of new code compared with my much smaller number.

The psychological aspects are less easy to define but in many ways are more important. Unfortunately I would concede that I am less capable of discussing them in anything like scientific manner. It’s not going to stop me trying, however!

The question at the centre of is all is: why do certain kinds of people like writing software? It’s a painstaking and fiddly task which requires unheard-of precision and attention to detail. Why would anyone want to do that?

I’m sure there are as many answers as there are programmers, but a common root cause would be the simple challenge of solving a complicated problem. There’s a certain satisfaction in completing something tricky and, even if you recognise that there are other ways of doing something, there’s usually a certain pride in the way you did it. It may not be the best, but it’s your way.

So really it’s one of those seven deadly sins: pride.

Also there is a large motivational aspect to it. At this point in the project I was already lacking my usual level of enthusiasm for a number of reasons that I won’t get into here. What kind of effect is someone unnecessarily rewriting my code going to have on me at this low ebb? Why would I want to write more code and make more fixes now? Surely it’s just going to get rewritten.

Note that I didn’t say “criticised.” I think an important aspect to get across here is that I am not above criticism. If the code had been improved, if the new author had discussed the options with me first and we’d agreed a way forward, I doubt that I would be writing this piece here and now.

Now there may well be some degree of bitterness here, however I think ll agree that there are both technical and human factors reasons why things should have been different.

Software Project Survival Guide

Introduction

For many people here, writing software is, if not a job, then a hobby. Our enthusiasm is a double-edged sword. Our technical knowledge is usually much greater than people who just develop software for a living. This sounds like a big advantage, but it’s not as large as you would have thought.

Let’s have a look in ‘Software Project Survival Guide’ (SPSG) to see what Steve McConnell, famed author of ‘Code Complete’ and ‘Rapid Development,’ has to say on the subject.

The most relevant bit I can find is in chapter two. It’s a project survival test. Out of thirty-three questions, only one (28: “Does the project team have all the technical expertise needed to complete the project?”) is directly related to our ability to effectively code.

That can’t be right, can it?

Content

While you’re still reeling over the consequences of that last paragraph, I’ll slip in some of the dull things that the book has to cover and finish with some of the more glamorous stuff at the end when you’re brain is back in gear.

Some of the most important stuff is ‘management.’ Whether it’s planning, quality assurance, tracking what’s been developed so far or checking on the number of defects, it’s not the ‘techies’ that are the most important.

At first this comes as a shock, but the more you think about it, the more it makes sense. Unless you know what’s going wrong, you can’t fix it. Unless the complete design is documented a team can’t effectively write the software quickly.

Think about Linux kernel development. It obviously work well, but it’s not terribly optimal. The same bug might get fixed several times and there’s not a huge amount of up to date documentation so people have to dive in at the code. For free software this works fine — since the time is usually free, but if you have deadlines and are under contract to deliver a working product on time you’re going to lose a lot of money.

Good bits

McConnell takes the three hundred pages to explain these past two paragraphs in detail. It sounds very dull, but he has a clear, friendly style that makes it, if not entertaining, then not as dull as it might have been.

He splits the project into stages, explains in broad terms what they are and documents each one in more detail in the later chapters. It’s all very clear and logical leaving you in no doubt what stage you’re currently at and what you should be doing for the successful execution of it.

Bad bits

‘Code Complete’ is a book that just about everyone that writes computer programs should read. ‘Software Project Survival Guide’ does not fall into the same category. The book should be read by “anyone who has a stake in a software project’s outcome” according to the preface, but that is only accurate once you define the type of project it covers.

To cut a long story short, if you write software professionally you should read it. It’s probably more use to managers and team leaders, but you can be a better developer if you know the kind of things that need to be done. Besides, almost everyone will be in charge of people at one point and will need a broader picture than merely what the other modules do. (Well where I work that’s true, anyway.)

It covers projects that have a customer whether that’s an internal customer, an external client or people that buy shrink-wrapped software. What it doesn’t cover are software projects like Linux, massive sprawling projects developed because someone is interested in doing it. This is not unreasonable. Half the challenge in a ‘normal’ project is tracking whether you’re running to schedule. If you don’t have a schedule and cost target it doesn’t matter. However, maybe we could take a few hints from the book to ‘streamline’ the process.

Overall

Steve McConnell has done it again. I don’t think that it will go down as a classic like ‘Code Complete,’ but SPSG is an indispensable book for any developer that has any interest in the process and not just lines of code.

The facts

Author: Steve McConnell

Cost: ?22.49

ISBN: 1-57231-621-7

Buy this book from Amazon.com of from Amazon.co.uk.