Tag Archives: unix

Dreadful Thoughts

Introduction

I have a terrible confession to make. I am not a spiritual man so, rather than seek penance through the church, I shall document my reasoning here on the web and you can make your own conclusions.

Please go easy on me.

My confession: I can see myself buying a Macintosh later this year.

To some, that may not sound like something to be ashamed of but it’s all a matter of perspective. By trade and education I’m an engineer or scientist; I have short hair, no piercings and virtually no artistic ability (I present the evidence of that here on this web site!). My computer of choice tends to have a barren, baroque user interface. Let’s be honest here, it’s Unix.

The Macintosh and Unix sit virtually on opposite ends of a spectrum. It’s usually called usability, but I think it’s even more precise than that. The Mac makes it easy for people to learn how to use it. Apple make things easy to use and they sacrifice power and flexibility in order to do that.

One of the best examples if this extreme position is the mouse. In Unix you tend to have three buttons. Windows originally had two but now seems to have spawned wheels and more buttons than a Space Shuttle. In each case although it only takes a short time to realise that the left button does most of the useful stuff, Apple decided that one button was less confusing. They’re right, of course. But it does limit your options as far as, say, short-cut menus are concerned.

Unix is the opposite. It has a huge learning curve, but an expert can quickly do just about anything. After spending a lot of time on that curve, I’d actually say that Unix was more usable than a Mac. I’m under no illusions, though, that it’s more difficult to learn.

I guess these extremes, to some extent, explain the success of Windows. Ignoring Apples mistakes and the fragmentation of the Unix market in the early nineties, we can see that Windows is easier to use than Unix but much less so than a Mac. It has the Start menu, Wizards, pop-up help and often hides information rather than bombard the user with difficult, unnecessary detail. On the other hand, it does have rough and ready multi-user facilities, solid TCP/IP networking and a command-line interface (for the brave). It fits the middle ground doing neither task especially well.

The battle ground

That’s how they stand right now, but six months from now things may be very different.

The Unix side probably isn’t going to change much. Linux, especially, will continue with its vast range of incremental upgrades. distributions will eventually come on-line with the new 2.4 kernel, and improvements will continue in both the KDE and GNOME environments.

In the same time-frame, the next desktop version of Windows, XP, will be unleashed on the world. The beta’s are currently doing the rounds and people seem generally impressed. The interface is easier, more consistent and more aesthetically pleasing, and its built on the Windowds 2000 core which has generally been well recieved.

Normally it might have been enough for me to upgrade from my old copy of Windows 95. But for two things: you can’t, you can only upgrade from Windows 98 or above; and MacOS X.

It would be an obvious choice to buy a new PC preloaded with Windows XP since I’ve had a small succession of similar machines over the last ten years, but I find the improvements in MacOS X to be so compelling than I’m considering moving to a completely new architecture. In a sound-bite those reasons are power and user-friendly in one.

MacOS X is based on a BSD Unix kernel (called Darwin and available under an Open Source licence) and has an enhanced Macintosh user interface grafted on top. This is truly the key. You have the complex internals available from a command-line when you need it and a state of the art GUI when you just need a word processor.

In conclusion

There is only one other operating system that supports such a neat combination of Unix and User Friendliness (the BeOS) and that has many problems: I have tried it on three machines and all have had some device that is unsupported, so this can hardly be a unique scenario; and the software support is worse. I may prefer not to use Microsoft Office, but I need to be able to exchange files with people that do.

No other operating system will have quite that level of flexibility. Microsoft won’t be adding more Unix like functionality to Windows and the open source community just can’t compete with the many years of experience that Apple have designing computers for people who don’t write code.

C

Introduction

Talking about C is not easy. Almost all professional programmers have used it at some point and many have a strong attachment to it. I don’t want to start by saying that it’s a poor language, alienating much of my audience, but I figure I’m going to end up doing that anyway so I may as well get it out of the way at the beginning.

Compared to many languages that have come since, and even some that came before, C just isn’t a very good language.

There, I’ve said it.

History

Perhaps more than almost any other language since, C has a rich and famous history. It’s almost impossible to discuss it without also talking a little about the history of Unix.

In 1969, Ken Thompson finally got hold of a PDP-7 and decided to write an operating system for it. (It happened more often than you’d think back then.) That operating system was Unix.

A few years later, Dennis Ritchie designed and built a language based on B (itself based on a language called BCPL) and, creatively, called it C. Unlike its immediate ancestors, C had types and a number of other useful bits and pieces.

C was so useful that Unix was quickly rewritten in it. These days that sounds obvious, but until that point operating systems had always been written in assembler or machine code. This is an important part of computer history.

Utility

C became popular largely because it filled a very useful niche. At one extreme you have all the ‘real’ languages. At the time, real computer scientists would have used Algol68. Structured and clever, Algol was a great language but you couldn’t do anything very low level with it.

At the other extreme there is assembler which people had to resort to if they wanted to do anything close to the hardware. Assembler is only one step removed from machine code which makes writing reliable, bug-free code very difficult, especially when you’re building something as large and complex as an operating system.

C fits in the middle. Described by some as a high-level assembler, it allows you to do low level coding, accessing particular memory addresses and the like, and use high-level constructs such as functions and types.

The following books and papers helped me learn to hate C.

Practical C Programming” by Steve Oualline.

Writing Solid Code” by Steve Maguire.

Code Complete” by Steve McConnell.

Hello world. How many nasty ways can you write “Hello World” in C?

Pointers

Key to C’s ability to mix high- and low-level constructs are pointers. Most ‘serious’ languages have some concept of a them. Some call them references, some call them links, but they all, basically, refer to something that identifies a chunk of memory. Most other languages only use them when you have to, but they are C. No pointers, no language.

You want an array? That’s really a pointer. Pass by reference? A pointer. Strings? Ah, they’re arrays! (Which are pointers.)

The advantages of using pointers for just about everything in the entire language are mainly one sided: it makes writing compilers easier. For the poor souls that actually end up using the language all is not so rosy. As Steve McConnell puts it, “pointers are one of the most error-prone areas of modern programming” (Code Complete, section 11.9).

Some of the side-effects of using pointers are not immediately apparent, either. In most languages, arrays have bound-checking (the ability for the language to raise an error if you try to access an element that doesn’t exist). But C doesn’t really have arrays, it has pointers and a little syntactic sugar that makes it look like it has arrays. Pointers don’t know how much memory is being pointed to so you don’t get bound-checking. If you’re lucky your program causes a segmentation fault, if not you might corrupt other data or, on some operating systems, your program.

It takes all types

Another one of C’s biggest problem is it’s typing. As most people will already know, C allows you to put numbers into character variables, integers into floating points and any number of other nasty combinations. To C they’re all valid, but what they do are not always well defined or consistent. Even the same compiler sometimes does different things depending on the level of optimisation in use, the phase of the moon, etc.

The odd thing is that a weakly typed language doesn’t allow you to do more than one with strong types, it merely allows you to do the wrong thing more easily. For a language used for large-scale software engineering projects the risk of poor code is just too great and weak typing should be outlawed.

I’m sure that people are going to mention the myriad of warnings that modern compilers are able to produce, or that ‘lint’ has been available for nearly as long as the language has been. My counter-argument: I don’t see why you should have to add extra tools or read through pages of warnings in order to correct deficiencies in the source language!

The good bits

If C was truly as appalling as I’ve made out so far, no-one would actually be using it. The main ‘win’ for C, as far as I can see, is that it is small, well defined and widely available.

All three merits are in many ways different sides of the same coin (if you can imagine a three sided coin). To make the language well defined, it helps if it’s small. If lots of people are to use it, it needs to be simple enough that they aren’t put off (Ada anyone? Thought not.). Small and well-defined make it easier to write compilers too, meaning that it’s available on everything from the lowliest PC right up to mainframes.

The theory also goes that your programs should recompile on this range of machines too, but that’s not as true as we’d all like to think. If it was true, we wouldn’t need Java or hundreds of ‘#ifdefs’ throughout. My code tends not to be that low-level, but I wouldn’t like to make any promises about its portability. However, that doesn’t make C’s wide availability useless. Even if your program isn’t cross-platform, the skills required are. A C programmer can quickly write code for just about any machine.

Summary

I’ve probably written more lines of C code than just about any other language, so when I say that I don’t like it I hope that you can see that I’m not being narrow-minded or prejudiced.

As I’ve mentioned above there are some things that I like about C, it’s just that there is so much to hate about it and, even at the time it was written, there wasn’t an excuse for it!

The languages main features are its weak typing and over-use of pointers, both of which allow developers to make truly horrendous coding errors with ease.

If those were C’s only problems I might be able to forgive it, but they aren’t. C has more ways for both experienced and novice programmers alike to hang themselves than any other language I can think of (with the possible exception of Intercal). Even if it didn’t have the ‘=’ and ‘==’ operators to confuse, there’s still the wonderful ‘?:’ and a whole host of spectacularly error prone API calls (does all your code check the value that ‘malloc’ returns?)

No, C, as a language, is a dinosaur. It deserves to whither and die. If you write anything other than an operating system kernel and use C, switch to another language. You’ll be far more productive when you start battling the problem rather than the language.

Installing Oracle 8i R2


Introduction

Everyone will be very pleased to hear that Oracle’s third attempt at producing a usable database product on Linux has largely been successful. The first two usually worked but only after much aggravation. Forget all the extras that 8.1.6 provides, you can get the thing installed with much less grief!

Of course, I wouldn’t go so far as to say that it was simple and straight-forward all the time. It is Oracle that we’re talking about here.

I’ll start by describing how I got Oracle installed on my box and finish off with some questions and answers, much in the same format as the HOWTO. It’s probably worth having a look at the HOWTO still as many of the problems are similar and the solutions given there may give you some idea of where to start looking.

My machine

First, some news on my ‘server’ configuration: I still have the same Celeron 466 with 128Mb of memory. On the software side I’ve upgraded to Mandrake 7.1 (if I’d been running a production Oracle server I wouldn’t have taken the risk). I didn’t remove my old installation of Oracle before starting on the new one and I didn’t attempt to perform an upgrade.

I did remove JRE (Blackdown 1.1.6v5) and my installation of JDK (1.2.2) from my path. Oracle now comes with its own JRE, so even having the risk of it using the wrong one made me paranoid.

The last thing to note is that this time I downloaded my copy rather than using a CD. This seems to be what most other people do, so my tale here should be closer to ‘real life.’

My successful install

The process was as follows:

  1. Download Oracle 8i R2
  2. Extract the archive
  3. Create the required users and groups
  4. Make sure X is set up correctly
  5. Start the installer
  6. Quick tests

Firstly, the download. It’s big, nearly 300Mb. Don’t attempt it without something like Gozilla or wget even if you’re on a fast corporate connection.

Secondly the extraction. You’ll find that it comes in a standard tar archive compressed with GNU Zip. This command should get all the files out:

tar zxvf oracle8161.tar.gz

When you extract it, remember that the files coming out are slightly bigger (301807K on my machine). So you need over 500Mb of disk space before you even start the installation!

Before you actually start the installation, you’ll need to switch to “root” for a couple of commands. Start by creating a group called “dba” and a user “oracle”. Your new user should be in the new group. Log in as your new “oracle” user and make sure your X Windows system is working properly. (If you can fire up a new ‘xterm’ you’re fine.) The Oracle installer, as before, works only in a graphical environment.

Go to where you extracted the software archive. You’ll find a directory has been created (“Oracle8iR2”). Move into it and you’re ready to start the installation!

(A quick note: in the same directory there’s “index.htm” which is the root page for all the Oracle installation document. This seems to be improved over earlier releases and is worth a read.)

Type:

./runInstaller

A splash screen should appear, followed by a Windows-style Wizard/Installer. I find the default options for almost everything to be fine. Broadly speaking, and assuming some common-sense is used, just clicking “Next” continually should result in a working installation. In slightly more detail…

(Note that there are a few points where the installer asks you to log in as “root” to run some shell scripts. To simplify the text below, I’ve missed these steps out. Simply do as it says and click “Retry” once it’s done.)

  1. Welcome screen. Click “Next.”
  2. File Locations screen. The top box should be correct; it displays the location of the archive containing all the software about to be installed. The second box is the “base” of you Oracle installation. I chose “/home/ora816” but this is not recommended. Have a read of the OFA (Oracle Flexible Architecture) documentation.
  3. Available Products screen. If you’re installing a server, select “Oracle 8i Enterprise Edition 8.1.6.1.0”; otherwise select “Oracle 8i Client 8.1.6.1.0”. I’m assuming that you’re building a server and click the first option.
  4. Installation Types screen. Do you want a “Typical”, “Minimal” or “Custom” installation. Unless you really know what you’re doing, pick “Typical”.
  5. Upgrading or Migrating an Existing Database screen. If you have a previous installation, Oracle will ask whether you want to upgrade your database to the new 8.1.6 format. I didn’t. I’d recommend doing this yourself once the installation process is complete even if you do.
  6. Database Identification screen. Here Oracle asks you for a Global Database Name and a SID. As before, this is something your DBA probably has an opinion on. If you’re the DBA and you don’t know what it’s asking for, enter “dev1” for both.
  7. Database File Location screen. Now Oracle knows what you want to call your database, it asks you where you want to put all the files that make up the database. Think back to your reading of the OFA documentation for this.
  8. Summary screen. Oracle now tells you what it’s planning on installing. Click “Install” if you’re sure, or go to the “Previous” screen an juggle the options around.
  9. Configuration Tools. First Oracle runs the Net8 Configuration Assistant and then runs the Database Configuration Assistant. Basically, it sets up your networking and creates the database you asked for. No user intervention is required. (Note: the SYS account password is “change_on_install” and the SYSTEM password is “manager”. You should change both using the SQL*Plus “password” command as soon as possible.)
  10. End of Installation. That’s it, you have a complete installation!

If you want to install Oracle Programmer (Pro*C, etc), you need to follow the same process as before: go back through the installation process, but this time following the “Oracle Client” route. The rest of the process is similar to the above and very straight-forward. The new installer even asks you if you want to start again once your database has been created.

And if you want to set up a network connection to another machine, the process is exactly the same as for Oracle 8i (and is covered in the main HOWTO).

Questions and answers

Java problems?

As before, many of the problems come from your choice of Java Virtual Machine. R2 actually comes with a runtime environment this time (JRE 1.1.8), which does make things easier. However I have heard reports that older versions sometimes work better. The older version is normally Blackdown’s 1.1.6v5 release, the same Oracle used to recommend with their 8.1.5 release.

Memory requirements

One thing that is exactly the same is the amount of memory required. I don’t remember seeing a figure in their old documentation, but they say you need 400Mb this time, either real or virtual, for 8.1.6. I have 384Mb in total on my machine and it was fine. The default database configuration seems to use more memory but, as always, you can change that.

Running Redhat 7 or another glibc 2.2 based distribution

Short answer: add “export LD_ASSUME_KERNEL=2.2.5” to your profile and then type “. /usr/i386-glib21-linux/bin/i386-glibc21-linux-env.sh”.

Long answer: look at my page on the subject.

Perl

Introduction

Many developers would hate to have their master-work described as a mess, but not Larry Wall, creator of Perl and celebrity hacker. The way he sees it, the language is a mess because the problem domain — real life — is also a mess. He has a point.

I first came across Perl a few years ago when I was writing a program that required a certain amount of ‘screen-scraping’ from a telnet session, the ability to retrieve files using FTP, some complex processing and interaction with an Oracle database. This is a fairly messy problem, and one that Perl looked eminently able to solve.

Originally I came up with a design involving precarious shell-script creations, plus PL/SQL and a pile of other logic thrown in for good measure. This is even more messy and, worse, I can’t see how it would ever have worked. Then, at various times, people would suggest Perl. It’s good at text handling one would say. You can connect it to a database another would say. But it was the telnet library that sold it to me. I still hadn’t figured out a reliable way of doing that in a shell script.

The philosophy

With the tools that I already had, the problem may well have been impossible. I could have done it in C, but I didn’t have the luxury of time. I might have figured out a way of doing it with shell scripts, but it’d be a nightmare to debug and support.

It sure looked possible in Perl, and it had to be easier than doing it in C.

The Perl philosophy, it turns out, is make the every day things easy and everything else possible. The kind of thing that I wanted to do wasn’t exactly typical Perl fare, but it had all the right elements.

Much of the time I use Perl like a super-shell script. Thinking of it as such is not completely wrong, but it does do the software a great disservice. Sure, you can use it like that. But you can also make ‘real’ programs, complete with declared variables, objects and a GUI user interface.

You can do a lot in Perl, and it doesn’t try to cramp your style. Want to use objects? Go ahead! Think they look too complex? You don’t have to use them! You can write programs like glorified shell-scripts, or like C. Roll your own, use a library, the choice is yours.

This neatly brings me to another one of Perl’s philosophies. (The Perl definition of a philosophy is as messy as the language and the problem you end up solving with it.) This other message is: There Is More Than One Way To Do It.

Let’s try a trivial example: the if statement. Most languages insist you do something like this:

if ($a == 5) {
  print "a is five!n";
}

Indeed, this is perfectly valid Perl. But so is this:

print "a is five!" if ($a == 5);

I don’t necessarily think that this represents good programming style, but then I don’t have to use it. My choice.

The syntax

As you can see in the above example, Perl bears more than a passing resemblance to C. It uses the double equals (“==”) to compare numbers and semi-colons as statement terminators.

It’s also a bit like Unix shell scripts. Note the use of the dollar to identify variables. However, Perl is much more insistent on their use than Bourne. You must use the dollar all the time, even in assignments. This gets annoying if you change between languages with any degree of regularity.

The good news is that Perl doesn’t always use a dollar sign to identify variables. The bad news is that it also uses the at sign (@), the percent sign (%) and the ampersand (&). (At least these are the common ones. There was a joke going round recently suggesting that Perl 5.6 only supported Unicode character because they’d run out of symbols on the normal keyboard. At least I hope it was a joke.)

Fortunately it’s not completely indiscriminate. It turns out that Perl has only three data types: scalars, lists and hashes. The dollar identifies scalars, variables that accept a single value, at’s are used when you want to put several values in a single variable and the percent sign is used to identify associative arrays (which Perl calls hashes).

For example:

# Scalars
$a = "hello";
$b = 1.2;
# Lists
@c = ("hello", "goodbye");
@d = ($a, $b);
# Hashes
%e = ( "var1" => "value 1", "var2" => "value 2" );
# Output
print "$a, $c[0], ", $e{var1}, "n";

There are several interesting things to note here. Firstly, you don’t have to declare your variables (although you can if you want). Secondly, scalars store any single value, whether number, character or string. They are weakly typed in the same way that Variants are in Visual Basic; the system knows what’s there and will do different things based on that information. We’ll see more of this later.

Lists can store any number of scalars (which, as you can see, don’t all have to be what most other languages would consider the same type), with Perl performing all the memory allocation and deallocation, much the same way as the much-vaunted Java garbage collector. Many of the same properties apply to hashes.

Perhaps the final interesting property is the way you read from the non-scalar types. As you can see, you must use the dollar — scalar — to access them. This does make sense when you stop to think about it: you’re reading a single value not the complete list (which is still represented using the at sign).

The Perl Difference

All this variable stuff is unusual, but it doesn’t make it stand head and shoulders above everything else (in fact, the weak-ish typing and lack of user-defined types make it much worse than many others). But Perl is used in just about every CGI script in existence for a very good reason.

Is it the same reason that Perl has a reputation for being unreadable. Generally Perl is no less readable than, say, C but the one aspect that confuses just about everyone the first time are regular expressions. If you like code that looks like line-noise, this is the feature for you!

Regular Expressions are a way of representing patterns. For example this line could represent UK postal codes:

^[A-Z]+[0-9]+ [0-9][A-Z]+$

Or in English: one or more letters at the beginning of the line, followed by some digits. Then there’s a space followed by a single digit and one or more letters. The pedantic might note that all the letters must be uppercase.

Like all the usual Unix command-line tools, Perl allows you to look for and manipulate patterns in files. Perl extends the usual array of tokens allowing you to make fantastical expressions that quickly become completely unreadable. The power comes from the fact that all this is embedded right into the language, no clumsy function calls.

while ($line = <>) {
  chomp $line;
  if ($line =~ /1234/) {
    print ":$linen";
  }
}

Note here that in the third line I compare the input (“<>” reads the next line from the currently open stream, which is stdin by default) to an expression. Clearly this is a very simple example, but you should be able to see that having this functionality builtin gives the language the ability to express some very complex ideas concisely.

Extras

One of the things that Perl does better than just about any other language is plug-in modules. Perl 5 added some clunky object-oriented-like features and, while they may not be elegant, they do seem to work.

A testament to their power is the number of modules available for free download at CPAN (the Comprehensive Perl Archive Network). There are modules for connecting to just about any relational database, libraries to talk to all the Internet protocols I could think of and code to deal with XML and configuration files (or XML configuration files). There are so many modules you rarely have to write much in the way of code yourself.

Summary

I’ve barely begun to scratch the surface of Perl here. There’s much more to it, but the beauty of the language is that you don’t actually need to know that.

The language itself has a tendancy to look ugly and, often, unreadable. But it’s used just about everywhere you can find a Unix box. There are few other languages that come close to Perl for hacking text files and automating boring system administration tasks. It takes over where shell scripts leave off and only starts to run out of steam for projects thousands of lines long (it’s technically able to cope with more, but it’s not a real software engineering language). System Administrators don’t call it a Swiss Army Chain-saw for no reason.

As a computer language purist, I really want to hate Perl. It has weak typing, no data structures, it’s proud of the fact that there’s more than one way to do everything and the syntax just looks plain ugly. But this is not a pure or perfect universe. These very ‘flaws’ make Perl the ideal tool for the jobs it was designed for.

I may not actually like it, but I use Perl for just about all my hacking activities. It’s just too useful to ignore.