Category Archives: Opinion

Thoughts on computers and the IT industry.

History

A few years ago I had a job where every new recruit would go through a long process of shock and gradual acclimatisation to the main software product.

What it did doesn’t matter as much as how it was built: it was an application developed on top of a proprietary programming language and user interface designer. The reaction was always the same. Why? Why?! Why would you reinvent Visual Basic on Unix? Why would you inflict a programming language even worse than Basic on developers?1

The answer, it turns out, is that the original developers were idiots.

No, of course that’s not true. But if that’s the case, then why did almost every developer start from that point of view when they first arrived at the company?

That brings us to Twitter and its new owner. One of his first public proclamations is to declare that there are too many micro-services running, and, worse, most of do nothing useful! The reply-guys all agree and, between them, argue that it’s entirely possible to rebuild Twitter from the ground-up in weeks, possibly even a weekend if given enough pizza and Blue Bottle.

Were the original developers of Twitter also idiots?

I don’t know as much about Twitter’s architecture, but I’d be willing to bet that, no, they were also not stupid.

If it’s not the original developers, what does it say about the critic? It says that they see the complexity but not the nuance. They see the current state but they do not see any of the decisions that lead up the current system. They see complexity, but without understanding the whole problem domain they don’t see why that complexity exists.

In the case of my job, the software predated Visual Basic, which is a pretty good reason for not using it. It also had to work on Unix and be editable on client sites without extra tooling. By the time I worked there, it may have been dated but it was in production at many clients. It worked. Sure, it’s not how you’d architect it now but the decisions that led to the design did make sense.

If it’s dated, then why not rewrite it? That has been covered many times before, but the short answer is that when you design it, you focus so much on the clean, new solution that you forget why you added the warts to the old system. The layers upon layers of fixes and enhancements represent real world experience. Those micro-services are there for a reason. Not understanding the reason doesn’t change that2.

This is not an argument against evolving the software, only that you should understand what you already have. Sometimes rewriting can be justified. Rationalising a bunch of micro-services isn’t always a ridiculous idea. But there’s an important difference between complex and complicated. Can you know which your inherited system is after a few days on the job?


  1. It was a stack-based language, along the lines of Forth and Postscript. Long time users could do amazing things with tiny amounts of code. I never quite got there. ↩︎

  2. Logical fallacy: argument from incredulity. ↩︎

Twitter

Sometimes it’s only when you start writing about a subject that you truly understand your opinion. That’s the approach I’m taking to answering the question: are you going to leave Twitter?

A few people have asked me in the last couple of months and the only response I have is that I’m not jumping ship and closing my account immediately.

But as the weeks have progressed, as I’ve written this piece, my thinking has evolved. It’s not that I’m going to immediately close my account but I can see The End approaching. Indeed, my usage of Twitter has dropped considerably.

When Twitter was delisted from the stock market, the concept at the top of my mind was this: can you judge a company on the person or entity that owns it?

Twitter has been badly managed or owned by incredibly rich people (or both) for a long time, but they still have millions of users. Is a change of rich person really that significant?

Is the fact that Musk isn’t terribly likeable a factor? Many people bought products from Apple even though Jobs was famous for pushing people to breaking point. You can appreciate the vision even if you couldn’t work for the individual.

To be clear, I’m not saying that no-one avoided Twitter and Apple for these reasons. I’m sure there are some, but not me and not millions of others. Is there a line that he could cross where I would leave immediately? Yes, and, in fairness, he’s got pretty close by allowing back some of the extremists who have been banned.

And, circling back to the management, Twitter has been a mess pretty much since the beginning. They seem to have difficulty shipping anything. They’ve largely eliminated the “fail whale” but what big, beneficial features have come since? The algorithmic timeline?1

Like it or not, maybe the company needs shaking up.

Though, starting on the “cons” side, shaking up the company like this likely isn’t what is needed. It is the tech equivalent of the Brexit “solution” to Britain’s problems. Needing change isn’t the same as supporting chaos.

I don’t understand what over seven thousand people do at Twitter, but neither did Musk, hence the call going out to some of those laid off, asking for them to come back. More slash and burn than measure twice, cut once.

And Twitter’s considered approach to changes is out, replaced by arbitrary deadlines and hunches. $20 for Twitter Blue? No, how about $8. Available on Monday. Or Tuesday. Could be next week.

One common reason that people have left Twitter previously is the volume of hate and harassment. While I don’t doubt their experience, it’s not something that I’ve seen personally. I stay in my little bubble with tech and jokes and a bit of politics.

But it doesn’t feel like we’re heading in the right direction. Musk’s naive views on free speech are perhaps the most worrying, not in the sense that they have the most direct, immediate effect but because they demonstrate that he doesn’t Get It.

My hope is that Musk quickly learns and pivots to a more sensible, nuanced position. But his recent tweets about American politics and abandoning putting warnings on COVID misinformation makes me think this isn’t likely. He seems to think that the problems at Twitter are about the technology, that removing a few micro-services and adding a few blade servers will make a difference. However, the problems are all about people, those who use the platform, those who advertise on it, and those who work there. Until he understands that, or defers to someone who does, things will continue to spiral.

In the end, as an end user, Twitter is all about the people I interact with every day. If they leave, it doesn’t matter whether it’s because of something that Musk said or did, or not. Their absence will make the site not worth visiting any more.

In short, I stay on Twitter despite the company that runs it and despite the person who owns it. I’m there for the geeky discussions, the dad jokes and despairing at the state of British politics. If that goes away, so do I. Find me here if that happens.


  1. Most long-time Twitter users think it’s terrible. While it does occasionally surface interesting Tweets, I do think I’d prefer the original reverse chronological timeline, too. ↩︎

Programming Pearls

Every year I try to complete the Advent of Code. Every year I fail to finish. I get about halfway through, and the exercises start taking longer to complete than I have time.

Every year I think about Jon Bentley’s Programming Pearls1, because the same kinds of challenges you find in Advent of Code can be found in the book. The main difference being the quality of the answers. At least in my case2. In the words of the preface: “Programming pearls whose origins lie beyond solid engineering, in the realm of insight and creativity.”

The format of the book involves presenting a programming problem and then iterating on the solution while discussing the trade-offs involved at each step. It’s quite an old book by computing standards – the second edition was published in 1999 – and you may be put off by the use of C to illustrate the solutions. I would urge you to continue anyway, even if you are not an expert in C. You may also find some of the solutions to be hard work. Honestly, that’s part of the fun. If you don’t like having your brain turned inside out, this isn’t the book for you!

As you work your way through the chapters, you realise that the key for most of them is not esoteric optimisations or low-level hacking made possible by the C programming language. Instead, it’s data structures. If you somehow manage to store your data in the “correct” way, the algorithm to process it becomes simpler, clearer and faster. It’s almost miraculous.

Of course, there’s a lively debate about “computer science” and whether it should be the subject of developer interviews. What I would say is that the kinds of people who like to attempt Advent of Code are very likely the kind of people who will also enjoy Programming Pearls.


  1. Not to be confused with Programming Perl. ↩︎
  2. In my defence, I usually use Advent of Code to learn (or brush up on) a new programming language rather than solve the puzzle in the best way. That’s my excuse, and I’m sticking to it. ↩︎

Panic

The whole team got this email today. Okay, it wasn’t today and these are not the exact words, but it was something like this:

We have a serious regression in build 456. We have set the project back rather than taken it forward. We need the utmost focus and commitment on fixing it. We’ve broken it and we stay in the office until it’s fixed.

I’ve had a few of those messages over the years and while it’s intended to focus minds it often has the opposite effect. Let’s examine why.

Projects see the same mistakes made over and over and this email encompasses many of those sins; it’s one message but represents a microcosm of large part of my career.

Here are a few problems that I see immediately:

  • No problem definition
  • No person accountable
  • No next action
  • A deadline but no understanding of the work involved

This has a number of consequences.

There are studies that show if you have a heart attack in a crowded area you are less likely to receive life-saving CPR from a stranger than if you’re in an area with one other person.

In this case the passers by (the project team) don’t know that they’re needed. Without a problem definition I don’t know if the regression was caused by one of my changes or even if it affects my code. Without a person accountable everyone likely assumes that it’s someone else’s code. “Someone would have mentioned it if it was my code.” And with no “next action” it’s easy to assume that someone else will handle it.

Arguably the deadline is not really a deadline. What if the fix would take a week to implement? Instead it’s a target. You can’t take an estimate and reduce it to fit to an externally imposed date. It doesn’t work like that. You may hit your deadline if you’re lucky, but a good plan doesn’t need luck.

Even worse, the arbitrary deadline and lack of direction gives the entire project a sense of panic. I think the intention was urgency but urgency implies you know what you need to do and that you need to do it quickly. As we’ve seen, the task above is neither well defined nor assigned. The only clear things in the original email are the version that is broken and the deadline.

However, the biggest sin is questioning the commitment and competence of the people needed to resolve the issue. In my experience, this is rarely the case, yet asking the question can make it true. If you’re not trusted, why put in the extra work? Next time you need to make a change, are you going to do it the “right” way or the way with the absolute lowest risk? Putting in a lot of good work and then getting kicked for your efforts is not a good incentive for doing a job well.

Project versus Product

With the fuss about the Log4Shell vulnerability finally dying down, it’s time to step back and take a good, long think about what happened and, more importantly, what can be done to stop it from happening again.

Sadly the prognosis is not good. The tl;dr is both simple and obvious: we simultaneously like free stuff and getting paid for our own work.

Most companies treat open source software exactly the same as commercial software but with a much lower purchase cost. When the software goes wrong, we want someone else to fix it for us. Unfortunately, sometimes we don’t even know where the software comes from. In the case of log4j, it’s run by volunteers. There is no 24/7 help desk with eager employees waiting to take your call.

But even if there is a company that backs the project, one that does have engineering, QA and support staff, is it reasonable to expect an immediate fix to a vulnerability?

Whether a company backs it or not, using open source software is more like being part of a community than being a customer. I came across this phrase about free software a few years ago: “If it breaks, then you get to keep both pieces.” It’s very apt here.

My small part in trying to fix this perception is by calling free software projects and commercially supported versions products or services.

The idea here is that the word “project” implies some degree of a work in progress, one that requires effort from all stakeholders.

In my Day Job1, I often see companies expecting to get commercial quality support for free because the software is free. By “commercial quality” I don’t mean “good.” The level of knowledge and support provided by most free projects is phenomenal. Instead, I mean that there are service-level agreements and guarantees of service within a particular time frame.

As it says in “Cloud Without Compromise”:

But there is something to the old adage that “You get what you pay for.” (… there’s a world of difference between building systems for pet projects versus designing for the needs of enterprise.)

Those service level agreements come with a cost. If you want a fix or an enhancement, you’re welcome to ask a project for it, but it might never come or, if it does, maybe not on a timescale that helps you. The rules change when you pay for the help.

My hope is that those companies that only ever take from open source projects and never contribute learned a lesson. By helping keep the project healthy, you don’t even need to be altruistic. Think of it as insurance.

If you use the software, there are many ways you can “pay it forward”: share enhancements and fixes, write documentation, share your knowledge and experience to bring in more users, or help other users with community support. But if there’s a mechanism to pay for it, the simplest way for most users is cold, hard cash.


  1. An open source database, so I have both skin in the game and bias. However, I think I’d be saying the same thing even if I worked for a vendor of closed source software. ↩︎

Security by Scapegoat

As is common these days, I was complaining about something on Twitter.

It’s easy to complain about security practices which, if I’m honest, is why I do it. But there is an important point, one that I included in a follow-up tweet:

The security team in many companies models itself on the DUP. Say no to everything. But – and this is the key – offer no alternative.

The tweet above is about passwords but I see it everywhere. Another common one is transferring files. I understand why sharing files can be problematic. Confidential data can be exported, either deliberately or accidentally. Viruses can be imported. Security defects can be exploited.

So yes, blocking OneDrive or DropBox is part of the job. What is missing is a legitimate alternative.

Security teams should be enabling staff to safely perform their jobs. Instead, they block the insecure methods and stop.

If I need to share a file or remember a complex password and you don’t provide suitable tools, you did not prevent a security problem. You forced people to write their password on a PostIt note and stick it on their monitor. You pushed someone to use some dodgy new file sharing service that you haven’t heard of.

In the attempts to make the system more secure, you, best case, prevented someone from doing their job. Worst case, you pushed someone into doing something insecure.

In either case, you effectively delegated security to everyone else.