Category Archives: Opinion

Thoughts on computers and the IT industry.

Opinion

Ops is undervalued

I made the mistake of suggesting that there was a blog to be made from this tweet. This is that post.

People still underestimate the value in (Developer|Operator) Experience when building platforms and honestly it’s kinda shocking to me.

If you want to win mindshare you need to make your tools actually usable. If you don’t want to lose it you need the same.

First: I agree with the sentiment. Maybe not to the same extent as Danielle, but I fight the same battles in my day job. I wanted to say this now because, on reading the rest, you may think the opposite. What follows is an explanation of why this is a common situation. I don’t mean it as a justification.

In summary, making software work for ops teams is not a focus either for software companies or internal development teams because of at least three reasons:

  • It’s not a business driver
  • Ops are not the buyer
  • Engineering is run by developers

Naturally there are exceptions to these rules. Every company is different. These are observations, not rules.

Working at the “coal face” it’s easy to forget, but people don’t buy software. They buy a solution to a business problem1. These days, that solution is often software but you don’t buy a new product because it’s cool2.

A driver might be to get a report generated more quickly, or to provide a new service to paying customers, or to reduce the costs of some infrastructure3.

But, you argue, the ops team are the people who keep the system up and running. How can this new system generate reports or deliver a service if it’s not running reliably or has not been provisioned correctly?

You’d be right. Sadly, organisations are often not structured to recognise that fact. The IT teams tasked with making everything run smoothly are frequently in a completely different reporting hierarchy from “the business.” I put “the business” in quotes as I hear it described that way all the time, but it’s this us-versus-them philosophy that brings many issues, including how ops get sidelined4.

With “the business” and “ops” being in separate reporting structures, one or the other has to sign off on the purchase of new software (unless it goes way up the organisation) and that’s always going to be “the business.”

The buyers normally consult the ops team, but ultimately they’re going to put their own needs first. Objections that the ops team have will end up in the business case, but likely in an appendix that no one will ever read.

This makes no sense because, as we all know, most of the expense of a system occurs after go-live. But monitoring, management and deployment are not things at the top of mind of developers and business users.

But even in the IT organisation, the ops team frequently don’t get the attention they deserve either. Anecdotally, this is because IT leadership come from the development team. Their outlook on IT is therefore skewed towards making things rather than keeping them running. I see the same challenges for testing teams. Or, indeed, the lack of testing teams.

This lack of buy-in from the ops team is endemic. I see it in companies buying and using software. I see it in companies that make and sell software5.

At this point, what I would like to be able to do is say, “And the solution to this terrible problem is…” Sadly there is no easy answer. Saying “You should listen to your ops team” is both obvious and unhelpful. Making tools useful for the ops team to get mindshare is a good way to get your software on a shortlist but maybe not enough to clinch that sale.

  1. I’m talking here about software used in a business of course, but the difference between this and personal use isn’t as great as you might initially imagine. You buy a game to solve the “problem” of boredom. Very few people buy software because it’s software. ↩︎
  2. You might buy it because this, specific solution is cooler than the other options. ↩︎
  3. That is, even in the case where it’s the infrastructure that is being improved, the business case is not “make the ops team more efficient” but “make the cost of operations lower.” ↩︎
  4. There’s another blog in how toxic “the business” as a concept is. This isn’t that blog. ↩︎
  5. This is a chicken and egg problem. Do software companies fall into this trap because they’re run by developers? Or is it because they’re selling to companies who have already fallen into the trap? ↩︎
Opinion

What are Registers?

When people say that Twitter is a cesspool of conspiracy and abuse, I don’t recognise it based on my experience. My Twitter timeline is all jokes and geeky chat1, and that’s where this post takes its cue:

When I started learning assembler, no site ever mentioned what registers were good for. Wish it had said:

CPU talking to a RAM chip is slow, registers are a bit of memory built into the CPU in which you load numbers from RAM, do several calculations, and only THEN write back.

I said that this was a RISC-centric approach and was challenged to come up with a better definition.

It’s a harder question to answer than I initially thought. Every time I came up with a pithy definition, I thought of an exception. And with every clarification I got further and further away from 280 characters.

With no character limit, what is a register?

Wikipedia says that it’s “a quickly accessible location available to a computer’s processor,” which is the definition that I was arguing against. Thanks, Wikipedia.

Nevertheless, I maintain that I’m right. Let’s dig into the definition further.

It wasn’t the “quick access” bit I didn’t like. Registers are faster than main memory2. They’re also faster than on-die caches3.

The RISC-y bit was in the second sentence: loading bits of memory, do a few calculations, write back.

To explain why that’s not always true we have to take a quick tour of CPU design history. By necessity I’ve missed out many details or made sweeping generalisations4. I don’t think this detracts from my point.

First, a quick aside: for the sake of completeness, I should point out that there’s nothing sacred about registers. There are CPU architectures that do not have them, primarily stack-based but there may be others. Let’s ignore them for now.

There have been a few waves of CPU and instruction set architecture design, and the use of registers has changed over that time.

Processor Registers Instructions
Intel 40045 16 46
MOS 6502 5 56
Intel 8086 8 ?6
Motorola 68000 15 77
PowerPC 32 ?
ARM 31 ?

In The Beginning, making a CPU at all was a challenge. Getting more than a few thousand transistors on a single chip was hard! The first chips designers optimised for what was possible rather than to make things easy for programmers.

To minimise the number of transistors, there were few registers, and those that were present had predefined uses. One could be used for adding or subtracting (the accumulator). Some could be used to store memory addresses. Others might record the status of other operations. But you couldn’t use the address registers for arithmetic or vice versa.

The answer to the question “what is a register for” at this point is that it saves transistors. By wiring up the logic circuits to a single register and having few of them anyway, you could have enough space to do something.

As it became easier to add transistors to a slice of silicon, CPU designers started to make things easier for programmers. To get the best out of the limited hardware, many developers wrote code in assembler7. Therefore, making it easier for programmers meant adding new, more powerful hardware instructions.

The first CPUs might have had an instruction to “copy address n from memory into a register” and another to “add register 2 to the accumulator.” The next generation built on those foundations with instructions like “add the value at address n to the accumulator and write to address m.” The complexity of instructions grew over time.

Working on these early machines was hard partly because they had few registers and the instructions were simple. The new instructions made things easier by being able to work directly with the values in memory.

These new instructions were not magic, however. Having a single instruction to do the work didn’t make copying data from memory quicker. Developers trying to eke out the best performance had an increasingly difficult time figuring out the best combination of instructions. Is this single instruction that takes twenty cycles faster than these other three instructions that nominally does the same thing?

Around the same time, writing code in higher level languages became increasingly popular. The funny thing with compilers and interpreters is that it’s easier to write and optimise them when you use a limited set of instructions. All those esoteric instructions that were designed for people were rarely used when computers wrote the assembler code.

CPU designers figured out that they could make real-world code execute more quickly by heavily optimising a small number of instructions.

This led to completely different CPU designs, called RISC, or reduced instruction set chips. Rather than have a small number of special purpose registers and a large number of complex instructions, RISC insisted on large numbers of general purpose registers and few instructions8. The reduction in the instruction count was partly achieved by making arithmetic operations only work on registers. If you wanted to add two numbers stored in memory, you first had to load them into registers, add them together and write them back out to memory.

At this point, the answer to the question “what is a register for” changed. It became a sensible option to throw away the transistors used to implement many of the complex instructions and use them for general purpose registers. Lacking instructions that worked directly on memory, the definition of a register became “temporary storage for fast computations” — pretty much what we started with.

Since then, the original designs, with lots of instructions and a small number of registers (CISC), and the newer one, with lots of registers and few instructions (RISC), have to some extent merged. RISC chips have become less “pure” and have more special purpose instructions. CISC chips have gained more registers9 and, internally at least, have taken on many of the attributes traditionally attributed to RISC chips10.

Let’s loop back to the original question. Are registers a bit of memory built into the CPU in which you load numbers from RAM, do several calculations, and only then write back?

We’ve seen how registers are not necessary. We’ve see that their importance has waxed and waned. But, if we had to distill the answer down to a single word or sentence, what would that be?

On current hardware, on most machines, much of the time, the answer is: yes, they are a bit of memory built into the CPU in which you load numbers from RAM, do several calculations, and only then write back.

Was I being pedantic for no reason?

  1. I appreciate that there’s an element of white, male privilege here. ↩︎
  2. Even on architectures like Apple’s M1 chip where the main memory is in the same package as the CPU. ↩︎
  3. I’m going to assume you know a fair few concepts here. My focus is more “how did we get here” than “what do these things do.” ↩︎
  4. While I’ve mostly done this deliberately, I’m sure I’ve done it by accident in a few places too. ↩︎
  5. Four bits wide! ↩︎
  6. It was surprisingly hard to get a count of the instructions for most of these. ↩︎
  7. Machine code is the list of numbers that the CPU understands. Assembler is one level above that, replacing the numbers with mnemonics. Even the mnemonics are considered impenetrable by many programmers. ↩︎
  8. Of all the gross simplifications in this piece, this is probably the biggest. ↩︎
  9. I don’t want to get into all the details here but how they’ve managed to do this without substantially changing the instruction set is both fascinating and architecturally horrifying. Rather than add new instructions to address the new registers, they have a concept called “register renaming” where the register names, but not the values, get reused. ↩︎
  10. Again, without wishing to get into too much detail, modern CISC CPUs typically convert their complex instructions into more RISC-like instructions internally, and execute those. ↩︎
Opinion

Meetings

After university, when I first started working, I jealously noticed people leaving their desks and attending meetings. I was left sitting at my desk, bashing out code. What was going on? What exciting things were being discussed without me? Sometimes they’d come back from the meeting and ask a random question. It was all very mysterious.

A while later I started getting invited to these meetings. I found what was being discussed. I discovered the mystery.

I’ve spent the rest of my career trying to avoid them.

Of course, meetings are not inherently bad. Sharing information, collaborating, making decisions are all vital functions of a company and you need meetings to do that. So why are they often so bad? And why do I spend so much time trying to avoid them?

Meetings are a cultural artefact. Good and bad etiquette isn’t evenly distributed. The companies with the worst meetings are also, ironically, the ones with the most.

What makes a good meeting? There are lots of articles on the web about this, so I don’t want to belabour the point, but, actually, I think it’s quite simple:

  • A defined function
  • The right people
  • The right duration

Missing any one of those means that the meeting is going to be a waste for at least some of those attending.

By “a defined function,” yes, I mean an agenda. It doesn’t necessarily have to be a full and formal written agenda, but all attendees should know the point of the meeting. If they don’t, maybe you do need to write it down. I encourage people to decline meetings with an unclear objective1.

The “right people” to invite to a meeting is often driven by the org chart, but this is completely the wrong metric. You need the fewest people that can meet the objective of the meeting. Don’t include someone just because they’re “important.” Don’t exclude someone because they’re too junior. Include everyone needed to share information or make a decision, or whatever the goal. But no more than that.

One thing that infuriates me is where people in a meeting have no “function.”2 Everyone should have a clearly defined role. If they don’t, they shouldn’t be there.

What about duration? I see two sides to it. First, work expands to fill the time available. Don’t do that. If you set aside an hour for a meeting but it actually only takes ten minutes, quit while you’re ahead. In fact, for people that tend to take a while to get to the point, I’ll deliberately book short meetings.

Conversely, if you’ve spent an hour going around in circles without making a real decision, maybe it’s time to call it a day. Your conclusion should be the information you need to actually make a decision, the people who are going to obtain it and, hopefully, when the next meeting will be.

Talking of “an hour,” that’s my benchmark for maximum meeting length. Anything significantly longer than that suggests to me that there isn’t sufficient focus or a tight enough agenda. And, perhaps more importantly, people are just not going to focus that whole time. They’re going to drift off into a dream world or check their phone. Why have them in the room physically if they’re not present mentally?

It all sounds so simple when you put it like that and yet we’re all guilty of Doing It Wrong. If there’s one thing to take away, it’s that meetings should be deliberate, just like any other corporate artefact.

  1. There are exceptions. For example, I wouldn’t decline a meeting with a client but I would seek clarification. ↩︎
  2. When Dilbert was good, there was a character called the Meeting Moth. I think we’ve all worked with people like that. ↩︎
Opinion

Mismatched

Here’s something I’ve seen a few times recently: a startup issues a patch for a critical issue seen by one of their large customers. The “enterprise,” however, takes a week to install and test it. Clearly, the startup concludes, if it takes a week to try a patch it can’t be that urgent or the staff are dumb, or, quite likely, both.

Separately, we all know that a big difference between a startup and an enterprise is process. So why do people suddenly get angry and start to lack empathy when that difference is exposed?

What we saw in the first paragraph is normal in big companies where you can’t just promote changes into UAT, much less production. It doesn’t matter how loudly you shout at their operations team, it’s not going to make any difference. Maybe the process requires writing test logs and rollback plans. Perhaps it has to be deployed and run in the pre-production environment first. It likely needs sign-off by the QA and security teams. With the best will in the world, this just can’t be done in a few hours, no matter how critical the issue is. Who is to say that the patch isn’t worse than the problem it’s trying to fix?

The difference is frustrating, but don’t mistake tedious process with a lack of urgency or incompetence. Circumventing process can take longer than following it and your client probably knows that. If nothing else, these people might lose their jobs by not following the right process!

Work with it, understand their constraints. This isn’t the time to lose that empathy. It would help if you also had humility and understanding. You know your product but they understand their systems, including how your software interfaces with the other applications they have running in their data centre.

And yes, working with their process is more complex and time-consuming. This is why we charge enterprises more for, ostensibly, the same features.

Opinion

Amazon Fire 7″ (9th gen)

A few years ago we got an Amazon Fire tablet and I could almost copy and paste that review for the ninth generation unit.

My biggest complaint this time around is the battery life. It feels like it’s always in need of recharging. Almost everything else from last time is improved. It’s slightly smaller. The build quality is much better. It’s faster.

Having said that it’s still no iPad. While faster it still feels sluggish compared with Apple’s tablet, the screen is a lot worse and the software library is laughable by comparison. But, as before, it’s also a tenth of the price. As an almost disposable consumption device, I have few complaints.

Opinion

Innovation department

When I see a company that has an “innovation team” or a “chief innovation officer” I immediately understand that it’s not the kind of company I want to work for.

Innovation isn’t found in a particular team, person or department. It’s your culture.

If you need a special team outside the normal management structure to innovate, what does that say?