Category Archives: Computing

Articles about computers and the IT industry.

Opinion

Adventures in iCloud Mail Hosting

How does switching email hosts disable your Bluetooth headphones? Read on to find out.

As many people did recently, I got The Email from Google telling me that my Apps for Domains (Legacy) account is going away and that I should either pay up or move away.

I’m not averse to paying. I use email a lot and I have my own domain, so I appreciate that I’m not a typical consumer. But I do object to paying Google because it feels like they’re double-dipping: both data mining my information and billing me for the service1.

In short, I’m not going to stay with Google.

Since I already pay for iCloud — as part of the Apple One bundle — moving my email there would be the obvious choice. Being able to use your own domain is a recent feature, part of iCloud to use their current branding.

Unfortunately the documentation isn’t great. And there are technical gotchas. Let’s walk through my experience.

Step 1, enter your domain name. Okay, check.

Step 2, enter any existing email addresses. I entered my main address and it told me that it wasn’t allowed. As others have noted, the system does return useful error messages, however those are not displayed on the screen! I was left guessing, which is incredibly frustrating.

I figured – partly luck, partly a process of elimination – that the email address was associated with another Apple ID. I logged into my other Apple IDs and removed references to my address, but to no avail. It continued to reject me.

The difficulty is that I’ve been using Apple’s web services for a long time. I have accounts that date back twenty years, before iCloud, before MobileMe, before your Apple ID even needed to be an email address. My main email address predates even that.

Luckily Apple has a “find your Apple ID” tool that tells you the Apple accounts that an email address is associated with. It turned up two accounts that I have no recollection of ever creating. One was a pre-email address account with a very bizarre name. It was an odd reference, but one I recognised so it was certainly me!

I requested that those two accounts get deleted. At that point, the “add domain” screen allowed me to continue.

My hope for step 3 What was to add some email aliases. The way I had my mail set up at Google was with a single mailbox but with multiple aliases. In practice, most of the aliases were just “wildcard” addresses, that is, if it found an address it didn’t know it would send it to the main mailbox. I knew that (bizarrely) Apple’s iCloud didn’t do that. I’m not totally happy with that but, for the way I use email, it’s not an absolute dealbreaker.

Except. You can’t have aliases. I can create an alias to my iCloud address, somethingelse@icloud.com, but not somethingelse@mydomain.com.

This is a problem. Over the years, I’ve removed almost all of the aliases, but one I use almost every day is the one attached to my Apple ID!

I go through my last few months of email – that was a fun afternoon – updated my address at a handful of companies so that the only remaining required alias was the one for my Apple ID.

I have payments and purchases and subscriptions for the entire family associated with this account. I know that in theory I can change the ID but in practice it fills me with dread.

If I’m to move my email, however, this is a necessary step. That it’s the one remaining required alias and that it might make things easier moving to any other email provider pushes me to attempt it.

It wasn’t as difficult or as problematic as I feared. All the work I did previously, removing all traces of the “old” Apple IDs, meant that the change largely Just Worked.

For the next day I found myself signing into all my devices again. The most surprising was when I started listening to a podcast and two minutes later my AirPods suddenly stopped working. I tried turning them off and on again – all the usual diagnostic tools – but couldn’t get them going. I was stood in the middle of the street so I didn’t want to get too in depth.

When I got home I realised they stopped working because they were connected to my Apple ID. I re-paired them with my phone and the audio immediately started to play. Of all the things I expected to stop working when I changed the account, my earbuds were not on the list!

As I write this, it has been about three weeks since I “flipped the switch” and moved over to iCloud email. My Google account is still live – I can switch back if something is utterly broken – but I have not done so. I guess it’s possible that I’ve missed some emails, but I would have no idea if I did, of course!

Despite the initial teething problems I’ve written about above, it’s been working well since then. It even supports push email rather than polling periodically. Not critical, but a nice feature.

Would I recommend it? With reservations. It’s not going to work as a replacement for Google for some people. And both the software and the documentation needs work. The feature set is limited, the backend supports more detailed diagnostics than the front-end and the documentation assumes that you don’t have a mess of Apple IDs like I do.

Let’s hope that Apple work on this. Sadly, I don’t think it’s going to be an overnight thing. There are a bunch of issues here, many of which appear to be a consequence of decisions made years ago.

  1. The argument may well go that they do not data mine paid accounts but I have no ability to verify that. Google have done enough dodgy things with data that I don’t trust them. Pick one. ↩︎
Opinion

Ops is undervalued

I made the mistake of suggesting that there was a blog to be made from this tweet. This is that post.

People still underestimate the value in (Developer|Operator) Experience when building platforms and honestly it’s kinda shocking to me.

If you want to win mindshare you need to make your tools actually usable. If you don’t want to lose it you need the same.

First: I agree with the sentiment. Maybe not to the same extent as Danielle, but I fight the same battles in my day job. I wanted to say this now because, on reading the rest, you may think the opposite. What follows is an explanation of why this is a common situation. I don’t mean it as a justification.

In summary, making software work for ops teams is not a focus either for software companies or internal development teams because of at least three reasons:

  • It’s not a business driver
  • Ops are not the buyer
  • Engineering is run by developers

Naturally there are exceptions to these rules. Every company is different. These are observations, not rules.

Working at the “coal face” it’s easy to forget, but people don’t buy software. They buy a solution to a business problem1. These days, that solution is often software but you don’t buy a new product because it’s cool2.

A driver might be to get a report generated more quickly, or to provide a new service to paying customers, or to reduce the costs of some infrastructure3.

But, you argue, the ops team are the people who keep the system up and running. How can this new system generate reports or deliver a service if it’s not running reliably or has not been provisioned correctly?

You’d be right. Sadly, organisations are often not structured to recognise that fact. The IT teams tasked with making everything run smoothly are frequently in a completely different reporting hierarchy from “the business.” I put “the business” in quotes as I hear it described that way all the time, but it’s this us-versus-them philosophy that brings many issues, including how ops get sidelined4.

With “the business” and “ops” being in separate reporting structures, one or the other has to sign off on the purchase of new software (unless it goes way up the organisation) and that’s always going to be “the business.”

The buyers normally consult the ops team, but ultimately they’re going to put their own needs first. Objections that the ops team have will end up in the business case, but likely in an appendix that no one will ever read.

This makes no sense because, as we all know, most of the expense of a system occurs after go-live. But monitoring, management and deployment are not things at the top of mind of developers and business users.

But even in the IT organisation, the ops team frequently don’t get the attention they deserve either. Anecdotally, this is because IT leadership come from the development team. Their outlook on IT is therefore skewed towards making things rather than keeping them running. I see the same challenges for testing teams. Or, indeed, the lack of testing teams.

This lack of buy-in from the ops team is endemic. I see it in companies buying and using software. I see it in companies that make and sell software5.

At this point, what I would like to be able to do is say, “And the solution to this terrible problem is…” Sadly there is no easy answer. Saying “You should listen to your ops team” is both obvious and unhelpful. Making tools useful for the ops team to get mindshare is a good way to get your software on a shortlist but maybe not enough to clinch that sale.

  1. I’m talking here about software used in a business of course, but the difference between this and personal use isn’t as great as you might initially imagine. You buy a game to solve the “problem” of boredom. Very few people buy software because it’s software. ↩︎
  2. You might buy it because this, specific solution is cooler than the other options. ↩︎
  3. That is, even in the case where it’s the infrastructure that is being improved, the business case is not “make the ops team more efficient” but “make the cost of operations lower.” ↩︎
  4. There’s another blog in how toxic “the business” as a concept is. This isn’t that blog. ↩︎
  5. This is a chicken and egg problem. Do software companies fall into this trap because they’re run by developers? Or is it because they’re selling to companies who have already fallen into the trap? ↩︎
Opinion

What are Registers?

When people say that Twitter is a cesspool of conspiracy and abuse, I don’t recognise it based on my experience. My Twitter timeline is all jokes and geeky chat1, and that’s where this post takes its cue:

When I started learning assembler, no site ever mentioned what registers were good for. Wish it had said:

CPU talking to a RAM chip is slow, registers are a bit of memory built into the CPU in which you load numbers from RAM, do several calculations, and only THEN write back.

I said that this was a RISC-centric approach and was challenged to come up with a better definition.

It’s a harder question to answer than I initially thought. Every time I came up with a pithy definition, I thought of an exception. And with every clarification I got further and further away from 280 characters.

With no character limit, what is a register?

Wikipedia says that it’s “a quickly accessible location available to a computer’s processor,” which is the definition that I was arguing against. Thanks, Wikipedia.

Nevertheless, I maintain that I’m right. Let’s dig into the definition further.

It wasn’t the “quick access” bit I didn’t like. Registers are faster than main memory2. They’re also faster than on-die caches3.

The RISC-y bit was in the second sentence: loading bits of memory, do a few calculations, write back.

To explain why that’s not always true we have to take a quick tour of CPU design history. By necessity I’ve missed out many details or made sweeping generalisations4. I don’t think this detracts from my point.

First, a quick aside: for the sake of completeness, I should point out that there’s nothing sacred about registers. There are CPU architectures that do not have them, primarily stack-based but there may be others. Let’s ignore them for now.

There have been a few waves of CPU and instruction set architecture design, and the use of registers has changed over that time.

Processor Registers Instructions
Intel 40045 16 46
MOS 6502 5 56
Intel 8086 8 ?6
Motorola 68000 15 77
PowerPC 32 ?
ARM 31 ?

In The Beginning, making a CPU at all was a challenge. Getting more than a few thousand transistors on a single chip was hard! The first chips designers optimised for what was possible rather than to make things easy for programmers.

To minimise the number of transistors, there were few registers, and those that were present had predefined uses. One could be used for adding or subtracting (the accumulator). Some could be used to store memory addresses. Others might record the status of other operations. But you couldn’t use the address registers for arithmetic or vice versa.

The answer to the question “what is a register for” at this point is that it saves transistors. By wiring up the logic circuits to a single register and having few of them anyway, you could have enough space to do something.

As it became easier to add transistors to a slice of silicon, CPU designers started to make things easier for programmers. To get the best out of the limited hardware, many developers wrote code in assembler7. Therefore, making it easier for programmers meant adding new, more powerful hardware instructions.

The first CPUs might have had an instruction to “copy address n from memory into a register” and another to “add register 2 to the accumulator.” The next generation built on those foundations with instructions like “add the value at address n to the accumulator and write to address m.” The complexity of instructions grew over time.

Working on these early machines was hard partly because they had few registers and the instructions were simple. The new instructions made things easier by being able to work directly with the values in memory.

These new instructions were not magic, however. Having a single instruction to do the work didn’t make copying data from memory quicker. Developers trying to eke out the best performance had an increasingly difficult time figuring out the best combination of instructions. Is this single instruction that takes twenty cycles faster than these other three instructions that nominally does the same thing?

Around the same time, writing code in higher level languages became increasingly popular. The funny thing with compilers and interpreters is that it’s easier to write and optimise them when you use a limited set of instructions. All those esoteric instructions that were designed for people were rarely used when computers wrote the assembler code.

CPU designers figured out that they could make real-world code execute more quickly by heavily optimising a small number of instructions.

This led to completely different CPU designs, called RISC, or reduced instruction set chips. Rather than have a small number of special purpose registers and a large number of complex instructions, RISC insisted on large numbers of general purpose registers and few instructions8. The reduction in the instruction count was partly achieved by making arithmetic operations only work on registers. If you wanted to add two numbers stored in memory, you first had to load them into registers, add them together and write them back out to memory.

At this point, the answer to the question “what is a register for” changed. It became a sensible option to throw away the transistors used to implement many of the complex instructions and use them for general purpose registers. Lacking instructions that worked directly on memory, the definition of a register became “temporary storage for fast computations” — pretty much what we started with.

Since then, the original designs, with lots of instructions and a small number of registers (CISC), and the newer one, with lots of registers and few instructions (RISC), have to some extent merged. RISC chips have become less “pure” and have more special purpose instructions. CISC chips have gained more registers9 and, internally at least, have taken on many of the attributes traditionally attributed to RISC chips10.

Let’s loop back to the original question. Are registers a bit of memory built into the CPU in which you load numbers from RAM, do several calculations, and only then write back?

We’ve seen how registers are not necessary. We’ve see that their importance has waxed and waned. But, if we had to distill the answer down to a single word or sentence, what would that be?

On current hardware, on most machines, much of the time, the answer is: yes, they are a bit of memory built into the CPU in which you load numbers from RAM, do several calculations, and only then write back.

Was I being pedantic for no reason?

  1. I appreciate that there’s an element of white, male privilege here. ↩︎
  2. Even on architectures like Apple’s M1 chip where the main memory is in the same package as the CPU. ↩︎
  3. I’m going to assume you know a fair few concepts here. My focus is more “how did we get here” than “what do these things do.” ↩︎
  4. While I’ve mostly done this deliberately, I’m sure I’ve done it by accident in a few places too. ↩︎
  5. Four bits wide! ↩︎
  6. It was surprisingly hard to get a count of the instructions for most of these. ↩︎
  7. Machine code is the list of numbers that the CPU understands. Assembler is one level above that, replacing the numbers with mnemonics. Even the mnemonics are considered impenetrable by many programmers. ↩︎
  8. Of all the gross simplifications in this piece, this is probably the biggest. ↩︎
  9. I don’t want to get into all the details here but how they’ve managed to do this without substantially changing the instruction set is both fascinating and architecturally horrifying. Rather than add new instructions to address the new registers, they have a concept called “register renaming” where the register names, but not the values, get reused. ↩︎
  10. Again, without wishing to get into too much detail, modern CISC CPUs typically convert their complex instructions into more RISC-like instructions internally, and execute those. ↩︎
Computing

Unsociable Christmas Tree

Last year I got myself a Raspberry Pi-powered Christmas Tree. It has eleven LEDs, and you can program the Pi to switch them on and off.

Naturally, doing all that takes time, and last year I just didn’t have very much. I just downloaded the sample project and set it up with a random flashing pattern.

It amused me, anyway.

This year I wanted to get a little more sophisticated. I decided that it should be interactive. My first thought was a web server where people could connect using their phones and change the LED patterns. Then I thought better of it. Because of COVID we have no guests, rendering it far less interesting. Also, setting up a web server is hardly very exciting.

I wanted it to detect something but options were somewhat limited. The tree connects to the Pi’s pin connectors, but it didn’t leave any pins free to plug in anything else.

Next, I looked at my Arduino components. Could I do the sensing on the Arduino and the lights on the Pi? A sensible argument would be that wiring up two small computers like that would be ridiculously and unnecessarily complicated. While you wouldn’t be wrong, the whole point of this exercise is ridiculous.

In my Arduino bag of tricks, I have distance, presence, light and moisture sensors, buttons, switches and displays. I could have cobbled together something but while reading Twitter I got a better idea: how about I plug in a webcam and have the Pi detect faces and show different patterns depending on who is looking at it?

Luckily, other people have done most of the hard work. I based most of what follows on a blog “How to train your Raspberry Pi for facial recognition.”

There are lots of steps, but they’re relatively easy to follow. It was all working fine until it asked me to build OpenCV from source, at which point I ran out of disk space.

I gave up for a while.

What I’m saying is that while I was planning on something a little more elaborate, I ran out of time. Again.

When I came back to it, I realised that OpenCV was something that many Raspberry Pi owners were likely to use and I was surprised that I had to build my own version.

Luckily my surprise was supported by the actual Raspberry Pi software archive: if you install python3-opencv you get all the libraries you need, and all without all the hassle of having to build your own1. As a side benefit, this removes about half of the steps in the tutorial!

The rest worked incredibly well “out of the box.” I ran it, trained it on a few unsuspecting family members and was very impressed that it worked the first time. It uses a lot of CPU on my Pi 4, so I’m not sure that it would work on any of the earlier models.

My next task was to hook in the Christmas Tree code so that the tree responds to changes in what the webcam could see. And… that’s where I ran out of time.

The interface between the facial-recognition and the tree lights is, well, minimal. If it finds someone it recognises, all the lights come on, otherwise, it goes dark. You can see the code on GitHub — only a handful of lines are mine.

It technically meets the requirements of an Unsociable Christmas Tree but is certainly less ambitious than I would have liked. Still, getting machine learning working on a Pi and connecting it to something physical was fun. Maybe next year I’ll get the time to bring everything together?

  1. In theory, installing python3-opencv means you can skip the whole of point 4, “Install OpenCV by running the following commands in your Terminal,” from the guide. In practice, I tried to build my own version of OpenCV so it’s possible that I have extra libraries installed that you also need. If I get the time, I’ll come back and try this on a default installation of Raspberry Pi OS. ↩︎

Opinion

Meetings

After university, when I first started working, I jealously noticed people leaving their desks and attending meetings. I was left sitting at my desk, bashing out code. What was going on? What exciting things were being discussed without me? Sometimes they’d come back from the meeting and ask a random question. It was all very mysterious.

A while later I started getting invited to these meetings. I found what was being discussed. I discovered the mystery.

I’ve spent the rest of my career trying to avoid them.

Of course, meetings are not inherently bad. Sharing information, collaborating, making decisions are all vital functions of a company and you need meetings to do that. So why are they often so bad? And why do I spend so much time trying to avoid them?

Meetings are a cultural artefact. Good and bad etiquette isn’t evenly distributed. The companies with the worst meetings are also, ironically, the ones with the most.

What makes a good meeting? There are lots of articles on the web about this, so I don’t want to belabour the point, but, actually, I think it’s quite simple:

  • A defined function
  • The right people
  • The right duration

Missing any one of those means that the meeting is going to be a waste for at least some of those attending.

By “a defined function,” yes, I mean an agenda. It doesn’t necessarily have to be a full and formal written agenda, but all attendees should know the point of the meeting. If they don’t, maybe you do need to write it down. I encourage people to decline meetings with an unclear objective1.

The “right people” to invite to a meeting is often driven by the org chart, but this is completely the wrong metric. You need the fewest people that can meet the objective of the meeting. Don’t include someone just because they’re “important.” Don’t exclude someone because they’re too junior. Include everyone needed to share information or make a decision, or whatever the goal. But no more than that.

One thing that infuriates me is where people in a meeting have no “function.”2 Everyone should have a clearly defined role. If they don’t, they shouldn’t be there.

What about duration? I see two sides to it. First, work expands to fill the time available. Don’t do that. If you set aside an hour for a meeting but it actually only takes ten minutes, quit while you’re ahead. In fact, for people that tend to take a while to get to the point, I’ll deliberately book short meetings.

Conversely, if you’ve spent an hour going around in circles without making a real decision, maybe it’s time to call it a day. Your conclusion should be the information you need to actually make a decision, the people who are going to obtain it and, hopefully, when the next meeting will be.

Talking of “an hour,” that’s my benchmark for maximum meeting length. Anything significantly longer than that suggests to me that there isn’t sufficient focus or a tight enough agenda. And, perhaps more importantly, people are just not going to focus that whole time. They’re going to drift off into a dream world or check their phone. Why have them in the room physically if they’re not present mentally?

It all sounds so simple when you put it like that and yet we’re all guilty of Doing It Wrong. If there’s one thing to take away, it’s that meetings should be deliberate, just like any other corporate artefact.

  1. There are exceptions. For example, I wouldn’t decline a meeting with a client but I would seek clarification. ↩︎
  2. When Dilbert was good, there was a character called the Meeting Moth. I think we’ve all worked with people like that. ↩︎
Opinion

Mismatched

Here’s something I’ve seen a few times recently: a startup issues a patch for a critical issue seen by one of their large customers. The “enterprise,” however, takes a week to install and test it. Clearly, the startup concludes, if it takes a week to try a patch it can’t be that urgent or the staff are dumb, or, quite likely, both.

Separately, we all know that a big difference between a startup and an enterprise is process. So why do people suddenly get angry and start to lack empathy when that difference is exposed?

What we saw in the first paragraph is normal in big companies where you can’t just promote changes into UAT, much less production. It doesn’t matter how loudly you shout at their operations team, it’s not going to make any difference. Maybe the process requires writing test logs and rollback plans. Perhaps it has to be deployed and run in the pre-production environment first. It likely needs sign-off by the QA and security teams. With the best will in the world, this just can’t be done in a few hours, no matter how critical the issue is. Who is to say that the patch isn’t worse than the problem it’s trying to fix?

The difference is frustrating, but don’t mistake tedious process with a lack of urgency or incompetence. Circumventing process can take longer than following it and your client probably knows that. If nothing else, these people might lose their jobs by not following the right process!

Work with it, understand their constraints. This isn’t the time to lose that empathy. It would help if you also had humility and understanding. You know your product but they understand their systems, including how your software interfaces with the other applications they have running in their data centre.

And yes, working with their process is more complex and time-consuming. This is why we charge enterprises more for, ostensibly, the same features.