Tag Archives: enterprise

Opinion

Project versus Product

With the fuss about the Log4Shell vulnerability finally dying down, it’s time to step back and take a good, long think about what happened and, more importantly, what can be done to stop it from happening again.

Sadly the prognosis is not good. The tl;dr is both simple and obvious: we simultaneously like free stuff and getting paid for our own work.

Most companies treat open source software exactly the same as commercial software but with a much lower purchase cost. When the software goes wrong, we want someone else to fix it for us. Unfortunately, sometimes we don’t even know where the software comes from. In the case of log4j, it’s run by volunteers. There is no 24/7 help desk with eager employees waiting to take your call.

But even if there is a company that backs the project, one that does have engineering, QA and support staff, is it reasonable to expect an immediate fix to a vulnerability?

Whether a company backs it or not, using open source software is more like being part of a community than being a customer. I came across this phrase about free software a few years ago: “If it breaks, then you get to keep both pieces.” It’s very apt here.

My small part in trying to fix this perception is by calling free software projects and commercially supported versions products or services.

The idea here is that the word “project” implies some degree of a work in progress, one that requires effort from all stakeholders.

In my Day Job1, I often see companies expecting to get commercial quality support for free because the software is free. By “commercial quality” I don’t mean “good.” The level of knowledge and support provided by most free projects is phenomenal. Instead, I mean that there are service-level agreements and guarantees of service within a particular time frame.

As it says in “Cloud Without Compromise”:

But there is something to the old adage that “You get what you pay for.” (… there’s a world of difference between building systems for pet projects versus designing for the needs of enterprise.)

Those service level agreements come with a cost. If you want a fix or an enhancement, you’re welcome to ask a project for it, but it might never come or, if it does, maybe not on a timescale that helps you. The rules change when you pay for the help.

My hope is that those companies that only ever take from open source projects and never contribute learned a lesson. By helping keep the project healthy, you don’t even need to be altruistic. Think of it as insurance.

If you use the software, there are many ways you can “pay it forward”: share enhancements and fixes, write documentation, share your knowledge and experience to bring in more users, or help other users with community support. But if there’s a mechanism to pay for it, the simplest way for most users is cold, hard cash.

  1. An open source database, so I have both skin in the game and bias. However, I think I’d be saying the same thing even if I worked for a vendor of closed source software. ↩︎
Opinion

Security by Scapegoat

As is common these days, I was complaining about something on Twitter.

It’s easy to complain about security practices which, if I’m honest, is why I do it. But there is an important point, one that I included in a follow-up tweet:

The security team in many companies models itself on the DUP. Say no to everything. But – and this is the key – offer no alternative.

The tweet above is about passwords but I see it everywhere. Another common one is transferring files. I understand why sharing files can be problematic. Confidential data can be exported, either deliberately or accidentally. Viruses can be imported. Security defects can be exploited.

So yes, blocking OneDrive or DropBox is part of the job. What is missing is a legitimate alternative.

Security teams should be enabling staff to safely perform their jobs. Instead, they block the insecure methods and stop.

If I need to share a file or remember a complex password and you don’t provide suitable tools, you did not prevent a security problem. You forced people to write their password on a PostIt note and stick it on their monitor. You pushed someone to use some dodgy new file sharing service that you haven’t heard of.

In the attempts to make the system more secure, you, best case, prevented someone from doing their job. Worst case, you pushed someone into doing something insecure.

In either case, you effectively delegated security to everyone else.

Opinion

Why Enterprise Software is Bloated

I confess that I’ve stolen the title of the post from elsewhere. My objective here is not to detract from that post, which is great, but to build on a few points that I thought it was going to make but didn’t. To make it clear where I’m coming from, here’s a Tweet that I wrote some time before I read that blog:

People who complain that “enterprise” software is too expensive have clearly never come across the bizarre, arbitrary and nonsensical policies and rules these companies have. It’s not unusual to have two customers with contradictory policies.

Alex’s post assumes that bloat is unnecessary and could be optimised away with better engineering. There are always counter-examples, but, in general, this is not my experience.

The key is to realise two facts:

  1. Large companies have lots of rules and procedures
  2. Large companies have a lot of leverage over software companies

My favourite rules and processes are always about corporate security:

The game of cat and mouse begins: I need to send a file to a client, but which method will actually make it through their corporate security? So far I’m down 2–0 but I’m not giving up yet.

Corporate security is focused very much on stopping Bad Things from happening. Undoubtedly that’s one part of the job. The part that usually gets forgotten is to allow people to do their jobs securely. Without these “safe routes,” the security is making everyone’s life harder and increasing costs but without improving security.

Here’s a lightly fictionalised example. A client has a production outage. We need to see their log files to diagnose the problem. Dropbox, Box and Google Drive are all blocked. They don’t have a corporate file share. It’s not possible to email them, because the logs are large and ZIP files are forbidden. We can’t screen share, since they blocked Zoom (our tool) and corporate policy means they can’t open a Team’s call with outside people. There is a special dispensation for this, but it’s only available for client-facing staff. And, in any case, needs to be applied for in advance. In the end, we went Old Skool, zipped the files and renamed them with a “TXT” file extension and that worked.

There’s a lot to unpack here. First: the security measures didn’t work. Despite the obstacles, we managed to transfer the files. It just took multiple well paid people a lot of time to figure out. There was no known supported tool available to facilitate this use case which, let’s not forget, was an outage of one of their production systems, something that customers pay them actual money to be able to access.

As ever, the individual rules make some sense, but together they make a system that doesn’t work for anyone. It annoys staff, adds delays to tasks and in many cases does not work anyway.

Let’s circle back to the original question: why is enterprise software bloated?

The short answer is that vendors need to be able to support enterprises despite all that unique processes. A task, like copying log files, that should take minutes might take hours. Ultimately that needs to be paid for.

Many of the processes and procedures are unique to an individual enterprise. This means that the vendor has no option but to write custom software to accommodate them. These changes get folded back into the product and every other customer gets a copy of them, but hidden behind a feature flag.

Multiply this by every customer and you end up with a vast number of options, the majority of which are not used by most customers. It’s a bit crazy. It’s bloated, hard to keep track of and hard for the vendor to test.

If everyone recognises the problem, what is the solution?

Some companies just lay down the law: this is the right way to do something; take it or leave it. You can do this for consumer software or if you have millions of customers. There’s only so much negotiation you can do with Microsoft for your Windows or Office licence.

Companies that do this for “enterprise” software tend not to be very successful. There’s a reason that Google Cloud is a distant third, well behind AWS and Azure.

You can make “consultingware,” that is, software with a core that it heavily modified for each customer. At a certain point you don’t even have a product any more, with disadvantages for both the vendor and the user.

Or you can make a single product with switches that enable or disable all these esoteric options. Either way, all your customers end up paying extra for the privilege.

In the end, as long as big companies continue to customise their environments in mutually incompatible ways, “enterprise software” is going to be expensive and bloated.

Opinion

Mismatched

Here’s something I’ve seen a few times recently: a startup issues a patch for a critical issue seen by one of their large customers. The “enterprise,” however, takes a week to install and test it. Clearly, the startup concludes, if it takes a week to try a patch it can’t be that urgent or the staff are dumb, or, quite likely, both.

Separately, we all know that a big difference between a startup and an enterprise is process. So why do people suddenly get angry and start to lack empathy when that difference is exposed?

What we saw in the first paragraph is normal in big companies where you can’t just promote changes into UAT, much less production. It doesn’t matter how loudly you shout at their operations team, it’s not going to make any difference. Maybe the process requires writing test logs and rollback plans. Perhaps it has to be deployed and run in the pre-production environment first. It likely needs sign-off by the QA and security teams. With the best will in the world, this just can’t be done in a few hours, no matter how critical the issue is. Who is to say that the patch isn’t worse than the problem it’s trying to fix?

The difference is frustrating, but don’t mistake tedious process with a lack of urgency or incompetence. Circumventing process can take longer than following it and your client probably knows that. If nothing else, these people might lose their jobs by not following the right process!

Work with it, understand their constraints. This isn’t the time to lose that empathy. It would help if you also had humility and understanding. You know your product but they understand their systems, including how your software interfaces with the other applications they have running in their data centre.

And yes, working with their process is more complex and time-consuming. This is why we charge enterprises more for, ostensibly, the same features.