Tag Archives: Opinion

A new CEO for Yahoo?

Rumour has it that Yahoo! are looking for a new CEO. Some people have been putting their name forward for the role, or at least offering suggestions for Carol Bartz’s successor. This post is in response to Joe Stumps list of ideas.

To be clear, I know that list is not completely serious. I know that he’s not really angling for the CEO role and I understand that many of the options would not be achievable even if they were the best thing for Yahoo! That’s not the point I’m trying to make.

The point, in summary, is that buying a bunch of companies to get smart people is not going to fix Yahoo!s problems.

Let’s look at some of the suggestions and, more importantly, how they inter-relate.

I’d buy Instagram and put them in charge of both Instagram and Flickr. They would have 100% autonomy over the entire “Yahoo! Photo” division.

Fine. Instagram has done really well. But what makes it successful? (Assuming that it is successful. So far it has managed to attract a lot of users but there’s no revenue stream as far as I can see.)

Is it the photo part? Well, partially. It has filters that people like playing around with. Another key to its success is the sharing, social side. But…

I’d buy Path and With for the sole reason of bringing Dave and his team on to lead the new “Yahoo! Social” division.

What’s the direction of the company if “Yahoo! Photo” and “Yahoo! Social” are both doing social stuff? Who decides how to share photos or videos?

And how is social distinct from mobile?

I would buy Twitter and Square in order to bring Jack Dorsey on full-time to run a new division called “Yahoo! Mobile.” He would have 100% autonomy over the entire mobile strategy.

Part of the success of both Instagram and Path are the fact that they’re “mobile.”

Mobile and social are not divisions any more than a technology company should have an “internet” division; they are fundamentals that need to influence all modern web “properties.”

Buying companies is not a solution. They’ve bought plenty over the years, but that didn’t help. What happened to Flickr? How “Yahoo!” is it? What about Delicious?

What Yahoo! lacks is not smart people or good technology, it’s a coherent way of tying everything together. Unfortunately that can’t be fixed by maintaining existing fiefdoms or importing new ones.

Crash

It’s nearly four years old now, so I do expect the odd beach ball occasionally. When my MacBook is doing something hard or complex or just opening iTunes, it often shows its “I’m too busy to respond to you right now” indicator. But this time it was different. The beachball appeared and didn’t really go away again. Sure, it occasionally hid but as soon as I instructed the machine to do anything it would return.

And I do mean anything. From accessing a menu item, to quitting, to switching application, everything resulted in the beachball returning.

I powered off for the night, optimistically hoping that it was transitory, maybe a software failure that a simple reboot would fix.

The picture above and the title of this post probably tells you that it wasn’t.

Fortunately I was pretty well prepared and I lost little more than a few days use of my MacBook and less than £50. My backup regime has changed a little since I last wrote about it so it’s probably worth discussing what I’m doing these days.

Previously I used SuperDuper to clone my main disk. The nice thing about this is that there is a full, bootable copy of my complete Mac. The disadvantage is the small print surrounding the word “complete.”

It’s absolutely complete, but only to the point that the last backup was actually made. Using SuperDuper is fairly quick and easy, but it’s still a manual process that needs to be initiated manually.

Fortunately I realised this before I lost any data. A couple of years ago, shortly after I upgraded to Mac OS X 10.5, I also picked up a Time Capsule. Long story short, a Time Capsule is a wireless router with a built-in disk. And 10.5 has a feature called “Time Machine” that automatically performs hourly, incremental backups.

Since the disk in my MacBook went so suddenly this was brilliant. I would lose a maximum of one hours work with only one proviso: that the backup actually worked.

But skipping straight to actually restoring the backup would be missing a few steps. First I had to replace the broken disk.

CPUs and disks are things that I understand more in theory than in practice. I know that CPUs come in different clock speeds and with varying numbers of cores but if you start to discuss Intel part number I’m going to reply with a blank look of incomprehension. Similarly, I know that disks come in a number of physical and capacity sizes. So when I was looking at the specifications question such as “What’s the difference between SATA and SATA-II?” were asked. I quickly became confused.

Answers were not entirely forthcoming. In the end I took a bit of a punt and bought twice the capacity (320Gb) with a faster rotational speed (7200rpm). Most disks were SATA-II so I crossed my fingers on that. And nowhere did I see a physical size mentioned so there was no way that I could check to see whether it would actually fit in the aperture left by the old, failed disk.

As luck would have it, it all worked out just fine. Replacing the old disk was suspiciously easy, so much so that having switched over I was wondering which steps I had missed or which vital component had dropped on the floor and whose absence would short out the motherboard.

I booted up from the Snow Leopard install disk and told it to restore from the Time Capsule. On first attempt it couldn’t find a disk to install on to. I missed a step. I went back and formatted the new, 320Gb disk.

I let out a big sigh of relief when I realised it could find and format the disk. I guess that meant by confusion over SATA and SATA-II wasn’t going to be a problem.

This time the restore started. It would, the installer said, take about eight hours to complete.

I don’t know exactly how long it took but it was ready the next morning.

I rebooted.

It looked good.

My familiar desktop was there. The backdrop was there, the few documents and folders that I had there before the crash were still there now. The dock showed all my old applications. I poked around my Documents folder.

All good.

Phew.

I clicked on Mail. Slight panic when the setup dialog appeared, and then I realised that Time Machine would not have backed up in the mail index. The messages were all there. It churned away for a while, processing all the messages. It finished. And then it crashed.

I restarted.

It re-crashed.

Okay, let’s Google the error message.

Ah, Safari crashes when launching too.

Everything with an embedded web view crashed either on launch or when the web component was used. Even the Software Update app. Worrying, but not cause for concern yet. Using Firefox, which did still work, I downloaded the 10.6.4 updater from the Apple website and proceeded to install.

In doing so, I learned something new. Did you know that the installer application has an embedded web view?

Luckily the command-line does not scare me — I have a history of voluntarily using Emacs — and I found the appropriate incantation to install the patch the hard way.

It didn’t work. Applications crashed in exactly the same place as before.

In hindsight I probably should have tried installing Safari again but instead I went straight to reinstalling the whole OS from scratch. From past experience I knew that my data was safe. It may have been a brute-force approach, but I was confident that it would work.

And it did.

It took about an hour to do the initial install, then I had to download and install a bunch of updates. Excluding the time it took to copy my data from the Time Capsule to the local disk, it took perhaps three hours to swap out my old disk and get my laptop to pretty much exactly the state it was in when the old disk stopped working.

It’s disappointing that the restore from Time Machine backup option didn’t work in its entirety but, to its credit, I didn’t lose any documents, emails or data. And in that sense it was a complete success.

Delicious Debrief (Part 5/5)

The story so far

Last year Yahoo! announced, with no notice, a significant change that had far reaching consequences for all third party applications including my iPhone program, Yummy. This is the third in a series of posts that discusses how I dealt with it.

We’ve already talked about most of the work, starting with an overview, the announcement, the low level technical challenges and the implementation (technical and UI). All that remains it to launch it, and that’s what we’re going to talk about today.

Availability

I switched my own Delicious account over to use my Yahoo! ID to shake out as many bugs as possible and I’d already had a few emails from people asking if they could help, so by the time I submitted the update to Apple it was in pretty good shape.

It launched on 13 December 2009, simultaneously for the full version and Yummy Browser. I decided against holding back for the free version.

Despite the volume of code that had changed, I decided to call is a “point” update (moving from version 2.3.1 to 2.3.2) because it didn’t really add any new user-level features. In hindsight maybe I should have made more of a fuss over it, since it was the first — and is still the only — Delicious iPhone app that supported Yahoo! IDs. But it felt wrong to kick my competitors over something that was just as outside their control as mine1.

Lessons

So what have we learned through all this process?

I’ve learned how vulnerable Yummy is to outside influence. I had been planning a new feature release before the end of last year but this slipped until the first quarter of 2010 because of all the work that I have outlined above. I’m glad that I’ve kept “agile” (an overused word in the software industry), sticking to relatively small and frequent releases as this made it much easier to change from one feauture set (mostly what you see in the current Yummy 2.4 release) to another (OAuth).

But I think most of the lessons need to be taken on the other side, by Yahoo! From my point of view there were multiple failures:

  1. No notice. It was impossible to have an application ready for day zero as we just didn’t know that a change was coming
  2. No documentation. Even when we found out about the change there was little to no documentation about how to make the change
  3. No concessions to desktop apps. It just is not possible to write a desktop application2 with a fluid, easy to use UI. OAuth works nicely for web apps but not so well for desktop (and mobile) ones. Twitter is pushing forward with xAuth which solves this problem. It’s not yet standardised, which is a problem, but guaranteeing a poor user experience is hardly a winning strategy either
  4. No transition. When Twitter moved from basic auth to OAuth3, both systems worked for over a year. This meant that there was no single, big switch over. True, they incentivised new apps to use the new scheme and the end of basic auth is going to orphan a bunch of apps but, still, there has been a year when developers can transition from one scheme to the other. For Delicious this change happened overnight, but not for all users. Instead we have to support both schemes and with no obvious method of knowing which one to use in advance

Luckily Yahoo! seem to have noticed the pain that their development community have been put through. They have since (albeit very recently) tried to get a handle on who is developing what against their APIs. Quite how they’ll use that information is anyones guess but it’s certainly a step in the right direction.

  1. Six months later and with no updates from any of them I’m feeling less charitable! []
  2. By “desktop” I mean something that doesn’t run in a web browser. iPhone apps in this sense are “desktop” applications. []
  3. Past tense as, while this process is still ongoing, it will be complete shortly. []

Delicious Debrief (Part 4/5)

The story so far

Last year Yahoo! announced, with no notice, a significant change that had far reaching consequences for all third party applications including my iPhone program, Yummy. This is the third in a series of posts that discusses how I dealt with it.

On Monday I gave an overview of the problem, Tuesday I looked at how those changes were announced and why they were tricky, and yesterday I looked at how I actually implemented those technical details.

After the low level technical stuff there was still a lot of work making it useful. That’s what I’d like to talk about today.

But that’s not all

What I have been discussing so far has been entirely about the low-level, very technical details of OAuth. Of course there’s another side to it and that’s how it is presented to the user. This, again, has two parts: first, the options made available to the developer and, secondly, the work involved in actually making it happen.

So, the options available. We now have two authentication methods. I’ll call them Delicious and OAuth. The first point to make is that they’re not interchangeable. If I have an OAuth account I can’t log in with a Delicious username and password. And vice versa.

This begs the question: how can Yummy tell which type of account a user has?

The short answer is: it can’t.

Worse, many, if not most, Delicious.com users also have a Yahoo! account even when their delicious.com account is not linked to it. This results in a very confused user, as their login works without error yet they find they have no bookmarks.

What I was looking for was a way to identify the type of account that was accurate and would not require the same information to be entered multiple times. I ended up watching a number of people logging into their Yahoo! accounts1 and, unfortunately, this invalidated most of my theories for making the process easier.

I thought, for example, that I was onto a winner when I noticed that I always typed “@yahoo.com” at the end of my username. Yummy could show the username screen as normal and, if the user typed “@yahoo.com” it would automatically remove the password field and offer to login using OAuth. However there are two problems with this. The minor, but annoying one, is that the user would be forced to enter their username twice, once in Yummy and then again in Safari. More significantly, I found that you don’t have to type “@yahoo.com” at the end of your Yahoo! ID. Although I do it all the time, most people seem to avoid the extra typing.

If people are going to avoid the extra typing, I would still need a way for them to override the default (Delicious) authentication method. And if they know that they have to overide it then, I reasoned, I may as well just come straight out and ask them how they’d like to proceed.

And this is exactly how it currently works.

Honestly, I’m really not very happy with it. How software talks to the server is entirely technical and is really not something that I should need to ask my users about. However this seems to be the least-bad approach. As I’ve noted before, I’m happy to entertain alternatives but I’ve not come across a better one yet.

The other significant user interface issue is one that I have already alluded to: should Yummy use a built-in web viewer or exit entirely and switch to Safari? The OAuth specification is clear on the matter: the users standard web browser should be used as it is both trusted and known2. However, OAuth is really designed with web applications in mind. Even on a Mac this switch from “my” app to a web browser and back again would be far from ideal. But on an iPhone, that can only display a single application at a time, it is confusing at best.

All else being equal I would be happy to go against the advice of the specification and use an embedded web view if it made for a better user experience. Unfortunately, all else was not equal.

What I found was that, although I could log in correctly using the web view there was no way for Yummy to automatically regain control afterwards. If you’re familiar with OAuth you will be aware that there is supposed to be a method for doing this. I spent a long time trying to get this to work but never did. I’m unclear whether this was a problem in my code or on the Yahoo! side3.

In practice, this would have meant that I’d have to provide instructions before the user was allowed to log in. Something like “press the ‘Done’ button when you have finished logging in and enter the code you were given.” Of course this is not dramatically better than copying the code from Safari and relaunching Yummy.

In the end I decided that I would completely follow the spec rather than not follow its advice and force my users to jump through a few hoops.

Again, this is not something that I am entirely happy with. Could I have implemented this differently? Absolutely! Would the user experience be better had I done so? Not significantly.

In addition to the big things there were also a number of smaller, more cosmetic difficulties.

There a few things that that Delicious.com API is not terribly good at. One of the problems I’ve had from day one is that it’s not really designed for applcations to synchronise bookmarks (which is kind of bizarre when you think about it). It can be done, but it’s pretty tricky, especially when you have limited memory as is the case on the iPhone.

The new Yahoo! ID authentication scheme brought another: how do you know who has just logged in? Previously the user entered their username, so Yummy would know immediately. However in this case, their username is entered into Safari and Yummy never sees it. It so happens that one API call does return the username, but only under certain conditions and not necessarily straight away.

I mention this not because it was impossible to solve — indeed it was not spectacularly hard — but because it was was unexpected. After working through the low-level technical details of OAuth; and after figuring out how to allow the user to exit Yummy and return to the same point (while still allowing the process to be cancelled); after all that, the process was still not complete. What appeared to be just a different username and password became a significant and fundamental change.

Coming tomorrow

With most of the hard work finished I just had to make sure it really worked, and then sit back and analyse what went wrong.

  1. Don’t worry, I wasn’t looking at what password you were typing. []
  2. We’re using an iPhone here so we can assume that the user is not using Internet Explorer. []
  3. I’ve got this working in the development version of Yummy now. I’m still not sure where the problem was. []

Delicious Debrief (Part 3/5)

The story so far

Last year Yahoo! announced, with no notice, a significant change that had far reaching consequences for all third party applications including my iPhone program, Yummy. This is the third in a series of posts that discusses how I dealt with it.

On Monday I gave an overview of the problem and yesterday I looked at how those changes were announced and why they were tricky. Today we’ll look at how I actually implemented those technical details — though not at the code level so don’t worry if you’re not a programmer!

Open Source OAuth

The two leading contenders are OAuthConsumer and MPOAuth. It’s only fair to note that my evaluation of both projects was in October and November 2009. Progress is likely to have been made on both projects since then.

I managed to get the sample projects for both up and running pretty quickly. However on trying to log in I immediately came across a problem. Yahoo! displayed a big warning message suggesting that you don’t authorise any new applications for security reasons. And, worse, that this method of authenticating was going to be removed in a few months.

Even if I could get it working this was going to be a show-stopper. I couldn’t in clear conscience ask users to ignore such a prominent message. It took a long time pouring over the code and the OAuth specification to realise that what changed was the OAuth specification itself.

As the error message noted, due to a security flaw in OAuth 1.0 they issued an update to the specification (called 1.0a) and it was this that Yahoo’s servers was objecting to. Neither library had been updated to the new version.

The change to both projects was small and straight-forward but I was worried that I actually needed to make any alterations at all. Were they being maintained? Was anyone using them?

It fairly quickly became clear that both projects were really optmised for connecting to, or at least developed against, Twitter. As with all standards, each implementation has their own peculiarities and specialisations. To be clear, there is nothing wrong with this. All code has its own axioms and its intended use defines its “shape.”

When I first looked at it, one of the things I liked about MPOAuth was that it did a lot of the hard work for you. It kept an internal state machine so that it knew which message it should be sending to the authentication server and what it should expect to see in return. I’m all for other people doing the work for me. As long as you followed the “correct” order everything worked just fine.

However, in a real world application you can’t expect users to do the right thing every time; there needs to be some way to recover from errors or allow people to change their minds. And, try as I might, I couldn’t quite figure out how to force MPOAuth to do what I needed. In trying to be clever it appeared to be making my life harder rather than easier.

OAuthConsumer got closer to being an integral part of Yummy. The way it’s structured was much more closely aligned with how I would have done it but I came across endless errors trying to connect to Yahoo’s servers. Most came back with very obscure errors and I spent a huge amount of time single-stepping (debugging) the framework — something that I’d hoped I could avoid by taking pre-existing code.

To be fair, much of this comes down to OAuth itself. On the server side there is no way to determine which parameter is incorrect, only that the request is not internally consistent.

OAuth in Yummy

Having spent a lot of time trying to understand why the existing libraries did not work with Yahoo! I had accumulated a good understanding of the OAuth 1.0a specification but a much less clear understanding of the frameworks themselves. There was something about the way that they had both been put together that didn’t quite mesh with either my way of thinking or the way that I was trying to implement it in Yummy — I’m still not quite sure which.

So I decided to go ahead and make my own OAuth library. In some respects this was quite a risky move as it was going to be difficult code to write and it would be in a very fundamental part of the software, traditionally somewhere that I try to use simple, well tested code.

It’s difficult to say whether this was the quickest option. Maybe I could have changed one line in, say, OAuthConsumer and got everything working in half the time. But the amount of time that I spent struggling with them suggests not and this way I have full control over and full understanding of every line of code1.

I ended up writing about four hundred lines of new code instead of using MPOAuth or OAuthConsumer. This is some reasonably low-level code and was tricky to debug as the server-side typically responded with a simple “invalid signature” error no matter what the actual problem was. (This is not to complain about Yahoo’s service. It’s just the nature of the beast, unfortunately.) However, once written it’s proven to be surprisingly resilient2.

Coming tomorrow

Having decided on the low-level technical details I discovered that there was still a lot of work to do. That’s what I’ll be talking about tomorrow.

  1. I’m also geeky enough to like writing this kind of code. So shoot me. []
  2. I’m only aware of one bug caused by the new code, and that problem, strictly speaking, was already present in Yummy, it just did not exhibit the same behaviour previously. []

Delicious Debrief (Part 2/5)

The story so far

Last year Yahoo! announced, with no notice, a significant change that had far reaching consequences for all third party applications including my iPhone program, Yummy. This is the second in a series of posts that discusses how I dealt with it.

Yesterday I spoke at a high level about my iPhone application and some changes that Yahoo! made to their side of the system. Today I’d like to talk in a little more detail about how those changes were announced and why they were tricky.

The announcement

On 19 October 2009, Yahoo! posted a blog announcing that all new accounts would be accessed with a Yahoo! ID.

As far as I know, this is the first time that anyone outside Yahoo! knew about this change. And the change took effect immediately.

At the time of the announcement, Apple were taking around two weeks to approve updates to iPhone applications. This means that even if it took no time at all to make the required changes — which as we’ll see is far from the case — it would still mean that a user of Delicious.com would have approximately two weeks to wait before any third party application would support their new scheme.

I don’t necessarily expect Yahoo! to support me personally but I do expect them to act in ways that are beneficial to their own users. Knowing that no third party applications would work with new accounts clearly does not meet this goal.

Maintaining a healthy ecosystem around their own software with well maintained, documented and supported APIs seems to work to their advantage, yet they did not do this. Just look at the flourishing market for Twitter clients for an example of it done well.

But, put bluntly, they failed.

It took over a month to update the API documentation. And when they did they simply added a one paragraph note saying that authentication can now be performed using OAuth. Until that point the only official resources were a few responses in the Delicious discussion forums.

Now, months after the change went live, the documentation is still barely adequate to code against. I think the lack of other third party clients that support Yahoo! ID authentication reinforces this point. (At the time of writing, I think I’m right in saying that Yummy and Yummy Browser are the only two iPhone clients that support Yahoo! ID logins.)

Unfortunately the haphazard approach to developers was only the first part of the problem.

Surely it’s just a different username and password?

You’d think that using a Yahoo! password rather than a dedicated Delicious password would be pretty easy, but you’d be wrong.

OAuth, as it’s called, doesn’t work like that.

The idea is that you shouldn’t have to trust anyone other than Yahoo! with your Yahoo! password. So, bizarrely, I had to make Yummy work without ever knowing what your username or password is!

Without getting too bogged down in detail, this is roughly how it works:

  1. The client launches a web browser. Since an iPhone can’t multi-task third-party applications, this means exiting and switching over to Safari1. The changes announced for iPhone OS 4.0 don’t help here
  2. The user logs into Yahoo! The server responds with a code
  3. The user launches the client again and enters the code

Then every time the client wants to access a protected resource it does some cryptographic magic using the code (generating what’s called a “hash”) and sends that along with the request.

But the devil, as ever, is in the detail. The detail in this case is a twelve page document that assumes that you know what things like “octet string, first base64-encoded” means. Therefore in practice you end up looking up every second paragraph which slows things down.

Of course, writing the code yourself would be the hard option. For a standard such as OAuth there are bound to be some Open Source libraries that can be used without too much trouble.

Next…

Tomorrow I’ll talk about those Open Source libraries and what I ended up using.

  1. The iPhone OS does allow applications to include an embedded web browser, indeed Yummy already has one to allow users to preview their bookmarks. However, the OAuth specification recommends against this approach for authentication. The idea is that you should only trust Yahoo! with your username and password and that entering them into a third-party application might break that trust. An early version of the change did include an embedded browser and I may yet revert back to that version. []