Innovation in “A Big Company” – Part 2

Can big companies innovate internally? Why do some products continue to innovate, while others stagnate and wither away? In the last installment, we explained that the innovation needed in a product that is already successful in the market is different than the kind that brought that product to life in the first place. This Evolutionary innovation is different from the original Revolutionary kind in two ways: first, the scope of the innovative idea is constrained because it must happen in the context of the product that already exists; and second, it is much harder to execute on the innovation due to the competing pressures resulting from having to support and grow the customer base the product already has. A failure to comprehend this Innovation Lifecycle is a big reason why big companies fail to innovate. It’s why engineers and executives get so frustrated with each other about their inability to innovate. The executives want evolutionary innovation in their aging multi-billion product while the engineers think they should be creating the next revolutionary innovation.

So let’s look at some real examples of the innovation lifecycle in play at EMC (The Big Company).

Innovation Lifecycle at EMC

It is clear that EMC has done a masterful job of Revolutionary innovation with many stellar acquisitions over the years. Many of us on the outside (including yours truly) used to mock this as EMC’s inability to innovate internally – that EMC could only buy and not build. In fact, looking at competitors struggling with their one-trick-pony product plans, this was a brilliant strategy to build up a culture and portfolio of technologies and products based on the insight that the diverse needs of this industry cannot be serviced effectively with just one.

However, there is an elephant in the room: is acquisition the only way EMC can innovate? This question actually breaks down into two questions:

  1. Can EMC do revolutionary innovation organically vs. by acquisition only?
  2. And can EMC do evolutionary innovation on those acquired products? Or is it the case that once acquired, these products are destined to stagnate and be replaced by the next shiny toy?

We’ll talk about the revolutionary build-vs-buy question later in the series; for now, let’s focus on the second question.

The Acquisition

EMC’s first groundbreaking acquisition was back in 1999 when EMC acquired Data General, with the CLARiiON product as the centerpiece. CLARiiON was already a very successful product – the leader in the new midrange storage market that it helped create. CLARiiON’s revolutionary innovation was that it offered the features and reliability of the big expensive arrays with a new architecture that lowered the cost of the solution dramatically*.

In parallel, EMC had also acquired some technology – a media-server stack comprising a real-time operating system and a file system. In response to the rapid rise of NAS as a market need, this technology was repurposed to become Celerra, EMC’s NAS solution (this entailed another technology tuck-in acquisition for the NAS protocol stack). Celerra had its own set of innovations such as N+M clustering for resiliency and scale as well as the industry’s best multi-protocol implementation (simultaneous NFS and CIFS).

Innovation pt2 image 1

Both CLARiiON and Celerra saw minor innovations following the acquisitions. By the end of the decade though, the complexity of the two separate pieces had become a real problem in a landscape where competitors were delivering radically simpler solutions at lower cost. It was a build-vs-buy moment and we opted to build, with a plan for large evolutionary innovation.

Big Evolution #1

We first evolved and re-launched the product as VNX in 2011. This significant evolutionary innovation obliterated the two distinct pieces to create a single unified solution with a single management interface. VNX shipped with industry-leading hardware for power and compactness. It was also the first midrange hybrid array combining SSD with HDD. To this day, it is still the only midrange array with both caching (FAST Cache) and tiering capabilities (FAST VP). Some would argue – and many did – that this is not really innovation, that it is not even truly unified like the competition, which had a single software stack versus a NAS gateway. While this viewpoint is intellectually defensible, the ultimate measure of success of innovation is financial and the fact is that VNX has generated many billions – yes, with a B – of dollars so far and still counting – showing a huge bump in the invigoration of the product line.

OK, fine, but is this all you can do? Is it possible to have more fundamental innovations?

Big Evolution #2

While the VNX mashup was happening, there were a two other vectors that threatened to make the product obsolete without some MAJOR evolutionary innovation. First, SSD’s and CPU cores were everywhere. Second, the world had gone 64-bit, and the 32-bit file system was reaching the end of its useful life. This set in motion a multi-year journey of a near-complete overhaul of the core software stack. Flash changed storage system design (an evolutionary shift) because it was so much faster than the spinning disks that software stacks had been built around. Whereas it took hundreds of HDDs to saturate a powerful storage controller, it took only tens of SSD’s to do the same – the controller was suddenly the new bottleneck. The Flare™ operating system at the heart of the CLARiiON stack was obsolete – it had to go! And that’s exactly what the VNX team did. They replaced the Flare stack with a new one called MCx, 100% designed from the ground up for multi-core CPUs. This wasn’t one of those “minimally intrusive surgery” projects where you keep the old code around while building the new stack next to it; it was a rip-and-replace of the core data path stack. The results were astounding – a 4x jump in performance (see chart). I joined the group just as MCx was getting launched and having seen a previous employer struggling with getting to multi-core, I was completely blown away by what the VNX team had pulled off. I must admit I was a bit skeptical of these performance claims myself, but every customer I have talked to about MCx has confirmed that they saw similar multipliers in their production environments. As far as engineering feats go, it was as herculean as the Boston Big Dig, but unlike the Big Dig, they did it in only a few years and without the business missing a beat. You may well ask how innovative is multi-core really? True, the concept is not new by any means, but the innovation is in the epic implementation. MCx delivered 96% linearity of scaling with cores and that means the stack will keep scaling in performance as Intel adds more and more cores. MCx breathed at least another decade of life into the product.

Innovation pt2 image 2

Big Evolution #3

Then there was the aging UFS32 file system which ran on an aging proprietary operating system on a separate NAS gateway. The file system and platform teams went on an overhaul mission as well. In their case, it was more like major organ transplant where they built a state-of-the-art transactional file system (UFS64 in picture) and transplanted it along with the legacy UFS32 file system as well as the recent MCx stack into a brand new platform based on the Linux operating system. The result, as you can see, is a new stack that has no resemblance to and almost no traces of the old one. Such complete overhauls are very few and far between and for good reason – they take many years to pull off and that means a thousand things can happen to kill them before they see light of day.

Innovation pt2 image 3

There are many other examples of evolutionary innovation across EMC’s products: Isilon’s multiprotocol HDFS and NFS access and Cloud Pools, VMAX’s Hypermax operating system and SLO-based management, and Data Domain’s ProtectPoint solution, to name a few.


Highly successful products offer many opportunities for evolutionary innovation to defend and extend their success. The scope of these innovations tends to be evolutionary compared to the original idea, but each one extends the life/success of the product. And every few years, major inflection points happen that threaten the very existence of the product. Successful teams anticipate them well in advance and put big-bang innovation projects in place to navigate them. But spotting the need for innovation in time and coming up with the idea is just half the problem, and I would argue the easier half (except don’t tell my boss that!). The hard part is successfully implementing them. In the next post, we’ll highlight some key ingredients for successful innovation execution in big companies.


* Gartner notes in their October 2015 report on Critical Capabilities for General-Purpose, Midrange Storage Arrays that “Dual-controller architectures will continue to dominate the midrange storage market during the next three to five years, even as new quad controller and scale-out storage arrays compete for market share.

-Sudhir Srinivasan @DoctorSudhir

One Reply to “Innovation in “A Big Company” – Part 2”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s