Does open source have a future in the clouds?

I have just read Stephen O'Grady's well-written post on open source and cloud computing. Everyone should read his post. He makes excellent arguments outlining a pessimistic future of open source in the cloud.

The cloud market is new and has a long way to go before a few dominant players control a mature, stable market. Many customers desire that the cloud market maturity is quickly reached, the dominant market leaders are obvious, standards are well-understood, prices are falling year-over-year, and the technology path forward is relatively risk free. Someone at the Boston All-Things-Cloud meet-up mentioned that innovation will happen quickly in the cloud and that the market will rapidly mature. Don't get me wrong. I want the same things. However, high-tech markets don't develop as quickly as desired, nor as projected.

While speed of technology development has been increasing, the pace in which humans can comprehend it, assess the risks, and plan its usage has been a relatively slow constant. Customers will take a while to comprehend this new market's potential. Even if customers understand everything about it, market maturity is a long way down the road.

What does this point have to do with predicting open source's sad fate? Cloud computing will take a lot of time to develop and refine itself giving open source projects time to adapt. Open source projects are built to adapt by their very nature. Advances with cloud computing will benefit how open source projects are done in ways that we cannot comprehend today. So, maybe the future isn't so bleak.

Let's look at the nature of open source projects. They are a mixture of leading edge (see Eric Raymond's The Cathedral and the Bazaar first lesson) and re-engineering (Eric's second lesson) efforts. Open source developers commonly start with a clear need and usage pattern. They publish the code under a license to encourage other developers to extend the project for additional uses and contribute the extension back to the original project. Successful projects change and adapt because different developers are 'scratching various itches'. All it takes is one motivated project member to adapt (or initiate) a project for the cloud.

A common open source project problem has been finding secure, affordable, and usable Internet-connected equipment. In the past, well-meaning hardware providers would contribute equipment to open source projects, only to find that the open source projects did not have the financial means to operate and maintain the equipment at an hosting provider. Cloud computing provides by-the-hour resources for testing and development that individual project members can easily afford. Previously, open source projects that required large amounts of resources for short periods of time were impractical. Now, they are affordable.

AWS's rise to early dominance in the market was due to their early appeal and familiarity to Linux developers. Open source projects can make their binaries available via Amazon Machine Images (AMIs). Other project members can instantiate the AMIs as their own AWS instances and quickly have the project code running. This has helped boost both the AWS developer ecosystem and open source projects. Here are two examples, cloudmaster project and AMI, and mapnik + tilecash projects and their AMI. While I don't have any specific examples, I would not be surprised if a few open source projects have put out a 'donation' AMI to create a cash stream to offset their development costs.

Stephen correctly pointed out that there is currently no turn-key IAAS open source project. I would expect that it will take time for a collective open source effort to piece the right open source projects together to address this need. There are reasons to believe that it will be done. For example, Amazon Web Services (AWS) used many open source projects to build their offerings. They have contributed back to open source projects. I was part of an effort to build an IAAS offering. We were surprised to find how much of the networking, security and virtualization infrastructure needed for an IAAS offering already existed in open source projects.

Open source innovators are not idle. New open source projects are providing needed Internet-scale technology that today are proprietary. Take a look at Chuck Wegrzyn's Twisted Storage Project (to be renamed FreeSTORE) as an example open source project contributing Internet-scale storage technology. I'm guessing that others are out there too.

To be fair, one cannot build an equivalent to AWS from open source projects today. Thus far, AWS has yet to contribute their wholly owned and developed source code to the open source community. If AWS determines that contributing the code to the open source community is a move that would be in their best interest, it's a play that would assure open source's future in the clouds. It may even accelerate standards and cloud maturity. Hmmmm, maybe the guy at All-things-cloud was right.