My love for computers and software started as a teenager growing up in Germany. I fondly remember my Apple II which followed my TI 59. I early on discovered that I could earn money with my programming skills and wound up doing some after school work for Siemens. While I mostly wrote code on a local machine, I got to do some mainframe stuff in Cobol. I was struck at the time by how incredibly cool and easy it was to have code running on a machine that was in a different city (Munich) over 100 miles away that reliably and rapidly provided access for thousands of Siemens endusers across the world. That was almost 25 years ago.
The last 10+ years many of us have struggled with how to create a similar experience for not thousands, but tens or hundreds of thousands or even millions of people over the Internet. By comparison to my mainframe experience, most of how we have done that has a Rube Goldberg-esque feel to it: Lots of machines running lots of different pieces of software strung together by tons of custom code and all of that is before you get to the application layer. Last year I wrote a post titled “I Want a New Platform” which describes in some detail what I believe is wrong with our present stack and what might replace it. A lot of exciting things have happened since the post in what is rapidly becoming known as “cloud computing.”
Cloud computing will fundamentally transform how software and services on the web are created. When the cloud is fully realized, developers will no longer have to worry about provisioning and monitoring machines (whether virtual or real) or whether they will be able to handle a 1000-fold increase in load on their service. A single individual will be able to create a site or service that can affect the lives of many millions or even billions of people.
The undoubted leader in the use of cloud computing is of course Google. In fact, it appears to be a crucial aspect of Google’s competitive advantage. Imagine coming up with a better webmail client a la gmail, but not being part of Google – leaving monetization aside for a moment, where would you have run the sucker and made it support millions of users? One might point to Amazon EC2. But while EC2 is a great platform for creating and managing machine instances, it does not by itself provide the kind of abstraction that is needed. Most people who use EC2 today still create instances that essentially contain the same stack that they might otherwise have on more expensive machines at a dedicated hosting provider.
Google has taken some steps of making their cloud computing platform accessible to developers through Google App Engine. But there are some serious issues with App Engine. Some of these Google will likely fix, such as having Python as the only available language. Others are harder to overcome. What if you are worried about Google competing with your business? What if you would like to run your code elsewhere? Given that cloud computing is part of Google’s competitive advantage it’s not clear that Google will want to ever make App Engine as fully fledged and portable as it might otherwise be.
Today we are excited to announce that we are backing a team working on an alternative, the amazingly talented folks at 10gen. They bring together experience in building Internet scale systems, such as DART and the Panther Express CDN, with extensive Open Source involvement, including the Apache Software Foundation. They are building an open source stack for cloud computing that includes an appserver and a database both written from scratch based on the capabilities of modern hardware and the many lessons learned in what it takes to build a web site or service. The appserver initially supports server side Javascript and (experimentally) Ruby. The database stores objects using an interesting design that balances fast random access with efficient scanning of collections.
10gen could be seen as a departure from the kind of investments that we have made. But in fact 10gen fits in well with many of our basic premises. First, 10gen is attempting to change the structure of a large market, the $20B+ a year hosting industry. Hosting providers running 10gen will be able to completely abstract away machines and instead provide developers with a platform a la Google App Engine. Second, it is doing so in a capital efficient manner. 10gen is not investing in physical hardware. Instead, 10gen is developing the software and services to run on existing hardware that has been built up and continues to accumulate. Third, 10gen is not looking to traditional sources of defensibility in the form of proprietary closed software or hardware. 10gen is opensourcing the key components of its stack and is actively recruiting contributors (if you are interested, check out the SDK and the contact page).
We look forward to working with Dwight, Eliot, Geir and the team they are building. 10gen has released an alpha SDK in time for OSCON 2008 this week.