Is it just us, or is there really a sea change in the nature of research and development.
For my entire career in information technology, research was all about cracking very hard technical problems. How to move more bits from here to there, how to store more bits, how to organize bits so they can be retrieved efficiently. It was often not clear that the problem could be solved at all, and speculative research was needed to explore novel approaches to the problem. Once the technical nut was cracked, the project would move to the development phase. Engineering would be assembled and the long march to a marketable product begun.
Back then, the most interesting companies would put their present their technical innovation front and center and describe their market entry strategy as an after thought. Today, it seems that most of the companies we see spend very little time convincing us that they have solved an important and difficult technical problem. They treat the entire technical problem as something that can be done. In short they treat the software engineering as a development problem not a research problem.
But they are doing research, just not technical research in the traditional sense. The most difficult problem is no longer a software engineering problem; it is a social engineering problem. The most innovative companies we see these days seem to spend all of their time trying to figure out why MySpace grows while Friendster plateaus: or why del.icio.us is more popular than Furl.
Some might argue that the technical research being done at places like Google is still an important and defensible advantage. They are probably right, but how much of that research is really focused on fundamental technical questions. Is the evolution of their search algorithms driven by the need for better and faster search, or is it driven by the need to stay ahead of the all of the people trying to game the algorithm. Is it an electrical engineering problem or a social engineering problem?
Research is no easier than it used to be, but it does now seem different. Very few entrepreneurs seem to highlight technical aspects of their software innovation any more. They assume that if they get the social engineering right, they can build it.
When Joshua Schacter recently suggested we retain an anthropologist here at Union Square Ventures, he was only partly kidding. Something is changing in the nature of research and development.