Tuesday, December 2, 2008

Open Source - Par for the Course

In high school our biology teacher explained to us that in the physical world, open systems adapt and evolve and that diverse gene pools have a higher chance of useful mutations, thereby creating better suited and stronger generations. Open organizations and open source software projects show similar characteristics.

The Open Source movement started with some programmers developing an application for their own use and then making it available freely to a wider audience. The reason for giving it away was that the assumed collective wisdom of a larger group, using and testing it, would improve the quality. Moreover, extensive use would lead to new features being added, thus creating a living and expanding body of software that would grow rapidly in functionality. A coordinator reviewed all contributions and decided on the next iteration of the product, which was then formally released back to the community. The benefits of this accrued to all participants and therefore each of the developers was willing to give up their right of ownership of the source code they developed.

These days, thousands of programmers from different parts of the world participate in large scale Open Source projects, working together on millions of lines of programming code. The LINUX operating system and Apache Web Server, which both have substantial market share, are examples of very robust, complex programs. Apache now runs a stunning 93M websites. Most of the people writing code for those projects have never met each other. Still, the software they produce tends to be of similar or better quality than commercial software. These programs can evolve much faster, because there are more eyes and brains involved to come up with new ideas and to diagnose and resolve problems. Interestingly none of these contributors are being paid for their efforts. For them it is par for the course. Peer recognition and being part of a community of like-minded geeks is more important than financial reward.

The "collective collaboration" model applies not only to software. Wikipedia has 75,000 active contributors, working on 10M articles in 250 languages, that get searched by 685M visitors a year. The quality of Wikipedia is considered to be similar to the venerated Encyclopedia Britannica. The ATLAS particle detector, at CERN in Geneva, will search for new discoveries in the head-on collisions of protons of extraordinarily high energy. This has the potential to rewrite the science of physics. Their website tells us: “ATLAS is a virtual United Nations of 37 countries. International collaboration has been essential to this success. These physicists come from more than 169 universities and laboratories and include 700 students. ATLAS is one of the largest collaborative efforts ever attempted in the physical sciences.” Using Internet collaboration and a highly excited community of academics around the globe, the way science is conducted is changing.

Some companies have seen the opportunity and built their business model around the concept of “crowd sourcing”, a term coined by Jeff Howe of Wired magazine. They essentially created completely open companies that are based on the contributions of their stakeholders and apply what is now called “open innovation”. Two online companies got quite a bit of buzz. The shirt retailer Threadless uses its customer base to design apparel online and they sell it through their catalogue. RYZ allows you to create your own sneakers (try it, it is fun!) and then runs a contest to select the best designs. These companies forge a tightly knit community of followers and a loyal customer base which gives continual feedback on their product lines. MIT professor Eric von Hippel states: “online design is becoming a substitute for in-house research and development while voting takes the place of conventional market research.”

Hybrid models are emerging as well. TopCoder organizes contests for the best software, for which it offers prize money. The quality is judged, not by some panel, but the candidate’s peers. They celebrate the developer of the month. Yes, as you already expected, this month’s winner in the algorithm category is Xao Xaoqing from China. Apple opened up its iPhone platform to allow 3rd party software developers to create applications and sell them through the iPhone Appstore. The Facebook Platform invites anyone to create applications to help them engage Facebook’s 120 M users. However, the Facebook platform may ultimately lose out because it is not as open and therefore not as popular as Google’s OpenSocial project.

How is it possible that in the average company you cannot get people who sit across the hall from each other to work together and that these guys collaborate “naturally”? How come that in your company most big projects are late, over budget or don’t complete at all, while these nerds create the best software?

The first step is to create a community of co-creators, testers and users. A community is non-hierarchical. It is a group of peers who share a common view and enthusiasm to jointly address a problem or create something new. There may be some friendly competition between members to show off skills and capabilities. The second step is to establish the architecture and processes to enable the work breakdown, so that bite-size activities can be parceled out. As the problem gets more complex, this will become more necessary, but also harder to do. Architecture design is usually done by a small tightly knit group, before it’s handed over for “crowd sourcing”. Then you need to ensure that the members keep moving in the targeted direction. Successful communities have a limited set of clear rules that all comply with. Their coordinators are seen as “primus inter pares”, leading among equals. There are published processes for conflict resultion. The last step is to ensure that once the product is out the bugs are removed. In Open Source this is a large part of the attraction of the model. In the words of the open-source guru Eric S. Raymond: “Given enough eyeballs, all bugs are shallow.” Most online companies, now release “beta” versions of their software to expose it as widely as possible but with lower user expectations. They would prefer for their users to locate and help resolve their problems.

There is no reason to assume that these models don’t apply to your organization. Most companies simply haven’t caught up yet with the opportunities of creative communities.

6 comments:

  1. Gary Hamel in Future of Management (HBS Press Book 1996) highlights Innocentive an Eli Lily subsidiary which has created a distributed network of independent researchers scattered around the globe where client companies have been able to document 20-fold improvements in R&D productivity.

    There's a significant opportunity in creating systems for organizations to tap their stakeholders knowledge and creativity.

    ReplyDelete
  2. "How is it possible that in the average company you cannot get people who sit across the hall from each other to work together" ...........

    Organization creates boundaries, there is no room for folks to choose - they just wait for an assignment, innovation is only talked about, element of fun is missing, risk avoidance is the mantra rather than encouraging bold risk taking attitude, reusability never even reaches beyond the drawing board. Lastly the most important element of management practice is ignored at all levels "Put the right people in right place with empowerment" which eventually results in an organization with very little collaboration.

    ReplyDelete
  3. I think that these communities also have a effect of natural selection on the people that join them. So not only the end product evolves into better quality, but the project team can also be of exceptional quality. People that are joining these communities are passionate about their profession and maybe these communities with their open approach naturally attract great developers. This is because the community is not looking for them, but these developers select the community or project they think they can be of value for by themselves. If I remember this right, Google has applied a similar strategy by letting new employees select their own projects to join in their first days at the company.

    Also, because each involved developer knows their code is going to be seen by a lot of peers, they have a stronger incentive to write better quality code from the start. With closed source projects, great code or quality enhancing methods (like pair-programming or test-first development) is often not the first objective. Planning and deadlines rule the project and a functioning application in time is more important than code quality, cause: who is going to see this code?

    ReplyDelete
  4. Haha, de BAARDAAP! Vertel het erbij..... This teacher looked a lot like a monkey himself, one with a beard, it was his nickname, that is why I remember it so well, hahaha...

    ReplyDelete
  5. The quality of Wikipedia is not always that high. The content encounters some serious weaknesses: you don't know who's the writer or editor of the information offered; information that is presented today and is (in fact) correct can be changed tomorrow into incorrect. The whisdom of crowds isn't a vbery reliable "source". (http://en.wikipedia.org/wiki/Andrew_Keen)
    Furthermore Wiki's don't have time restrictions attached to them - the whisdom of crowds hasn't too. Wiki's can be however an effective collaboration tool, if embedded in a organizational structure with time and money driven boundaries.

    So my question is: to what extend the communities you mention are managed by time/money restrictions, pace setting, required quality levels? Primus inter pares in a business environment: does it work when a customer is waiting?

    ReplyDelete
  6. Reaction on Erik's comment:

    Of course with anyone being able to add and modify you invite all kinds of rubbish. My kids used to run contests to see how long it would take for Wikipedia to spot the nonsense they posted and rectify it. Wikipedia is using extensive Patrols to watch over a class of pages and take any appropriate actions. “Most patrol actions are performed by individual Wikipedians, but some are performed by bots or other tools that monitor for potential integrity breaches. Patrols focus on various pages, noticeboards and feeds. Many of the well-known patrols have hundreds of users, and are directly responsible as a first line against vandalism, or other potential problems” , according to Wikipedia. See http://en.wikipedia.org/wiki/Wikipedia:Patrols about Patrols, http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Spam , about fighting Spam and http://en.wikipedia.org/wiki/Wikipedia:DR about dispute resolution. The mechanisms to control the quality of content have become truly sophisticated and continue to evolve.

    Wiki’s are only a tool. Like all technology, it is the way we use it that derives its effectiveness. So you have to embed the systems into the proper organization structure, incentive system, processes and policies to make them effective. Just implementing Wiki’s without these structures are no guarantee for success. They may well be distracting employees from doing their regular job. On the other hand, I have seen the use of Wiki’s in projects dramatically improving the collaboration and communication and thereby playing a key role in the success of projects.

    There is indeed wisdom in the crowds; it is how you bring it out and have it rise among the clutter. The collective knowledge of all folks at the front, dealing daily with customers, probably gives you the best insight in what customers need. If you combine this with the ability of your customers to provide direct feedback on your products and services, you probably get a good pulse on quality and competitiveness. This insight will help you improve on your products and services with the highest impact, at the lowest cost. When we launched online banking for Citibank in the mid-90s, I suggested that we would allow customers to gives direct feedback on the service through a bulletin board (fashionable at that time) and vote on the features they wanted to see in the product. Marketing was against this, as it would put out our dirty laundry for all to see. When we finally did this (on a much smaller scale than I suggested), it turned out that Marketing’s assumptions about what customers want, was completely different from what customers expressed as their priorities.

    To answer your last question: yes you need to drive communities. This means setting clear objectives, having the architecture to break down the work and parcel it out, setting the deadlines, having the quality control and monitoring progress. With other words, having community doesn’t negate the need for old-fashioned project or operations management practices. But, it allows for a much higher level of engagement and input from all participants that in traditional organization forms. And yes the primus inter pares in a business environment, will need to have the authority to move things along, when necessary. So while the carrot plays the biggest role, there is nothing like a good stick to be waved at the right time.

    ReplyDelete