Content Log

Remembering Jeffrey McManus

My brother-in-law, Jeffrey McManus (on Facebook), passed away on July 5th at the age of 46. It was way too soon and unexpected. This came at a very bittersweet time in our lives as my son, Michael, was about to graduate from secondary school on July 6th and my daughter was just finishing her middle school on the same day. We took time to reflect on our long planned holiday in Greece prior to Michael going away to Georgetown in August. 

Jeffrey McManus

I can't imagine what my sister and her kids are going through right now. It is great to know that they have been surrounded by friends and family that celebrated Jeffrey's life. We truly wished that we could be at his memorial in San Francisco on the 19th. We celebrated his life with Singapore Slings on the same day and I felt like I was the only guy in Greece wearing an Aloha shirt - one of the items specified in the dress code for his memorial.

My kids knew him just as Uncle Jeffrey. The kind of uncle that shows up in movies made by SNL alumni and who any kid would love to have. There were very rarely any mopey faces around when Jeffrey was around. But with all the notices and comments on Facebook, my kids were wondering why he was so important.

That can be summed up on one word - Community. Jeffrey and my sister were part of the nascent social movement that predated any of this Facebook stuff. They were active members on the Well while Jeffrey earned a living at coding, teaching and writing on Microsoft technologies such as ASPs, SQL Server, C# and VB. See Jeffrey's books on Amazon. (http://www.amazon.com/Jeffrey-P.-McManus/e/B000APSY3K) Microsoft didn't really quite organize community the way that you would recognize today, that didn't stop Jeffrey. He would speak at events around the world evangilizing Microsoft technologies the way an open source community would do today. Using these technologies, he pioneered an API and social community with Insero, a company he co-founded with my sister.

From there he went on to establish the developer community for E-Bay, at the time a real innovation. Through trial and error, he helped develop the best practices in engaging developers and have them form a community. He did this with the understanding of someone who was a developer himself. He did this through developing the tools that developers need such as APIs and connections to IDEs. Most of all, he did it through his big, extroverted personality that developers found infectious. That's not to take away from the technical innovations he made while there, such as developing RESTful APIs only a couple of years after Roy Fielding's paper describing REST. In addition, he helped figure out the taxonomies that vendors and buyers could access products from the E-Bay platform. He really helped make E-Bay a platform, not just a site.

He became the expert in community development and naturally socialized with the connected technology crowd. People sought him out to participate in conferences. That's one of the first ins that I had to meet up with Tim O'Reilly. I would often name drop my brother-in-law as at least a conversation starter. I don't remember who went to Yahoo first, Jeffrey or my sister, but Yahoo recruited Jeffrey to develop their developer community. Unfortunately, this meant trying to tackle some of Yahoo's basic issues such as usability, which he was the first to admit was not his forte. He made an impact there, but Yahoo wasted the opportunity that they had with him on board.

So he struck out on his own with some of the many ideas of a very fertile mind. He had the brilliant idea of Approver.com, which was either too ahead of its time or he was not interested enough in taking venture money. He later developed CodeLesson to provide an online developer instruction experience that bridged the gap between online instruction and in-person instruction. He still consulted on community for companies like Twilio. We consulted with him at Alfresco in the early days of the development of our community.

Jeffrey and my sister are part the GenX crowd that invented social networking and with it all the social connections that easily explains the many kind thoughts that have been put onto his profile. It's just a minor sign of how much we will miss him, his wit and his ideas. RIP!

8 ECM Predictions for 2010

At the recent AIIM Board Meeting that I attended this month, AIIM President John Mancini asked us to blog using the number 8. Why 8? Because no one else is using that number and AIIM can brand around it. So what better way to start than with my 8 predictions for ECM and Open Source - emphasis on the former. I found that I had a lot more that I could have written, so I cheated and added two more on section 7+1 and section 8+1 for a total of 10. Still, I didn't get around to my thoughts on Apple, mobile, Google, Google Wave, New Applications, Mergers and Acquisitions, etc. Since I have to go home and I am the last one in the office, this will have to do! I know it's a lot, but I have been doing a lot of thinking lately.

In no particular order...

1. Economy's affect on ECM.

Nothing affects the ECM industry more than the economy for good or bad. Any sudden downturn has an immediate effect on practically all enterprise software spending regardless of the payback. Until the consequences of the downturn are understood, financial controllers will hold off on purchases above a certain amount. Generally bad for large enterprise purchases. On the flip side, a recovery can and has been an excellent opportunity for ECM as automation is often a better investment than bringing on the old hires that you let go. Middling growth leads to middling results.

The prognosis for the economy is not entirely clear right now. Most recessions endure a double dip, although this one was so sharp it may have washed out most of the bad stuff that a double dip gets rid of. There is a great deal of uncertainty for both downside and upside opportunity in the economy and know one knows for sure what will happen. Most leading indicators are good, but a similar situation arose a couple of years after the 1929 crash and the economy dropped again in the mid-30s. If it gets bad, it can very, very bad as over-stretched governments have very little room for maneuvre. In addition, Tim Geithner has said that prospects for increased employment will not be great in 2010.

My guess is that economic recovery will continue and this could be a good year for ECM and enterprise software in general. Regulation is up, corporate purchasing is up, optimism is up. Content management will be an easier sell than most as investment for the future with less risk than hiring. I'd put the probability at 60-70% for good conditions for ECM. Downside is that is still a pretty risky environment. As the Boy Scouts say, "Be Prepared!"

Having said all that, government will be a growth area for ECM in 2010. Stimulus packages will kick in and new regulations will emerge to put yet more content pressure on organizations. This is one of the reasons we invested so heavily in Compliance, Governance and Records Management in 2009.

2. ECM in the developing world.

Growth in Asia, particularly India and China, is obviously outpacing Europe and North America. While the developed world spent, the developing world saved with more reserves for investment in infrastructure and IT. Even eastern Europe is doing better than expected because of lower debt. Could the developing world be a major growth area for ECM. Pressure on space and skyrocketing real estate prices in rapidly developing urban areas means that digitization of information can be a valuable tool for freeing up space. ECM is ready for the developing world.

If I were a traditional software vendor, I wouldn't count on this for revenue though. These guys got to where they are by being cheap and bargaining ruthlessly. This is no place for a traditional enterprise software sales person. Also, the size and scope of the developing world means that traditional distribution channels won't work and relationship selling can take forever to develop. And it is not just software that is feeling this - so do hardware manufacturers, manufacturing equipment makers, even medical device manufacturers.

I think this is an area where open source will absolutely shine. Internet distribution and download is a much more effective way of getting software to people, but will probably develop new models of monetization. We're seeing a lot of activity in India as an example, but demand is going to come in the form of developer support rather than traditional licenses, particularly in various outsourcing houses. Microsoft too will benefit from the extensive reach of its indirect distribution channels, although the software pirates may benefit as much, if not more.

3. SharePoint in 2010.

You have to hand it to Microsoft; they seem to have scared the bejesus out of the traditional ECM vendors. They have all seemed to rolled over and played dead in the wake of SharePoint 2007. Either that or they have just buried their heads in the sand in complete denial. To claim that SharePoint is not really ECM is ignoring the messages that are coming from the podium of Microsoft conferences. Most seem to say - "Oh we support SharePoint, we offer archiving, records management, etc." Yeah, but who owns the data? SharePoint is not just a front end to your system. It is becoming the platform for knowledge worker applications.

This is probably a make or break year for the ECM industry. If you are an ECM vendor, beware the ever tighter integration with office, better web support, new records management, and claims to better scalability and administration. Still, it is not invulnerable. Microsoft has chosen not to address its fundamental architectural flaw of storing content in the database and it is still an exclusively Microsoft-centric platform. Forget whatever database, operating system, language, browser you have - you better get used to SQL-Server, .NET, Windows, IE and don't forget Silverlight. If the traditional vendors can't battle the crap out of that, then they deserve to lose. The next 12 months will be critical during the transition to SharePoint 2010.

4. The E in ECM.

The whole idea of Enterprise software in the 21st century seems anachronistic. The term Enterprise really only took hold in the 90s in order to describe systems that were able to scale beyond the department. It meant big, powerful, flexible, but it also meant big, clunky and expensive. As Web 2.0 sites with their cheap (read free), simple, but scalable platforms scaled to millions of users in a matter of months, the whole idea of only being able to support thousands of users and take years to implement became ludicrous. Being Enterprise meaning you can support your heavyweight infrastructure of other Enterprise parts also seems less interesting when you consider that the largest databases on the planet run on MySQL using a concept called Sharding.

Support the Enterprise concept if you want, but Brand Enterprise has lost a lot of its value. And while we are at it, so is Management. In a world of wikis, blogs, Facebook and Google, a strictly controlled environment is destined to be locked up in a closet with just a couple of people using it. It's not that control is not valuable, but you need to really consider what would really happen if information were freed up. Transparency and tracking are more valuable tools today than locks, keys and authorization slips. Leave those to the information that really, really needs it. However, the Content in ECM will be more important than ever as more and more of it gets created at an exponential rate. Best to guide it rather than restrain it.

So what replaces Enterprise Content Management? I don't know, but my guess is that we start to find out in 2010. it might even have a new name by December. Hint: I wouldn't jump to the conclusion it is Social Software. Also see Application vs. Platform below.

For the sake of readability of the rest of my predictions though, I am going to stick with ECM.

5. WCM and ECM.

What a funny relationship these two have. ECM was born in the early 90s out of document management. WCM was born in the late 90s because ECM couldn't get out of document management. The in 1999 and the early 2000's (Noughties here in the UK - what do you call them?), both ECM and WCM decided that they were in the same business and started buying up each others capabilities and then jury rigging the architectures together. The logic went that both sides were managing content, they had repositories and their way of handing content was better. From a pure dollar perspective, that one didn't play out as well for the WCM vendors. The largest players from that time, Vignette and Interwoven, are now subsumed by OpenText and Autonomy.

Still WCM didn't do too badly in 2009. The web site is the store front of the 21st century. And most apps are web apps and most web apps are web sites, at least as far as the user is concerned. Companies still invested in branding and applications for customers as a way to weather the recession. Because of the plethora of web sites, styles and characteristics, there seems to be a plethora of WCM vendors as well. It must be the most fragmented industry that I have ever seen in the enterprise software space. Due to these characteristics and the fractious nature of WCM, if you see a WCM vendor and an ECM vendor in the same account, often it means that somebody shouldn't be there.

However, I think WCM should play a very important role for and in ECM and hopefully this will become clearer in 2010. ECM as repositories of trusted information predominantly deliver their content via a web interface. If its up to the end user, it would be the web site that they are using. If web applications are presented as web sites, are content rich and want to dynamically deliver content to end users, then the marriage of WCM and ECM makes a lot of sense. This is why Alfresco is investing in its WCM to work well with the Spring Java Framework - to help build web applications that are web sites and provide dynamic content and repository services for content-rich applications.

I hope the mapping of the WCM industry becomes clearer as well for 2010. We believe our positioning of focusing on WCM for Java web applications, particularly Spring applications, means that we can work with other parts of the WCM industry, such as Drupal/Acquia, Joomla and others. In 2010, some other ECM vendors should just go ahead and get out of the WCM business and stick to the knitting. They should cooperate with, rather than compete with the WCM vendors. I think CMIS will make this a lot easier.

6. The Cloud and ECM.

As my colleague Ian Howells has pointed out, the buzz around Cloud is many times more intense in Silicon Valley as it is outside. And it is a level of buzz that feels like the Web in mid 1990s that was not felt outside the valley. Could we be seeing the start of a huge land rush that we did in 1995? There is a lot of commonality with the experimentation, trials and shear enthusiasm.

The economics of the Cloud seem almost no brainers. Two years ago, the cost of running systems in the cloud were 1/3 the cost of on premise. Now according to "Above the Clouds", a report from the University of California on Cloud Computing, the cost is about 1/5 to 1/7. I know it is a stretch to say that if this trend continues we can see computing at 10% and ultimately 1% the cost of on premise, but this is what I believe. The reason is that we are only just beginning to see the industrial scale of Cloud facilities come on line that have enormous buying power, very cheap energy, and nearly free cooling being in cold climates or near cold rivers. The ever increasing bandwidth available means that location of computing is less and less relevant. There are still lots of obstacle to be overcome, particularly in perception of security and reliability as well as real legal issues of data domicile. But the implications of 100X compute power for the same cost as in house is enormous and unknowable.

A lot of people have predicted that 2010 is the year of the cloud. I think that 2010 will be the start of ECM in the Cloud. Steve Ballmer at the SharePoint conference in October made a big point of talking about SharePoint in the Microsoft Cloud. We have been and will be doing a lot of work in using the elastic power of the Cloud with the Alfresco platform. I expect that you will see other ECM platforms working in this area. I would also expect to see lots of open source in the Cloud. This can't be good for

7. CMIS.

I guess I have been talking about this longer and louder than anyone else out there, so you wouldn't be surprised to see me say I think CMIS will have a significant impact in 2010. In the past two years, I have learned a lot about the standards process, particularly OASIS, and I have learned not to be so optimistic when lots of players are involved, no matter how motivated. But I can see the end is near to getting CMIS 1.0 to an official standard in the Spring 2010. It will not be long before the ECM vendors have their implementations out. But I have also seen portal vendors also building portlets as well to integrate with the ECM systems. I would really be surprised if these aren't out by the end of 2010 and already having an impact.

CMIS is both and an opportunity and a threat for traditional ECM vendors. It is a threat because it provides SharePoint in particular an opportunity to ease users out of those traditional platforms and into SharePoint. SharePoint being a value player at the low end will naturally try to gobble up ECM low end implementations. It is an opportunity for a number of different reasons. This is a chance for the larger vendors to consolidate the small holdings of the lesser or defunct competitors. It means that ISVs and system integrators will be able to reuse solutions from other vendors and apply them to other ECM vendors solutions.  More solutions will mean more money spent on ECM as it solves real business problems, thus making a bigger ECM pie. This in turn will create more solutions. All this played out with the standardization of the DBMS market and there is no reason to expect that it won't in the ECM market.

Whenever I talk to anyone about integrating with Alfresco, I suggest that they integrate using CMIS. CMIS is very rich and usually good enough to do what they need to do store, access and search content. When the integration is done, it works not just with Alfresco, but any CMIS-enabled repository. We have done this with Drupal, Joomla and Confluence in the last year. Expect lots of early integrations in 2010, particularly with portals. We see lots of Liferay used in conjunction with Alfresco and I would encourage portlets developed by customers to be done with CMIS as an example. Expect CMIS to be on tenders, RFIs and RFPs. Finally, you may see the first companies being formed around CMIS and content applications.

7+1. Content Platform vs. Content Application.

I once had a bizarre conversation with Gartner about what constituted "Visionary" in ECM. The answer is that you buy into their vision of CEVAs (Content Enabled Vertical Applications) and have lots of them. Hmmm - visionary means you buy into someone else's vision? But then I asked why SharePoint was considered visionary when they didn't invest in CEVAs. The answer was because of all the interest from their clients in SharePoint as a platform. Hmmm - so you are visionary buying into their vision unless you are Microsoft. Now I understand that CEVAs are gone and replaced or "rebranded" as Composite Content Applications. Does this align with being a provider of CEVAs or being a platform that allows others to create applications?

My belief is that we are seeing a natural swing that has pervaded content management from the very beginning. Is content management an application or a platform? Is it middleware or a fundamental subsystem? A lot of this has to do with how you create, capture and use content. Some of the process of content management is so generic that it makes sense to have an application to perform specific tasks. However, many so called out-of-the-box solutions are so heavily customized that it really is more a platform than an application. Delivery and consumption of content via the web can either be an applet, gadget, portlet, web part or an entirely new piece of HTML/JavaScript/Java/.NET piece of coding.

Content management was never like an ERP or CRM system - plop it in, conform your business processes to that system and configure the rest. Content management was in many ways something closer to a database. Sure it has a rich domain model, but ultimately customers want to use it for all sorts of different applications. They want to write queries, build relationships, define business processes and create rich and elaborate user interfaces, particularly for the web and web sites. The better way to look at it is, that some "CEVAs" may be considered applets that are part of a larger application or business process. ECM systems should make those applets work independently as portlets where they can be provided in context. Capture and consumption can really come from anywhere, so provide the APIs that can provide the content access and manipulation required in the language and development environment of the application that needs the content. The reality is that applications need content as much as they need data in a database, so let's give them the same sort of tools that they have with databases. If portions of the application can be packaged as applets, that is part of the job done, but not all of it.

Three trends will get us to think about ECM as platform more than application in 2010. First, CMIS as a platform-style API will get us to think of ECM and repositories as a platform. CMIS will highlight the areas where we need to build and extend applications by initially system integrators creating solutions with CMIS that will be portable to other ECM systems. Second, the desire to build mash-up applications as a result of experience with Web 2.0 will encourage developers to be pragmatic about pulling in the right tools, including ECM into mashed up or Composite applications. The ECM system is not necessarily to going to be the locus of the Composite application, the web site may. From this perspective the ECM system will be perceived as a platform in support of the web site or web application. Finally, the 5 parts service to 1 part product cost ratio (plus or minus) that has been around since the early days of ECM can not endure in our cost conscious times. If we stop pretending it is a whole app and provide the tools that developers need to pull together the whole app in a fraction of the time, particularly with Web 2.0-style scripting over hard core coding, everyone will benefit.

This is not a massive swing of the pendulum, but it is heading in the clear direction of platform. This is where we are thinking. This is also where Microsoft is thinking based upon comments from Steve Ballmer at the SharePoint 2009 conference.

8. Open Source makes strange bedfellows.

We go into 2010 as what a lot of open source experts have described us as the largest private open source company and cash generative to boot. Not bad going as we celebrate our 5th anniversary in January. In those 5 years, we have learned a lot about the politics, business and religion of open source and it has been absolutely fascinating and fun.  We have had to think about licensing, pricing, subscriptions, alliances, packaging, downloads, new marketing models, confrontations and who are friends and enemies are. Throw a recession in there as well and it makes for interesting times.

As time has passed and we think about how we can apply the open source model, we have focused more on our core competence - Content Services and Content Repository. The more CMIS has become real, the more we can cooperate with others with out locking either party in. Thus we can move away from competition with many players and become best of breed in content services. This has allowed us to more work with a Drupal or a Liferay when in the past there may have been overlap in our product sets. CMIS opens up even more opportunities to work with others as well. Working with others creates a stronger and multiplicative network effect.

Because open source stands parallel to other stacks and encourages others to integrate, particularly through contributions. This opens up alliances that may not require integration work because it has been done by others. We didn't have to build the integration to Kofax or SAP and you will see more integrations in early 2010. Portals - open source or closed - have always worked hand in hand with content management, while not necessarily encroaching on each other's space - SharePoint excepted. These types of integrations in turn allow us to work with other vendors that may need to bolster part of their product set and not feel threatened. Expect to see more integrations and alliances between open source vendors and closed source vendors in 2010.

This is not new for open source. MySQL and JBoss have worked with many non-open source vendors and bolstered their capabilities. In 2010, this will take on a new significance though. The Cloud is requiring many vendors to bolster their product sets for new models of demand and elasticity. The growth and threat of SharePoint and Microsoft product sets can make some products seem incomplete. As we see in the next section, Social software and content management can be very complementary. The ease with which it is now possible to pull together an entire stack of software means that a lot of that stack may be open source and present some really interesting offerings in 2010.

8+1. Social Software and ECM.

Should you buy social software from your ECM vendor asked CMS Watch. Conclusion was maybe not. I personally think that social software will be a category in itself with vendors whose core competency will be social software. Just like any web site could have been Facebook, they weren't because they didn't concentrate on that. A lot of social networking centers on content-centric networking - discussing, arguing and collaborating on content. However, it doesn't necessarily mean that the social platform will own that content. Look at all the YouTube video that show up in social networking sites.

I do think that ECM vendors should concentrate on Content Collaboration - the act of working together to create and manage content, but that in and of itself is not Social Computing. It's about sharing ideas and content in the context of a common problem and it should be presented as such. This collaboration could benefit from more focused social computing platforms, such as instant messaging, on-line conferencing, activity feeds a la Facebook, walls, wikis, blogs and forums. By concentrating on providing content services for those platforms, we are creating a win-win situation. More than that, we are focusing on what we are good at - Content Services. These Social Computing or Networking platform usually have only the most rudimentary content services, which make for very complementary products.

SharePoint has a lot to do with why this question would even come up. For lack of anything else to use in the enterprise, people are trying to build these types of solutions. However, what was once labeled as collaboration in SharePoint is now Social Computing, but the reality of it is that it is generally still content collaboration. At least that is what most customers use it for. This may or may not become a stretch too far for SharePoint, but it is one that we are not going to try. I didn't always feel like this and felt that we had a big opportunity in Social Computing. But as we progress the company, we have more to offer integrating with others and sticking to our core. We can get more from others that do the same.

Introducing Alfresco Labs 3

I am pleased to announce the beta release of Alfresco Labs 3, our new version of Alfresco targeted at collaborative content management and the first open source alternative to Microsoft SharePoint. Alfresco Labs 3 is a natural evolution of the Alfresco platform that started with the first preview release of Alfresco three years ago. As Alfresco has grown more powerful in capabilities, we have strived to simplify both usage and development of Alfresco web clients, web sites and web applications. This release has leveraged the Web Script architecture that we introduced last year and provides simple user interface components allowing knowledge workers to collaborate on and share content. You can find out more here and download this latest version here.

New Alfresco Share application previewed in Alfresco Labs 3

We believe that the Enterprise Content Management market is evolving from specialized software for information intensive activities to become a major component of business and infrastructure for knowledge workers. Although over 90% of the Fortune 1000 have at least one type of ECM system, less than 10% of the employees in most of these companies use ECM, despite the huge increase in regulatory compliance and information explosion. Microsoft SharePoint, through its hooks to Microsoft Office, has made significant inroads into this yawning gap in the ECM market, and growing at 35% per annum as a result. We believe that open source has an even greater opportunity to reach into this portion of the market through its ubiquitous internet reach. With Alfresco Labs 3, our opportunity is to provide an alternative to, provide interoperability with, and be complementary to Microsoft SharePoint.

New Alfresco Surf RESTful platform uses Yahoo's YUI AJAX library for a more interactive experience. New thumbnail service provides previews before opening.

Much of the development of Alfresco Labs 3 has focused on building the Alfresco Surf platform, a robust, enterprise-class web application and site assembly framework that bundles a full site construction object model and toolkit for building class-leading web and collaborative applications. The Surf platform is designed to work in a number of different web environments, including as a Web Part in Microsoft SharePoint Portal. It includes content oriented components designed around the YUI (Yahoo! User Interface) AJAX Library and Adobe Flex for dynamic uploads and previewing of content and other information. The new user interface components make it much simpler for users to develop new collaborative web applications.

New Activities, Tagging and People services provide a social collaboration experience for the enterprise. Collaborative sites also have wikis, blogs, calendars and discussion forums.

With Alfresco Labs 3, we have also introduced a few important concepts as New Content Services to support collaborative applications and social computing including Sites, Thumbnails, Tagging and Activities. The Surf platform uses the Sites to construct a collaborative site and store framework information on how pages and components are constructed. The Thumbnail service generates previews of documents and content to show what you will open before you open it, enhancing the more limited view of an icon that only tells you what time you have. The Tagging service provides a simple folksonomy view of content as well as providing tools for constructing tag clouds. Activities are similar to Feed inside of Facebook, providing a stream of information on what activities people are working or commenting on. To support these services, the People services have been enhanced with more information and the ability to connect people to one another for social networking. These services are accessible through REST-based APIs and from JavaScript allowing their use in construction of new Surf components.

Integration with Flex and YUI provides simultaneous loading of files.

Alfresco Share, which we are previewing with Alfresco Labs 3, is built on the Alfresco Surf platform and is designed to be modified by end users or through programming to fit a wide range of collaborative and social computing applications. Share provides a simple web site paradigm for creating collaborative applications by aggregating Surf components and incorporating new Surf components as they are developed. This is an early release of the Share application, but we encourage our open source community to develop new components based upon the examples of the basic components provided. With Share, users can create a collaborative site either inside or outside the organization, invite users and share and collaborate around content. Share includes document libraries, calendars, wikis, blogs, and discussions.

Microsoft Office can access content in Alfresco as though it were a Microsoft SharePoint Server

Also previewed in Labs 3 is Microsoft SharePoint compatibility from within Microsoft Office that makes Office applications think they are talking to a SharePoint server. Microsoft released the SharePoint protocols this spring as part of its compliance with the European Commissions’ decision issued on March 24, 2004 along with an increasing tendency by Microsoft toward interoperability. Alfresco is the first ECM system to implement the Microsoft Office and Windows SharePoint Services protocols as a compatible server. This allows Office users to browse and find documents within the repository, checkout / checkin / version documents, share the documents in shared workspaces and access the additional menus and task panes reserved for SharePoint. All of this is available with no additional software needed to be added to Office.

The entire team is getting way too big to congratulate individually, but everyone has done a great job so far getting this beta to realization. For this, I am extremely grateful. We look forward to providing the next update in early September where the following will be provided:

• New REST APIs based upon ATOM Publishing
• New look and feel for Alfresco Share
• Additional collaboration components, including instant messaging
• A new properties framework for extended metadata
• New people and site search capabilities built around social networking
• Web views support for Microsoft Office SharePoint emulation
• Associated technical documentation

Our current plan is that the Enterprise version of Alfresco 3 will be available in mid-October.

We think that Alfresco 3 will provide our users and customers choice in developing collaborative application and in supporting basic content management and collaborative sharing for Microsoft Office users. Rather than a single fixed platform, Alfresco offers a choice of operating system, database, application server, development environment, and web browser, including those provided by Microsoft. In addition, through our work with Unisys, we have demonstrated a platform that is much more scalable. By providing this new platform as open source, we hope that you will also help us in making it a better platform and contribute new collaborative capabilities and components that we could not even have conceived.

How Web 2.0 will change the face of business

The following is an article that I wrote recently for a magazine...I'm sure the marketing team will shorten it.:-) Feedback welcome...unless of course you are posting a URL to cheap watches.

Speaking to the IT architects of multi-national corporation about their plans around using Web 2.0 in enterprises from the largest oil and pharmaceutical companies to global banking and accounting firm, it is clear that most, but not all, have plans and intentions to use the new technologies that have changed the way we use the internet. Forrester predicts that by 2013, social software, the application of Web 2.0 for the enterprise, will grow at an annual rate of 43% per year to $4.5B by 2013. This is quickly becoming the fastest growing sector in the Enterprise software industry. However, many people are generally confused by what Web 2.0 is and it's significance in the workplace and in culture, including those planning to adopt it. The term has been for around four years old now and was coined by technology gurus Tim O'Reilly and John Battelle to describe the resurgence in activity, venture capital and huge audiences that surrounded new emerging web sites. However, in that time, there is no concise definition of what Web 2.0 is.

Web 2.0 is explained more by example than by defining the technologies that make it up. A collection of brands provide the metaphors for what exactly is different in the way we use new web technologies, such as Google for search, YouTube for video, Flickr for photos, MySpace and Facebook for social networking and Wikipedia for wikis. There are a few more examples and there may be new sites, like Twitter, that may expand our understanding of Web 2.0, but we are coming close to a complete list. These brands as metaphors become the nouns and verbs of describing Web 2.0 as a new way of socializing, communicating and sharing with each other in huge, consumer-scale markets. By being the first to create critical mass in the internet space, these brands have been able to define the way we will live and the way we will work. After all, it wasn't really a PC until IBM named their desktop computer a Personal Computer.

However, Web 2.0 is not really so much a revolution in technology, but in how people use technology and how people interact with each other as a result of that technology. The amazing technological innovations have really been happening behind the scenes with the huge build out of inter-networking and creation of new scalability technologies through open source. The open source sharing of code used to build these sites have made it possible to build and manage the sites on a modest budget and deliver incredible new content and services to absolutely anyone with great performance. This in turn has allowed a whole new class of people have come to use technology that wouldn't have had access to it before. Now even your granny was connected to the internet. The internet and desktop technology was no longer the domain of the geeks and nerds. Real people, average people, artistic people, old people, young people were connecting to the internet and discovering each other. The web sites, in turn, were reacting and evolving a very rapid rate to adapt to these new users.  Sites that appealed to these new users grew out of a Darwinian natural selection and became a lot more facile and adaptable in the process and very large and profitable.

As a result of the introduction of the internet, rapid infrastructure build-out and the new generation of Web 2.0 sites, we have seen one of the most dramatic democratizations of technology since at least the PC, if not the telephone. Through universal access, users discovered that computers could be used for far more than information; that they could be used as a medium of expression, sharing and revelation. Although the PC gave access to computing power to most office workers and many home users, it was generally within the constraints of software created by others using information created by others in business domains defined by others. The information that users created and shared tended to be very textual, columnar, organized and very factual. If you wanted to liven this information up at all, you would add in a few sappy clip art pictures to express what you were really trying to say. In short, it was an environment that was invented by geeks and nerds (like me!) and generally appealed to similar personality types.

By broadening access to information and technology and broadening the types of users accessing that information, new and more expressive types of people looked for ways to use the technology that suited their personality. Users voted with their mouse for sites that appealed to their personal sense of expression. If you were musical, visual, artistic, auditory, adventurous, sympathetic or caring, you could share art, prose, music, visuals or images with those who shared your interest, rather than a production studio in the State of Washington. The place that these people met were generally at sites that catered to introducing one friend to another - first Friendster, then MySpace and slightly more recently Facebook. Depending on your mode of expression, you may end up going to Flickr for photos, YouTube for video, or Last.fm for music and then linking it all into your personal page on your favorite social networking site. The socialization and communication enjoyed by this expression and sharing created a truly different feeling that no spreadsheet or presentation could ever possibly provide. You were Flickring, YouTubing, Googling, Twittering on your MySpace or Facebook, sites that almost have a feel like a real geographic location. For many people, these activities have become compulsive and addictive in the process. This is why the average age of television watchers in the US has now risen to 50 years old and a new generation of users will not be willing to go back.

These compelling experiences have attracted huge numbers of people, which in turn makes the experience even more compelling in a positive feedback loop. Just as in any revolution, once critical mass is achieved, the revolution takes on its own momentum and is self-adapting. A critical mass of people were dictating what is interesting and what is not. What is acceptable is far more likely to be those sites and capabilities that minimize constraints and empower people. Those that put constraints on reasonable behavior were quickly discarded. Sites that allowed individuals to write, like WordPress blogs, to edit, like Wikipedia, or to tag interesting content, like Digg, rose quickly up the internet charts of popularity. Once Facebook removed restrictions on who could join or who your friends could, it quickly grew exponentially to over 60 million users. Web 2.0 became a very democratic revolution with core principles of freedom of speech and freedom of assembly on the internet. The notions of controls are not determined by central authorities, but by the users themselves in terms of their access to the site.

By lowering the barriers to participation, this provides platforms for anyone to contribute to a site like Wikipedia or YouTube. Actual active contribution of content can be a relatively small number, less than 1% of all users according to an article in Time on the 25th April 2007 and much lower for YouTube, but as high as 4.5% for Wikipedia visitors. However with a critical mass of large numbers of people, this can still represent hundreds of thousands of authors and contributors. It also doesn't take into account social networking sites where everyone is a contributor by simply creating their home page and being compelled to enhance and adorn it in response to their friends doing the same thing. I have seen several large corporation that have skills or profile pages for their users, but even senior executives may be more likely to update their Facebook or LinkedIn profiles than their corporate skills page.  In addition, this doesn't include what might be the most powerful aspect of this participation - feedback of the masses. Many people comment on blogs when they feel passionate about a particular subject and are more likely to rate information provided such as reviews or products available on-line. In a world swamped in information, feedback on popularity and rating, hallmarks of Web 2.0 sites, provides a valuable indicator of what is important and what are leading trends.

Dubbed "Wisdom of the Crowds" by James Surowiecki in 2004, a mass of individuals on average is smarter than anyone person or expert could ever be. At some point excesses of disclosure and impropriety go off in their own direction and those seeking refuge of appropriateness find it in communities of like minded people. Those looking for accurate or appropriate information just move off to more appropriate communities. This provides the feedback loop of control through democracy of participation. Wikipedia is a good example of this as it has demonstrated that it can be just as accurate as encyclopedia Britannica, but much faster at correcting mistakes. "Lord of the Rings" provided feedback to the community  on their website on how the story was evolving prior to filming and ultimately became one of the most popular series of films. The "Da Vinci Code" kept everything secret and, despite the popularity of the book, did not fare well in the box office.

Michael Lynch, CEO of Autonomy, suggested in Financial Times on the 30th of June, that Web 2.0 was something that must be tamed. Perhaps this is missing the point. Web 2.0 is not anarchic nor is it necessarily bad for business. Web 2.0 To try to control Web 2.0 is like trying to put one's finger in the dyke. It is happening and there is nothing that business can do to prevent it. In fact, when companies try to restrict access to Web 2.0 they either find that the roadblocks have been circumscribed or that potential employees will go elsewhere. Generation Y, the generation born between 1978 and now, is expected to grow from 25% of the US workforce to 47% by 2014. This is a generation that has only known the empowerment of the internet and have become accustomed to their vote counting. To try and control it now would only disenfranchise them. To empower them would yield not only an optimistic workforce, but also provide an engaging conversation between employees, their customers and their partners in a participatory and enlightened collaboration. In addition, my generation, the Baby Boomers, are starting to retire now, taking with them some of the most valuable knowledge ever accumulated in some of the biggest numbers ever. Knowledge management programs over the last two decades have failed to capture that knowledge, is Web 2.0 our last hope of retaining it? Interestingly, the Time article suggests that those over 35 are more likely to contribute content to Web 2.0 sites, so this may be an indicator that older generations want to contribute their knowledge from experience and are willing to do this through Web 2.0.

Software vendors are now jumping on the bandwagon with social software and collaborative features smelling a bit opportunity. Many are repackaged capabilities from another era of enterprise software. Some are looking at their portfolios and asking whether this is what they were doing all along. This misses the point. Web 2.0 has so far outstripped enterprise software as we know it in usability, accessibility and empowerment, that it causes mass rolling of eyeballs at its mere sight of not just the new generation, but most others as well. Those who are familiar with the ease of use and empowerment of Web 2.0 sites like YouTube, Wikipedia and Facebook are aware of what is possible and have much higher expectations. I believe that the enterprise software vendors will get there, but with much coaxing and coaching of a new generation. It will take a few years, but eventually they will figure out that Web 2.0 is not just a few new collaboration features and highly interactive web technologies, but empowerment of their users and the ability to draw in a critical mass of users from outside the trusted circle.

Enterprise systems won't change immediately, but it will probably change faster than people expect. Care should be taken in what is opened up. However, rather than treating Web 2.0 content and technology suspiciously, corporations should ring fence the information that must be controlled and open up the rest to participation. At the Enterprise 2.0 conference in Boston this June, Pfizer presented how they were using open source technology to enable Web 2.0 collaboration. This is a brave move in the highly regulated world of pharmaceuticals, but they have recognized clear boundaries of what must be regulated in content, particularly in manufacturing and research practices and what can be opened up, such as redefining process or identifying new potential areas of research. The scenarios of "Doctor 2.0" are available on the internet, but they have created a vision and a reality that uses the same technology as Wikipedia to create Pfizerpedia, a wiki of process and ideas that feed into the main areas of research and manufacturing.

Change in the Enterprise is more likely to come from outside as well. In all likelihood, if you are information worker, you use Google more than any of your internal IT systems. You may even rely on blogs to track what is happening in your industry more than you rely on industry press. You and your co-workers are likely to use these and other Web 2.0 technologies to track what is happening in your business world as well as your social world. These web sites will set further expectations on the internal systems you use and a requirement to integrate internal information with these external sources of information. Web 2.0 has an answer for this as well with an integration technique known as "mash up", the ability mix information from multiple sources using the web browser itself as the point of integration. These external sources of information also provide something that our internal information systems could never provide, a critical mass of opinion utilizing the Wisdom of the Crowds. We will ultimately need to combine external opinion with our internal opinion to get more accurate predictive decision making with our own unique insights inside the enterprise.

Ultimately, the most profound effect Web 2.0 will have is on the way we do business rather than just the technology we use. Employees will use this freedom of speech to provide valuable and fearless feedback for the business. Employees will have the freedom to assemble teams with customers without interference. Customers will become part of the decision making process and allow us to design the most imaginative products and services. Control will be limited those areas that ultimately must have control and free up the creative process to speed and enhance business. Empowered employees will build more productive businesses and become more fulfilled participants in the business. With any opportunity comes risk and embracing Web 2.0 is not without its risks. However, smart businesses can already see the opportunities and are willing to take those risks.

Impressions of Enterprise 2.0 in Boston

Boston at five in the morning before I had to take off on Friday.

This past week, I was on a panel at Enterprise 2.0 in Boston with Bob Bickel of Ringside Networks and Jeff Whatcott of Acquia, the commercializer of Drupal. The topic was open source options for delivering an Enterprise 2.0 Experience. Both Bob and Jeff are excellent speakers and bring a wealth of experience behind new companies. I think that Kathleen Reidy at the 451 Group did a very good job of covering the panel, so I will move on to my impressions of the conference.

On the panel, we spent much more time talking about open source and less about Enterprise 2.0. However, this doesn't mean that there was a lot of clarity on the meaning of the term Enterprise 2.0 at the conference. Although Web 2.0 had no less than Tim O'Reilly and John Battelle to define what that term means (barely), Enterprise 2.0 has no such authority. Consensus says that it is just Web 2.0 for the enterprise. However, researching the concept a couple of years ago, E2.0 is about taking the social aspects of Web 2.0, collaboration, social networks, user contribution, wisdom of crowds and social tagging and voting and applying it to information, documents and content in the enterprise. There are no fixed patterns for how to do this, although popular Web 2.0 sites, such as Facebook, Google Maps, Digg, YouTube and Wikipedia, provide at least paradigms for how these can be accomplished in the enterprise. It is very difficult to describe Enterprise 2.0 without drawing analogies to these web properties.

On Wikipedia, the topic "Enterprise 2.0" redirects to Enterprise Social Software. In August 2007, a "Ruud Koot" permanently redirected it from Enterprise 2.0.  The last direct version of an Enterprise 2.0 article in Wikipedia extolls an Alan Wurms as the person who apparently coined the term in 2001. That is the power of Wikipedia, it can get rid of the rubbish. Most people's problem with the term is that it does not describe what it does and it sounds like it is just riding on the tailcoat of the Web 2.0 phenomenon. Is this really a new version of the Enterprise? I sat in a session with Carl Frappaolo where he equated Enterprise 2.0 with the evolution of Knowledge Management, but made the point that enterprises have not fundamentally changed as a result.

Some people believe that Enterprise 2.0, like Web 2.0, must be delivered as a whole hosted platform on the internet in order to be Enterprise 2.0. For some people this is absolutely true, but a majority still look to keep this information under enterprise control for bandwidth and security reasons. The majority of vendors in the exhibit area provided Software as a Service solutions. Most likely they used open source in creating those solutions.

I apparently missed the highlight of the show, which actually occurred on Monday before the opening. This was a shoot out between Microsoft SharePoint and IBM's counter to SharePoint, Connections. As an IBM product manager at the IBM booth said, "We don't fuck up demos." Everyone seemed to agree with him. Poor Lawrence Liu of Microsoft was not so lucky. The Microsoft demo did not have the business process coherence in which IBM is very well versed. There was a lot of hand-waving about how various Enterprise 2.0 features were supplied by partners. The performance issues that Lawrence faced may very well be related to the terrible internet connectivity provided by the Westin Hotel. Imagine an Enterprise 2.0 conference where no one is connected. Both companies are talking more about Social Software, Social Computing and Social Networking more than Enterprise 2.0, so my feeling is that this emerging market will be named more along those lines rather than E2.0.

Peter Fields of Wachovia

There were three customer presentations on their usage of Enterprise 2.0 and these present probably the best understanding of what these collections of technologies are, what they are trying to accomplish and what market is forming as a result. Despite the fact that he was using Microsoft SharePoint, I really liked the presentation from Peter Fields of Wachovia. Peter seems to think about the business problems and technology solutions the way I do. (Or probably the other way around.) He described the need to empower employees as a way of tapping into the intuitive sense of employees and he is the only other person I have ever seen that has uses Myers-Briggs to describe this paradigm shift. In a session just before, I got shot down in flames for daring to suggest that the change in enterprise software is the result of shifting demographics and a new, incoming generation of worker - the Millennials. Here Peter was backing it up with data that suggests that in less that five years, this generation will move from 25% of the working population of the US to 41% of the working population. He discussed an imperative that I had not really considered as well, which is that the baby boomers are retiring and this will represent the single largest loss of implicit knowledge in industrial society. Enterprises MUST facilitate capturing what the baby boomers know now and lower the barriers dramatically toward capturing that information.

Simon Revell from Pfizer

Peter is roughly my age, but Simon Revell from Pfizer, who looks a lot younger, presented a view of what the new generation wants - seemingly both Generation X and Generation Y. Pfizer has created a couple of sets of slides describing life in a networked world. Pfizer does use the 2.0 word and even describes a "Doctor 2.0" as a female researcher who also seems to spend a lot of time on Facebook. But rather than trivializing what that means, Simon presented a set of tools that Pfizer is using (open source by the way) that allow researchers to collaborate. Pfizerpedia is a mediawiki implementation modeled on Wikipedia and used as a single instance. Its primary purpose is to capture best practice in an informal way, which Pfizer codifies and controls after the process is discovered or developed. The result is actually a knowledge base of information that can be used for many other purposes. My sense has been that wikis that are single, highly interconnected instances rather than many team or project wikis. Bob talks about one wiki, Peter is hoping for 10,000 wikis. My take is that we need two words for what is now described as a wiki.

I wish I could have seen more the conference, although many of the break out sessions didn't add a lot. The subject of wikis and blogs have been covered better at Web 2.0 conferences. Some of the sessions on community didn't really say anything at all. Neither did a session by Mark Woollen from Oracle CRM. Mark is a good speaker and there was some good content at the end. Too bad that the beginning didn't say much except that social networking will probably be important in the future.

The direction that Oracle's Mark Woollen's presentation took

Enterprise 2.0 is not just about wikis, blogs and forums. These do not make communities. A whole bunch of the vendors in exhibit area are likely to be gone in a couple of years if not sooner. Open source, although played down in this conference, will likely be one, if not the, major driver of this new market. Companies are actually using this and it is those that hope to attract and retain not just a younger generation of employee, but also customer. The smart ones are also recognizing that they need this stuff to make sure that critical knowledge is not retired when their baby boomer employees have.

A Manifesto for Social Computing in the Enterprise

Investment in the infrastructure of the internet has dramatically increased bandwidth to everyone in the developing world and created home computers that are not only inexpensive, but very powerful. This change has expanded the usage of the internet exponentially and introduced new demographics and generations of users that had not used computing prior to the expansion of the internet. These users have themselves created the content and applications that feed the internet and have set expectations of the applications that we use in web browsers and new mobile devices. The increased bandwidth has made this experience much more interactive and visual experience encompassing video and visual elements. Web properties such as YouTube, Google, Amazon, Facebook, MySpace, and Flickr have set the benchmark for expression, accessibility and social interaction of computing systems.

Dubbed Web 2.0, this revolution in computing has shifted the face of software from a logical, linear, and introverted science to an expressive, graphical and social art. New designers of web sites, unschooled in traditional software techniques, are nonetheless able to create software that scales to millions of users and billions of objects of information and still meld those users into an artistically aware community. The next generation of enterprise employees who started using the internet in their early teens have only known this evolving culture of free and creative development of the internet and now demand better of the enterprise software that they meet. Older employees also know that that the software that they use on a day to day basis can be better. Enterprise 2.0 seeks to emulate the success of Web 2.0 in the creation of new software for the enterprise.

Social Computing

The shift of computing power from business logic and calculation to socialization and people-orientation has been dubbed by some as Social Computing. The term Social Computing has been used interchangeably with Enterprise 2.0 or Enterprise Social Applications, however, IBM and Microsoft have created Social Computing research centers and Forrester has started to use the term in describing next generation enterprise collaboration. Social Computing is the use of technology to support sharing of information and enabling collaboration through social networks and to tap into the value of the “Wisdom of Crowds”, a concept made famous by James Surowiecki in 2004 to explain how many people are smarter than individual experts. Social Computing exploits software oriented toward people and Social Networks, the extended relationships of individuals, to connect to more people and access the Wisdom of Crowds.

To tap into the wisdom and awareness of social networks and empower people to collaborate at any time or place, Social Computing platforms need the following capabilities:

• People - Support information about people, their preferred communications, their relationships and affiliations, since social networking is all about people rather than just systems, data and objects. The more information available about other users, the more likely they can be found as a source of knowledge.
• Context of Networks - Social networks organized around projects, teams and departments provide the context of work and relevance of information as it spreads from creation to the people that need that information. Social networks, especially networks extended beyond the enterprise, provide the greatest differentiation of social computing from previous generations of collaboration.
• Social Collaboration - Provide an environment where people can share ideas, contribute knowledge and solve problems in creative, unstructured socialization as opposed to rigorous workflows that are required for control of information. Next generation tools use techniques developed by Web 2.0, particularly those tools that empower social knowledge, such as social tagging, integration of communication and awareness of changes in social networks.
• Content as a Service - Content is the container of knowledge and information and is core to the socialization of information. Content needs to be accessible everywhere, not just in large, monolithic applications. Content capabilities need to be accessible as reusable service components. Social computing can happen inside the enterprise or outside and a channel can be a web site, web application, mobile device or even external web platforms such as Facebook or Google applications. Mashups can occur inside the enterprise or outside and the channel will require content as a service that can securely be accessed wherever it is needed or wherever it is contributed.
• People-centric Tools - As Web 2.0 has spread new paradigms of user interaction, the consumerization of software has created expectations that enterprise software becomes easier and empowers user to contribute, correct and classify content and information within the context of social networks. AJAX and next generation rich internet application interfaces such as Adobe Flex will provide users with a much richer, more intuitive user experience and the ability to scan much more social knowledge to find ideas and solutions. These tools should themselves be componentized and accessible as a service so that they may be mashed up with other sources of social knowledge.

This does not mean that the need for traditional enterprise content technologies such as document and records management goes away. They are still repositories of the truth and verifiable information and thus play an important role in sharing knowledge within social networks. However, these traditional technologies lack the usability, empowerment, and breadth of reach that Web 2.0 sites provide. They lack the collaborative nature that invites in people without barriers and restrictions to contribute to the sharing of knowledge and information. Web content management for creating a richer Web 2.0-style user interface becomes even more important to this collaboration to provide a compelling face to the interaction and to simplify the access and navigation of shared information. Enterprise Content Management cannot become one of the principle platforms of Social Computing unless it addresses the requirements of Social Applications.

Use of Social Computing

The balance is shifting from contained and controlled companies to engaged and empowered collaborative enterprises driven by Web 2.0-inspired social computing. At the center of the shift from old models of computing in the enterprise to new social models are companies that are inspired to innovate or to engage more with their customers. This includes companies not just using their internet or intranet web sites, but engaging in social networking channels such as Yahoo, Google, YouTube, Facebook and MySpace. Those using social computing are interested in engaging people, such as customers, employees or partners. They are using new people-centric tools and facilitate creating or extending existing social networks.

Major ECM vendors are all planning their Social Computing efforts and to a large extent are being dragged in this direction by their more forward-looking customers. Enterprises that have discovered the value of Social Computing are:

• Consumer-oriented companies that particularly address a younger demographic must engage their customers as part of both the marketing process as well as the development of new products. For example, games and film companies that engage their viewers in plot and scene development do much better than those that keep everything under wraps until the game or film is ready.
• Enterprises hiring a new generation of knowledge workers who grew up on the internet must provide tools as empowering as those available from Web 2.0. Turning these tools off forces these workers to seek employment elsewhere and forcing them to use tools that do not meet their expectations of usability and engagement.
• Financial Services firms are leading the shift in usage of these technologies. Financial Services have always been innovators in developing new technologies and investing in providing better service for their clients. Speed in innovation in these services becomes a major competitive advantage where churn of clients can be very high in turbulent times. Internally, competition for talent is intense and providing better support is important for attracting and retaining employees. In particular, young and ambitious brokers and managers are more likely to be sociable themselves and seek out Social Computing inside and outside the enterprise.
• Government and Non-Profit organizations that provide services and citizen feedback online find increasing their IT budgets much easier than those that merely arbitrated by a front-line service. It is now inconceivable for an American politician to run for office without an extended internet presence such as Facebook or YouTube.
• Enterprises that have faster cycles of product innovation, especially high tech, are looking to their customers and partners to participate in the development of new products and services. In previous generations, the field acted as a filtering mechanism of new customer requirements and ideas. However, today technology can provide a frictionless way of getting the entire enterprise to exchange ideas and improvements with the customer communities.

Integrating Social Computing

Because Social Computing is unlikely to come from a single source, especially because of the diversity of sources of knowledge and social networks available on Web 2.0, it is extremely important for the enterprise infrastructure for Social Computing to be integrated with those sources. This means bring these sources into the enterprise and bring the enterprise sources out to Web 2.0. No matter where the people collaborating are, the tools they want should be available. To facilitate this, the Social Computing should be:

• Open Source - Through being developed through social computing paradigms and sharing best of breed components with the open source community, open source systems have evolved rapidly and encompass social computing capabilities developed by the open source community. Social tools such as MediaWiki, the wiki that powers Wikipedia, and WordPress, the most popular blogging software were developed using open source.
• Integrating the Inside Out - By providing content as a service and enabling light-weight, Web-Oriented scripting development, the Social Computing platform should quickly integrate content services into external channels and web sites, such as Facebook and iGoogle, to allow enterprises to engage customers, partners and home workers.
• Integrating the Outside In - If the Social Computing platform is modular and supports a Web 2.0-style mashup-oriented architecture, it enables users and teams to integrate external open source tools and social networking web services, such as Facebook, LinkedIn or other Open Social-enabled properties, to tap into the wisdom of crowds available on the internet and to make customers and partners part of team collaboration.
• REST-style Architecture - A Web 2.0-style or REST-style of architecture using easy, light-weight scripting languages and integrated through internet standards-based APIs can easily mash-up content services into any web-oriented application or web site. These architectures should be scalable, fault-tolerant and high performance to meet any enterprise or internet requirement.
• Choice - The Social Computing platform should be based upon open interfaces developed by the open source community to provide choice of operating system, database, application server, content authoring tools or APIs.

Over the past year and continuing into the coming year, Alfresco is dedicated to expanding its architecture and applications to enable this vision of Social Computing. We will work with partners and open source community to provide best of breed open source tools for enabling this architecture. We will integrate with external Social Computing properties such as Facebook and the Open Social alliance to expand the breadth of social networks and the ability to collaborate through those networks. We will be expanding the Alfresco system’s understanding of users as people and facilitate sharing of information and content through their networks. We will be open in the process and seek and encourage your feedback and participation.

Scaling Out Like Technorati

My fellow World Economic Forum Technology Pioneer, David Sifry, the founder of Technorati, was also in Dalian, China for the “Meeting of New Champions” or “Summer Davos” as the Chinese like to call it. During Davos in January, I had the great misfortune of pitching Alfresco against Technorati in a competition between tech pioneer companies. As fantastically well as Alfresco is doing, Technorati has the temerity to compete against Google in blog search and win.

I got the chance to talk to Dave during the conference and ask him some questions on the technology and architecture behind Technorati, the internet blog search site. I thought that someone who could take ordinary computer components and build a huge internet architecture could possibly teach something to people running enterprise architectures that are puny in comparison.

Technorati is a web site that tracks blogs, pictures and any user generated content and allows you to search those sites about what people are thinking, seeing and hearing. When a new or urgent situation breaks out, you can do worse than to search Technorati for immediate reaction. Every day, every hour, every second, Technorati is indexing over 10 million blogs with over 10 billion objects. Technorati’s user base is doubling every six months and quick and accurate response is critical for retaining those users.

David Sifry, Founder and Chairman of Technorati

I asked Dave about his architecture and what applicability their might be for enterprise architectures.

John Newton: In building Technorati, what were some of the issues that you had in architecting your systems.

David Sifry: I was looking at just temporal information. I had no idea how big it could get. When I looked at the architecture, instead of architecting it right, I architected it for right now. I had no big budget and I didn’t want to wait six months to build it. Also, I had no idea what the killer app would be.

I focused on data flexibility. At the time, that meant putting everything into a relational database. That was okay while the size of the indexes is less than RAM and about a million blocks of data. That was okay while there were less than 20 million blogs.

The next generation took advantage of data parallelism. That meant upon update send a signal to all the other systems. We expanded the data over several “shards” [segments of data partitioned on different databases on separate machines].

What was surprising was that we were writing as much data as we were reading. At this point Technorati was as big as some of the biggest OLTP. Even so, maintaining data integrity was important, because you would want the link count [count of how many other blogs point to a particular URL] to be out of sync. This put real pressure on the system. At the same time, we realized that time was more important dimension than URL. People didn’t want to sort or search on URL, they wanted to search on time. [i.e. what are the latest blogs on a particular subject?]

By this point, we understood the application more and more. The app [Technorati] is about real time access. You need to be able to count on finding latest information on a subject. That’s when we built the third architecture. Scaling was well understood and we build the shards on time rather than on URLs. Instead of putting data into a DBMS, we put it into special purpose databases. It was more of a bus-based architecture. Each database could be scalable and grow as big as we needed.

JN: The notion of shards - did you call it that at the time? I have been looking into shards and I was only aware of or heard of them for about the last year.

DS: Back in 2002 when we were pitching this to VCs, I at least explained the theory. All I just thought through the problem carefully. Doing it this way, we could add hundreds of systems, lots of cheap CPUs, RAM and disks. It provides inherent parallelism. I can’t believe that I was the first one to think this up.

JN: How big does this architecture scale?

DS: We are loading one terabyte a day into Technorati. That’s 100 million blogs or about 10 billion objects. A lot of is new types of tagged data. There are about a half billion videos and photos.

With all that data, you have to think about what do you throw away?  We can’t really delete anything, because we are potentially losing an asset. We don’t delete anything. So we take data out of the spin cycle. [Transitory data used in preparation.] We take the long-term data and put it into low latency storage.

When data is doubling in size every six months, that means that only one quarter is a year old. We don’t need to worry old data.

JN: How do you deal with large number of users with very large data sets?

DS: Any off the shelf tools falls over. There is a lot of interesting analysis on old data, but no off the shelf tools can handle that much data. It’s only just now that some tools can handle it.

JN: What are those tools?

DS: One is Green Plum by a bunch of O’Reilly guys. If you use ordinary data warehouse tools, they would just scream and shout.

JN: Actually what I was originally referring to was the fact that you are showing lots of data that are not users used to enterprise information management tools. How do you present this information to consumer-level users? How do you deal with the user interface and visualization of all this data?

DS: Gotcha. It depends on what the user wants to get out of Technorati. If the user wants search results, then we give it to them. Sometimes they want to browse or discover information. We have spent a lot of time on visual design. Then we give them lots of bright, shiny things for them to click on.  Things like metadata, video or other links.

We have used enterprise class web tools to analyze what users are doing? We look at the click stream and see what is successful or not. That helps to make the information contextual.

One of the big mistakes that we made is to not do this [buy click stream analysis tools] sooner. It was only $80K. Up to that point it was so much trial and error. I’m glad we finally did it. Now we can see how much time a user spends on a feature. We can see page views, goals per visitor.

JN: So what do you measure on Technorati?

DS: Measuring a web site is like forecasting the weather. Yesterday it’s sunny and today it is cloudy. Why is it cloudy?  Sometimes you have no idea. Sometimes you realize that that a change in barometric pressure has a lot to do with it.

We look at the number of newbies, number of reports, session lengths and then measure them against prior periods. It’s not always consistent.

I had never built a B2C site before. I just focused on me, on what I wanted. That worked well for a while when I was the target audience. But we have to build for a broader audience.

JN: At Alfresco, we measure conversions. Are you measuring things like performance? Does that affect retention of users?

DS: Of course, but if the system is falling down, then even performance doesn’t matter. So I don’t get too stressed out about it.

JN: When we met at Davos you wanted to move Technorati to be the Internet Now! Is that still the case?

DS: Everything is shifting. I wanted it to be a site that everyone is able to use. We forgot about the core users that just wanted to find out about blogs and any real time information. In an attempt to jump the chasm, we chased after 100 million users and tried to be everything to everyone. Now we try to make blogs and user driven content available for those looking for that.

Also performance is improved significantly. Now I notice how slow other sites are. This is a total tribute to the engineering team. Everything is easier and faster.

Pretty soon we will have a whole lot of stuff that we have been working for a year.

JN: Can you say what it is?

DS: I don’t pre-announce.

JN: What does the Technorati brand stand for today?

DS: Good question. What’s popping up now on the internet, especially user generated content? It’s about users tagging user generated content and finding it.

JN: Who are your competitors?

DS: I probably sound like the typical entrepreneur, but nobody really seriously. Google provides blog search, but other than that nobody really. Other people are trying to identify and tag information like Digg and del.icio.us, but they aren’t really competition.

JN: What do you want Technorati to be in two years time? Five years would be ridiculous.

DS: I would like Technorati to be a profitable business that is strongly differentiated. It will be the place that you would go for mobile, RSS or push information. For all that you would come to Technorati.

IBM's Many Eyes

I found this reference to IBM's Many Eyes. Many Eyes some of the concepts of Wisdom of Crowds and Web 2.0 visualization, this site encourages anyone to create graphs, data visualizations and tag clouds by uploading their data sets. You can try it out here.

Imagine if this used Flex.

I want my Joost TV

While I was in Davos, I got to meet Niklas Zennstrom, the founder of Skype. Somehow he neglected to mention that he was about to revolutionize TV. Later on in the conference, I heard rumors about Joost, his new service, but unfortunately I couldn't ask Niklas for a beta invite. It would help if remembered who I was along with other couple thousand people clambering around Davos.

The buzz is that Joost is the future of TV. I signed up for the Joost beta a couple of weeks after I returned and I got the download notice in my inbox two weeks ago. I finally have managed to download the beta today.

After a couple of sputtering starts on our home 5Mb DSL line, the thing finally got going. At first I thought this is not going to work as the image started and stopped, but when it did get going it was presenting me with near DVD quality images on my PC. I hear they have a new proprietary compression algorithm, but the image and sound are pretty good.

You are first confronted with a user interface for selecting channels of information. Since I live in the UK, I don't think I have heard of any of the channels or the programs on the channels, but I understand that they are signing up some pretty amazing media deals.

From these channels, you can select specific programs to watch on demand.

Once you start watching, you get a full screen viewing experience by default. It is absolutely nothing like YouTube. It is full screen with all the normal controls that you would expect.

While you are watching, you can access a set of widget for adding comments or instant messaging a friend. If they open up the platform the way that Skype has, you can imagine all sorts of widgets being created for searching for related shows, looking up references to and from the show, historical references or simulcasting your own commentary and voice over.

One thing you should watch out for. As soon as I downloaded this, I started taking screen shots. I live in the UK, so I don't know anything about a series called Total Recall 2070, but it looked interesting. After taking this screen shot, the couple take their clothes off and start getting it on when my wife walks in. I am only talking about the first few minutes of the program. I have to explain that I am doing this in the name of research! It could probably use some sort rating on the programs, at least while I am trying it out at home and not in the office.

Anyway, I have no more invitations for 3 friends to invite to the Joost beta. Please let me know if you are interested.

Add Salesforce.com to the Enterprise Content Management List

Today, Salesforce.com announced that they will be getting into the ECM business by acquiring Koral, one of our neighbors here in Maidenhead. Some of the guys there are our friends and we wish them luck in this new turn in the ECM market. They scored a real coup showing up at Demo last year and it obviously got the attention of Salesforce. The system has not been around long, but they have added some interesting Web 2.0 twists. It is focused on document management and was born out of the efforts of BuildOnLine, a specialist online content management provider for the construction industry that has recently merged to create CTSpace.

This is a significant shift for both Salesforce.com and for the ECM market. As we all know, although most of the Fortune 1000 have ECM, penetration into those accounts can generally figured in single digit percentages and practically non-existent in smaller organizations. Increasingly, ECM will be delivered either as a software or physical appliance or as software as a service in a utility like form. Smaller enterprises or organizations that have generic content management requirements will find this service useful. This will allow Salesforce to leverage its brand and start with the sales organizations. It also give Salesforce a means to expand its business beyond the sales and marketing organization. It also further validates the Software as a Service model for simple utility functions.

Although everyone is looking for simplicity, not everyone will be looking for a utility-like approach to ECM. It is up to the ECM vendors to simplify the installation and set up process to make it as easy as flipping a switch, but keeping the content inside. Organizations that are not comfortable putting their documents outside the firewall, such as financial services and government organizations are more likely to look for an internal system. Also, as the BuildOnLine guys found out, once you move to the area of specialist content applications, the sale gets much harder and configuring systems becomes even worse.  Records management, engineering applications and specialist publishing applications fall into this category. It will be entirely up to each organization which makes sense for them.

Software as a Service is a model that Salesforce.com didn't invent, but has become its biggest proponent and greatest success story. This acquisition will raise the profile of SaaS as a model for ECM. Salesforce will not be alone in delivering content management services as others are developing their solutions with systems like Documentum and Alfresco. Likewise, Microsoft is taking Sharepoint on-line with the Office Live offering. Other companies are now looking at providing an SaaS model for web content management and collaborative content management as well. No doubt we will be seeing feature by feature comparisons between these various solutions soon. Depending on the breadth of functionality, integration with internal systems and scope outside of sales and marketing content, we will see how Salesforce does.