Content Log

New Alfresco 3.2 was designed for the Great Recession

Back at the height of the market downturn in October, we looked at how Alfresco should address the rapidly changing economic situation. Rather than being pessimistic, we believed that this was a real opportunity for us. Times like these wipe the decks clean and provide openings for companies that provide value for money and can replace existing older technologies. This is the fourth recession in my career and what past experience has shown is that management and IT are under pressure to do more with less, cut costs, make remaining people more productive, and implement new technology if there is a clear zero-sum gain in cost reduction. Regulation also always comes after the disaster hits, but the new regulatory regime must be addressed with fewer resources. It seemed like a perfect time to be in open source!

Today we are releasing Alfresco Community Edition 3.2 and it really is an ECM designed for the Credit Crunch. We have been expanding the capabilities that generally been out of reach of anyone who could not afford a traditional ECM system, but who can now use one to reduce costs, improve productivity, reduce long-term costs of development of content applications or prove compliance. This release tackles  records management capabilities, handling and archiving of emails, mobile access for the worker on the go, the latest and greatest implementation of CMIS, and new extranet collaboration capabilities. All of these are targeted at what we felt would be important factors in a lean economic environment. All are also available as open source to help reduce the cost of managing content in enterprises struggling to do more with fewer resources.

New records management capabilities are very important for us, because this is the platform with which we will be going to the US government to certify for DoD 5015.2, but also because it provides a level of control that any organization facing regulatory requirements will find useful, such as life cycle management, retention policies, review process and disclosure and transparency controls. Built upon the new Share and SURF platforms these records management capabilities are the basis for a new records management application that is planned for certification at the end of September. To support this, we have added a new interactive forms system based upon the Yahoo YUI Ajax library and now allows both types and aspects to be applied and used in Share, including new records or regulatory metadata. By basing these capabilities on Share, we get a lot of the benefits of Share, including in browser viewing without downloading the record, URL-addressability of records information, and collaborative capabilities such as commenting, tagging and discussions. A new import and export capability is designed to simplify archiving records sets and import them to separate records repositories if necessary. A new records life cycle management automatically handles the physical storage of records to offline or tertiary storage. This is also the first records management system designed to be queryable by the proposed OASIS CMIS standard.

New records capabilities are destined for DoD 5015.2 certification

Related to and required by records management is a new ability to manage and archive emails using the IMAP email protocol. Virtually any email client can access, archive and categorize documents, records, attachments and other content with no plug-in required, because the Alfresco repository supports the IMAP protocol natively just as it does CIFS, WebDAV and NFS. This email integration is designed for two purposes. The first is for archiving and managing email, especially records. This interface allows you to manage email according to your organizational policies. If the policy is to archive everything and figure out organization later, the Alfresco rules can accept all content and can invoke rules to help organize, classify and apply the appropriate retention policies. However, if it is important for users to help classify the email as records, then users can drag and drop emails into the appropriate repository folders within their standard email client.

Manage and archive emails as records or access Alfresco from your email client

The second purpose for IMAP integration is to allow users who live in email to be able to access content from the repository without leaving email. Most browsers are IMAP capable, such as my Mac Mail client, and we have done extensive testing on Microsoft Outlook. Metadata and context of content is presented to the user with Freemarker script templates, which are very easy to configure. This metadata appears as email text in folders from the repository and the actual content appears to be an attachment to the email. This makes it easy to forward or send documents as either simple attachments or who content with metadata. It is also easy to use Alfresco from devices designed specifically for email such as the Blackberry, iPhone, Palm Pre or other mobile mail devices.

Every recession seems to create a step change in technology usage and this recession is probably no different. Smart phones now outnumber the number of laptops sold and for many tasks they can be just as effective. That is why we felt that content will be increasing consumed, processed and created on these small devices. However, the smaller form factor means that you can’t just take a big app and make it smaller. With Alfresco 3.2, we have looked at the tasks that people perform today and what content management tasks they could perform on mobile devices. The result is a version of Alfresco Share designed to fit the form-factor of these new smart phones, starting first with the iPhone. According to StatCounter, between the iPhone and iTouch, Apple has approximately 37% of the mobile browser market. It is also the first ECM application designed for business processes on the go. We have focused not only on browsing and access of content, but also on the business processes by allowing users to start and track workflows and activities around content collaboration.

Alfresco Share is now designed for use on the go with the Apple iPhone

This release also has the most complete implementation of the latest release of CMIS, version 0.61 of the OASIS CMIS Technical Committee, of which David Caruana and I are members. We have implemented both the REST-based Atom Pub and SOAP Web Services protocols. Dave has spent a lot of time on these capabilities and it is the future of our API. A lot of the work has gone into the new query language that provides SQL-like query capabilities along with other capabilities that had previously required using Lucene. Alfresco’s implementation of CMIS has been the basis of integrations with Joomla, Drupal, Atlassian’s Confluence, CMIS Spaces and CMIS Explorer and we expect more in the future. Dave and Gabriele Columbro are planning on contributing some our experience to the  Apache Chemistry project. Dave is hoping to contribute our CMIS client test harness, which may be used against any CMIS and currently contains over 100 tests covering all aspects of the spec including schema validation.

Some of the CMIS integrations to Alfresco include Joomla, Drupal and Confluence

Following release of Share last year and updates earlier this year, we have added a number of capabilities to support the use of Share in an extranet in order to provide content collaboration outside the enterprise firewalls or in the cloud. We have been testing scaling Share to tens of thousands of concurrent users. Since extranet use cases are more people-oriented, we have extended the contextual information available about users in the users profiles. We have also simplified the administration of users, groups and sites from a new administration framework integrated into Share. Share can also take advantage of some of the advanced metadata management capabilities of the Alfresco repository with the new forms system mentioned earlier and explicit support for types and aspects in Share. Some of the new user interface components available include Content Favorites and a new Image Gallery. This release of share is Cloud-ready for EC2 and other cloud services.

Alfresco’s Web Content Management Platform has been improved to support larger authoring and deployment environments better. A new parallel deployment engine uses multi-threaded updates to web farms for higher performance updates of web sites. A new web clustering architecture allows authoring installations to scale to more users and allows the deployed servers to also be clustered for shared services. A change to the architecture of previewing means that changes to either content or even code can be instantaneously previewed in a test environment. Improved rendering and transformation of web content provides better support for XML includes, XSL transforms and execution of Freemarker and Web Scripts in the web tier. A new pluggable deployment architecture allows you to deploy to multiple delivery environments such as to file systems, other Alfresco servers or external and web-edge delivery channels.

New WCM capabilities include better scaling of authoring environment and new parallel deployment to web farms

Sorry for the long post, but there is a lot that we have been working on while the Great Global Recession has been raging on. Our goal has been to help you cut costs of traditional ECM, improve your productivity through mobility and efficiency in handling content, address new compliance issues with the first open source records management system, help tame the great email beast, and engage your customers, partners and employees with new WCM and extranet capabilities. We also hope to future-proof your content applications by delivering the first and best with CMIS implementations as they appear from the OASIS technical committee. Last year, we showed you how we can save costs by providing apples to apples comparisons between ECM vendors using the US GSA Schedule 70 pricing. Now we hope to help you beat even those saving with new capabilities that we are releasing with the Alfresco Community 3.2 download. We hope you give it a try at http://wiki.alfresco.com/wiki/Download_Community_Edition

Follow me on Twitter @johnnewton

Alfresco Labs 3 Special Inaugural Release

 

Photograph: Win McNamee/Getty Images

"It has been the risk-takers, the doers, the makers of things -- some celebrated, but more often men and women obscure in their labor -- who have carried us up the long, rugged path towards prosperity and freedom." - President Barack Obama, January 20, 2008

How do you put out a release on one of the biggest days in history? The answer is you don’t, you wait until the day after and create a new beginning.

I’m pleased to announce that the final release of the open source Alfresco Labs 3.0. Alfresco Labs 3 has been our most important version of Alfresco yet. Combining new technologies, new techniques, new standards and new levels of ease of use, we have been fulfilling a lot of the vision that we had when we started Alfresco four years ago. This release combines the stabilization work that we have been working on in the enterprise release with new innovations specifically for the open source community.

The Alfresco Labs 3 releases that started in summer of last year have been aimed at expanding our collaboration and social computing initiatives and to providing an open source alternative to Microsoft SharePoint. In doing so, we have introduced a number of new capabilities that have not been seen in either commercial or open source systems:

• The first implementation of the new CMIS specification that is now in the standardization process with OASIS and promises to become the SQL of content management. Both the REST-based Atom Publishing Protocol and SOAP Web Services are provided.
• The first ECM implementation of the Microsoft SharePoint protocol after publication of the protocol by Microsoft in April 2008.
• New SURF, REST-enabled web runtime to provide an AJAX-enabled set of content management and collaboration components that utilize the rich Yahoo YUI AJAX libraries.
• Alfresco Share, which is a new collaboration application built using our WCM technology and provides on-demand collaboration sites integrated with SURF collaboration and Microsoft Office integration through the SharePoint protocol.
• New SURF collaboration components including wikis, blogs, forums, calendars, discussions, and social tagging.
• A new document management experience with the SURF-based document library that includes thumbnail and preview generation to avoid long downloads and eliminates the need to have the application locally with a Flash-based preview.
• Continued innovation of our WCM platform introducing features from the most recent enterprise release including web farm deployment, virtualization and reuse of assets between web sites. We have also converted much of the WCM functionality into Web Scripts that are accessible from you SURF applications.
• Multi-tenant capability to create virtual instances of Alfresco from a single machine.
• Managing in-bound email to collaboration sites for email-based collaboration.
• Integration of native PHP interpretation with the Caucho Quercus interpreter for integration of PHP applications and development of Web Script and SURF components in PHP.
• The new Web Studio drag and drop web site development tool designed to develop SURF sites and applications.

Perhaps most important is the fact that the enterprise and open source lines were merged to provide bug fixes found by our customers or through the certification process on a multitude of platforms. Since the majority of our enterprise customers start using Alfresco as a result of using the open source version first, it is in our interests to have a viable, robust open source implementation of Alfresco. We also believe that ultimately the primary reason people are buying the enterprise subscription is for support, warranty and indemnification and the fact that Alfresco is open source is what draws them in the first place. We therefore recommend that the community upgrade to this version of Alfresco as soon as possible.

There are still new features being introduced for the first time as part of this release. Web Studio is new separate application using the new SURF web framework to provide a drag and drop design experience to build SURF web sites and applications. Web Studio allows you to work with and view the web site you are developing, but uses the Yahoo YUI AJAX libraries. Web Studio is also designed to develop and edit web sites safely by using the new WCM REST-based Web Scripts and is fully interoperable with the Alfresco WCM application.

With the economy in the dumps, this is a particularly propitious time to release the stable open source version of Alfresco 3.0. Open source should thrive in this environment. Where companies and IT organizations may not even take a call from a traditional enterprise salesperson, they will download open source software. We want to be able to build market share in the ECM market with open source that is robust and thriving.

Brainstorming at Fortune Brainstorm in Half Moon Bay, CA

Groucho Marx once said, "I would not join any club that would have someone like me for a member." That was my original concern about attending the Fortune Brainstorm, a three day technology conference in Half Moon Bay California. I met one of the conference organizers, David Kirkpatrick at a couple of the World Economic Forum events and he is a very bright and insightful guy. But I thought it might be another one of these conferences where you spend a lot of money just to attend. Our marketing team convinced me otherwise.

Brainstorm last week was a very connected event with some very influential people. I recognized at least a quarter of them from Davos. The bloggers were also there in force with Robert Scoble, Om Malik and Kara Swisher participating as the first evenings entertainment. The subject matter was also generally more relevant that Davos as well given its proximity to Silicon Valley. Given how far I came, I used part of the time on Monday to do some interviews in San Francisco on our Alfresco Labs 3 launch, so I missed the opening sessions with Jeff Bezos of Amazon, Michael Dell, Mark Benioff from Salesforce and Brad Smith from Intuit. The setting for most of this was somewhat intimate with main room laid out like a giant conference room with lots of Herman Miller executive chairs. (Herman Miller was a sponsor.)

The environment was set up like an office. So everyone ends up doing work.

Although this was a tech conference, some of the focus was on how relevant tech is. The level of discussion was what you would expect in an article in Fortune magazine. That level of discussion is good to understand how the big trends are moving to affect everyone's lives, not just those in tech. The first breakfast session that I attended was on Cloud Computing with Marissa Mayer from Google, Adam Selipsky from Amazon, Kevin Lynch from Adobe, and Zach Nelson from NetSuite. Cloud Computing is increasingly attractive from a cost perspective, but enterprise customers are still worried about security and reliability. Google does not allow encryption and Amazon does not guarantee recovery and recommends that you back up your data on S3. So I asked the panel when we can expect security AND reliability. Marissa didn't answer and the other panelists answered the part of the question that they wanted to.

Cloud Computing: Michael Copeland (Fortune), Charles Fitzgerald (Pi/EMC), Zach Nelson (NetSuite), Kevin Lynch (Adobe), Adam Selipski (Amazon Web Services), Marissa Mayer (Google)

Mark Anderson hosted a session on the direction of technology with several CTO and technology leaders including Sophie Vanderbroek from Xerox, Padma Warrior of Cisco, and Bob Iannucci of Nokia. The discussion range as far as their technologies. Xerox is looking at collaboration and sees the future in intelligently understanding the content being handled in that collaboration in something called content-centric networking. This is something that Alfresco can get its head around. Cisco sees the internet morphing from a primarily messaging based platform to more of an entertainment platform. Video being a more natural form of communication will become more pervasive and that the real requirement will be more filtering rather than generating. Nokia took us on a cosmological tour of technology trying to show us that the bigger issues are around mobility and the data that affects our lives. In addition, data privacy will become one of the biggest issues to tackle.

CTO Forum: Mark Anderson (Strategic News Service), Sophie Vanderbroeken (Xerox), Padmasree Warrior (Cisco), Bob Iannucci (Nokia)

Vinton Cerf, inventor of the Internet and now at Google, kept sitting in front of me and always asking questions. It made it difficult to get a question in myself since the moderators would always move to another part of the audience. Get your own seat Vint!

Nicholas Negroponte showed off for the very first time a dual-boot One Laptop Per Child XO machine (the famous $100 laptop) that is now configured to run both the OLPC operating system and Microsoft Windows.

David Kirkpatrick and Nicholas Negroponte showing of the latest dual boot OLPC XO

In "How Green is Your (Silicon) Valley?", VJ Joshi from HP, Rob Lloyd from Cisco,  Jonathan Schwartz from Sun, and Michael Spliter from Applied Materials discussed the role that Silicon Valley can play in creating green solutions. By and large, as you would expect, IT makes things more efficient and eliminates the need for travel and face to face meetings. VJ kept it relatively small with the practical steps that HP is taking like making two-sided printing the default. Jonathan kept it big getting us to think about what will happen when IT services as a utility will be 10 times bigger. Jonathan Schwartz talked about MySQL's approach of no office buildings at all. But, I found Rob's and Cisco's point more insightful. When we start architecting our buildings (and homes) with information systems and controls in place, that is when we will really start to reduce our greenhouse footprint. Meeting rooms, transportation and environmental control facilitated by IT (and of course internet routers) will allow us to reduce transportation of ourselves and our resources.

Greening the Valley: James Manyika (Fortune and hidden by the Fortune sign), VJ Joshi (HP), Rob Lloyd (Cisco), Michael Spliter (Applied Materials), Jonathan Schwartz (Sun)

David Kirkpatrick is writing a book on Facebook called The Facebook Effect and used this position to interview new COO Sheryl Sandberg. She puts on a much more media savvy front than Mark Zuckerberg, but not quite as much out-of-the-garage appeal. It all sounded a little too glib. Apparently she used to work in the White House at some point and Google. I was surprised to learn that co-founder Matt Cohler had left Facebook, although he was attending the conference. She announced that he was becoming a partner at Benchmark Capital. I bet Peter Fenton had something to do with that. I had interacted with Matt in relation to some work that we were doing with the World Economic Forum and it seemed that at least part of his role at Facebook was now taken up by Sheryl.

David Kirkpatrick and new Facebook COO Sheryl Sandberg

At this point, my camera ran out of battery :-( I participated in a Lunch Lab on Healthcare facilitated by Marissa Mayer of Google and Zoe Baird of the Markle Foundation, but they did most of the talking. There were a few experts who got a word in edgewise, but I didn't really feel that I could participate despite having working with pharmaceutical companies on information management and classification for the last 15 years.

In World without Exits, Andrew Braccia (Accel), Danny Rimer (Index), Dana Settle (Greylock), David Siminoff (Venrock) and Quincy Smith (CBS Interactive) all talked about how there are no exits aside from acquisitions. Danny, one of the best investors in Europe and backer of MySQL and Skype, said that there are still acquirers for good companies. He said all of his portfolio that had been acquired since the beginning of the year had been enterprise software companies. Andrew said that he is telling his portfolio companies to hunker down for the next couple of years. I have heard such talk since 1994 before the big launch of Netscapes IPO.

Life on the Net 2018 was pretty interesting, although probably a little too diverse like several of the sessions at the conference. We had Larry Lessig - open source intellectual property specialist, Phil Rosedale - founder of Second Life and Half Moon Bay resident, Joichi Ito - chairman of SixApart. Larry's "the sky's falling and your privacy is gone" position was pretty scary. He said that the Patriot Act came in so fast that it must have been pre-written before 9/11. He said that he had sources that have told him that is exactly what happened and that there is an internet-based Patriot Act waiting to be put in place as a result of any sort of catastrophic disaster on the Net. Phil believes that virtual worlds of some sort will be the future of user interfaces to the internet because they are easier for older and non-experienced users to use. Obviously, we need more technology improvements for this to happen. Perhaps the biggest technical obstacle to be overcome is the notion of a single identity that is secure and non-repudiable.

The evening event started with a Tweet Out in the platform overlooking the Pacific Ocean. I now have more respect and interest in Twitter as a result of meeting so many Tweeters. Dinner was hosted by the government of Singapore and I met so many people that evening I can't recount it all. However, I didn't stay up so late that I didn't get a chance to get up early and take some pictures of the harbor I was staying next to. I waited too late to stay at the Ritz-Carlton in Half Moon Bay, but sometimes that can be an advantage, especially cost wise.

   

An early morning stroll around the harbor at Half Moon Bay.

I was up early enough to sit in on a breakfast roundtable on Programming for the Web and Social Networking. Now I thought this meant programming like programming a computer. They meant programming like NBC is setting up it's Fall line-up. Still interesting. I am getting a sense that there is still a new big wave of video and much richer content coming than just YouTube. And the opportunity to manage it, organize it, categorize it and monetize it will be huge.

Finally, I sat in on a real programming session with the Future of Code, featuring David Hansson of BaseCamp and Ruby on Rails fame, Grady Booch - early pioneer of Object Oriented Programming and at IBM, and Charles Simonyi of Intentional Software, creator of Microsoft Office and Space Tourist. They all take very different approaches to architecture and they spent a lot of the time telling each other that the others were wrong. They all have points and my biggest insight from this is what prejudiced guys they all are.

Neville Roy Singham (ThoughtWorks and moderator), David Hansson (37signals), Roger Simonyi (Intentional), Grady Booch (IBM)

Grady Booch: "There are systems built upon systems and then you have a lot of crap. But that's enough about the Bush Administration."

I missed the rest because I had to head off to Sun Computers, but you can read more below about Eric Schmidt and Neil Young. We had gone to meet with Jonathan Schwartz who had left the conference the day before. He said that although there were interesting things there, he does go to a lot of those things. I asked him if they were all the same. He nodded. Lucky guy. Living 5000 miles away from the valley, I don't get to go to that many events of this quality.


Philip Rosedale Doesn’t See Browser-Based Virtual Worlds As A Threat to Second Life.  Is He In Denial? - Erick Schonfeld, TechCrunch

A glimpse into the future of the Internet - Tom Forenski, ZDNet Blogs

Facebook's Sheryl Sandberg: Making money isn't a priority (except for her) - Nicholas Carlson, Valleywag

Liveblogging Eric Schmidt/Google Interview at Brainstorm - Erick Schonfeld, TechCrunch

The best Fortune Brainstorm Tech Talk: Neil Young challenges tech industry - Robert Scoble, Scobleizer

The blog editing system in action - Robert Scoble, Scobleizer

Introducing Alfresco Labs 3

I am pleased to announce the beta release of Alfresco Labs 3, our new version of Alfresco targeted at collaborative content management and the first open source alternative to Microsoft SharePoint. Alfresco Labs 3 is a natural evolution of the Alfresco platform that started with the first preview release of Alfresco three years ago. As Alfresco has grown more powerful in capabilities, we have strived to simplify both usage and development of Alfresco web clients, web sites and web applications. This release has leveraged the Web Script architecture that we introduced last year and provides simple user interface components allowing knowledge workers to collaborate on and share content. You can find out more here and download this latest version here.

New Alfresco Share application previewed in Alfresco Labs 3

We believe that the Enterprise Content Management market is evolving from specialized software for information intensive activities to become a major component of business and infrastructure for knowledge workers. Although over 90% of the Fortune 1000 have at least one type of ECM system, less than 10% of the employees in most of these companies use ECM, despite the huge increase in regulatory compliance and information explosion. Microsoft SharePoint, through its hooks to Microsoft Office, has made significant inroads into this yawning gap in the ECM market, and growing at 35% per annum as a result. We believe that open source has an even greater opportunity to reach into this portion of the market through its ubiquitous internet reach. With Alfresco Labs 3, our opportunity is to provide an alternative to, provide interoperability with, and be complementary to Microsoft SharePoint.

New Alfresco Surf RESTful platform uses Yahoo's YUI AJAX library for a more interactive experience. New thumbnail service provides previews before opening.

Much of the development of Alfresco Labs 3 has focused on building the Alfresco Surf platform, a robust, enterprise-class web application and site assembly framework that bundles a full site construction object model and toolkit for building class-leading web and collaborative applications. The Surf platform is designed to work in a number of different web environments, including as a Web Part in Microsoft SharePoint Portal. It includes content oriented components designed around the YUI (Yahoo! User Interface) AJAX Library and Adobe Flex for dynamic uploads and previewing of content and other information. The new user interface components make it much simpler for users to develop new collaborative web applications.

New Activities, Tagging and People services provide a social collaboration experience for the enterprise. Collaborative sites also have wikis, blogs, calendars and discussion forums.

With Alfresco Labs 3, we have also introduced a few important concepts as New Content Services to support collaborative applications and social computing including Sites, Thumbnails, Tagging and Activities. The Surf platform uses the Sites to construct a collaborative site and store framework information on how pages and components are constructed. The Thumbnail service generates previews of documents and content to show what you will open before you open it, enhancing the more limited view of an icon that only tells you what time you have. The Tagging service provides a simple folksonomy view of content as well as providing tools for constructing tag clouds. Activities are similar to Feed inside of Facebook, providing a stream of information on what activities people are working or commenting on. To support these services, the People services have been enhanced with more information and the ability to connect people to one another for social networking. These services are accessible through REST-based APIs and from JavaScript allowing their use in construction of new Surf components.

Integration with Flex and YUI provides simultaneous loading of files.

Alfresco Share, which we are previewing with Alfresco Labs 3, is built on the Alfresco Surf platform and is designed to be modified by end users or through programming to fit a wide range of collaborative and social computing applications. Share provides a simple web site paradigm for creating collaborative applications by aggregating Surf components and incorporating new Surf components as they are developed. This is an early release of the Share application, but we encourage our open source community to develop new components based upon the examples of the basic components provided. With Share, users can create a collaborative site either inside or outside the organization, invite users and share and collaborate around content. Share includes document libraries, calendars, wikis, blogs, and discussions.

Microsoft Office can access content in Alfresco as though it were a Microsoft SharePoint Server

Also previewed in Labs 3 is Microsoft SharePoint compatibility from within Microsoft Office that makes Office applications think they are talking to a SharePoint server. Microsoft released the SharePoint protocols this spring as part of its compliance with the European Commissions’ decision issued on March 24, 2004 along with an increasing tendency by Microsoft toward interoperability. Alfresco is the first ECM system to implement the Microsoft Office and Windows SharePoint Services protocols as a compatible server. This allows Office users to browse and find documents within the repository, checkout / checkin / version documents, share the documents in shared workspaces and access the additional menus and task panes reserved for SharePoint. All of this is available with no additional software needed to be added to Office.

The entire team is getting way too big to congratulate individually, but everyone has done a great job so far getting this beta to realization. For this, I am extremely grateful. We look forward to providing the next update in early September where the following will be provided:

• New REST APIs based upon ATOM Publishing
• New look and feel for Alfresco Share
• Additional collaboration components, including instant messaging
• A new properties framework for extended metadata
• New people and site search capabilities built around social networking
• Web views support for Microsoft Office SharePoint emulation
• Associated technical documentation

Our current plan is that the Enterprise version of Alfresco 3 will be available in mid-October.

We think that Alfresco 3 will provide our users and customers choice in developing collaborative application and in supporting basic content management and collaborative sharing for Microsoft Office users. Rather than a single fixed platform, Alfresco offers a choice of operating system, database, application server, development environment, and web browser, including those provided by Microsoft. In addition, through our work with Unisys, we have demonstrated a platform that is much more scalable. By providing this new platform as open source, we hope that you will also help us in making it a better platform and contribute new collaborative capabilities and components that we could not even have conceived.

Adobe embeds Alfresco Repository

It's been quite a while in the making, but I am very pleased with the news today that Adobe will be embedding Alfresco technology as part of its LiveCycle Suite. A while ago, I wrote a blog about embeddable content repositories. It was clear then and more clear now that the old generation of content repositories is not really designed to be embedded as part of content-oriented applications. Yet, we all know that there is more information in content than there is in databases. Why can't applications use a set of services for managing content the way they manage data in embedded databases?

On this particular news, ComputerWorld reports Raja Hammond, Group Manager for Adobe LiveCycle, as saying, "Alfresco has a fantastic lightweight installation. It is J2EE server-based, so it is very much aligned with our architecture. We're able with this release to totally embed it. We've done extensive customization to the UIs to add additional capabilities to them. We've integrated them tightly with the various solution components within LiveCycle."

At InfoWorld, Brian Wick, Director of Product Marketing at Adobe said, "It's much easier, much quicker for our customers to build LiveCycle apps with the content services piece built in." This should be the sentiment of any product manager whose product handles content. This clearly the case of LiveCycle which handles potentially huge numbers of PDFs and forms.

Over at CMS Watch, Alan Pelz-Sharpe, a long-time ECM observer, blogged on the announcement that, "It's been a while since there was a big product announcement in the ECM world, but today's announcement by Adobe that they will be embedding Alfresco into their LiveCycle Enterprise Suite will doubtless garner a few headlines. Alfresco, the UK-based open source ECM company, has certainly done a great job of marketing themselves since their launch a couple of years back, stealing some limelight from more established and much bigger vendors such as Interwoven, Vignette, and OpenText. The question we have to ask is whether this announcement is another marketing   triumph, or whether it suggests something more substantial. First off is the fact that it is a real OEM (Original Equipment Manufacturer) deal, and the technology will actually get embedded into the Adobe offering, so it is more than simply a paper partnership."

It is also significant that the Alfresco platform is open source. Open source allowed Adobe and our dozens of other OEMs to try out Alfresco before even approaching us. Open source also provides a level of comfort and confidence in a platform for services like content services and content repository. It is much better than providing code in escrow. it actually provides a community as well to ensure the long-term success of the platform.

We look forward to a fruitful and simbiotic relationship with Adobe. We believe that this is the beginning of looking at content management as a peer of database management of an essential component of any enterprise-class application. Congratulations to Adobe on all the hard work and the new release.

MySQL Acquisition and Enterprise Software

In a software industry that had little innovation and created obstacles for the next class of rising companies, open source is turning enterprise software on its head. Xen Source, Zimbra and JBoss are now part of larger companies acquiring new technologies and new distribution models by leveraging the power of open source. Now we see that MySQL has been acquired by Sun for $1 billion. Sun has been embracing open source more and more under Jonathan Schwartz's watch as CEO and this can be seen as a logical next step in that strategy.

Marten Mickos, a happy man and a really nice guy.

When we started Alfresco, we came in with the assumption that one of the only things that is working in enterprise software is open source. The past year or so have proven this prediction right. Although it wasn't really my prediction. A meeting with Marten Mickos, CEO of MySQL in 2002, helped me understand that, yes, open source really could work. Up to that point, I was of the same opinion as Bill Gates, that open source is equivalent to communism. MySQL helped me understand the power of huge numbers of people using software and the value that support can provide to fund the development of professional software. The fact that the model works means that small open source companies can thrive in an environment of behemoths consolidating stacks and actually create an environment of innovation.

David Axmark and Monty Widenius, founders of MySQL

When a category has been around long enough that customers know what they want, then open source works really well. MySQL provided a simple, cost-effective database system that meant that you didn't have to install a big, hulking Oracle, DB2 or SQL Server and more importantly, you didn't even have to pay for it. You just pay for support. JBoss did the same thing for app servers, Xen Source for virtualization systems and Zimbra for email. Some people question whether MySQL was really innovating, after all the set of SQL is the same as Oracle had in the early 90's. In reality, there wouldn't be a Web 2.0 or possibly even a Web 1.0 without MySQL. MySQL pioneered the model of Scale Out rather than Scale Up to provide web properties like Facebook, Google, Yahoo, etc. to scale to levels that were unthinkable in Oracle back in the 90s. JBoss, Xen and Zimbra were doing the same to their respective industries and bigger companies were willing to pay for that.

From our perspective at Alfresco, Sun is a great company to acquire MySQL. Sun has proven their alliance and cooperation with open source. And this doesn't change our plans to become a public company. We have created public companies in the past and we intend to in the future. Our sales of support and development of our community have exceeded our expectations and events like this make us even more determined that IPO can be successful for the development of the Alfresco system and the Alfresco community.

Congratulations to Marten and team and good luck in the future. We are looking forward to more successful collaborations and joint deployments of Alfresco and MySQL.

Scaling Out Like Technorati

My fellow World Economic Forum Technology Pioneer, David Sifry, the founder of Technorati, was also in Dalian, China for the “Meeting of New Champions” or “Summer Davos” as the Chinese like to call it. During Davos in January, I had the great misfortune of pitching Alfresco against Technorati in a competition between tech pioneer companies. As fantastically well as Alfresco is doing, Technorati has the temerity to compete against Google in blog search and win.

I got the chance to talk to Dave during the conference and ask him some questions on the technology and architecture behind Technorati, the internet blog search site. I thought that someone who could take ordinary computer components and build a huge internet architecture could possibly teach something to people running enterprise architectures that are puny in comparison.

Technorati is a web site that tracks blogs, pictures and any user generated content and allows you to search those sites about what people are thinking, seeing and hearing. When a new or urgent situation breaks out, you can do worse than to search Technorati for immediate reaction. Every day, every hour, every second, Technorati is indexing over 10 million blogs with over 10 billion objects. Technorati’s user base is doubling every six months and quick and accurate response is critical for retaining those users.

David Sifry, Founder and Chairman of Technorati

I asked Dave about his architecture and what applicability their might be for enterprise architectures.

John Newton: In building Technorati, what were some of the issues that you had in architecting your systems.

David Sifry: I was looking at just temporal information. I had no idea how big it could get. When I looked at the architecture, instead of architecting it right, I architected it for right now. I had no big budget and I didn’t want to wait six months to build it. Also, I had no idea what the killer app would be.

I focused on data flexibility. At the time, that meant putting everything into a relational database. That was okay while the size of the indexes is less than RAM and about a million blocks of data. That was okay while there were less than 20 million blogs.

The next generation took advantage of data parallelism. That meant upon update send a signal to all the other systems. We expanded the data over several “shards” [segments of data partitioned on different databases on separate machines].

What was surprising was that we were writing as much data as we were reading. At this point Technorati was as big as some of the biggest OLTP. Even so, maintaining data integrity was important, because you would want the link count [count of how many other blogs point to a particular URL] to be out of sync. This put real pressure on the system. At the same time, we realized that time was more important dimension than URL. People didn’t want to sort or search on URL, they wanted to search on time. [i.e. what are the latest blogs on a particular subject?]

By this point, we understood the application more and more. The app [Technorati] is about real time access. You need to be able to count on finding latest information on a subject. That’s when we built the third architecture. Scaling was well understood and we build the shards on time rather than on URLs. Instead of putting data into a DBMS, we put it into special purpose databases. It was more of a bus-based architecture. Each database could be scalable and grow as big as we needed.

JN: The notion of shards - did you call it that at the time? I have been looking into shards and I was only aware of or heard of them for about the last year.

DS: Back in 2002 when we were pitching this to VCs, I at least explained the theory. All I just thought through the problem carefully. Doing it this way, we could add hundreds of systems, lots of cheap CPUs, RAM and disks. It provides inherent parallelism. I can’t believe that I was the first one to think this up.

JN: How big does this architecture scale?

DS: We are loading one terabyte a day into Technorati. That’s 100 million blogs or about 10 billion objects. A lot of is new types of tagged data. There are about a half billion videos and photos.

With all that data, you have to think about what do you throw away?  We can’t really delete anything, because we are potentially losing an asset. We don’t delete anything. So we take data out of the spin cycle. [Transitory data used in preparation.] We take the long-term data and put it into low latency storage.

When data is doubling in size every six months, that means that only one quarter is a year old. We don’t need to worry old data.

JN: How do you deal with large number of users with very large data sets?

DS: Any off the shelf tools falls over. There is a lot of interesting analysis on old data, but no off the shelf tools can handle that much data. It’s only just now that some tools can handle it.

JN: What are those tools?

DS: One is Green Plum by a bunch of O’Reilly guys. If you use ordinary data warehouse tools, they would just scream and shout.

JN: Actually what I was originally referring to was the fact that you are showing lots of data that are not users used to enterprise information management tools. How do you present this information to consumer-level users? How do you deal with the user interface and visualization of all this data?

DS: Gotcha. It depends on what the user wants to get out of Technorati. If the user wants search results, then we give it to them. Sometimes they want to browse or discover information. We have spent a lot of time on visual design. Then we give them lots of bright, shiny things for them to click on.  Things like metadata, video or other links.

We have used enterprise class web tools to analyze what users are doing? We look at the click stream and see what is successful or not. That helps to make the information contextual.

One of the big mistakes that we made is to not do this [buy click stream analysis tools] sooner. It was only $80K. Up to that point it was so much trial and error. I’m glad we finally did it. Now we can see how much time a user spends on a feature. We can see page views, goals per visitor.

JN: So what do you measure on Technorati?

DS: Measuring a web site is like forecasting the weather. Yesterday it’s sunny and today it is cloudy. Why is it cloudy?  Sometimes you have no idea. Sometimes you realize that that a change in barometric pressure has a lot to do with it.

We look at the number of newbies, number of reports, session lengths and then measure them against prior periods. It’s not always consistent.

I had never built a B2C site before. I just focused on me, on what I wanted. That worked well for a while when I was the target audience. But we have to build for a broader audience.

JN: At Alfresco, we measure conversions. Are you measuring things like performance? Does that affect retention of users?

DS: Of course, but if the system is falling down, then even performance doesn’t matter. So I don’t get too stressed out about it.

JN: When we met at Davos you wanted to move Technorati to be the Internet Now! Is that still the case?

DS: Everything is shifting. I wanted it to be a site that everyone is able to use. We forgot about the core users that just wanted to find out about blogs and any real time information. In an attempt to jump the chasm, we chased after 100 million users and tried to be everything to everyone. Now we try to make blogs and user driven content available for those looking for that.

Also performance is improved significantly. Now I notice how slow other sites are. This is a total tribute to the engineering team. Everything is easier and faster.

Pretty soon we will have a whole lot of stuff that we have been working for a year.

JN: Can you say what it is?

DS: I don’t pre-announce.

JN: What does the Technorati brand stand for today?

DS: Good question. What’s popping up now on the internet, especially user generated content? It’s about users tagging user generated content and finding it.

JN: Who are your competitors?

DS: I probably sound like the typical entrepreneur, but nobody really seriously. Google provides blog search, but other than that nobody really. Other people are trying to identify and tag information like Digg and del.icio.us, but they aren’t really competition.

JN: What do you want Technorati to be in two years time? Five years would be ridiculous.

DS: I would like Technorati to be a profitable business that is strongly differentiated. It will be the place that you would go for mobile, RSS or push information. For all that you would come to Technorati.

Open Source and Business Pleasure vs. Business Pain

A European PR firm was pitching my company for business last week and putting out a few ideas on how to generate demand in different countries across Europe. One of the ideas that they presented was a “business pain barometer” to indicate how much pain companies might be feeling using existing enterprise systems. This didn’t exactly resonate as a value proposition for open source, but it is a tried and true campaign strategy for traditional enterprise systems. Selling pain relief has worked for the last three decades to sell enterprise software, but has it run its course?

Read the rest of this entry »

When Collaboration is an Emergency

Tsunami, Earthquake, Hurricane, Flood - everyone’s nightmare disaster can also create the biggest challenges in collaboration and employing information technology. The same Communications of the ACM that had the 7 Habits article had a whole series of articles on Emergency Management Systems.  Surprisingly, the techniques that are required to cope with the flood of information in case of disaster don’t seem all that different from those required by business today. The Indonesian Tsunami and Hurricane Katrina taught the whole planet about the need to invest in preparedness and reaction capabilities regardless of how poor or how rich we are.

One article I found particularly interesting was The Human and Computer as a Team in Emergency Management Information Systems. In fact, nothing in this article seemed to be limited what is required in a disaster, but what is necessary for coping with daily business pressure and information overload. The primary process of coping with a disaster is Build the Picture, Understand the Picture, and Change the Picture in a Goal-Oriented Fashion. Sounds like good business strategy to me.

The article talks about the people who are involved in the command, control, and analysis for emergencies that are built on trust of others who also are working in 14-24 hour shifts knowing that mistakes can cost lives and immediate action is essential. As if describing the persona in a use case, these workers:

• Feel they are exercising control
• Have total focus on the problem at hand and ignore all that is not relevant
• Improvise with unconventional solutions to appraise information and formulate decisions
• Enjoy the challenge and curiosity of the effort
• Are highly motivated due to the critical nature of the problem

This sounds like a typical Silicon Valley start-up or anyone else in a highly competitive field where people enjoy what they are doing. The design of the emergency response system then creates challenges that are not a typical of other collaborative systems in a highly reactive environment:

• Obtain accurate and timely perceptions of reality through communication structures that track and facilitate open exchange of information
• Enhance focus without interruption and require minimum effort to carry out a task
• Encourage creativity and improvisation of both the individual and the team

In disaster management systems under development, the emergency manager has the following tools at hand:

• Information prioritization - rules to prioritize situational information defined by context
• Decision support and modeling tools - impact analysis and support for decision execution
• Representation of a common operating picture - visualization of what is happening and where resources are open to everyone

Wouldn’t it be great to have a system like this in any business? It requires a good understanding though of the participation of the people and computers. What is each good at and what is each bad at? People are good at:

• Perceiving patterns
• Improvising flexible procedures
• Exercising judgment
• Inductive reasoning
• Detecting small changes by sight and sound
• Storing large amounts of information for long periods of time and recalling facts at the right time

Machines are good at:

• Responding quickly to control signals
• Applying great force smoothly and precisely (like landing a 747)
• Repetitive, routine tasks such as monitoring
• Handling highly complex operations and multi-tasking
• Deductive reasoning and computation
• Storing information briefly

In other articles, there were extending these systems to use community participation using open source and mashup to collect information not just from officials but the public at large. Those that were prepared for the Tsunami were often ready because they were alerted by mobile phones. The internet can also play an important role in collecting intelligence. After all, the internet was originally designed to withstand thermo-nuclear war and breakdowns in individual communication links. Here are some examples of mashups for accessing and collecting information:

Automation and collaboration have a role in emergency management. Just as triage methods were invented in time of war and moved on to ordinary civil use, emergency systems can probably help teach us what is important in collaboration and process automation. The primary lesson that the Human and Computer as a Team article conveys is that we ignore the human role at our peril and that the computer supports people and helps build trust between people by increasing trust in the information that they are sharing.

Convergence of Content and Data Management?

Tony Byrne announced that he is hosting a panel on convergence between enterprise data and content management and poses it as a question - will structured and unstructured information management converge? My short answer is no, but that answer has a complicated reason behind it. Much of it has to do with the fact that the larger stack of enterprise software is consolidating around it. Here are some of Apoorv Durga's comments on convergence as well.

"Oh really?"

I have lived in both worlds having worked with relational databases since 1977, being one of the founding engineers at Ingres and then co-founding Documentum with Howard Shao. While at Documentum, we explored what content was and how it was different from databases. Over the years my early bigotry in favor of a purely relational view of the world has given way toward a more relaxed view of how content is structure, indexed and managed. While starting Alfresco, we had the opportunity to start from scratch but still used some of the concepts that have proven effective in capturing and delivering information to users.

The relationship between relational databases and content management is like nuclear physics and organic chemistry. Relational database provides the mechanics to make data and information happen and content management builds upon that. Relational databases provide the transaction controls to ensure data integrity, the back-up tools to make sure that information is recoverable, replication to move data from one location to another, and the query, data manipulation and relationship tools to handle much more complex structures. Content management is more like the organic chemistry of information, combining information and relating it to human beings to make it more usable and consumable. The structures, processes, and models of content are different from other classes of information management. However, just like organic chemistry, content management may combine with other classes of application just as relational databases have. We are just missing the standardization and theoretical foundations of content management that have supported relational constructs.

What makes content management different from data management is how close it is to people. To make content useful, the people who create the information need to understand how it will be used. Content needs to be compelling, original, concise and understandable. Content has context that only humans can provide and only humans can use. This means that the services around content are more about change than integrity. Integrity is important, but that’s why the database is there. There is a whole rich set of services there to deal with transformation, change process, classification, publishing, versioning, content to content relationships, links and a whole bunch of other things that databases just don’t "think" about. Search may be yet another system that has no relational database at all, but should use the concepts that have been built up by the content management system. That’s why content management systems are separate systems built upon relational databases and integrated with separate search systems.

Since the inception of content management, the content management vendors have by and large continued to support the notion of a repository sitting on top of a relational database and integrated with a separate search system. Interestingly, many of the main vendors of ECM are now the database management companies - IBM, Oracle, and Microsoft. This should not be surprising since content management is now one of the fastest growing segments of database applications. Even so, these companies have chosen to layer their content management software on top of their relational database systems. The database groups are then free to focus on data management as their core competency. Databases support not just content management, but transactional systems and analytical business intelligence systems. Internal to these companies, the database groups have not really subsumed the content groups. Microsoft flirted with the idea of combining everything into one server group, but unwound that decision to have Sharepoint in the Office group. IBM’s content group reported into the DB2 group, but remained independent and it remains to be seen where it ends up after the FileNet acquisition. Oracle’s content group has wandered all over the organization since Oracle first attempted to build content systems in the late 1980s.

The non-database vendors of content management - EMC, OpenText, Interwoven, Vignette and Alfresco - still use relational databases in the management of content and layer their services above a database. Interwoven tried to not use databases to improve performance in the early days and took a very XML-based approach to managing, categorizing and controlling content, but this ended up being a losing proposition to companies worried about integrity. EMC sees a future that is independent of all these stack war issues in that people will always need storage and that content management is really about managing storage. They are essentially above (or below) the stack wars, but don’t be surprised to see them try to architect the database out of the equation. OpenText, Interwoven and Vignette look to either get acquired or get out of the way. At Alfresco, we believe that open source is the open alternative to the stack wars, which I will speak about later. The motivation of each is not the convergence of content and data, but the consolidation of the ECM stack at one level and the entire enterprise software stack at another with fewer and fewer players.

From Kathy Sierra's blog

What is happening at the macro business layer is that entire application stacks are consolidating to manage the data of record. IBM, Oracle, Microsoft and SAP are all vying to own the data and make themselves as sticky as possible. Each has Service-Oriented Architecture to make it possible to surround that data and to integrate it with other stacks when necessary. Data in the case of content management is simply the data about the content and is not a whole lot different than customer data as far as these stacks are concerned. These stacks need the checklist of the big items that enterprise customers are buying in order to build or integrate applications. This includes relational database, content management, business intelligence, build and test environment, system administration, and all sorts of XML stuff. Most of these, with the exception of IBM, have gobbled up the top application layer including CRM and ERP. SAP flirted with the database layer in alliance with MySQL, but seems to have abandoned this strategy. It could be though content management may be a common stack component if SAP goes out and purchases an ECM vendor. Content has become an important part of the data being managed and these SOA stacks will just link it like any other data.

Despite the relentless consolidation of these stacks, sucking in the ECM market with it, total integration of all systems into a single stack is impossible. At best, these stacks are fighting for a bigger piece of the enterprise pie by displacing smaller players. Enterprises are trying to go from a choice of 25 different systems to 3, but not down to one. Microsoft building Sharepoint organically can exclude other databases other than SQL Server, but lose a chunk of the market in the process. Will IBM really limit FileNet to only DB2? Will Oracle lock Stellent only to its database? Well maybe, but a totally integrated stack does not solve all problems of enterprise process or control. Likewise, SOA has not delivered on the promise of interoperability, despite the billions of dollars spent by IBM, Microsoft and major enterprises. Nor does it move far outside of back-office systems and into the front-office systems and web sites where most of the value is presented to an enterprise’s customers. It does not deliver the conversation with its customers that enterprises are increasingly demanding.

From Kathy Sierra's blog

There is a lot happening out in the world of the Internet that is making this whole notion of data versus content irrelevant. Web 2.0 has moved the conversation from the whole notion of bits and bytes into what matters is the content, people and the relationships between people and content. Web 2.0 says that people don’t care about data and structure, but in communicating with each other and building closer relationships. This notion is seeping into the enterprise software space with the class of software known as Enterprise 2.0. It is still early days, but billions of dollars of value have already been built upon the foundations of Web 2.0 and those foundations are at least 90% open source.

Open source has provided an alternative view of the vertical stacks that are being created by IBM, Oracle, Microsoft and SAP. In this view, open source is the stack and dominated by no one vendor. Each layer of the stack can be substituted with a best of breed open source component. These layers have been constantly rising from the operating system to the database to the app server and now the application layers. These application layers look a lot different than the enterprise stacks though. Rather than integrating at the depths of the infrastructure in a structured SOA, they are “mashing up” near the user and making it much easier for more providers to create new services and applications not depending on any particular stack. In fact, the stack is irrelevant as long as it is freely available. How many people really know what is behind Amazon, Google, Yahoo or Saleforce.com? The answer is a lot of open source, but which open source doesn’t matter a bit to the end users of those systems. At Alfresco, we are one layer in that open source stack and the user is free to choose that component or any other in the open source stack.

I plan on attending this session and seeing what others think.