CMIS Face to Face at Microsoft in Redmond

Dave Caruana and I just finished participating the in the first Face to Face of the OASIS CMIS Technical Committee here in Redmond, WA. Companies participating in person were IBM, Oracle, EMC, Alfresco, Exalead, OpenText, SAP, Day, Nuxeo, Dennis Hamilton (representing himself), and of course, Microsoft. Gary Gershon, Julian Reshke from Greenbytes and Betsy Fanning from AIIM participated by phone. (Sorry if I missed anyone.) Virtually everyone in the Enterprise Content Management market. The tone of the meeting was very cooperative and collaborative. David Choy from EMC is the chair of the committee and did a good job of staying neutral on the issues. Al Brown from IBM/FileNet is the secretary for the bindings and Ethan Gur-esh from Microsoft is the secretary for the data model. All three did an excellent job keeping things on track and discussing all the open issues. Most of us have all worked together before in both the JCR meetings and iECM meetings and that made collaboration a lot easier. Dennis and I even go back to ODMA and DMA - over 15 years ago.

(BTW, I micro-blogged what was going on Twitter. CMIS Tweets are below. You can follow me on http://twitter.com/johnnewton)

The CMIS effort so far has been use case driven with the main use cases being collaborative content management, integration into portals, mashups and search. There are a number of use cases that explicitly out of scope, such as records management and web content management. Some use cases lie in between where the main application may not be addressed, but we want to allow developers to create applications that may access the information that may be created by other out-of-scope applications. An example that we discussed a lot was records management. We don't want CMIS to be so complex that you can build a records management system, but you may want to access records information for the purposes of eDiscovery.

These use cases that are in-scope seem to be clear and compelling enough that we all agreed that we urgently want to get CMIS to market to get people building these applications. Although this was one of the last thing that we discussed, it is the most impressive outcome of the meeting. There was so much agreement and consensus in the meeting that we all felt it is important to meet the deadline of having CMIS complete before the end of the year. David Choy did a great job outlining the OASIS process and what must be done. The bureaucracy (and I mean that in the nicest possible way) of the OASIS process is such that the minimum time that it takes to get from public review to standard is 6 months. So we all agreed a timeline that would get this done. This means that the spec will be ready for public review by the end of Spring. That's a mean feat, but all agreed doable with few obstacles in the way. It also means that the 0.5 spec going into the OASIS process was pretty solid.

That doesn't mean that there wasn't plenty to discuss. It was a three day event after all and we used it all up. The most controversial issues were around security and what is truly RESTful. The Security proposal was presented by SAP who have a real need to secure documents consistently with the SAP system. This was an excellent use case and provided a real-world example from which to work and focused on Access Control Lists. Most of us have been involved in round and round discussions on security and I'm not quite sure how we got to a general consensus. We are not quite done, but we all seem to have a great deal of agreement. A subcommittee will work on this some more. I had proposed a more abstract Policy interface prior to OASIS submission based upon work that we have done in iECM and JSR-283. After discussing this with David Pitfield of Oracle, I think we need to think about that one some more. (Sorry James McGovern, but XACML was shot down in flames. For the reasons why, please see this IBM article on implementing ACLs with XACML and look how horrible ACLs look. http://www.ibm.com/developerworks/xml/library/x-xacml/)

David Nuescheler, who is the chair of the JSR-283 committee and CTO of Day Software, presented a couple of discussions in collaboration with Julian Reschke. Julian is also on the WebDAV committee and brings a unique perspective. David presented some of the thoughts that originated with Roy Fielding, also from Day, on whether the REST binding is really RESTful. Although there are arguments to not call the Atom Pub protocol REST bindings, we all felt it was important to communicate our intentions toward REST. In the end, we will call the protocol support RESTful Atom Pub bindings or something like that. David suggested perhaps exploring protocols other than Atom Pub, such as WebDAV. Apparently, these were ruled out quite a while ago and there are already WebDAV standards bodies. Decision is to continue with Atom Pub. Julian, who also seems to know HTTP extremely well, logged many issues around how CMIS uses / abuses HTTP and most of his recommendations are being incorporated into the spec.

David also raised the issue of reference implementations. In the end, we all agreed that we are all making progress on our own implementations and that a reference implementation wasn't necessary. There was interest and probably continues to be interest in an open Test Compliance Kit (TCK), but we all agreed that a clearer spec should be the primary point of reference rather than a TCK. It may be up to us, the open source vendors, to come up with a TCK. We have already created both a Java and .NET test harness to test CMIS access. Even Microsoft said that if you need a free CMIS test platform, you can get Alfresco. :-)

As the spec has developed, we have all been prototyping our implementations. The early implementors (Alfresco, EMC, IBM, Microsoft, OpenText, Oracle and SAP - in alphabetical order :-)) participated in an interoperability session or "plug-fest" at a Redmond interop lab in August prior to making the spec public. And it worked! It is a real testament to the fact that this is becoming a real standard. In other words, it works. This is part of the reason that there is not as much demand for a reference implementation. It is also part of the reason that it continues to develop and have such momentum. By working with each other to test each others implementations, we are discovering what needs to be tweaked in our software and what needs to be fixed in the spec. Another interop session is planned sometime around AIIM as well as a demo that the AIIM conference is organizing and sponsoring.

In addition, we were able to demonstrate our integrations with CMIS. IBM showed Quickr accessing CMIS. Day showed a prototype that they are working on that sits on Jackrabbit. Dave showed some new components that we are working on with Share and the Joomla and Drupal integrations with Alfresco using CMIS. Microsoft and EMC had shown their integrations in previous interop sessions.

There were a few other issues that we discussed. OpenText presented hierarchical properties. I presented a proposal on how to deal with Aspects or Mixins. A little more than half the systems represented, some like Oracle and IBM had more than one, support some notion similar to Aspects - think of them as data type plug-ins. The vendors that have broad search requirements had a great deal of interest in Universal Search (really a new use case) and want to see access to change logs to simplify indexing new or changed documents in ECM systems. However, I think we all agree that we don't want any of these to hold up a 1.0 of the specification and getting CMIS out to market.

CMIS has momentum. It will be a real specification. We all seem to agree on what we want and what our customers need. Companies are implementing it. IBM, EMC and Alfresco have released 0.5 implementations. Microsoft can't talk about what they are doing, but they will. (They would probably have to kill me if I knew exactly what they are doing. ;-)) It could be more and it can probably be better, but CMIS that is out there and implemented will be better than the perfect spec coming out much later.

Microsoft were gracious hosts and provided with good facilities and food. (Just replace the projector bulb. :-)) Thanks very much Ethan!

If you are interested, CMIS Tweets follow:

Discussing CMIS use cases: Collab Content Mgmt, Portals, Mashups, Search
Open use case issues: Observation, tagging, data dict., better REST bindings - less verbose
Can't use MSN Messenger inside of Microsoft on their guest network. Bizarre.
Going thru introductions, including favorite color. (Hmm.) Should note that SAP and OpenText are also here.
Interesting discussion on reference implementation. Decision that Tech Committee will not implement ref impl.
TC agrees that we need well documented test cases. Lots of interest in a Test Compat Kit.
Show and tell coming up. Day and Lotus.
Tomorrow will be REST design (with input from Roy Fielding via David Nuescheler), schema design and various issues.
Now discussing REST issues. David Nuescheler is bringing up the issues first raised by Roy Fielding who first described REST.
Discussed REST at length. Staying with Atom Pub, but will look at browser-based Javascript libraries.
Gary Gershon is reviewing the SOAP bindings for CMIS.
David Nuescheler: "There is no such thing as a REST binding. REST is an architecture."
It is very difficult to concentrate on the minutiae of SOAP web services. (zzzzzz)
Ethan (MSFT) now discussing unified search.
Long discussion on universal search and relationship to observation. Universal search indexing looks like a MUST. Obs is SHOULD.
Dave Caruana is demonstrating Alfresco SURF and CMIS.
Dave Caruana just demonstrated Joomla and Drupal integrated with Alfresco thru CMIS.
Just got caught out. I'm supposed to talk about data dictionaries and I don't remember what the issue was. :( Jet Lag!
Long discussion on ACLs proposed by SAP. No appetite for XACML. Sorry James McGovern. Keeping it very simple.
I was very surprised at Microsoft's interest in content crawling and Universal Search. Lumped into Observation discussion y'day.
Just finished presenting how to do Mixins / Aspects in CMIS. Some want to model as related objects. How would you do it?
Just finished OpenText proposal on "Hierarchical Properties"
Question on the table - What do we need to do to finish the proposal?
After lunch, group photo! Courtesy of Dennis Hamilton.
Now batching.
Discussing timeline for CMIS and flowcharted process. We are remarkably close! But min 6 months.
Let me clarify. Get to public review in under 6 months.
Closing out most open issues on spec from the Technical Committee.
Would love to see Adobe here.
Successful CMIS face to face complete! :-)