Draft plan, see the discussion on dev@
Situation
In Jackrabbit 2.x we have a solid and feature-rich content repository that works well especially for the needs of traditional web sites and integrated content management applications. However, the trends in user expectations (especially for personalized, interactive and collaborative content), application architectures (distributed, loosely coupled, multi-platform solutions with lots of data) and hardware design (horizontal rather than vertical scaling) have rendered some of the original Jackrabbit design decisions (that date back almost a decade) obsolete and there's no easy way to incrementally update the design.
This problem has been a know issue for years (the first proposals to address it date back to 2008), and over the last year we've been actively prototyping promising ideas to gather more experience on how they work in practice. Now is the time to move to the next phase where we take that experience and all the knowledge we've gathered from working with existing Jackrabbit versions and use it to build a new repository that better matches today's needs.
Target
We want to implement a scalable and performant hierarchical content repository for use as the foundation of modern world-class web sites and other demanding content applications. The repository should implement standards like JCR, WebDAV and CMIS, and be easily accessible from various platforms, especially from JavaScript clients running in modern browser environments. The implementation should provide more out-of-the-box functionality than typical NoSQL databases while achieving comparable levels of scalability and performance.
By 2013 the new repository should have stabilized its internal architecture and the implementation should be ready for production use in major deployments that don't need all the designed features to be ready yet. By 2014 the repository should have reached comparable level of functionality on all Jackrabbit 2.x features that we haven't explicitly decided not to support. It should be possible to migrate existing Jackrabbit applications and deployments to the new repository implementation with little effort.
Path
Since the goals of the new repository implementation are somewhat different than the traditional goals of Jackrabbit (i.e. to be "a fully conforming JCR implementation"), we should organize the implementation work as a new Jackrabbit subproject with its own identity (i.e. not just "Jackrabbit 3"). This subproject shall have it's own dev@ mailing list, source repository, issue tracker and release cycle. For now the effort will remain a part of Jackrabbit and fall within the oversight of the Jackrabbit PMC, but down the line we may want to branch it off to a separate TLP, possibly through the Incubator.
The new repository shall be implemented in an open, collaborative fashion based on the Apache principles. The effort shall be open to all contributors and actively strive to make it easy for new people to get involved. To achieve this, the new architecture should be designed to be modular and easily extensible. Cross cutting concerns shall be identified and planned for early in the process. Extra care shall also be taken to make the new repository trivially easy to deploy and use for the most common use cases.
To enable a rapid feedback cycle and to make it easy to track our progress, we shall adopt a "release early, release often" model of making new releases at least once a month and maintain a growing set of automated integration and performance tests to maintain release quality. A key design constraint for all features is to make them easily testable.
To get started on all this, I propose that we select a name for the new repository and set up the proposed subproject infrastructure over the next two weeks. Then we'll target at releasing the first 0.1 version of the new repository by the end of March. The minimum requirements for that release are only that it's accessible through HTTP and JCR ("Hello, World!" level OK), come with a runnable jar for easy deployment, and have a basic integration test suite that verifies that the release meets the above requirements. We'll build on from there.