Apache Jackrabbit : Oakathon August 2017

Where and When

  • 2017.08.21 - 25
  • Location: Adobe Basel, Meeting Room: Joggeli (2nd floor)

Attendees

Who

When

Marcel Reutegger

21. - 24.

Robert Munteanu

22. - 24.

Alex Deparvu

21. - 24.

Michael Dürig

21. - 23. + either 24. or 25.

Kevin Wellenzohn

21. - 24.

Tommaso Teofili

21. - 24.

Thomas Mueller

21. - 24. (tentative)

Chetan Mehrotra

21. - 25. (remote)

Valentin Olteanu

21. - 25. (partially available)

Matt Ryan

21. - 25. (afternoons - remote)

Tomek Rekawek

21. - 25.

Julian Sedding

21. + 22.

Topics/Discussions/Goals

Title

Summary

Effort

Participants

Example

Describe what the topic is about. Is is mainly a discussion, implementing a feature, or a POC? Add links to further information and details.

The expected effort. 2h, 1d, full week, etc.

Add names of participants.

Key Signing Party

To update the Oak team's WOT.

1h

Open to anyone.

Update OSGi Annotations in Oak

Proposal to update the OSGi annotation to latest standard as described here. Mostly removal or org.apache.felix.scr.annotations, others can be tackled if found.

1 day

AlexD, Robert

Workload-aware Property Index

The workload-aware index aims to improve update performance and reduce the number of index conflicts. Discussion of an implementation in Oak.

2h

Kevin Wellenzohn, Thomas Mueller (tentative)

Hybrid Index v2

Next phase of hybrid index to enable support for unique index and sync indexes

2 d

Chetan Mehrotra, Thomas Mueller (tentative)

Journal basedAsync Indexer

New scalable design for async indexer

2 d

Chetan Mehrotra, Thomas Mueller (tentative)

Finalize and review composite node store

Make sure the current version is ready for production

2d

Robert Munteanu, Tomek Rekawek

Implement metrics

Implement Metrics across Oak and prototype RRDTool persistence

3d

Marcel, Thomas Mueller (tentative), Alex D

Modularization

Anythign left to be done here? Robert: Split out oak-core more possibly - document-based storage is a candidate

3d

Robert (if composite and OSGi annotations are done in time), Marcel, Thomas Mueller (tentative), Alex D

Manage Jenkins configurations in SVN

Stop relying on the Jenkins UI for configuration, but instead use a Jenkinsfile or the DSL Job plugin

0.5d

Robert

Segment store analyser

Implement a tool to analyse the content of a segment store

2-5d

Michael

Provide JCR node info to blob store

Prototype a way to provide JCR node information - path, node type, properties and property values - to blob store, for Composite Blob Store

2 d

Matt Ryan

Unmanaged binary support

Deliver a prototype demo of at least one unmanaged binary support use case (see JCR Binary Usecase UC1, UC5, UC6, UC13). UC13 is proposed but could be another.

5 d

Matt Ryan

Prefetching the DocumentStore cache using machine learning

Work out a way to: (1) gather data about sequences of nodes read from the DocumentStore, (2) use this data to predict what nodes will be requested soon.

5d

Tomek Rekawek, Thomas Mueller (tentative)

Agenda Proposal

There are two long running tracks not listed in the below Agenda: Segment store analyser lead by Michael and Blob store & Unmanaged binaries lead by Matt. Those two tracks are mostly independent from the other proposed topics and will run for the full week of the Oakathon. The General track is mostly administrative stuff and intended for all participants.

Mon

General

Index Track

Misc 1

Misc 2

9:00-12:30

9:00 Setup
9:30 5 min overview per topic

10:30 Workload-aware Property Index| | |

13:30-17:00

 

Hybrid Index v2

Update OSGi annotations

Metrics

 

 

Tue

 

 

 

 

9:00-12:30

 

9:30 Hybrid Index v2 intro by Chetan

Composite NS

Metrics

13:30-17:00

 

Hybrid Index v2

14:00 Composite BlobStore & Unmanaged Binary intro by Matt

 

 

Wed

 

 

 

 

9:00-12:30

 

Hybrid Index v2

Composite NS

Modularize DocNS

13:30-17:00

13:30 Key Signing Party (1h)

Journal based async indexer

 

 

Thu

 

 

 

 

9:00-12:30

 

Journal based async indexer

Composite NS

Modularize DocNS

Lunch sponsored by Adobe

 

13:30-17:00

15:00- Wrap-up results

 

 

 

 

 

Fri

 

 

 

 

9:00-12:30

 

Journal based async indexer

 

 

13:30-17:00

 

 

 

Prep Work

Notes from the Oakathon

Oak Tooling API

See also OAK-6584.

Current situation

Current segment store related tools are implemented ad-hoc by potentially relying on internal implementation details of Oak Segment Tar. This makes those tools less useful, portable, stable and potentially applicable than they should be.

Goal

Provide a common and sufficiently stable Oak Tooling API for implementing segment store related tools. The API should be independent of Oak and not available for normal production use of Oak. Specifically it should not be possible to it to implement production features and production features must not rely on it. It must be possible to implement the Oak Tooling API in Oak 1.8 and it should be possible for Oak 1.6.

Typical use cases

Validation

Reimplement Script Oak on top of the tooling API.

API draft

Metrics

A metrics reporter using RRD4J has been implemented in Apache Sling (SLING-7055). It writes metrics periodically into a time-series database on the local file system.

Open questions

  • How expensive is it to gather metrics? What if reading a metric is too expensive because the custom metric implementation is troublesome? Should the reporter detect and blacklist such metrics? Alternatively, the reporter could automatically increase the interval for reading such metrics.
  • It would be useful to also have RRD archives with MAXIMUM consolidation function. E.g. for tracking the observation queue length.
  • How should timer and histogram metrics be written to RRD? Those are not simple values, but expose already aggregated data.

More metrics

It may be useful to have more metrics OOTB in Oak.

  • MongoDB metrics exposed by the MongoDocumentStore (storage size, free blocks, caches, etc.)
  • RDB metrics exposed by the RDBDocumentStore

Other areas?

Composite Node Store

A high-level review of the composite approach was done. A number of Jira issues were opened as a result:

Open questions

  • Do we need additional ops tooling to help with diagnosing/recovering from failed mount-time checks?

Update OSGi Annotations in Oak

Work in progress, tracked under OAK-6741 - Switch to OSGi R6 annotations

Attachments: