Apache Jackrabbit : Oakathon May 2019

Where and When

  • Date: May 13th - 17th
  • Location: Adobe Basel, Barfüsserplatz 6 (guests please register on the 2nd floor)
  • Meeting Rooms: Rhein (5th floor, except Friday on 3rd floor)

Attendees

Who

When

Matt Ryan

14.5.19 - 17.5.19

Marcel Reutegger

13.5.19 - 16.5.19

Julian Reschke

13.5.19 - 17.5.19 (remote)

Stefan Egli

13.5.19 - 16.5.19 (onsite except 15.5.)

Alex D

13 - 16 (remote 17)

Topics/Discussions/Goals

Title

Summary

Effort

Participants

Proposed by

Build timeouts

Jenkins builds hit the 2h timeout every now and then. Discuss options to reduce the build time and implement.

2d

Julian, Marcel

Marcel

Migrate Wiki

The Moin Wiki goes out of service by the end of May. Migrate to Confluence.

 

Marcel, Davide

Matt

Direct Binary Access API

Some early uses of the Direct Binary Access feature hint at possible API changes. Since the current use is pretty minimal and changing the API in the future will be much more painful, should we consider some of these changes now? (Matt to prepare use cases for discussion.)

1h discussion / 1-2 days to implement?

Matt

Matt

Managing Expectations for Oak Changes

We've recently changed our release cadence. I'd like to discuss setting and managing appropriate expectations for outside parties wrt how long it will take to see bug fixes and new features in an Oak release.

30m - 1h

Matt

Matt

CI/CD

How can allow developers to run their own CI/CD pipeline with additional integration tests that are not part of Oak?

2h

 

Marcel

Jackrabbit

Branch or backport API changes in Jackrabbit for Oak 1.14.0

15m

 

Julian

Async Commits

Discuss feasibility of revisiting async conflict resolution (

https://oak.markmail.org/thread/p2ljif2vag3alktf

) which could be used to implement async commits

2d

 

Stefan

Oakathon Cadence and Purpose

Discussion about Oakathon frequency, and to ensure we maintain the purpose and spirit of Oakathons moving forward.

1h

All interested

Matt

Agenda Proposal

Monday

  • 11:00 Kick off. Short intro of proposed topics and form groups.

Tuesday

  • 9:45 Intro to remaining topics proposed by Matt
  • 13:15 Managing Expectations for Oak Changes

Wednesday

  • 9:30 Oakathon Cadence and Purpose
  • 13:30 CI/CD

Thursday

Friday

Prep Work

Build timeouts

The intro for this topic went a bit beyond the proposed problem. There are various other Oak build jobs on Apache Jenkins. Julian is taking care of a number of matrix jobs that are failing rather frequently but do not send notifications to the list. We agreed to update the documentation with links to the existing jobs and look into making them more stable. Some fail with docker errors. The plan is to ask Apache Infra what Jenkins slaves actually support Docker and are recommended to use.

Reducing build time could then be achieved by splitting the whole build into a matrix that runs modules individually and in parallel.

Jackrabbit

When Oak needs a new Jackrabbit API, there are basically two options on the table. 1) backport the change to the current stable branch or 2) create a new stable release from trunk. Currently that would be a 2.20.

Discussion started about pros and cons. Going with a new stable release from trunk would mean a new release policy that first has to be discussed on the Jackrabbit dev list. There also the open question whether a new stable release from Jackrabbit trunk means a new maintenance branch has to be created. For Oak we decided this is something we don't do anymore automatically, unless needed because of backward incompatible changes. There was a concern that this may result in frequent Jackrabbit stable releases whenever the API must be enhanced for Oak. Though, last year, this was only needed once for the new direct binary access feature. Alternatively we may consider moving the Jackrabbit API to Oak.

Notes from the Oakathon

Migrate Wiki

The consensus of the group was to not migrate the Moin wiki to Confluence and instead just create a browsable archive of the existing Moin wiki. The archive would then be available as part of the Jackrabbit website. The archived wiki should have a disclaimer that those pages are not meant as documentation and link to the official documentation. The current Confluence wiki contains an outdated version of the Jackrabbit site and will be deleted.

The Confluence wiki pages that represented an old version of the Jackrabbit website are now deleted. Marcel did a test migration of the Moin to the Confluence wiki with the help of http://selfserve.apache.org/

Further progress/work is tracked in https://issues.apache.org/jira/browse/JCR-4435

CI/CD

One option for running additional non-Apache integration tests would be to fork the Jackrabbit Oak GitHub mirror, create feature branches and work on those branches until a change is ready. Then a PR back to the fork could trigger additional integration tests that act as an additional quality gate until the change is committed back to the Apache SVN. The last step would still be manual. The development workflow can potentially be simplified if the Oak code base was on GitHub. Then it might be possible to further automate the process by automatically triggering a PR against the Oak origin on GitHub when the fork applied a validated PR with the additional external integration tests. The final PR obviously would still have to pass all Oak unit and integration tests.

Moving the code base to GitHub seems to make it easier for a developer even if there are some manual steps needed to get a change into Oak. It would make the use of git-svn obsolete.

Async Commits

Guava Update

Spent some time on "Guava references in public apis (OAK-7217)" as it seems to be blocking the Guava update decisions (OAK-7182). Tried a bash script but wasn't enough as it provided general "use of Guava apis" but not focused on oak apis, so changed gears and decided to go for an oak-run approach. this seemed a better option as it allows for solving a more generic issue of "foreign apis exported by oak apis" and it turns out we also expose commons-io objects (but also a lot of others). Latest version is attached to the issue.