gerritcp/README.md

gerritcp

A utility to copy changesets from one gerrit instance to another

Theory of operation

The tool is built primarily for the use case of downstreaming coreboot into Chromium OS. If feasible, extension of its scope to other scenarios is welcome.

The tool must:

• copy submitted changes from upstream to reviewable change sets in downstream;
• mask out changes to a configured set of files;
• keep a configured set of files around that only exists in downstream;
• provide multiple work streams with well-defined semantics so that changes, to a given subdirectory (originally: util/crossgcc) can be handled separately from everything else;
• un-abandon change sets in downstream if they would be revived by some transaction of this tool;
• skip change sets that are marked WIP in downstream;
• keep already downstreamed change sets alone, e.g. if they have been reordered.

gerritcp keeps a bare git repo for tracking both upstream and downstream. Its new commits are created by directly adding objects to the git repo so there's no current checked-out tree that can get out of sync. It works its way backward in the source branch, collecting information about which work stream a change belongs to and if it is already present in downstream and usable. As soon as all work streams have a root change to work from, gerritcp works forward through the collected changes, creating new change sets and immediately pushing them to downstream gerrit (marking them unabandoned as necessary).

A change is "usable" if it is not marked WIP and if it contains metadata of upstream's change submission process: WIP allows skipping changes, even the top-of-patchtree change, in case they need to be put aside for a while, e.g. when waiting for an accompanying fix. Checking for upstream's metadata ensures that the downstream change in question is a downstreamed change: There are sometimes changes that have been pushed (and reviewed) downstream first and then put upstream for submission when ready. These should be integrated with the history, but they shouldn't derail other upstream changes.

When applying a change to downstream, several things needs to be taken care of:

• Since it's possible that changes have been taken out of downstream's patch train, gerritcp can't simply adopt the toplevel tree object of the upstream commit. Even files can't be adopted 1:1 but need to be merged because there might be changes to skip.
• Since downstream handles a few files differently, these might have to be skipped entirely, by dropping the downstream object ids into the upstream-ish tree objects that result from the prior step.
• For work streams, the right parent commit needs to be chosen: It must be the work stream specific parent (if that one isn't merged already) or the upstream parent (in other cases) to account for situations in which commits in the work stream touch files outside its direct responsibility.

Pushes to downstream need to be authenticated and there needs to be enough access to gerrit to be able to unabandon changes, while pulls from upstream can be anonymous. Future additions such as watching upstream's event stream might require authentication on the upstream side as well.

Configuration format

Configuration is stored in a single file in yaml format:

sites:
  upstream|downstream:
    url: string
    repo: string
    branch: string
    authentication: none|cookie
    cookieName: string
    cookieVal: string

workstreams:
  {name}:
    onlyIfTouching:
      - string # paths
      - ...
    neverModify:
      - string # paths
      - ...

The coreboot.org -> chromiumos configuration can be found in chromium-coreboot.yaml.