brew/docs/Homebrew-linuxbrew-core-Maintainer-Guide.md

# Homebrew linuxbrew-core Maintainer Guide

## Merging formulae updates from Homebrew/homebrew-core

Linuxbrew-core is a fork of Homebrew-core and, therefore, it has to periodically
merge changes made by Homebrew developers and contributors. Below we
describe the steps required to merge `Homebrew/homebrew-core` into
`Linuxbrew/homebrew-core`, possible conflicts and ways to resolve
them. Note, that instructions below have been written for a "clean"
environment and you might be able to skip some of the steps if you
have done them in the past.

### Preparation

First of all, we want to enable developer commands and prevent
automatic updates while we do the merge:

```bash
export HOMEBREW_DEVELOPER=1
export HOMEBREW_NO_AUTO_UPDATE=1
```

Once we've done that, we need to get access to the `merge-homebrew`
command that will be used for the merge. To do that we have to tap the
[`linuxbrew/homebrew-developer`](https://github.com/Linuxbrew/homebrew-developer)
repo:

```bash
brew tap linuxbrew/developer
```

Next, we have to navigate to the repository where we want to do the
merge and make sure that there are 3 remotes:

* a remote named `origin` pointing to Linuxbrew-core,
* a remote named `homebrew` pointing to Homebrew-core, and
* a remote pointing to your GitHub fork of Linuxbrew-core.

Remote names `origin` and `homebrew` are hard-coded in
`merge-homebrew`, while the remote pointing to your fork must be the
same as your GitHub username, as it will be used to submit a pull
request for the merge. Set the name to the `$GITHUB_USER` environment
variable, or let `hub fork` add a remote for you.

```bash
brew install hub
cd $(brew --repo homebrew/core)
git remote add homebrew https://github.com/Homebrew/homebrew-core.git
hub fork --remote-name=$GITHUB_USER
```

Now, let's make sure that our local branch `master` is clean and that
your fork is up-to-date with Homebrew/linuxbrew-core:

```bash
git checkout master
git fetch origin master
git reset --hard origin/master
git push --force $GITHUB_USER master
```

Strictly speaking, there is no need for `git reset --hard
origin/master` and simple `git merge origin/master` would have been
sufficient if you didn't mess with your local `master` branch.
However, hard reset makes sure that these instructions are correct
even if you did mess something up. The same is true for the `--force`
flag for the `git push` command above.

By default, the following command will attempt to merge all the
changes that the upstream Homebrew developers have made.

```bash
brew merge-homebrew --core
```

Merging all the changes from upstream in one go is usually
undesireable since our build servers will time out. Instead, attempt
to only merge 8-10 modified formulae.

`git log --oneline master..homebrew/master` will show a list of all
the upstream commits since the last merge, from oldest to newest.

Pick a commit SHA-1 that will merge between 8-10 formulae (16-20 commits
including bottles). Once you're satisfied with the list of updated
formulae, begin the merge:

```bash
brew merge-homebrew --core --skip-style <sha>
```

The `--skip-style` argument skips running `brew style`, which saves
time and in some cases avoids errors. The style errors can be fixed in
bottle PRs later in the process when CI flags them.

#### Simple Conflicts

Once you issue the above command, the merge will begin and in the very
end you will see the list of (conflicting) formulae that
`merge-homebrew` could not merge automatically:

```bash
==> Conflicts
Formula/git-lfs.rb Formula/gnutls.rb Formula/godep.rb
```

Note, that you can also get a list of unmerged files (*i.e.* files with conflicts) using:

```sh
git diff --name-only --diff-filter=U
```

Of course, conflicts will be different every merge. You have to
resolve these conflicts either manually in a text editor, or by using
tools like `diffuse`, `tkdiff`, or `meld`, some of which are available
from Homebrew. Frequently, conflicts are caused by the new versions
of macOS bottles and look like:

```ruby
<<<<<<< HEAD
    sha256 "bd66be269cbfe387920651c5f4f4bc01e0793034d08b5975f35f7fdfdb6c61a7" => :sierra
    sha256 "7071cb98f72c73adb30afbe049beaf947fabfeb55e9f03e0db594c568d77d69d" => :el_capitan
    sha256 "c7c0fe2464771bdcfd626fcbda9f55cb003ac1de060c51459366907edd912683" => :yosemite
    sha256 "95d4c82d38262a4bc7ef4f0a10ce2ecf90e137b67df15f8bf8df76e962e218b6" => :x86_64_linux
=======
    sha256 "ee6db42174fdc572d743e0142818b542291ca2e6ea3c20ff6a47686589cdc274" => :sierra
    sha256 "e079a92a6156e2c87c59a59887d0ae0b6450d6f3a9c1fe14838b6bc657faefaa" => :el_capitan
    sha256 "c334f91d5809d2be3982f511a3dfe9a887ef911b88b25f870558d5c7e18a15ad" => :yosemite
>>>>>>> homebrew/master
```

For such conflicts, simply remove the "HEAD" (Linuxbrew's) part of the
conflict along with `<<<<<<< HEAD`, `=======`, and `>>>>>>>
homebrew/master` lines. Later, we will submit a request to rebuild
bottles for Linux for such formulae.

The `merge-homebrew` script will stage resolved conflicts for you.

#### Complex Conflicts

Of course, from time to time conflicts are more complicated and you
have to look carefully into what's going on. An example of a slightly
more complex conflict is below:

```ruby
<<<<<<< HEAD
    if OS.mac?
      lib.install "out-shared/libleveldb.dylib.1.19" => "libleveldb.1.19.dylib"
      lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.dylib"
      lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.1.dylib"
      system "install_name_tool", "-id", "#{lib}/libleveldb.1.dylib", "#{lib}/libleveldb.1.19.dylib"
    else
      lib.install Dir["out-shared/libleveldb.so*"]
    end
=======
    lib.install "out-shared/libleveldb.dylib.1.19" => "libleveldb.1.19.dylib"
    lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.dylib"
    lib.install_symlink lib/"libleveldb.1.19.dylib" => "libleveldb.1.dylib"
    MachO::Tools.change_dylib_id("#{lib}/libleveldb.1.dylib", "#{lib}/libleveldb.1.19.dylib")
>>>>>>> homebrew/master
```

Note, that in the "HEAD" (Linuxbrew's) part we see previous code of
the Homebrew's formula wrapped in `if OS.mac?`. To resolve such a
conflict you have to replace the contents of `if OS.mac?` part up
until `else` with the contents of the bottom part of the conflict
("homebrew/master"). You also have to check if there are any obvious
modifications that have to be made to the `else` part of the code that
deals with non-macOS-related code.


#### Finishing the merge

Once all the conflicts have been resolved, a text editor will open
with pre-populated commit message title and body:

```text
Merge branch homebrew/master into linuxbrew/master

# Conflicts:
#       Formula/git-lfs.rb
#       Formula/gnutls.rb
#       Formula/godep.rb
```

Leave the title of the message unchanged and uncomment all the
conflicting files. Your final commit message should be:

```text
Merge branch homebrew/master into linuxbrew/master

Conflicts:
        Formula/git-lfs.rb
        Formula/gnutls.rb
        Formula/godep.rb
```

#### Submitting a PR

The `merge-homebrew` command will create a pull-request for you, using `hub`.

Once the PR successfully passes the tests and/or is approved by other
Homebrew developers, you can finalize the merge with:

```bash
brew pull --clean <PR-NUMBER>
git push origin master
```

The merge is now complete. Don't forget to update your GitHub fork by
running `git push your-fork master`

## Building bottles for updated formulae

After merging changes, we must rebuild bottles for all the PRs that
had conflicts.

To do this, tap `Linuxbrew/homebrew-developer` and run the following
command where the merge commit is `HEAD`:

```sh
for formula in $(brew find-formulae-to-bottle); do
  brew build-bottle-pr --remote=$GITHUB_USER $formula
done
```

The `find-formulae-to-bottle` command outputs a list of formulae
parsed from the merge commit body. It also performs some checks
against the formulae:

And it skips formulae if any of the following are true:
- it doesn't need a bottle
- it already has a bottle
- the formula depends on macOS to build
- the formula's tap is Homebrew/homebrew-core (the upstream macOS repo)
- there is already an open PR for the formula's bottle
- the current branch is not master

If a formula you are expecting to bottle is skipped, there may be an
error; by default, this script won't output the errors. To see them,
run `brew find-formulae-to-bottle --verbose` separate to the `for`
loop above.

The `build-bottle-pr` script creates a branch called `bottle-<FORMULA>`, adds `# Build a bottle
for Linuxbrew` to the top of the formula, pushes the branch to GitHub
at the specified remote (default: `origin`), and opens a pull request using `hub
pull-request`.

## Common build failures and how to handle them

### Bottling errors

## Handling `brew bump-formula-pr` PRs

### Formulae that exist in Homebrew/homebrew-core

The `brew bump-formula-pr` command will raise PRs against the Linux
formula repo for upstream Mac formulae when running on Linux. This
isn't how version bumps are done for _most_ formulae. Until
[Homebrew/brew issue
6341](https://github.com/Homebrew/brew/issues/6341) is implemented - a
feature that will raise PRs against the correct repo for macOS
formulae bumps on Linux - we have to redirect users to raise their PRs
in Homebrew/homebrew-core:

> Thanks for your PR.
>
> However, this formula is not Linux-specific. Its new versions are merged from the [Homebrew/homebrew-core](https://github.com/Homebrew/homebrew-core) repository daily [as documented in CONTRIBUTING.md](https://github.com/Homebrew/linuxbrew-core/blob/master/CONTRIBUTING.md). Please submit this change as a PR to that repository.
>
> We look forward to your PR against Homebrew/homebrew-core for the next version bump!

### Linux-only formulae

If the formula is a Linux-only formula, it either:
- will contain the line `# tag "linuxbrew"`
- won't have macOS bottles

These formulae are fine for users to bump with `brew bump-formula-pr`,
but you should request that they remove the existing `x86_64_linux`
bottle SHA line so that CI will build a bottle for the new version
correctly. If the bottle SHA isn't removed, CI will fail with the
following error:
> `--keep-old` was passed but there are changes in `sha256 => x86_64_linux`