Import these from the homebrew/aliases tap and deprecate that tap.
This required a little messing around with class/module/constant names
to get `brew tests` and `brew typecheck` to play nicely.
I added also added Sorbet type signatures and integration tests.
- Clarify the comment of why we have SimpleCov special logic for
parallel tests
- use a nicer ParallelTests API for checking which process to output
the coverage format on
There were a few tests which require core to be tapped and fail
if it isn't. This is annoying if someone is trying to contribute
to the project and they're using the JSON API instead of having
the core repo tapped locally.
I'm just skipping these because it's the simplest thing to do.
The tests that failed are mostly rubocop tests so it's fine
if they only run on CI.
- This is cleaner than vendoring a whole bunch of new gems and pinning `psych`.
- Thanks for the pointer, Bo!
- It doesn't work, though?
```
❯ brew tests --only=migrator --debug
Error: cannot load such file -- debug/debug.so
Warning: Removed Sorbet lines from backtrace!
Rerun with `--verbose` to see the original backtrace
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug/frame_info.rb:16:in `require'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug/frame_info.rb:16:in `rescue in <module:DEBUGGER__>'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug/frame_info.rb:13:in `<module:DEBUGGER__>'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug/frame_info.rb:3:in `<top (required)>'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug/session.rb:31:in `require_relative'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug/session.rb:31:in `<top (required)>'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug.rb:3:in `require_relative'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/gems/3.1.0/gems/debug-1.6.3/lib/debug.rb:3:in `<top (required)>'
/opt/homebrew/Library/Homebrew/dev-cmd/tests.rb:48:in `require'
/opt/homebrew/Library/Homebrew/dev-cmd/tests.rb:48:in `run'
/opt/homebrew/Library/Homebrew/brew.rb:89:in `<main>'
```
- Byebug was introduced in [2020](https://github.com/Homebrew/brew/pull/7577) for hooking into tests for debugging.
- It does not work anymore in so far as it does not stop at breakpoints when following the instructions to trigger them in tests.
This sets the HOMEBREW_NO_INSTALL_FROM_API environment variable
to prevent the selected tests from using the API. We will need
this as we transition to having the API be enabled by default
when running the tests but it's also nice as a sanity check
with the :needs_utils_curl scope in a few places.
Any test that is not tagged as :needs_network and that makes
a call to an unapproved method in the `Utils::Curl` module
will raise an error unless that method gets mocked somehow.
tests: add exceptions for tests that use curl to download local files
These are acceptable ways to use curl in local, non-network tests.
For those edge cases we allow you to bypass the check with :needs_utils_curl.
I added two new methods to cache both installed and all taps.
All taps includes core taps no matter if they're installed locally
since they're always provided by the API anyway.
This makes it easier to cache `Tap.each` while making the code
easier to reason about. It also will be useful because we'll
be able to avoid the `Tap.select(&:installed?` pattern that has
recently invaded the codebase.
Note: I also stopped clearing all tap instance caches before
tests. Running `Tap.each` would cache existing taps which would
lead to unexpected behavior since the only existing tap before
each test is the core tap. This is the only tap whose directory
is not cleaned up between tests so we just clear it's cache directly.
We also now clear all tap instances after tests as well regardless
of whether the API was used that time.
The core taps exist outside of the normal cache busting cycle
so they need to clear explicitly at the instance level.
Just to be sure we should clear all of them each time.
This essentially reverts part of the change in this PR.
- https://github.com/Homebrew/brew/pull/16746
Now we only include the cachable registry when running tests.
We basically just load it first and add a bunch of methods to
it before loading the rest of the formula files when we require
global.rb. I added a check to make sure this require order is
preserved.
I also made a bunch of methods private, stop excluding classes
that inherit from casks since it's unnecessary and add more docs.
This adds a registry for all modules and classes that
cachable is included in. The registry allows us to
programmatically clear all caches in between tests
so that we don't forget to do that when adding a new
class or refactoring code. The goal here is to reduce
the number of flaky tests in the future.
- These are arbitrary length limits that had a load of disables in code.
- The limits were only increasing over time rather than decreasing.
- Fixing the problematic code to be shorter would take a long time for
questionable gain since the problem has been around so long.
- We're not going to make the really long things be any shorter any time soon.
- The instructions in issue 14685 say, pragmatically, "disable all the rubocop
rules we're never going to realistically fix e.g. Metrics/ClassLength". But
that felt like a slippery slope to more _really_ long modules/classes/blocks,
and the limits are here for a reason.
- Extract update tests into separate tests.
- Make `brew tests --online` run only once in its own job. This job
could be made non-required to fight flakiness.
- Split up the various macOS runs into several steps now that we have
more parallel macOS workers available.
- Cleanup some flaky tests.
- fix `check_for_config_scripts` test, no idea why this wasn't working
- don't autoretry tests if `focus` or `byebug` are enabled
- add a ARM64 Ventura bottle symlink
