I previously added the 8 curl exit code (weird server reply) to the
list of non-success exit codes that `#curl_headers` will handle.
We're now seeing failures with a 56 exit code (failure in receiving
network data), where the server returns a 4xx response for a `HEAD`
request but the same request using `GET` works as expected (e.g.,
casks like `beeper`, `get-api`, `odrive`, `ui`, etc.).
This adds 56 to the list of exit codes in `#curl_headers`, so a
response with a 4xx HTTP status will be automatically retried using
`GET`.
- Previously I thought that comments were fine to discourage people from
wasting their time trying to bump things that used `undef` that Sorbet
didn't support. But RuboCop is better at this since it'll complain if
the comments are unnecessary.
- Suggested in https://github.com/Homebrew/brew/pull/18018#issuecomment-2283369501.
- I've gone for a mixture of `rubocop:disable` for the files that can't
be `typed: strict` (use of undef, required before everything else, etc)
and `rubocop:todo` for everything else that should be tried to make
strictly typed. There's no functional difference between the two as
`rubocop:todo` is `rubocop:disable` with a different name.
- And I entirely disabled the cop for the docs/ directory since
`typed: strict` isn't going to gain us anything for some Markdown
linting config files.
- This means that now it's easier to track what needs to be done rather
than relying on checklists of files in our big Sorbet issue:
```shell
$ git grep 'typed: true # rubocop:todo Sorbet/StrictSigil' | wc -l
268
```
- And this is confirmed working for new files:
```shell
$ git status
On branch use-rubocop-for-sorbet-strict-sigils
Untracked files:
(use "git add <file>..." to include in what will be committed)
Library/Homebrew/bad.rb
Library/Homebrew/good.rb
nothing added to commit but untracked files present (use "git add" to track)
$ brew style
Offenses:
bad.rb:1:1: C: Sorbet/StrictSigil: Sorbet sigil should be at least strict got true.
^^^^^^^^^^^^^
1340 files inspected, 1 offense detected
```
Take 2 of https://github.com/Homebrew/brew/pull/17692 but with:
- provide and document `HOMEBREW_NO_VERIFY_ATTESTATIONS`
- don't try to run unless there's GitHub credentials
- don't try to run unless `gh` is installed
- don't try to run in CI
While we're here:
- split out a `Homebrew::EnvConfig.devcmdrun?` helper method
- add some missing `Homebrew::EnvConfig.github_api_token` presence
checks
I recently noticed that ~23 `livecheck` blocks using the `HeaderMatch`
strategy were failing. Looking into it, these fail when using a `HEAD`
request and retry with `GET` but the resulting response with the
headers we want is simply discarded because the `exit_status` from
curl is 8 ("weird server reply").
This resolves the issue by adding a special case for this exit status,
so `#curl_headers` will return the headers in this scenario.
`#curl_headers` was recently introduced into `Strategy#page_headers`
but only the call was modified and the method wasn't updated to
correctly work with the new return value, so all `HeaderMatch` checks
immediately started failing with an error.
This commit includes changes that return `#page_headers` to a working
state. I've removed the `result.assert_success!` call because it
prevents a few checks from being retried with `GET` (`firefox-cn`,
`krisp`, `prepros`).
- Some casks have URL arguments like "referer" (spelled wrong, that's
intentional in the HTTP spec).
- The audit for one such cask, `iThoughtsX`, was failing because the
"referer" wasn't getting passed through to cURL so the access would
404.
----
Before:
```
❯ brew audit --cask --online --appcast --signing 'ithoughtsx'
[...]
audit for ithoughtsx: failed
- The binary URL https://cdn.toketaware.com?download=iThoughtsX.zip is not reachable (HTTP status code 404)
- Version '9.2.0' differs from '9.3.0' retrieved by livecheck.
- Version '9.2.0' differs from '9.3.0' retrieved by livecheck.
Error: 2 problems in 1 cask detected
```
After:
```
❯ brew audit --cask --online --appcast --signing 'ithoughtsx'
[...]
audit for ithoughtsx: failed
- Version '9.2.0' differs from '9.3.0' retrieved by livecheck.
- Version '9.2.0' differs from '9.3.0' retrieved by livecheck.
Error: 1 problem in 1 cask detected
```
Update base URL when there is an absolute location, so that following
relative locations are considered relative to the new base.
Consider below cURL output for https://example_one.com:
HTTP/1.1 302 Moved Temporarily
Location: https://example_two.com
HTTP/1.1 302 Moved Temporarily
Location: /foo/
HTTP/1.1 200 OK
The final URL should be https://example_two.com/foo/ rather than
https://example_one.com/foo/.
The response from a URL protected by Cloudflare may only provide a
relevant cookie on the first response but
`#curl_http_content_headers_and_checksum` only returns the headers of
the final response. In this scenario, `#curl_check_http_content` isn't
able to properly detect the protected URL and this is surfaced as an
error instead of skipping the URL.
This resolves the issue by including the array of response hashes in
the return value from `#curl_http_content_headers_and_checksum`, so
we can check all the responses in `#curl_check_http_content`.
The return hash from `#curl_http_content_headers_and_checksum`
contains a `:status`, which is the status code of the last response.
This string value comes from `#parse_curl_response`, where the key is
`:status_code` instead.
Aligning these keys technically allows us to pass either of these
hashes to the `#url_protected_by_*` methods, as both contain
`:status_code` and `:headers` in the expected format.
Before `#parse_curl_output` was introduced and related methods were
updated to use it, `#url_protected_by_cloudflare?` and
`#url_protected_by_incapsula?` were checking a string of all the
headers from a response and using a regex to check related header
values.
However, when `#curl_http_content_headers_and_checksum` was updated
to use `#parse_curl_output` internally, the `:headers` value became
a hash generated by `#parse_curl_response`. The `#url_protected_by_*`
methods were updated to work with the hash value but this wasn't able
to fully replicate the previous behavior because
`#parse_curl_response` was only keeping the last instance of a given
header (maintaining pre-existing behavior). This is an issue for
these methods because they check `Set-Cookie` headers and there can
be multiple instances of this header in a response.
This commit updates these methods to handle an array of strings in
addition to the existing string support. This change ensures that
these methods properly check all `Set-Cookie` headers, effectively
reinstating the previous behavior.
Past that, this updates one of the early return values in
`#url_protected_by_cloudflare?` to be `false` instead of an implicit
`nil`. After adding a type signature to this method, it became clear
that it wasn't always returning a boolean value and this fixes it.
`Curl#parse_curl_response` only includes the last instance of a given
header in its `:headers` hash (replicating pre-existing behavior).
This is a problem for headers like `Set-Cookie`, which can appear more
than once in a response.
This commit addresses the issue by collecting duplicate headers into
an array instead. Headers that only appear once in the response will
still have a string value but headers that appear more than once will
be an array of strings. Whenever headers from `#parse_curl_response`
are used (directly or indirectly), it's important to conditionally
handle the expected types.
