89 Commits

Author SHA1 Message Date
Carlo Cabrera
65be47848d
github_packages: adjust upload retries
We still regularly see upload failures here, so let's adjust this to see
if we can try to reduce these.
2023-05-30 11:55:08 +08:00
Carlo Cabrera
9744b69071
github_packages: use exponential backoff when retrying
The retry behaviour in `publish_commit_bottles.yml` [1] is often
successful after the second try, so it's likely that we're not waiting
long enough in between retries here.

Let's fix that by retrying with exponential backoff instead of adding a
fixed interval of five seconds after each failure.

[1] 3241035b2a/.github/workflows/publish-commit-bottles.yml (L431-L443)
2023-05-03 16:58:16 +08:00
Carlo Cabrera
31a152208b
github_packages: improve upload error handling
Erroring out in the middle of uploading multiple bottles results in a
state that is tedious to recover from.

Let's try to avoid these situations by performing checks for all the
bottles first before trying to upload any.
2023-05-03 16:40:42 +08:00
Douglas Eichelberger
24cf6076e8 brew style --fix 2023-04-24 20:42:39 -07:00
Ruoyu Zhong
a196d33482
github_packages: fix bottle manifest schema violation
According to [^1] [^2], image manifest annotations need to be a
string-string map.

This should fix the errors seen in Homebrew/homebrew-core#128517,
Homebrew/homebrew-core#128558, et al.

[^1]: 2879913ce4/schema/image-manifest-schema.json (L35)
[^2]: 2879913ce4/schema/defs-descriptor.json (L22)
2023-04-17 23:17:20 +08:00
Colin Dean
2787eead89 Add bottle file size to GHP manifest annotations
This will reduce the number of requests necessary to ascertain the size
of formulas' bottle archives for analysis purposes.

Currently, getting the size of each bottle requires 1 request for the
formula.json and followed by N requests per formula-version, which
for most formulae is 7— more than 47k requests!

After this change, size retrieval can ascertain all bottle sizes for a
formula-version in a single request, at the cost of one additional
request per formula-version if that formula-version has not been
rebuilt since this change was introduced.

To start, size retrieval will incur an additional P requests where
P is the number of packages. Over the next few weeks and months,
the retrieval will go a lot faster as all new and updated packages will
require only one request.
2023-04-16 21:32:47 +00:00
Issy Long
1e64a658d4
Revert "Update GitHubPackages for new Tab implementation" 2023-03-15 23:12:37 +00:00
Douglas Eichelberger
ea65b7264a Update GitHubPackages for new Tab implementation 2023-03-15 09:49:45 -07:00
Dawid Dziurla
c4cf2d208e
github_packages: reduce skopeo retry times to 2 2023-03-09 09:42:43 +01:00
Dawid Dziurla
de98ad53cc
github_packages: additional retry of skopeo copy with backoff 2023-03-09 09:17:24 +01:00
Dawid Dziurla
244006f15a
github_packages: retry skopeo copy 5 times 2023-03-08 15:32:31 +01:00
Mike McQuaid
fe37be6c30
github_packages: improve schema debugging. 2023-02-10 09:05:10 +00:00
Rylan Polster
68bbe03d04
Remove remaining formula_api_path references 2023-01-06 02:46:21 -05:00
Mike McQuaid
c294dcc616
glibc related cleanup
Extracted from https://github.com/Homebrew/brew/pull/13577
2022-08-23 12:42:02 +01:00
Bo Anderson
ecc705803f
github_packages: add comment for image-spec pin 2022-05-16 16:15:17 +01:00
Bo Anderson
354e5b60a7
github_packages: pin image spec schema 2022-05-11 04:28:02 +01:00
Bo Anderson
ac9af0dbbc
github_packages: fix OOM with large bottles 2022-03-03 16:26:47 +00:00
XuehaiPan
9ef52080e3 utils: extract common word "for" in reason 2021-11-23 23:59:09 +08:00
XuehaiPan
c0826f1890 utils: add method ensure_executable! 2021-11-23 23:32:32 +08:00
XuehaiPan
d749173adc utils: mark reason as keyword argument in ensure_formula_installed! 2021-11-23 23:31:22 +08:00
XuehaiPan
3376479e95 utils: add method ensure_formula_installed! 2021-11-23 22:48:39 +08:00
fn ⌃ ⌥
93b8fa7838 pr-upload: deprecate --github-org= 2021-11-19 11:48:23 -08:00
Michka Popoff
1cc7ca33c0
update: migrate everyone from linuxbrew-core to homebrew-core
Co-authored-by: Mike McQuaid <mike@mikemcquaid.com>
2021-10-20 11:01:38 +01:00
Bo Anderson
861dea9ada
Bump various type strictnesses 2021-09-11 01:00:23 +01:00
Bo Anderson
02724df0d6
Move json_schemer install to Gemfile 2021-09-10 03:08:21 +01:00
yahavi
0335d8c0bc Fix code review comments + disable authorization on redirections 2021-07-26 19:07:23 +03:00
Bo Anderson
d78dc014d1
github_packages: add missing root mkpath 2021-05-04 16:21:23 +01:00
Bo Anderson
ae91ec2772
github_packages: handle "manifest unknown" 2021-04-12 18:58:28 +01:00
Bo Anderson
503f6dc049
github_packages: safer handling of skopeo inspect 2021-04-11 05:04:15 +01:00
Bo Anderson
57e4209e9c
github_packages: support --keep-old 2021-04-09 17:55:12 +01:00
Bo Anderson
342a697459
github_packages: fix bottle tag handling 2021-04-09 01:55:42 +01:00
Mike McQuaid
eb25bf3668
github_packages: fix "Uploaded to" organisation.
This is hardcoded to Homebrew but we've started uploading bottles to
the Linuxbrew org.
2021-04-08 11:48:46 +01:00
Mike McQuaid
4cbc34d31d
Merge pull request #11071 from MikeMcQuaid/linuxbrew-url-fix
Fix Linuxbrew URL handling
2021-04-08 11:12:39 +01:00
Mike McQuaid
598c230e12
Fix Linuxbrew URL handling
After this `HOMEBREW_BOTTLE_DOMAIN=https://ghcr.io/v2/linuxbrew/core`
will work as expected.

While we're here, do a bit of cleanup of constant usage; we don't want
some of these leaking outside the `GitHubPackages` class.

In doing so `docker://` will continue to work for `ghcr.io` but won't
work for arbitrary other Docker hosts; this isn't something we want to
support yet.
2021-04-08 10:54:28 +01:00
Mike McQuaid
0ff9bf6cb4
github_packages: fix source for Linuxbrew bottles.
Addresses https://github.com/Homebrew/brew/pull/11056#issuecomment-815261503
2021-04-08 10:22:30 +01:00
Mike McQuaid
aafc5b6e98
github_packages: rewrite more invalid versions.
Needed for `picat` and `srmio`.
2021-04-08 09:32:23 +01:00
Mike McQuaid
7b8d97aead
github_packages: quiet skopeo inspect
We want this to fail so don't print out the (expected) `stderr` output.
2021-04-07 17:34:05 +01:00
Mike McQuaid
1435018e11
Merge pull request #11057 from MikeMcQuaid/github_packages_more_fixes
More GitHub Packages bulk upload fixes
2021-04-07 17:26:03 +01:00
Mike McQuaid
8800b85f64
More GitHub Packages bulk upload fixes
- replace invalid non-ASCII characters from the descriptions
- handle invalid macOS versions on attempting to load old formulae from
  old bottles
- let `rubocop` auto-remove a no-longer needed `rubocop:disable`
- fix `skopeo` credentials argument
2021-04-07 17:03:44 +01:00
Mike McQuaid
c141dd4480
github_packages: don't allow linuxbrew- prefix.
Given https://github.com/Homebrew/linuxbrew-core/pull/22874 we no longer
need/want to allow a `linuxbrew-` repository prefix.
2021-04-07 16:34:59 +01:00
Nanda H Krishna
11de11025b
github_packages: fix image_formula_name issue 2021-04-07 19:33:16 +05:30
Mike McQuaid
4613d368d9
Merge pull request #11054 from MikeMcQuaid/github_packages_image_tag_cleanup
github_packages: remove invalid docker tag characters.
2021-04-07 14:00:01 +01:00
Mike McQuaid
d707c0bbd8
github_packages: remove invalid docker tag characters.
Some versions have `+` in them.
2021-04-07 13:32:31 +01:00
Mike McQuaid
ee491928e4
github_packages: fix typo. 2021-04-07 13:20:24 +01:00
Mike McQuaid
b0ba92466f
github_packages: fix more bulk upload errors.
- replace `+` in formula names (because it's an invalid character)
- do a `skopeo inspect` first to avoid overwriting existing packages
2021-04-07 12:48:34 +01:00
Mike McQuaid
fd251c9109
github_packages: fix versioned bottle names.
`@` cannot be used in Docker image names. Use `/` instead (which we
already use in image names so has some precedent).

Make `mktemp` use `AT` (consistent with `Formula` subclasses), too.
2021-04-06 19:14:08 +01:00
Mike McQuaid
480d943035
Merge pull request #11036 from sjackman/sj/ghcr-compact
GHCR: Use reject to remove empty hash values
2021-04-06 08:52:55 +01:00
Shaun Jackman
d6c0bbc7f4 GHCR: Add constant OS::CI_OS_VERSION
Rename constant OS::GLIBC_CI_VERSION to OS::CI_GLIBC_VERSION.
2021-04-05 22:39:41 -07:00
Shaun Jackman
2852d9f0de
GHCR: Use reject to remove empty hash values 2021-04-05 10:08:03 -07:00
Mike McQuaid
cdaeee03c4
GitHub Packages cleanup
- `download_strategy`: only request image index JSON for downloading
  the manifest for the tab
- use a shared `OS` constant for the version of `glibc` we use in CI
- fix `skoepeo` typo
- ensure that blank hash values are deleted (again) rather than just
  `nil` ones
- use a shared `Hardware::CPU` constant for oldest CPU we're
  supporting/using on Intel 64-bit
- re-add comment to `software_spec`
2021-04-05 14:58:17 +01:00