Write tabs to bottle JSON, optionally not bottle

- Write a subset of the tab required for bottles as an annotation.
- Add option on new bottle creation to skip writing tab into bottle
  and instead add it (and other useful metadata) to bottle JSON.
- Read formula information and tab from bottle JSON.
- Write prettier JSON to disk.
- Don't write `HEAD` to tab; this duplicates `HOMEBREW_VERSION`.
- Allow `brew bottle` to use `--json` to generate JSON files from a
  local bottle file.
This commit is contained in:
Mike McQuaid 2021-03-30 17:35:13 +01:00
parent 6be0b57a82
commit d8a2cf9efc
No known key found for this signature in database
GPG Key ID: 48A898132FD8EE70
18 changed files with 222 additions and 112 deletions

View File

@ -14,7 +14,7 @@ Lint/NestedMethodDefinition:
# TODO: Try to bring down all metrics maximums. # TODO: Try to bring down all metrics maximums.
Metrics/AbcSize: Metrics/AbcSize:
Max: 250 Max: 275
Metrics/BlockLength: Metrics/BlockLength:
Max: 100 Max: 100
Exclude: Exclude:

View File

@ -64,7 +64,7 @@ module Homebrew
def stale_formula?(scrub) def stale_formula?(scrub)
return false unless HOMEBREW_CELLAR.directory? return false unless HOMEBREW_CELLAR.directory?
version = if to_s.match?(Pathname::BOTTLE_EXTNAME_RX) version = if HOMEBREW_BOTTLES_EXTNAME_REGEX.match?(to_s)
begin begin
Utils::Bottles.resolve_version(self) Utils::Bottles.resolve_version(self)
rescue rescue

View File

@ -70,6 +70,9 @@ module Homebrew
depends_on: "--write", depends_on: "--write",
description: "When passed with `--write`, a new commit will not generated after writing changes "\ description: "When passed with `--write`, a new commit will not generated after writing changes "\
"to the formula file." "to the formula file."
switch "--only-json-tab",
depends_on: "--json",
description: "When passed with `--json`, the tab will be written to the JSON file but not the bottle."
flag "--root-url=", flag "--root-url=",
description: "Use the specified <URL> as the root of the bottle's URL instead of Homebrew's default." description: "Use the specified <URL> as the root of the bottle's URL instead of Homebrew's default."
@ -252,9 +255,15 @@ module Homebrew
end end
def bottle_formula(f, args:) def bottle_formula(f, args:)
return ofail "Formula not installed or up-to-date: #{f.full_name}" unless f.latest_version_installed? local_bottle_json = args.json? && f.local_bottle_path.present?
unless (tap = f.tap) unless local_bottle_json
return ofail "Formula not installed or up-to-date: #{f.full_name}" unless f.latest_version_installed?
return ofail "Formula was not installed with --build-bottle: #{f.full_name}" unless Utils::Bottles.built_as? f
end
tap = f.tap
if tap.nil?
return ofail "Formula not from core or any installed taps: #{f.full_name}" unless args.force_core_tap? return ofail "Formula not from core or any installed taps: #{f.full_name}" unless args.force_core_tap?
tap = CoreTap.instance tap = CoreTap.instance
@ -266,39 +275,95 @@ module Homebrew
return return
end end
return ofail "Formula was not installed with --build-bottle: #{f.full_name}" unless Utils::Bottles.built_as? f
return ofail "Formula has no stable version: #{f.full_name}" unless f.stable return ofail "Formula has no stable version: #{f.full_name}" unless f.stable
if args.no_rebuild? || !f.tap bottle_tag, rebuild = if local_bottle_json
rebuild = 0 _, tag_string, rebuild_string = Utils::Bottles.extname_tag_rebuild(f.local_bottle_path.to_s)
[tag_string.to_sym, rebuild_string.to_i]
end
bottle_tag ||= Utils::Bottles.tag
rebuild ||= if args.no_rebuild? || !tap
0
elsif args.keep_old? elsif args.keep_old?
rebuild = f.bottle_specification.rebuild f.bottle_specification.rebuild
else else
ohai "Determining #{f.full_name} bottle rebuild..." ohai "Determining #{f.full_name} bottle rebuild..."
versions = FormulaVersions.new(f) versions = FormulaVersions.new(f)
rebuilds = versions.bottle_version_map("origin/master")[f.pkg_version] rebuilds = versions.bottle_version_map("origin/master")[f.pkg_version]
rebuilds.pop if rebuilds.last.to_i.positive? rebuilds.pop if rebuilds.last.to_i.positive?
rebuild = rebuilds.empty? ? 0 : rebuilds.max.to_i + 1 rebuilds.empty? ? 0 : rebuilds.max.to_i + 1
end end
filename = Bottle::Filename.create(f, Utils::Bottles.tag, rebuild) filename = Bottle::Filename.create(f, bottle_tag, rebuild)
local_filename = filename.to_s
bottle_path = Pathname.pwd/filename bottle_path = Pathname.pwd/filename
tar_filename = filename.to_s.sub(/.gz$/, "") tab = nil
tar_path = Pathname.pwd/tar_filename keg = nil
tap_path = tap.path
tap_git_revision = tap.git_head
tap_git_remote = tap.remote
root_url = args.root_url
formulae_brew_sh_path = Utils::Analytics.formula_path
relocatable = T.let(false, T::Boolean)
skip_relocation = T.let(false, T::Boolean)
prefix = HOMEBREW_PREFIX.to_s prefix = HOMEBREW_PREFIX.to_s
cellar = HOMEBREW_CELLAR.to_s cellar = HOMEBREW_CELLAR.to_s
ohai "Bottling #{filename}..." if local_bottle_json
bottle_path = f.local_bottle_path
local_filename = bottle_path.basename.to_s
tab_path = Utils::Bottles.receipt_path(f.local_bottle_path)
tab_json = Utils.safe_popen_read("tar", "xfO", f.local_bottle_path, tab_path)
tab = Tab.from_file_content(tab_json, tab_path)
# TODO: most of this logic can be removed when we're done with bulk GitHub Packages bottle uploading
tap_git_revision = tab["source"]["tap_git_head"]
if tap.core_tap?
if bottle_tag.to_s.end_with?("_linux")
tap_git_remote = "https://github.com/Homebrew/linuxbrew-core"
formulae_brew_sh_path = "formula-linux"
else
tap_git_remote = "https://github.com/Homebrew/homebrew-core"
formulae_brew_sh_path = "formula"
end
end
_, _, bottle_cellar = Formula[f.name].bottle_specification.checksum_for(bottle_tag, exact: true)
relocatable = [:any, :any_skip_relocation].include?(bottle_cellar)
skip_relocation = bottle_cellar == :any_skip_relocation
if bottle_tag.to_s.end_with?("_linux")
prefix = HOMEBREW_LINUX_DEFAULT_PREFIX.to_s
cellar = Homebrew::DEFAULT_LINUX_CELLAR
elsif bottle_tag.to_s.start_with?("arm64_")
prefix = HOMEBREW_MACOS_ARM_DEFAULT_PREFIX.to_s
cellar = Homebrew::DEFAULT_MACOS_ARM_CELLAR
else
prefix = HOMEBREW_DEFAULT_PREFIX.to_s
cellar = Homebrew::DEFAULT_MACOS_CELLAR
end
else
tar_filename = filename.to_s.sub(/.gz$/, "")
tar_path = Pathname.pwd/tar_filename
keg = Keg.new(f.prefix)
end
ohai "Bottling #{local_filename}..."
formula_and_runtime_deps_names = [f.name] + f.runtime_dependencies.map(&:name) formula_and_runtime_deps_names = [f.name] + f.runtime_dependencies.map(&:name)
keg = Keg.new(f.prefix)
relocatable = T.let(false, T::Boolean)
skip_relocation = T.let(false, T::Boolean)
keg.lock do # this will be nil when using a local bottle
keg&.lock do
original_tab = nil original_tab = nil
changed_files = nil changed_files = nil
@ -318,7 +383,11 @@ module Homebrew
tab.HEAD = nil tab.HEAD = nil
tab.time = nil tab.time = nil
tab.changed_files = changed_files tab.changed_files = changed_files
tab.write if args.only_json_tab?
tab.tabfile.unlink
else
tab.write
end
keg.find do |file| keg.find do |file|
if file.symlink? if file.symlink?
@ -342,7 +411,7 @@ module Homebrew
mv "#{relocatable_tar_path}.gz", bottle_path mv "#{relocatable_tar_path}.gz", bottle_path
end end
ohai "Detecting if #{filename} is relocatable..." if bottle_path.size > 1 * 1024 * 1024 ohai "Detecting if #{local_filename} is relocatable..." if bottle_path.size > 1 * 1024 * 1024
prefix_check = if Homebrew.default_prefix?(prefix) prefix_check = if Homebrew.default_prefix?(prefix)
File.join(prefix, "opt") File.join(prefix, "opt")
@ -400,8 +469,6 @@ module Homebrew
end end
end end
root_url = args.root_url
bottle = BottleSpecification.new bottle = BottleSpecification.new
bottle.tap = tap bottle.tap = tap
bottle.root_url(root_url) if root_url bottle.root_url(root_url) if root_url
@ -417,7 +484,7 @@ module Homebrew
end end
bottle.rebuild rebuild bottle.rebuild rebuild
sha256 = bottle_path.sha256 sha256 = bottle_path.sha256
bottle.sha256 sha256 => Utils::Bottles.tag bottle.sha256 sha256 => bottle_tag
old_spec = f.bottle_specification old_spec = f.bottle_specification
if args.keep_old? && !old_spec.checksums.empty? if args.keep_old? && !old_spec.checksums.empty?
@ -448,7 +515,7 @@ module Homebrew
output = bottle_output bottle output = bottle_output bottle
puts "./#{filename}" puts "./#{local_filename}"
puts output puts output
return unless args.json? return unless args.json?
@ -456,8 +523,15 @@ module Homebrew
json = { json = {
f.full_name => { f.full_name => {
"formula" => { "formula" => {
"pkg_version" => f.pkg_version.to_s, "name" => f.name,
"path" => f.path.to_s.delete_prefix("#{HOMEBREW_REPOSITORY}/"), "pkg_version" => f.pkg_version.to_s,
"path" => f.path.to_s.delete_prefix("#{HOMEBREW_REPOSITORY}/"),
"tap_git_path" => f.path.to_s.delete_prefix("#{tap_path}/"),
"tap_git_revision" => tap_git_revision,
"tap_git_remote" => tap_git_remote,
"desc" => f.desc,
"license" => f.license,
"homepage" => f.homepage,
}, },
"bottle" => { "bottle" => {
"root_url" => bottle.root_url, "root_url" => bottle.root_url,
@ -465,10 +539,12 @@ module Homebrew
"cellar" => bottle.cellar.to_s, "cellar" => bottle.cellar.to_s,
"rebuild" => bottle.rebuild, "rebuild" => bottle.rebuild,
"tags" => { "tags" => {
Utils::Bottles.tag.to_s => { bottle_tag.to_s => {
"filename" => filename.bintray, "filename" => filename.bintray,
"local_filename" => filename.to_s, "local_filename" => local_filename,
"sha256" => sha256, "sha256" => sha256,
"formulae_brew_sh_path" => formulae_brew_sh_path,
"tab" => tab.to_bottle_hash,
}, },
}, },
}, },
@ -479,7 +555,7 @@ module Homebrew
}, },
} }
File.open(filename.json, "w") do |file| File.open(filename.json, "w") do |file|
file.write JSON.generate json file.write JSON.pretty_generate json
end end
end end

View File

@ -92,6 +92,12 @@ module Homebrew
hash.deep_merge(JSON.parse(IO.read(json_file))) hash.deep_merge(JSON.parse(IO.read(json_file)))
end end
if args.root_url
bottles_hash.each_value do |bottle_hash|
bottle_hash["bottle"]["root_url"] = args.root_url
end
end
bottle_args = ["bottle", "--merge", "--write"] bottle_args = ["bottle", "--merge", "--write"]
bottle_args << "--verbose" if args.verbose? bottle_args << "--verbose" if args.verbose?
bottle_args << "--debug" if args.debug? bottle_args << "--debug" if args.debug?

View File

@ -538,8 +538,8 @@ class CurlGitHubPackagesDownloadStrategy < CurlDownloadStrategy
private private
def _fetch(url:, resolved_url:) def _fetch(url:, resolved_url:)
raise "Empty checksum" if checksum.blank? raise CurlDownloadStrategyError, "Empty checksum" if checksum.blank?
raise "Empty name" if name.blank? raise CurlDownloadStrategyError, "Empty name" if name.blank?
_, org, repo, = *url.match(GitHubPackages::URL_REGEX) _, org, repo, = *url.match(GitHubPackages::URL_REGEX)

View File

@ -81,9 +81,6 @@ class Pathname
include DiskUsageExtension include DiskUsageExtension
# @private
BOTTLE_EXTNAME_RX = /(\.[a-z0-9_]+\.bottle\.(\d+\.)?tar\.gz)$/.freeze
# Moves a file from the original location to the {Pathname}'s. # Moves a file from the original location to the {Pathname}'s.
sig { sig {
params(sources: T.any( params(sources: T.any(
@ -243,7 +240,7 @@ class Pathname
def extname def extname
basename = File.basename(self) basename = File.basename(self)
bottle_ext = basename[BOTTLE_EXTNAME_RX, 1] bottle_ext, = HOMEBREW_BOTTLES_EXTNAME_REGEX.match(basename).to_a
return bottle_ext if bottle_ext return bottle_ext if bottle_ext
archive_ext = basename[/(\.(tar|cpio|pax)\.(gz|bz2|lz|xz|Z))\Z/, 1] archive_ext = basename[/(\.(tar|cpio|pax)\.(gz|bz2|lz|xz|Z))\Z/, 1]

View File

@ -1139,14 +1139,21 @@ class FormulaInstaller
) )
end end
tab.tap = formula.tap # fill in missing/outdated parts of the tab
# keep in sync with Tab#to_bottle_json
tab.used_options = []
tab.unused_options = []
tab.built_as_bottle = true
tab.poured_from_bottle = true tab.poured_from_bottle = true
tab.time = Time.now.to_i
tab.head = HOMEBREW_REPOSITORY.git_head
tab.source["path"] = formula.specified_path.to_s
tab.installed_as_dependency = installed_as_dependency? tab.installed_as_dependency = installed_as_dependency?
tab.installed_on_request = installed_on_request? tab.installed_on_request = installed_on_request?
tab.time = Time.now.to_i
tab.aliases = formula.aliases tab.aliases = formula.aliases
tab.arch = Hardware::CPU.arch
tab.source["versions"]["stable"] = formula.stable.version.to_s
tab.source["path"] = formula.specified_path.to_s
tab.source["tap_git_head"] = formula.tap&.git_head
tab.tap = formula.tap
tab.write tab.write
end end

View File

@ -4,6 +4,7 @@
require "digest/md5" require "digest/md5"
require "extend/cachable" require "extend/cachable"
require "tab" require "tab"
require "utils/bottles"
# The {Formulary} is responsible for creating instances of {Formula}. # The {Formulary} is responsible for creating instances of {Formula}.
# It is not meant to be used directly from formulae. # It is not meant to be used directly from formulae.
@ -458,7 +459,7 @@ module Formulary
def self.loader_for(ref, from: nil) def self.loader_for(ref, from: nil)
case ref case ref
when Pathname::BOTTLE_EXTNAME_RX when HOMEBREW_BOTTLES_EXTNAME_REGEX
return BottleLoader.new(ref) return BottleLoader.new(ref)
when URL_START_REGEX when URL_START_REGEX
return FromUrlLoader.new(ref) return FromUrlLoader.new(ref)

View File

@ -51,8 +51,8 @@ class GitHubPackages
load_schemas! load_schemas!
bottles_hash.each_value do |bottle_hash| bottles_hash.each do |formula_full_name, bottle_hash|
upload_bottle(user, token, skopeo, bottle_hash, dry_run: dry_run) upload_bottle(user, token, skopeo, formula_full_name, bottle_hash, dry_run: dry_run)
end end
end end
@ -119,10 +119,8 @@ class GitHubPackages
exit 1 exit 1
end end
def upload_bottle(user, token, skopeo, bottle_hash, dry_run:) def upload_bottle(user, token, skopeo, formula_full_name, bottle_hash, dry_run:)
formula_path = HOMEBREW_REPOSITORY/bottle_hash["formula"]["path"] formula_name = bottle_hash["formula"]["name"]
formula = Formulary.factory(formula_path)
formula_name = formula.name
_, org, repo, = *bottle_hash["bottle"]["root_url"].match(URL_REGEX) _, org, repo, = *bottle_hash["bottle"]["root_url"].match(URL_REGEX)
@ -139,27 +137,27 @@ class GitHubPackages
blobs = root/"blobs/sha256" blobs = root/"blobs/sha256"
blobs.mkpath blobs.mkpath
# TODO: ideally most/all of these attributes would be stored in the git_revision = bottle_hash["formula"]["tap_git_head"]
# bottle JSON rather than reading them from the formula. git_path = bottle_hash["formula"]["tap_git_path"]
git_revision = formula.tap.git_head
git_path = formula_path.to_s.delete_prefix("#{formula.tap.path}/")
source = "https://github.com/#{org}/#{repo}/blob/#{git_revision}/#{git_path}" source = "https://github.com/#{org}/#{repo}/blob/#{git_revision}/#{git_path}"
documentation = if formula.tap.core_tap?
formula_core_tap = formula_full_name.exclude?("/")
documentation = if formula_core_tap
"https://formulae.brew.sh/formula/#{formula_name}" "https://formulae.brew.sh/formula/#{formula_name}"
elsif (remote = formula.tap.remote) && remote.start_with?("https://github.com/") elsif (remote = bottle_hash["formula"]["tap_git_remote"]) && remote.start_with?("https://github.com/")
remote remote
end end
formula_annotations_hash = { formula_annotations_hash = {
"org.opencontainers.image.created" => Time.now.strftime("%F"), "org.opencontainers.image.created" => Time.now.strftime("%F"),
"org.opencontainers.image.description" => formula.desc, "org.opencontainers.image.description" => bottle_hash["formula"]["desc"],
"org.opencontainers.image.documentation" => documentation, "org.opencontainers.image.documentation" => documentation,
"org.opencontainers.image.license" => formula.license, "org.opencontainers.image.license" => bottle_hash["formula"]["license"],
"org.opencontainers.image.ref.name" => version_rebuild, "org.opencontainers.image.ref.name" => version_rebuild,
"org.opencontainers.image.revision" => git_revision, "org.opencontainers.image.revision" => git_revision,
"org.opencontainers.image.source" => source, "org.opencontainers.image.source" => source,
"org.opencontainers.image.title" => formula.full_name, "org.opencontainers.image.title" => formula_full_name,
"org.opencontainers.image.url" => formula.homepage, "org.opencontainers.image.url" => bottle_hash["formula"]["homepage"],
"org.opencontainers.image.vendor" => org, "org.opencontainers.image.vendor" => org,
"org.opencontainers.image.version" => version, "org.opencontainers.image.version" => version,
} }
@ -167,42 +165,17 @@ class GitHubPackages
formula_annotations_hash.delete(key) if value.blank? formula_annotations_hash.delete(key) if value.blank?
end end
created_times = []
manifests = bottle_hash["bottle"]["tags"].map do |bottle_tag, tag_hash| manifests = bottle_hash["bottle"]["tags"].map do |bottle_tag, tag_hash|
local_file = tag_hash["local_filename"] local_file = tag_hash["local_filename"]
odebug "Uploading #{local_file}" odebug "Uploading #{local_file}"
tar_gz_sha256 = write_tar_gz(local_file, blobs) tar_gz_sha256 = write_tar_gz(local_file, blobs)
tab = Tab.from_file_content( tab = tag_hash["tab"]
Utils.safe_popen_read("tar", "xfO", local_file, "#{formula_name}/#{version}/INSTALL_RECEIPT.json"),
"#{local_file}/#{formula_name}/#{version}",
)
os_version = if tab.built_on.present?
/(\d+\.)*\d+/ =~ tab.built_on["os_version"]
Regexp.last_match(0)
end
# TODO: ideally most/all of these attributes would be stored in the
# bottle JSON rather than reading them from the formula.
os, arch, formulae_dir = if bottle_tag.to_s.end_with?("_linux")
["linux", "amd64", "formula-linux"]
else
os = "darwin"
macos_version = MacOS::Version.from_symbol(bottle_tag.to_sym)
os_version ||= macos_version.to_f.to_s
arch = if macos_version.arch == :arm64
"arm64"
else
"amd64"
end
[os, arch, "formula"]
end
platform_hash = { platform_hash = {
architecture: arch, architecture: tab["arch"],
os: os, os: tab["built_on"]["os"],
"os.version" => os_version, "os.version" => tab["built_on"]["os_version"],
} }
tar_sha256 = Digest::SHA256.hexdigest( tar_sha256 = Digest::SHA256.hexdigest(
Utils.safe_popen_read("gunzip", "--stdout", "--decompress", local_file), Utils.safe_popen_read("gunzip", "--stdout", "--decompress", local_file),
@ -210,18 +183,16 @@ class GitHubPackages
config_json_sha256, config_json_size = write_image_config(platform_hash, tar_sha256, blobs) config_json_sha256, config_json_size = write_image_config(platform_hash, tar_sha256, blobs)
created_time = tab.source_modified_time formulae_dir = tag_hash["formulae_brew_sh_path"]
created_time ||= Time.now documentation = "https://formulae.brew.sh/#{formulae_dir}/#{formula_name}" if formula_core_tap
created_times << created_time
documentation = "https://formulae.brew.sh/#{formulae_dir}/#{formula_name}" if formula.tap.core_tap?
tag = "#{version}.#{bottle_tag}#{rebuild}" tag = "#{version}.#{bottle_tag}#{rebuild}"
title = "#{formula.full_name} #{tag}"
annotations_hash = formula_annotations_hash.merge({ annotations_hash = formula_annotations_hash.merge({
"org.opencontainers.image.created" => created_time.strftime("%F"), "org.opencontainers.image.created" => Time.at(tag_hash["tab"]["source_modified_time"]).strftime("%F"),
"org.opencontainers.image.documentation" => documentation, "org.opencontainers.image.documentation" => documentation,
"org.opencontainers.image.ref.name" => tag, "org.opencontainers.image.ref.name" => tag,
"org.opencontainers.image.title" => title, "org.opencontainers.image.title" => "#{formula_full_name} #{tag}",
}).sort.to_h }).sort.to_h
annotations_hash.each do |key, value| annotations_hash.each do |key, value|
annotations_hash.delete(key) if value.blank? annotations_hash.delete(key) if value.blank?
@ -254,6 +225,8 @@ class GitHubPackages
platform: platform_hash, platform: platform_hash,
annotations: { annotations: {
"org.opencontainers.image.ref.name" => tag, "org.opencontainers.image.ref.name" => tag,
"sh.brew.bottle.checksum" => tar_gz_sha256,
"sh.brew.tab" => tab.to_json,
}, },
} }
end end
@ -263,7 +236,7 @@ class GitHubPackages
write_index_json(index_json_sha256, index_json_size, root) write_index_json(index_json_sha256, index_json_size, root)
# docker/skopeo insist on lowercase org ("repository name") # docker/skopeo insist on lowercase org ("repository name")
org_prefix = "#{URL_DOMAIN}/#{org.downcase}" org_prefix = "#{DOCKER_PREFIX}#{org.downcase}"
# remove redundant repo prefix for a shorter name # remove redundant repo prefix for a shorter name
package_name = "#{repo.delete_prefix("homebrew-")}/#{formula_name}" package_name = "#{repo.delete_prefix("homebrew-")}/#{formula_name}"
image_tag = "#{org_prefix}/#{package_name}:#{version_rebuild}" image_tag = "#{org_prefix}/#{package_name}:#{version_rebuild}"

View File

@ -71,6 +71,7 @@ HOMEBREW_PULL_API_REGEX =
%r{https://api\.github\.com/repos/([\w-]+)/([\w-]+)?/pulls/(\d+)}.freeze %r{https://api\.github\.com/repos/([\w-]+)/([\w-]+)?/pulls/(\d+)}.freeze
HOMEBREW_PULL_OR_COMMIT_URL_REGEX = HOMEBREW_PULL_OR_COMMIT_URL_REGEX =
%r[https://github\.com/([\w-]+)/([\w-]+)?/(?:pull/(\d+)|commit/[0-9a-fA-F]{4,40})].freeze %r[https://github\.com/([\w-]+)/([\w-]+)?/(?:pull/(\d+)|commit/[0-9a-fA-F]{4,40})].freeze
HOMEBREW_BOTTLES_EXTNAME_REGEX = /\.([a-z0-9_]+)\.bottle\.(?:(\d+)\.)?tar\.gz$/.freeze
require "fileutils" require "fileutils"

View File

@ -33,17 +33,17 @@ class Tab < OpenStruct
"poured_from_bottle" => false, "poured_from_bottle" => false,
"time" => Time.now.to_i, "time" => Time.now.to_i,
"source_modified_time" => formula.source_modified_time.to_i, "source_modified_time" => formula.source_modified_time.to_i,
"HEAD" => HOMEBREW_REPOSITORY.git_head,
"compiler" => compiler, "compiler" => compiler,
"stdlib" => stdlib, "stdlib" => stdlib,
"aliases" => formula.aliases, "aliases" => formula.aliases,
"runtime_dependencies" => Tab.runtime_deps_hash(runtime_deps), "runtime_dependencies" => Tab.runtime_deps_hash(runtime_deps),
"arch" => Hardware::CPU.arch, "arch" => Hardware::CPU.arch,
"source" => { "source" => {
"path" => formula.specified_path.to_s, "path" => formula.specified_path.to_s,
"tap" => formula.tap&.name, "tap" => formula.tap&.name,
"spec" => formula.active_spec_sym.to_s, "tap_git_head" => formula.tap&.git_head,
"versions" => { "spec" => formula.active_spec_sym.to_s,
"versions" => {
"stable" => formula.stable&.version.to_s, "stable" => formula.stable&.version.to_s,
"head" => formula.head&.version.to_s, "head" => formula.head&.version.to_s,
"version_scheme" => formula.version_scheme, "version_scheme" => formula.version_scheme,
@ -188,17 +188,17 @@ class Tab < OpenStruct
"poured_from_bottle" => false, "poured_from_bottle" => false,
"time" => nil, "time" => nil,
"source_modified_time" => 0, "source_modified_time" => 0,
"HEAD" => nil,
"stdlib" => nil, "stdlib" => nil,
"compiler" => DevelopmentTools.default_compiler, "compiler" => DevelopmentTools.default_compiler,
"aliases" => [], "aliases" => [],
"runtime_dependencies" => nil, "runtime_dependencies" => nil,
"arch" => nil, "arch" => nil,
"source" => { "source" => {
"path" => nil, "path" => nil,
"tap" => nil, "tap" => nil,
"spec" => "stable", "tap_git_head" => nil,
"versions" => { "spec" => "stable",
"versions" => {
"stable" => nil, "stable" => nil,
"head" => nil, "head" => nil,
"version_scheme" => 0, "version_scheme" => 0,
@ -330,17 +330,33 @@ class Tab < OpenStruct
"changed_files" => changed_files&.map(&:to_s), "changed_files" => changed_files&.map(&:to_s),
"time" => time, "time" => time,
"source_modified_time" => source_modified_time.to_i, "source_modified_time" => source_modified_time.to_i,
"HEAD" => self.HEAD,
"stdlib" => stdlib&.to_s, "stdlib" => stdlib&.to_s,
"compiler" => compiler&.to_s, "compiler" => compiler&.to_s,
"aliases" => aliases, "aliases" => aliases,
"runtime_dependencies" => runtime_dependencies, "runtime_dependencies" => runtime_dependencies,
"source" => source, "source" => source,
"arch" => Hardware::CPU.arch, "arch" => arch,
"built_on" => built_on, "built_on" => built_on,
} }
attributes.delete("stdlib") if attributes["stdlib"].blank?
JSON.generate(attributes, options) JSON.pretty_generate(attributes, options)
end
# a subset of to_json that we care about for bottles
def to_bottle_hash
attributes = {
"homebrew_version" => homebrew_version,
"changed_files" => changed_files&.map(&:to_s),
"source_modified_time" => source_modified_time.to_i,
"stdlib" => stdlib&.to_s,
"compiler" => compiler&.to_s,
"runtime_dependencies" => runtime_dependencies,
"arch" => arch,
"built_on" => built_on,
}
attributes.delete("stdlib") if attributes["stdlib"].blank?
attributes
end end
def write def write

View File

@ -42,6 +42,8 @@ describe Tab do
"head" => "HEAD-1111111", "head" => "HEAD-1111111",
}, },
}, },
"arch" => Hardware::CPU.arch,
"built_on" => DevelopmentTools.build_system_info,
) )
} }
@ -360,6 +362,7 @@ describe Tab do
specify "#to_json" do specify "#to_json" do
json_tab = described_class.new(JSON.parse(tab.to_json)) json_tab = described_class.new(JSON.parse(tab.to_json))
expect(json_tab.homebrew_version).to eq(tab.homebrew_version)
expect(json_tab.used_options.sort).to eq(tab.used_options.sort) expect(json_tab.used_options.sort).to eq(tab.used_options.sort)
expect(json_tab.unused_options.sort).to eq(tab.unused_options.sort) expect(json_tab.unused_options.sort).to eq(tab.unused_options.sort)
expect(json_tab.built_as_bottle).to eq(tab.built_as_bottle) expect(json_tab.built_as_bottle).to eq(tab.built_as_bottle)
@ -368,13 +371,26 @@ describe Tab do
expect(json_tab.tap).to eq(tab.tap) expect(json_tab.tap).to eq(tab.tap)
expect(json_tab.spec).to eq(tab.spec) expect(json_tab.spec).to eq(tab.spec)
expect(json_tab.time).to eq(tab.time) expect(json_tab.time).to eq(tab.time)
expect(json_tab.HEAD).to eq(tab.HEAD)
expect(json_tab.compiler).to eq(tab.compiler) expect(json_tab.compiler).to eq(tab.compiler)
expect(json_tab.stdlib).to eq(tab.stdlib) expect(json_tab.stdlib).to eq(tab.stdlib)
expect(json_tab.runtime_dependencies).to eq(tab.runtime_dependencies) expect(json_tab.runtime_dependencies).to eq(tab.runtime_dependencies)
expect(json_tab.stable_version).to eq(tab.stable_version) expect(json_tab.stable_version).to eq(tab.stable_version)
expect(json_tab.head_version).to eq(tab.head_version) expect(json_tab.head_version).to eq(tab.head_version)
expect(json_tab.source["path"]).to eq(tab.source["path"]) expect(json_tab.source["path"]).to eq(tab.source["path"])
expect(json_tab.arch).to eq(tab.arch.to_s)
expect(json_tab.built_on["os"]).to eq(tab.built_on["os"])
end
specify "#to_bottle_hash" do
json_tab = described_class.new(JSON.parse(tab.to_bottle_hash.to_json))
expect(json_tab.homebrew_version).to eq(tab.homebrew_version)
expect(json_tab.changed_files).to eq(tab.changed_files)
expect(json_tab.source_modified_time).to eq(tab.source_modified_time)
expect(json_tab.stdlib).to eq(tab.stdlib)
expect(json_tab.compiler).to eq(tab.compiler)
expect(json_tab.runtime_dependencies).to eq(tab.runtime_dependencies)
expect(json_tab.arch).to eq(tab.arch.to_s)
expect(json_tab.built_on["os"]).to eq(tab.built_on["os"])
end end
specify "::remap_deprecated_options" do specify "::remap_deprecated_options" do

View File

@ -24,14 +24,22 @@ module Utils
def file_outdated?(f, file) def file_outdated?(f, file)
filename = file.basename.to_s filename = file.basename.to_s
return if f.bottle.blank? || !filename.match?(Pathname::BOTTLE_EXTNAME_RX) return false if f.bottle.blank?
bottle_ext = filename[native_regex, 1] bottle_ext, bottle_tag, = extname_tag_rebuild(filename)
bottle_url_ext = f.bottle.url[native_regex, 1] return false if bottle_ext.blank?
return false if bottle_tag != tag.to_s
bottle_url_ext, = extname_tag_rebuild(f.bottle.url)
bottle_ext && bottle_url_ext && bottle_ext != bottle_url_ext bottle_ext && bottle_url_ext && bottle_ext != bottle_url_ext
end end
def extname_tag_rebuild(filename)
HOMEBREW_BOTTLES_EXTNAME_REGEX.match(filename).to_a
end
# TODO: remove when removed from brew-test-bot
sig { returns(Regexp) } sig { returns(Regexp) }
def native_regex def native_regex
/(\.#{Regexp.escape(tag.to_s)}\.bottle\.(\d+\.)?tar\.gz)$/o /(\.#{Regexp.escape(tag.to_s)}\.bottle\.(\d+\.)?tar\.gz)$/o

View File

@ -380,6 +380,7 @@ _brew_bottle() {
--merge --merge
--no-commit --no-commit
--no-rebuild --no-rebuild
--only-json-tab
--quiet --quiet
--root-url --root-url
--skip-relocation --skip-relocation

View File

@ -365,6 +365,7 @@ __fish_brew_complete_arg 'bottle' -l keep-old -d 'If the formula specifies a reb
__fish_brew_complete_arg 'bottle' -l merge -d 'Generate an updated bottle block for a formula and optionally merge it into the formula file. Instead of a formula name, requires the path to a JSON file generated with `brew bottle --json` formula' __fish_brew_complete_arg 'bottle' -l merge -d 'Generate an updated bottle block for a formula and optionally merge it into the formula file. Instead of a formula name, requires the path to a JSON file generated with `brew bottle --json` formula'
__fish_brew_complete_arg 'bottle' -l no-commit -d 'When passed with `--write`, a new commit will not generated after writing changes to the formula file' __fish_brew_complete_arg 'bottle' -l no-commit -d 'When passed with `--write`, a new commit will not generated after writing changes to the formula file'
__fish_brew_complete_arg 'bottle' -l no-rebuild -d 'If the formula specifies a rebuild version, remove it from the generated DSL' __fish_brew_complete_arg 'bottle' -l no-rebuild -d 'If the formula specifies a rebuild version, remove it from the generated DSL'
__fish_brew_complete_arg 'bottle' -l only-json-tab -d 'When passed with `--json`, the tab will be written to the JSON file but not the bottle'
__fish_brew_complete_arg 'bottle' -l quiet -d 'Make some output more quiet' __fish_brew_complete_arg 'bottle' -l quiet -d 'Make some output more quiet'
__fish_brew_complete_arg 'bottle' -l root-url -d 'Use the specified URL as the root of the bottle\'s URL instead of Homebrew\'s default' __fish_brew_complete_arg 'bottle' -l root-url -d 'Use the specified URL as the root of the bottle\'s URL instead of Homebrew\'s default'
__fish_brew_complete_arg 'bottle' -l skip-relocation -d 'Do not check if the bottle can be marked as relocatable' __fish_brew_complete_arg 'bottle' -l skip-relocation -d 'Do not check if the bottle can be marked as relocatable'

View File

@ -453,6 +453,7 @@ _brew_bottle() {
'--merge[Generate an updated bottle block for a formula and optionally merge it into the formula file. Instead of a formula name, requires the path to a JSON file generated with `brew bottle --json` formula]' \ '--merge[Generate an updated bottle block for a formula and optionally merge it into the formula file. Instead of a formula name, requires the path to a JSON file generated with `brew bottle --json` formula]' \
'--no-commit[When passed with `--write`, a new commit will not generated after writing changes to the formula file]' \ '--no-commit[When passed with `--write`, a new commit will not generated after writing changes to the formula file]' \
'(--keep-old)--no-rebuild[If the formula specifies a rebuild version, remove it from the generated DSL]' \ '(--keep-old)--no-rebuild[If the formula specifies a rebuild version, remove it from the generated DSL]' \
'--only-json-tab[When passed with `--json`, the tab will be written to the JSON file but not the bottle]' \
'--quiet[Make some output more quiet]' \ '--quiet[Make some output more quiet]' \
'--root-url[Use the specified URL as the root of the bottle'\''s URL instead of Homebrew'\''s default]' \ '--root-url[Use the specified URL as the root of the bottle'\''s URL instead of Homebrew'\''s default]' \
'--skip-relocation[Do not check if the bottle can be marked as relocatable]' \ '--skip-relocation[Do not check if the bottle can be marked as relocatable]' \

View File

@ -823,6 +823,8 @@ value, while `--no-rebuild` will remove it.
Write changes to the formula file. A new commit will be generated unless `--no-commit` is passed. Write changes to the formula file. A new commit will be generated unless `--no-commit` is passed.
* `--no-commit`: * `--no-commit`:
When passed with `--write`, a new commit will not generated after writing changes to the formula file. When passed with `--write`, a new commit will not generated after writing changes to the formula file.
* `--only-json-tab`:
When passed with `--json`, the tab will be written to the JSON file but not the bottle.
* `--root-url`: * `--root-url`:
Use the specified *`URL`* as the root of the bottle's URL instead of Homebrew's default. Use the specified *`URL`* as the root of the bottle's URL instead of Homebrew's default.

View File

@ -1133,6 +1133,10 @@ Write changes to the formula file\. A new commit will be generated unless \fB\-\
When passed with \fB\-\-write\fR, a new commit will not generated after writing changes to the formula file\. When passed with \fB\-\-write\fR, a new commit will not generated after writing changes to the formula file\.
. .
.TP .TP
\fB\-\-only\-json\-tab\fR
When passed with \fB\-\-json\fR, the tab will be written to the JSON file but not the bottle\.
.
.TP
\fB\-\-root\-url\fR \fB\-\-root\-url\fR
Use the specified \fIURL\fR as the root of the bottle\'s URL instead of Homebrew\'s default\. Use the specified \fIURL\fR as the root of the bottle\'s URL instead of Homebrew\'s default\.
. .