What’s new in Bazel 6.0
Bazel 6.0 includes over 3,100 changes since 5.4.[1] It’s the latest major release, following the release of 5.0 in January of this year, and it’s Bazel’s third LTS release.
Since there were so many changes, many of them quite impactful, I felt I needed to review them all and provide a nice summary for y’all. Similar to Bazel 5.0’s changes, the end result was quite big, so I’ve included a table of contents to allow easy navigation to the changes that interest you the most:
Command-line flag changes
Bazel’s LTS strategy allows for breaking changes between major versions. In particular, it allows for command-line flags to be removed, renamed, made to do nothing, or have their default values changed. In the following sections I collected all such flag changes I could find.
Renamed
--experimental_build_transitive_python_runfilesis now--incompatible_build_transitive_python_runfiles. (36afffa)--experimental_debug_spawn_scheduleris now--debug_spawn_scheduler. (e38c73f)--experimental_desugar_java8_libsis now--desugar_java8_libs. (e38c73f)--experimental_enable_bzlmodis now--enable_bzlmod. (f106d5c)--experimental_local_execution_delayis now--dynamic_local_execution_delay. (e38c73f)--experimental_worker_max_multiplex_instancesis now--worker_max_multiplex_instances. (e38c73f)
Default Values Changed
--analysis_testing_deps_limit=1000(21dfe4c)--experimental_allow_unresolved_symlinks=true(3d5c5d7)--experimental_keep_config_nodes_on_analysis_discard=true(60523c7)--experimental_collect_local_sandbox_action_metrics=true(60523c7)--experimental_incremental_dexing_after_proguard=50(ce55639)--experimental_incremental_dexing_after_proguard_by_default=true(ce55639)--experimental_inmemory_dotd_files=true(d44f11b)--experimental_inmemory_jdeps_files=true(822e049)--experimental_keep_config_nodes_on_analysis_discard=false(2f9f842)--experimental_use_dex_splitter_for_incremental_dexing=true(ce55639)--incompatible_always_include_files_in_data=true(0caf488)--incompatible_enforce_config_setting_visibility=true--incompatible_existing_rules_immutable_view=true(74b7dd5)--incompatible_remote_results_ignore_disk=true(4c56431)--incompatible_use_platforms_repo_for_constraints=true(f137e64)--use_top_level_targets_for_symlinks=true(6452024)--use_workers_with_dexbuilder=true(0c3f528)
Deprecated
No-op
These flags now do nothing, but still exist to allow for migration off of them:
--distinct_host_configuration(78d0fc9)--dynamic_worker_strategy(db64e7e)--experimental_allow_top_level_aspects_parameters(bccbcbf)--experimental_dynamic_skip_first_build(db64e7e)--experimental_keep_config_nodes_on_analysis_discard(75bb463)--experimental_multi_cpu(85bfefe)--experimental_skyframe_prepare_analysis(f22e63d)--incompatible_disable_managed_directories(cbf8159)--incompatible_disable_third_party_license_checking(0aa750b)--incompatible_disallow_legacy_py_provider(f068b31)--incompatible_override_toolchain_transition(2d5375c)
Removed
--all_incompatible_changes(8e6c469)--analysis_warnings_as_errors(8e6c469)--default_android_platform(ed7a056)--experimental_delay_virtual_input_materialization(c887c2a)--experimental_enable_cc_toolchain_config_info(8e6c469)--experimental_local_memory_estimate(361ce67)--experimental_persistent_test_runner(1f33504)--experimental_profile_cpu_usage(8e6c469)--experimental_required_aspects(8e6c469)--experimental_shadowed_action(8e6c469)--experimental_skyframe_eval_with_ordered_list(8e6c469)--experimental_worker_allow_json_protocol(09df7c0)--extra_proguard_specs(ab51d2e)--forceJumbo(9cb551c)--incompatible_applicable_licenses(29e4aee)--incompatible_disable_depset_items(bf30d81)--incompatible_disable_late_bound_option_defaults(8e6c469)--incompatible_disable_legacy_proto_provider(8e6c469)--incompatible_disable_proto_source_root(8e6c469)--incompatible_do_not_emit_buggy_external_repo_import(8e6c469)--incompatible_enable_exports_provider(01a46f0)--incompatible_linkopts_to_linklibs(34ce6a2)--incompatible_proto_output_v2(6b6c63e)--incompatible_use_native_patch(8e6c469)--ios_cpu(684fb0a)--json_trace_compression(8e6c469)--master_bazelrc(96c8a90)--max-bytes-wasted-per-file(e846207)--minimal-main-dex(63ddfc4)--remote_allow_symlink_upload(4f557e8)--set-max-idx-number(e846207)--show_task_finish(2e48994)--ui(8e6c469)
Remote
One of Bazel’s most powerful features is its ability to use remote caching and remote execution. Numerous improvements and fixes to Bazel’s remote capabilities are included in Bazel 6.0.
Remote caching (RBC)
Using a remote cache is one of the most popular ways of speeding up a Bazel build. Thankfully these changes make using a remote cache both more performant and more reliable:
- Added the
--experimental_remote_downloader_local_fallbackflag, which causes the remote downloader to fallback to normal fetching. (7b141c1) - Added the
--incompatible_remote_downloader_send_all_headersflag, which causes all values of a multi-value header to be sent with Remote Downloader. (b750f8c) - The
--experimental_action_cache_store_output_metadataflag is now enabled by default when using the--remote_download_*flags. (e5f92a4) - Outputs downloaded with
--experimental_remote_download_regexno longer block spawn execution. (e01e7f5) --experimental_remote_download_regexnow matches on execution root relative paths. (bb8e6ec)- Top-level outputs are now always downloaded when using
--remote_download_toplevel. (b6f3111) bazel runnow works with--remote_download_minimal. (845077f)- When using Build without the Bytes, AC results in the disk cache are now validated to be complete. (21b0992)
- Outputs and stdErr are now always downloaded. (b303cd1)
- Symlinked targets are no longer unnecessarily downloaded. (32b0f5a)
- Downloads now use a priority system to improve critical path performance. (05b9739)
- HTTP remote caches now support client TLS authentication. (aaf65b9)
- Headers are no longer included in
FetchBlobRequest. (7aa69a9) - AC uploads will no longer error if
--remote_upload_local_results=trueandGetCapabilitiesreturnsupdate_enabled=false. (f589512) - Fixed hanging when failing to upload action inputs. (8b61746)
Remote execution (RBE)
For some projects, using remote execution is the ultimate performance unlock for their Bazel builds. In addition to the remote caching changes covered above, which also apply to remote execution, the following changes improve the remote execution experience:
- Added the
--experimental_remote_mark_tool_inputsflag, which marks tool inputs for remote executors, which in turn allows them to implement remote persistent workers. (72b481a) - Added the
--incompatible_remote_dangling_symlinksflag, which allows symlinks in action outputs to dangle. (5b46a48) - Added the
--remote_print_execution_messagesflag, which allows control over when to print remote execution messages. (8b57c58) - Added the
exclusive-if-localtag, for disabling parallel local test execution but allowing remote execution if requested. (8936828) - Added support for remotely generating unresolved symlinks. (ca95fec)
- Fixed formatting of non-ASCII text. (c55b01e)
Dynamic execution
Dynamic execution allows for Bazel to race remote and local execution of actions, potentially allowing you to get the benefit of both modes, without the drawbacks of either. Bazel 6.0 included a number of changes to dynamic execution, almost all of them behind new experimental flags:
- Added the
--experimental_dynamic_exclude_toolsflag, which prevents dynamic execution for tools. (9a52a27) - Added the
--experimental_dynamic_local_load_factorflag, which controls how much load from dynamic execution to put on the local machine. (d5c4f55) - Added the
--experimental_dynamic_slow_remote_timeflag, which starts locally scheduling actions above a certain age in dynamic scheduling. (88f605c) - Added the
--experimental_prioritize_local_actionsflag, which prioritizes local-only actions over dynamically executed actions for local resources. (8fa3ccb)
Local execution
In buck2, local execution is a specialization of remote execution. So I’m going to use that as the reason that this section is listed under the “Remote” section, and not because I don’t want to reorganize this post just to fit in a couple neat local execution changes 😉:
- Announced at BazelCon, SkyMeld allows merging the analysis and execution phases with the the
--experimental_merged_skyframe_analysis_executionand--experimental_skymeld_uiflags. - Local actions now fetch inputs before acquiring the resource lock. (17276d4)
Build Event Service (BES)
Using a build event service can give you unparalleled insight into your Bazel builds at scale. There were some nice changes to BES support, though I think the improvements to how it interacts with the remote cache are especially noteworthy.
- Added the
--bep_maximum_open_remote_upload_filesflag, which allows control over the maximum number of open files allowed during BEP artifact upload. (46104c6) - Added the
--bes_check_preceding_lifecycle_eventsflag, which tells BES to check whether it previously receivedInvocationAttemptStartedandBuildEnqueuedevents matching the current tool event. (14b5c41) - Added the
--experimental_build_event_upload_max_retriesand--experimental_build_event_upload_retry_minimum_delayflags, which allow for configuring the behavior of BES uploader retires. (e7218d5) - Added the
--experimental_remote_build_event_uploadflag, which controls the way Bazel uploads files referenced in BEP to remote caches. (6b52772) - BES RPC calls now include request metadata. (dbcf260)
Logging
Bazel offers various methods to gain insight into your build. I cover some of the notable changes to those methods below.
Build Event Protocol (BEP)
The build event protocol is used by build event services, so all of these changes could have also been listed in that section as well. The BEP can also be collected locally with --build_event_json_file and --build_event_binary_file.
The vast majority of changes added additional information to the BEP, though some are fixes and improvements:
- Added the
digestandlengthfields toFile. (da6d949) - Added the
BuildMetrics.network_metricsfield. (9bc9096) - Added the
MemoryMetrics.peak_post_gc_tenured_space_heap_sizefield. (a9ac2b6) - Added the
WorkerMetrics.last_action_start_time_in_msfield. (d233c89) - Changed the semantics of
build_event_stream.BuildMetrics.PackageMetrics.packages_loadedto be the number of packages successfully loaded. (d8c25fc) - Deprecated
AnomalyReport. (286fb80)
Timing profile
The action timing profile, which is enabled by default with --profile, is viewable both locally in Chrome and on build event services. These changes add more detail and clarity to the profile:
- Added action mnemonics. (e78fd2e)
- Added the
--experimental_collect_load_average_in_profilerflag, which adds the system’s overall load average. (b4dbed0) - Added the
--experimental_collect_system_network_usageflag, which adds system network usage. (e382cb2) - Added the
--experimental_collect_worker_data_in_profilerflag, which adds worker memory usage. (be5354b) - Improved reporting of critical path components. (3d2bb2a)
- Made the
sort_indexvalue always a string. (081f831)
Execution log
Bazel logs all of the spawns it executes in the execution log, which is enabled with the --execution_log_json_file or --execution_log_binary_file flags. This feature is relatively stable, with just a few noticeable additions:
- Added the
--experimental_execution_log_spawn_metricsflag, which causes spawn metrics in be included in the execution log. (b4b8b26) - Added the
SpawnExec.digestfield. (b2cbc9a) - Added the
SpawnExec.target_labelfield. (9f908ca)
Query
bazel build wasn’t the only command to get improvements in this release. Here are some changes that were made to the query family of commands:
- Added the
--incompatible_package_group_includes_double_slashflag, which removes the leading//frompackage_group.packageoutput. (1473988) - Added an optional second argument to the
someoperator, specifying number of nodes returned. (cc71db2) - Labels are decanonicalized if possible. (47b1cad)
cquery
- Added support for queries over incompatible targets. (73b22b6)
- Added
struct,json,proto, anddepsetto--output=starlark. (d693465) - Added
ConfiguredRuleInputwhen using--transitions, which reports the configuration dependencies are configured in. (9994c32) - Added more information about configurations to proto output. (29d46eb)
- Added the
Configuration.is_toolattribute to proto output. (fb92e2d) - Complete configurations are now included in proto output. (ac48e65)
--output=filesnow also outputs source files. (ca8674c)- Starlark transitions now report source code location. (5de9888)
- Updated
AnalysisProtosV2. (46a36d6)
aquery
- Added the
--include_file_write_contentsflag, which includes file contents for theFileWriteaction. (6d73b96) - Added the
Configuration.is_toolattribute to the proto output. (fb92e2d) - Fixed formatting of non-ASCII text. (c55b01e)
- Fixed non-deterministic sorting of execution requirements. (ec1ac2f)
Dependency management
A new dependency system named Bzlmod was added in Bazel 5.0, and made non-experimental in Bazel 6.0. Besides all of the changes needed to support Bzlmod, these were some other notable dependency management related changes:
- Added the
--experimental_check_external_repository_filesflag, which allows disabling checking for modifications to files in external repositories. (123da96) - Added the
build_fileandbuild_file_contentattributes togit_repository. (f5a899f) - Added the
add_prefixattribute tohttp_*rules. (87c8b09) - Added the
integrityattribute tohttp_fileandhttp_jar. (e51a15f) - Added the
workspace_rootattribute torepository_ctx. (8edf6ab) - Added the
successparameter torepository_ctx.download. (5af794b) - Added the
rename_filesparameter torepository_ctx.extract. (2b02416) - Added host arch to repository rule markers. (16c89c1)
--override_repositorynow accepts tildes. (cc55400)http_*rules now honour theNETRCenvironment variable. (a15f342)- Download progress now displays human readable bytes. (801e01c)
- Removed support for managed directories. (cbf8159)
- When Bzlmod is enabled, canonical label literals are used.
Platforms and toolchains
The C++, Android, and Apple rules are being migrated to support building with Platforms. While progress has been made, they don’t fully support it yet in Bazel 6.0. For C++ projects, it’s recommended that the --incompatible_enable_cc_toolchain_resolution flag is used, to help the Bazel team discover any issues in the wide variety of projects that exist.
Here are some of the platforms and toolchains related changes which weren’t tied to any of those migrations:
- The
hostconfiguration is deprecated, and a (6e6c4cf) lot (32fc7ca) of (790d7a7) changes (48e8868) were (715c9fa) made (d988e8b) to migrate away from it and to theexecconfiguration. - Added the
--incompatible_disable_starlark_host_transitionsflag, which prevents rules from using thehostconfiguration. (6464f1c) - Added the
--experimental_exec_configuration_distinguisherflag, which changes how theplatform_suffixis calculated. (51c90c7) - Added the
exec_groupargument totesting.ExecutionInfo(). (423fb20) - Toolchain dependencies can now be optional.(Issue #14726)
shpath is now selected based on execution platform instead of host platform, making it possible to executeshactions in multi-platform builds. (eeb2e04)- The
exectransition no longer resets anyTestConfiguration.TestOptionsoptions. (2adf003) platform.exec_propertiesnow become execution requirements. (e4c1c43)
Execution platforms
Execution platforms are platforms which build tools execute on. These include the host platform on which Bazel runs.
In the following sections I collected notable changes for Linux and macOS. I’m sure there were some for Windows as well, but since I don’t use Bazel on Windows, none of the changes stood out to me as pertaining only to it.
Linux
- Added the
--incompatible_sandbox_hermetic_tmpflag, which causes the sandbox to have its own dedicated empty directory mounted as/tmprather than sharing/tmpwith the host filesystem. (ae6a90a) - Added the
--sandbox-explicit-pseudoterminalflag, which allow processes in the sandbox to open pseudoterminals. (9a13051) - Fixed handling of large UIDs. (467f32d)
macOS
- Remote Xcode version is now matched more granuarly. (2ff4124)
clean --asyncis now available on macOS. (b8d0e26)- The
@bazel_tools//tools/cpp:compilerflag now has the valueclangfor the auto-configured Xcode toolchain rather than the generic valuecompiler. (f99319f) - Removed some aborts from
wrapped_clang. (3451774) - Fixed
llddetection on macOS. (d9f20dc)
Target platforms
Target platforms are platforms which you are ultimately building for. I cover the Android and Apple platforms in the following sections, as they still have some functionality provided by Bazel core, instead of being fully supported by standalone Starlark rules.
Android
- D8 is now the default desugarer. (ff311f6)
- D8 is now the default dexer. (66d07f0)
- The D8 jar from Maven is used instead of Android Studio’s. (ae24714)
- Added the
android_binary.min_sdk_versionattribute. (4c3219e) - Added the
ApkInfo.signing_min_v3_rotation_api_versionattribute. (f6dbd1e) - Added the
--mobile_install_run_deployerflag to themobile-installcommand, which allows skipping the deployer after building. (0216657) - Added the
--bytecode_optimization_pass_actionsflag, which allows splitting the optimization pass into N parts. (2b44482) - Added the
--persistent_multiplex_android_tools,--persistent_multiplex_android_resource_processor,--persistent_android_dex_desugar, and--persistent_multiplex_android_dex_desugarexpansion flags, which allow for easy enabling of persistent and multiplexed Android tools (dexing, desugaring, resource, and processing) (63aace2) - Added the
--incompatible_android_platforms_transition_updates_affectedflag, which causesAndroidPlatformsTransitionto also updateaffected by Starlark transitionwith the changed options. (563664e) - Added the
--experimental_persistent_multiplex_busybox_toolsflag, which enables multiplex worker support inResourceProcessorBusyBox. (4cfd32d) - Added worker support to
AndroidCompiledResourceMergingAction. (3df19e8) - Added worker support to
CompatDexBuilder. (7ce1c57) - Added worker support to the D8 desugarer. (f32b99f)
- Added the merged manifest to the
android_deploy_infooutput group. (c60eb5d) - Added
application_resources’s output groups toandroid_binary. (bb97b02) - Removed support for
android_binary.multidex=off. (7de4fab) - Reduced
AndroidAssetMergerintermediate outputs. (b76cc3a) - Resources are now sorted to produce a consistent zip (which helps cache hit rates). (2c25f73)
- Incremental APKs are now zipaligned before they are installed. (1b2cf8d)
- Split APKs are now zipaligned before they are installed. (6613f6f)
minsdkis now added to dexing and desugaring artifacts paths. (4829960)- Certain Android actions now have their output paths stripped of config information. (2907d15)
- Fixed Android’s
armeabi-v7aconstraint. (46e0be4)
Apple
- Added the
watchos_device_arm64cpu. (ce61164) - Added the
watchos_device_arm64ecpu. (531df65) - Added the
apple_common.link_multi_arch_static_libraryfunction. (8778455) - Added
-no_deduplicatewhen linking with Darwindbg/fastbuild. (9608414) - Added support for the
static_link_cpp_runtimesfeature in Apple linking. (2cc3dc4) cpus for tvOS and watchOS are now correctly inferred when running on an Apple Silicon host. (da8a327)- 32-bit watchOS architectures are no longer included in mulit-arch builds when targeting watchOS 9.0 or greater. (01d46bb)
- Moved
-frameworkflags to be after the-filelistflag. (7866fd9) - Static frameworks are now linked into fully linked static libraries. (ccb2cc0)
- macOS dynamic libraries now have
.dylibextensions instead of.so. (6e1b440) - Linking
cc_infois now exposed fromAppleExecutableBinaryandAppleDynamicFramework. (9feeb1d) - Removed the
AppleDylibBinaryandAppleLoadableBundleBinaryproviders. (092884b) - Removed the
should_lipoargument fromapple_common.link_multi_arch_binary. (3073f1b) - Removed the native
apple_binaryrule. (0535477) - Removed the native
apple_static_libraryrule. (589354c)
Languages
While there are lots of programming languages that are supported through standalone Starlark rules, some are still written as “native” rules in Bazel core, or are bundled Starlark rules while Starlarkification is in progress. In the following sections I summarize the notable changes in support of these languages.
C and C++
- Added the
--experimental_unsupported_and_brittle_include_scanningflag, which enables C/C++ include scanning. (6522472) - Added the
--host_per_file_coptflag. (4919d4a) - Added the
archive_param_filefeature, which allows turning off param file for archives. (bff9730) - Added the
default_link_libsfeature, which can be disabled to support pure C. (5ebb0d6) - Added the
gcc_quoting_for_param_filesfeature, which enables gcc quoting for linker param files. (a9e5a32) - Added the
treat_warnings_as_errorsfeature, which treats warnings as errors. (f802525) - Added the
separate_module_headersparameter tocc_common.compile. (94f83f4) - Added the
languageparameter tocc_common.configure_features. (d308c17) - Added the
main_outputparameter tocc_common.link. (8ac29d7) - Added the
BAZEL_CURRENT_REPOSITORYlocal define tocc_*rules. (eb18166) - Added
Action.argvsupport toCppCompileAction. (aaba5be) - Added support for vendor’ed
clang(17ed57a) - Reverted
cc_library.interface_depsback toimplementation_deps. (abe6667) - Default flags features can now be disabled. (25d17f5)
- The
per_object_debug_infofeature is now enabled by default. (5f51d21) - The
@bazel_tools//tools/cpp:compilerflag now has the valuegccif the configured compiler is detected to begcc. (ef3f058) - The
mallocattribute ofcc_*rules now accepts anyCcInfoproviding target. (1746a79) - The C++ archive action now has the
CppArchivemnemonic. (d519fec) - Coverage can now be collected for external targets. (32e61b3)
- Make variable substitution now accepts
datadependencies. (46a8e09) cc_common.linkno longer stamps actions for tool dependencies. (1799842)- Windows interface libraries can now use the
.libextension. (af4a150) - Fixed
cc_testto apply all compilation flags. (06f9202) - Fixed dynamic library lookup with remotely executed tools. (e3dcfa5)
Java
- Upgraded JDK11. (8f3d99a)
- Upgraded JDK17. (fbb0958)
- Added JDK18. (a7f1c71)
- Added the
--incompatible_disallow_java_import_empty_jarsflag, which disallows emptyjava_import.jarsattributes. (1acda6b) - Added the
--multi_release_deploy_jarsflag, which causes_deploy.jaroutputs ofjava_binaryto be Multi-Release jar files. (7f75df2) - Added the
com.google.devtools.build.buildjar.javac.enable_boot_classpath_cacheproperty, which disables the bootstrap classpath cache. (fbb68e9) - Added the
@AutoBazelRepositoryannotation processor. (0f95c8a) - Added the
add_exportsandadd_opensattributes tojava_*rules. (2217b13) - Added the
hermetic_srcsandlib_modulesattributes tojava_runtime. (79badc0) - Added the
classpath_resourcesargument tojava_common.compile. (37d08ed) - Added the
resource_jarsargument tojava_common.compile. (80ca10b) - Added the
--add_exportsand--add_opensoptions tosinglejar. (4ff441b) - Added the
--hermetic_java_homeoption tosinglejar. (341d7f3) - Added the
--jdk_lib_modulesoption tosinglejar. (276fb09) - Added the
--multi_releaseoption tosinglejar. (f33ce3d) - Added the
--output_jar_creatoroption tosinglejar. (8b5ed8a) - Added a tag-based suppression mechanism for
java_import.depschecking. (2930dd3) - Coverage can now be collected for external targets. (acbb9e1)
- The Java runtimes now have
target_compatible_withset instead ofexec_compatible_with. (d5559c1) ijar/java_importnow preserve classes with@kotlin.Metadataannotations. (a32a0fd)- Hermetic packaged JDK modules now record file size in deploy JAR manifest
JDK-Lib-Modules-Sizeattribute. (756be22) TurbineTransitiveJarattributes are now recognized inijar. (b64f734)- The stub template now defaults to a UTF-8 locale. (17cfa01)
Objective-C
- Added support for Objective-C specific features to
cc_common.configure_features. (68f29c6) objc_librarynow requiresCcInfoin its deps. (540892d)- Removed the
ObjcProvider.direct_headersattribute. (8a2b711)
Python
- Added the
coverage_toolattribute topy_runtime. (9d01630) - Added the
requires-darwinexecution requirement to macOSpy_testtargets. (32364dc) - Added
CurrentRepository()to Python runfiles library. (d60ce2c) - The stub now also considers
RUNFILES_DIRwhen no runfiles directory can be found. (c3425fe) - Reduced the number of imports used in the stub. (c33e44c)
py_*.srcs_version="PY2"is now the the same as"PY2ONLY". (ecd4c90)- Removed UNC path prefix on Windows-10.1607 or later. (40b95c3)
Rules authoring
Bazel’s extensibility is powered by the ability to write custom rules. Most rules used in a project will be provided by open source rule sets, but projects may also define rules themselves. Bazel 6.0 includes numerous changes that make custom rules more performant, easier to write, or even expand what is possible to do with them.
Aspects
Aspects allow augmenting build dependency graphs with additional information and actions. These changes expanded their capabilities:
- Added the
--aspects_parametersflag, which allows passing parameters to command-line aspects. (3771072) - Added support for using
attr.bool(). (30fd508) - Added support for using
attr.int(). (14292d1) - Added support for setting exec_compatible_with and exec_group on aspects. (7e3755d)
- Aspects now inherit fragments from required aspects and from targets they are attached to. (b3e12ba)
Persistent workers
Persistent workers improve build performance by sending multiple requests to long-running processes. Here are some notable changes to persistent worker support:
- Added the
--experimental_total_worker_memory_limit_mbflag, which sets a limit on total worker memory usage. (8e67432) - Added the
--experimental_worker_multiplex_sandboxingflag, which controls whether to sandbox multiplex workers that support it. (fb19a28) - Added the
--experimental_worker_strict_flagfilesflag, which checks if the worker argument list conforms to the spec. (cb2cd9f)
Starlark
As mentioned at Bazelcon, progress is being made on migrating natives rules out of Bazel and into standalone Starlark rules. In the Bazel 6.0 release progress was made on the Android, C++, Java, Objective-C, Protobuf, and Python rules.
In addition to changes directly needed for Starlarkification, for which there were many and I’m not going to list them here, the Starlark language itself received performance and feature improvements:
- Added the
--incompatible_disallow_symlink_file_to_dirflag, which disallowsctx.actions.symlinkfrom symlinking a file into a directory. (54f11fe) - Added the
--incompatible_remove_rule_name_parameterflag, which disables therule.nameparameter. (6449105) - Added the
--experimental_debug_selects_always_succeedflag, which causesselectfunctions with no matching clause to return an empty value, instead of failing. (b615d0d) - Added the
--experimental_get_fixed_configured_action_envflag, which causesaction.envto return fixed environment variables specified through features configuration. (e82beda) - Added the
--experimental_lazy_template_expansionflag, which adds support for lazily computed substitutions toctx.actions.expand_template(). (cc74b11) - Added
dictunion operators (|and|=). (b1deea4) - Added the
coverage_support_filesandcoverage_environmentparameters tocoverage_common.instrumented_files_info. (0a13dd6) - Added the
initparameter toprovider(), which allows for performing pre-processing and validation of field values. (fc13ba2) - Added
loadvisibility. dict()anddict.update()now accept arbitrary maps, not justdicts. (dfa9c62)testing.ExecutionInfocan now be used as a provider key. (40a6cb0)- The
symlinksandroot_symlinksparameters of thectx.runfilesfunction now acceptdepsets. (aaf87f4) - Starlark flags no longer warn for incompatible commands. (f717d6a)
- Moved
analysis_testtotesting.analysis_test. (5e80514) - The
\a,\b,\f, and\vescape sequences are now supported. (d0fde13) print()statements are now emitted only if the line of code is executed. (3bda5c9)native.existing_rule/s()with--incompatible_existing_rules_immutable_viewcan now be encoded as json and passed as**kwargs. (cf99f84)- Fixed incremental builds that transition on flags when the default value of those flags changed. (2f7d965)
- Fixed preserving the relative order of explicit and expanded Starlark flags. (9f2542f)
- Fixed
ctx.actions.symlink(target_path = ...)incorrectly making paths absolute. (d834905)
Misc
There were a handful of changes that I couldn’t find a nice home for in the sections above, but I still felt were important or interesting enough to call attention to:
- Added the
--experimental_output_directory_naming_schemeflag, which modifies the way the output directory is calculated. (52d1d4a) - Added the
--incompatible_check_testonly_for_output_filesflag, which checkstestonlyfor prerequisite targets that are output files by looking uptestonlyof the generating rule. (65388c3) - Added the
--incompatible_fix_package_group_reporoot_syntaxflag, which changes the meaning of"//..."to refer to all packages in the current repository instead of all packages in any repository. (1473988) - Added the
--incompatible_package_group_has_public_syntaxflag, which allows writing"public"or"private"in thepackage_group.packagesattribute to refer to all packages or no packages respectively. (1473988) - Added the
--incompatible_unambiguous_label_stringificationflag, which enables unambiguous stringification ofLabels. (Issue #16196) - Added the
bazel leafcommand. (b82a8e9) bazel confignow shows output path prefixes. (507d85f)BAZEL_TEST=1is now set as a test environment variable. (830d464)- Progress updates no longer have their delay increased when there is no cursor control. (60e9bf3)
- Fixed native rules not merging default outputs into the transitive runfiles. (7cc786a)
- Fixed
--nobuild_runfiles_linkswhen used with--run_under(3badca3)
Summary
As you can see, Bazel 6.0 was another massive release. Thankfully, through the rolling releases process, people were able to test, or even actively depend on, these changes well before the first 6.0 release candidate was cut.
I expect Bazel 6.1 to be a fast follow with some changes that missed the final release candidate. Work on Bazel 7.0 is well underway as well, and I look forward to summarizing its changes next year.
The GitHub UI shows some commits that already exist in the 5.x series of releases due to cherry-picking, so the number of unique commits is actually smaller, closer to 2,800. ↩︎