Skip to content

Commit dec889f

Browse files
marvin-hansenmarvin-hansen
authored
Restored bzlmod documentation. (#2798)
Somehow in PR #2713, the bazelmod docs that I wrote went missing. This PR restores the bazelmod docs. Should be easy to review. --------- Signed-off-by: Marvin Hansen <[email protected]>
1 parent bef8d2d commit dec889f

File tree

2 files changed

+631
-9
lines changed

2 files changed

+631
-9
lines changed

‎crate_universe/docs_bzlmod.bzl

Lines changed: 317 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,328 @@
1-
"""# Crate Universe
1+
"""
2+
# Crate Universe
23
34
Crate Universe is a set of Bazel rule for generating Rust targets using Cargo.
45
56
This doc describes using crate_universe with bzlmod.
67
78
If you're using a WORKSPACE file, please see [the WORKSPACE equivalent of this doc](crate_universe.html).
89
9-
There are some examples of using crate_universe with bzlmod:
10+
There are some examples of using crate_universe with bzlmod in the [example folder](../examples/bzlmod).
11+
12+
# Table of Contents
13+
14+
1. [Setup](#Setup)
15+
2. [Dependencies](#dependencies)
16+
* [Cargo Workspace](#cargo-workspaces)
17+
* [Direct Packages](#direct-dependencies)
18+
* [Vendored Dependencies](#vendored-dependencies)
19+
3. [Crate reference](#crate)
20+
* [from_cargo](#from_cargo)
21+
* [from_specs](#from_specs)
22+
23+
24+
## Setup
25+
26+
To use rules_rust in a project using bzlmod, add the following to your MODULE.bazel file:
27+
28+
```starlark
29+
bazel_dep(name = "rules_rust", version = "0.49.3")
30+
```
31+
32+
You find the latest version on the [release page](https://github.com/bazelbuild/rules_rust/releases).
33+
34+
35+
After adding `rules_rust` in your MODULE.bazel, set the following to begin using `crate_universe`:
36+
37+
```starlark
38+
crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")
39+
// # ... Dependencies
40+
use_repo(crate, "crates")
41+
```
42+
43+
## Dependencies
44+
45+
There are three different ways to declare dependencies in your MODULE.
46+
47+
1) Cargo workspace
48+
2) Direct Dependencies
49+
3) Vendored Dependencies
50+
51+
### Cargo Workspaces
52+
53+
One of the simpler ways to wire up dependencies would be to first structure your project into a Cargo workspace.
54+
The crates_repository rule can ingest a root Cargo.toml file and generate Bazel dependencies from there.
55+
You find a complete example in the in the [example folder](../examples/bzlmod/all_crate_deps).
56+
57+
```starlark
58+
crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")
59+
60+
crate.from_cargo(
61+
name = "crates",
62+
cargo_lockfile = "//:Cargo.lock",
63+
manifests = ["//:Cargo.toml"],
64+
)
65+
use_repo(crate, "crates")
66+
```
67+
68+
The generated crates_repository contains helper macros which make collecting dependencies for Bazel targets simpler.
69+
Notably, the all_crate_deps and aliases macros (
70+
see [Dependencies API](https://bazelbuild.github.io/rules_rust/crate_universe.html#dependencies-api)) commonly allow the
71+
Cargo.toml files to be the single source of truth for dependencies.
72+
Since these macros come from the generated repository, the dependencies and alias definitions
73+
they return will automatically update BUILD targets. In your BUILD files,
74+
you use these macros for a Rust library as shown below:
75+
76+
```starlark
77+
load("@crate_index//:defs.bzl", "aliases", "all_crate_deps")
78+
load("@rules_rust//rust:defs.bzl", "rust_library", "rust_test")
79+
80+
rust_library(
81+
name = "lib",
82+
aliases = aliases(),
83+
deps = all_crate_deps(
84+
normal = True,
85+
),
86+
proc_macro_deps = all_crate_deps(
87+
proc_macro = True,
88+
),
89+
)
90+
91+
rust_test(
92+
name = "unit_test",
93+
crate = ":lib",
94+
aliases = aliases(
95+
normal_dev = True,
96+
proc_macro_dev = True,
97+
),
98+
deps = all_crate_deps(
99+
normal_dev = True,
100+
),
101+
proc_macro_deps = all_crate_deps(
102+
proc_macro_dev = True,
103+
),
104+
)
105+
```
106+
107+
For a Rust binary that does not depend on any macro, use the following configuration
108+
in your build file:
109+
110+
```starlark
111+
rust_binary(
112+
name = "bin",
113+
srcs = ["src/main.rs"],
114+
deps = all_crate_deps(normal = True),
115+
)
116+
```
117+
118+
You have to repin before your first build to ensure all Bazel targets for the macros
119+
are generated.
120+
121+
Dependency syncing and updating is done in the repository rule which means it's done during the
122+
analysis phase of builds. As mentioned in the environments variable table above, the `CARGO_BAZEL_REPIN`
123+
(or `REPIN`) environment variables can be used to force the rule to update dependencies and potentially
124+
render a new lockfile. Given an instance of this repository rule named `crates`, the easiest way to
125+
repin dependencies is to run:
126+
127+
```shell
128+
CARGO_BAZEL_REPIN=1 bazel sync --only=crates
129+
```
130+
131+
This will result in all dependencies being updated for a project. The `CARGO_BAZEL_REPIN`
132+
environment variable can also be used to customize how dependencies are updated.
133+
For more details about repin, [please refer to the documentation](https://bazelbuild.github.io/rules_rust/crate_universe.html#crates_vendor).
134+
135+
### Direct Dependencies
136+
137+
In cases where Rust targets have heavy interactions with other Bazel targets ([Cc](https://docs.bazel.build/versions/main/be/c-cpp.html), [Proto](https://rules-proto-grpc.com/en/4.5.0/lang/rust.html),
138+
etc.), maintaining Cargo.toml files may have diminishing returns as things like rust-analyzer
139+
begin to be confused about missing targets or environment variables defined only in Bazel.
140+
In situations like this, it may be desirable to have a “Cargo free” setup. You find an example in the in the [example folder](../examples/bzlmod/hello_world_no_cargo).
141+
142+
crates_repository supports this through the packages attribute,
143+
as shown below.
144+
145+
```starlark
146+
crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")
147+
148+
crate.spec(package = "serde", features = ["derive"], version = "1.0")
149+
crate.spec(package = "serde_json", version = "1.0")
150+
crate.spec(package = "tokio", default_features=False, features = ["macros", "net", "rt-multi-thread"], version = "1.38")
151+
152+
crate.from_specs()
153+
use_repo(crate, "crates")
154+
```
155+
156+
Consuming dependencies may be more ergonomic in this case through the aliases defined in the new repository.
157+
In your BUILD files, you use direct dependencies as shown below:
158+
159+
```starlark
160+
rust_binary(
161+
name = "bin",
162+
crate_root = "src/main.rs",
163+
srcs = glob([
164+
"src/*.rs",
165+
]),
166+
deps = [
167+
# External crates
168+
"@crates//:serde",
169+
"@crates//:serde_json",
170+
"@crates//:tokio",
171+
],
172+
visibility = ["//visibility:public"],
173+
)
174+
```
175+
176+
Notice, direct dependencies do not need repining.
177+
Only a cargo workspace needs updating whenever the underlying Cargo.toml file changed.
178+
179+
### Vendored Dependencies
180+
181+
In some cases, it is require that all external dependencies are vendored, meaning downloaded
182+
and stored in the workspace. This helps, for example, to conduct licence scans, apply custom patches,
183+
or to ensure full build reproducibility since no download error could possibly occur.
184+
You find a complete example in the in the [example folder](../examples/bzlmod/all_deps_vendor).
185+
186+
For the setup, you need to add the skylib in addition to the rust rules to your MODUE.bazel.
187+
188+
```starlark
189+
module(
190+
name = "deps_vendored",
191+
version = "0.0.0"
192+
)
193+
###############################################################################
194+
# B A Z E L C E N T R A L R E G I S T R Y # https://registry.bazel.build/
195+
###############################################################################
196+
# https://github.com/bazelbuild/bazel-skylib/releases/
197+
bazel_dep(name = "bazel_skylib", version = "1.7.1")
198+
199+
# https://github.com/bazelbuild/rules_rust/releases
200+
bazel_dep(name = "rules_rust", version = "0.49.3")
201+
202+
###############################################################################
203+
# T O O L C H A I N S
204+
###############################################################################
205+
206+
# Rust toolchain
207+
RUST_EDITION = "2021"
208+
RUST_VERSION = "1.80.1"
209+
210+
rust = use_extension("@rules_rust//rust:extensions.bzl", "rust")
211+
rust.toolchain(
212+
edition = RUST_EDITION,
213+
versions = [RUST_VERSION],
214+
)
215+
use_repo(rust, "rust_toolchains")
216+
register_toolchains("@rust_toolchains//:all")
217+
218+
###############################################################################
219+
# R U S T C R A T E S
220+
###############################################################################
221+
crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")
222+
```
223+
224+
Note, it is important to load the crate_universe rules otherwise you will get an error
225+
as the rule set is needed in the vendored target.
226+
227+
Assuming you have a package called `basic` in which you want to vendor dependencies,
228+
then you create a folder `basic/3rdparty`. The folder name can be arbitrary,
229+
but by convention, its either thirdparty or 3rdparty to indicate vendored dependencies.
230+
In the 3rdparty folder, you add a target crates_vendor to declare your dependencies to vendor.
231+
In the example, we vendor a specific version of bzip2.
232+
233+
```starlark
234+
load("@rules_rust//crate_universe:defs.bzl", "crate", "crates_vendor")
235+
236+
crates_vendor(
237+
name = "crates_vendor",
238+
annotations = {
239+
"bzip2-sys": [crate.annotation(
240+
gen_build_script = True,
241+
)],
242+
},
243+
cargo_lockfile = "Cargo.Bazel.lock",
244+
generate_build_scripts = False,
245+
mode = "remote",
246+
packages = {
247+
"bzip2": crate.spec(
248+
version = "=0.3.3",
249+
),
250+
},
251+
repository_name = "basic",
252+
tags = ["manual"],
253+
)
254+
```
255+
256+
Next, you have to run `Cargo build` to generate a Cargo.lock file with all resolved dependencies.
257+
Then, you rename Cargo.lock to Cargo.Bazel.lock and place it inside the `basic/3rdparty` folder.
258+
259+
At this point, you have the following folder and files:
260+
261+
```
262+
basic
263+
├── 3rdparty
264+
│ ├── BUILD.bazel
265+
│ ├── Cargo.Bazel.lock
266+
```
267+
268+
Now you can run the `crates_vendor` target:
269+
270+
`bazel run //basic/3rdparty:crates_vendor`
271+
272+
This generates a crate folders with all configurations for the vendored dependencies.
273+
274+
```
275+
basic
276+
├── 3rdparty
277+
│ ├── cratea
278+
│ ├── BUILD.bazel
279+
│ ├── Cargo.Bazel.lock
280+
```
281+
282+
Suppose you have an application in `basic/src` that is defined in `basic/BUILD.bazel` and
283+
that depends on a vendored dependency. You find a list of all available vendored dependencies
284+
in the BUILD file of the generated folder: `basic/3rdparty/crates/BUILD.bazel`
285+
You declare a vendored dependency in you target as following:
286+
287+
```starlark
288+
load("@rules_rust//rust:defs.bzl", "rust_binary")
289+
290+
rust_binary(
291+
name = "hello_sys",
292+
srcs = ["src/main.rs"],
293+
deps = ["//basic/3rdparty/crates:bzip2"],
294+
visibility = ["//visibility:public"],
295+
)
296+
```
297+
Note, the vendored dependency is not yet accessible because you have to define first
298+
how to load the vendored dependencies. For that, you first create a file `sys_deps.bzl`
299+
and add the following content:
300+
301+
```starlark
302+
# rename the default name "crate_repositories" in case you import multiple vendored folders.
303+
load("//basic/3rdparty/crates:defs.bzl", basic_crate_repositories = "crate_repositories")
304+
305+
def sys_deps():
306+
# Load the vendored dependencies
307+
basic_crate_repositories()
308+
```
309+
310+
This is straightforward, you import the generated crate_repositories from the crates folder,
311+
rename it to avoid name clashes in case you import from multiple vendored folders, and then
312+
just load the vendored dependencies.
313+
314+
In a WORKSPACE configuration, you would just load and call sys_deps(), but in a MODULE configuration, you cannot do that.
315+
Instead, you create a new file `WORKSPACE.bzlmod` and add the following content.
316+
317+
```starlark
318+
load("//:sys_deps.bzl", "sys_deps")
319+
sys_deps()
320+
```
321+
322+
Now, you can build the project as usual.
323+
324+
There are some more examples of using crate_universe with bzlmod in the [example folder](https://github.com/bazelbuild/rules_rust/blob/main/examples/bzlmod/).
10325
11-
* https://github.com/bazelbuild/rules_rust/blob/main/examples/bzlmod/hello_world/MODULE.bazel
12-
* https://github.com/bazelbuild/rules_rust/blob/main/examples/bzlmod/override_target/MODULE.bazel
13-
* https://github.com/bazelbuild/rules_rust/blob/main/examples/bzlmod/all_crate_deps/MODULE.bazel
14326
"""
15327

16328
load(

0 commit comments

Comments
 (0)