The document discusses the Rust compiler and borrow checker. It provides an overview of the stages of compilation including lexical analysis, parsing, semantic analysis, optimization, and code generation. It then does a deep dive on the borrow checker, explaining that it tracks variable initializations and moves in order to catch errors like attempting to use a variable after it has been moved.

1. The Rust Borrow Checker
A Deep Dive
Nell Shamrell-Harrington, Mozilla
@nellshamrell

2. Is the Borrow Checker a friend or a foe?

8. The Borrow Checker becomes your
friend through experience...

9. ...understanding how it works also helps.

10. Nell Shamrell-Harrington
● Sr. Staff Research Engineer at Mozilla
● Lead Editor of This Week in Rust
● Host of the This Week in Rust podcast on the
Rustacean Station
● @nellshamrell

11. Agenda
● Rust Compiler Overview
● Borrow Checker Deep Dive

12. Overview of the Rust Compiler

13. fn main() {
let numbers = vec![1, 2, 3, 4, 5];
for n in numbers {
println!("{}", n);
}
}
src/main.rs

14. fn main() {
let numbers = vec![1, 2, 3, 4, 5];
for n in numbers {
println!("{}", n);
}
}
src/main.rs

15. fn main() {
let numbers = vec![1, 2, 3, 4, 5];
for n in numbers {
println!("{}", n);
}
}
src/main.rs

16. $ cargo run

17. $ cargo run
1
2
3
4
5

18. Stages of Compilation
● Lexical Analysis
Source: "The Imposter's Handbook" by Rob Conery

19. Stages of Compilation
● Lexical Analysis
● Parsing
Source: "The Imposter's Handbook" by Rob Conery

20. Stages of Compilation
● Lexical Analysis
● Parsing
● Semantic Analysis
Source: "The Imposter's Handbook" by Rob Conery

21. Stages of Compilation
● Lexical Analysis
● Parsing
● Semantic Analysis
● Optimization
Source: "The Imposter's Handbook" by Rob Conery

22. Stages of Compilation
● Lexical Analysis
● Parsing
● Semantic Analysis
● Optimization
● Code Generation
Source: "The Imposter's Handbook" by Rob Conery

23. Wait a minute - isn't the Rust
compiler query based?

24. Yes...but that is out of
the scope of this talk.

25. Check out the "Guide
to Rustc Development"
for more info!
https://rustc-dev-guide.rust-lang.org/

26. Stages of Compilation
● Lexical Analysis
● Parsing
● Semantic Analysis
● Optimization
● Code Generation
Source: "The Imposter's Handbook" by Rob Conery

27. Lexical Analysis
Lexer
Program
Source: "The Imposter's Handbook" by Rob Conery
A program called a lexer

28. fn main() {
let numbers =
vec![1, 2, 3, 4, 5];
for n in numbers {
println!("{}", n);
}
}
Lexer
Program
Source: "The Imposter's Handbook" by Rob Conery
Lexeme
Lexical Analysis
A program called a lexer takes the raw source code (called a lexeme)

29. fn main() {
let numbers =
vec![1, 2, 3, 4, 5];
for n in numbers {
println!("{}", n);
}
}
Lexer
Program
Tokens
Source: "The Imposter's Handbook" by Rob Conery
Lexeme
Lexical Analysis
A program called a lexer takes the raw source code (called a lexeme)
and splits it into tokens

30. Stages of Compilation
● Lexical Analysis
● Parsing
● Semantic Analysis
● Optimization
● Code Generation
Source: "The Imposter's Handbook" by Rob Conery

31. Parser
Program
Source: "The Imposter's Handbook" by Rob Conery
Parsing
A program called a parser

32. Parser
ProgramTokens
Source: "The Imposter's Handbook" by Rob Conery
Parsing
A program called a parser analyzes the tokens

33. Parser
ProgramTokens
Source: "The Imposter's Handbook" by Rob Conery
Parsing
A program called a parser analyzes the tokens and translates them into
an abstract syntax tree (AST)
Abstract
Syntax
Tree

34. Source: "Guide to Rustc Development"
Lowering to HIR
The Rust compiler
Rust
Compiler

35. Source: "Guide to Rustc Development"
The Rust compiler takes the AST
Rust
Compiler
Abstract
Syntax
Tree
Lowering to HIR

36. And...
● Expands any macros included in the code
Source: "Guide to Rustc Development"

37. fn main() {
let numbers = vec![1, 2, 3, 4, 5];
for n in numbers {
println!("{}", n);
}
}
src/main.rs

38. println!("{}", n);
src/main.rs

39. ::std::io::_print(::core::fmt::Arguments::new_v1(
&["", "n"],
&match (&n,) {
(arg0,) => [::core::fmt::ArgumentV1::new(
arg0,
::core::fmt::Display::fmt,
)],
},
));
src/main.rs

40. And...
● Expands any macros included in the code
● Desugars certain constructs
Source: "Guide to Rustc Development"

41. for n in numbers {
}
src/main.rs

42. let result = match IntoIterator::into_iter(numbers) {
mut iter => loop {
let next;
match iter.next() {
Some(val) => next = val,
None => break,
};
(...)
},
};
src/main.rs

43. And...
● Expands any macros included in the code
● Desugars certain constructs
● Resolves any imports in the code
Source: "Guide to Rustc Development"

44. Source: "Guide to Rustc Development"
The Rust compiler takes the AST
Rust
Compiler
Abstract
Syntax
Tree
Lowering to HIR

45. Source: "Guide to Rustc Development"
The Rust compiler takes the AST and converts it into a Higher-level
Intermediate Representation (HIR)
Rust
Compiler
Abstract
Syntax
Tree HIR
Lowering to HIR

46. Source: "Guide to Rustc Development"
HIR Data Structures
Node (HirId)

47. Source: "Guide to Rustc Development"
HIR Data Structures
Definition (DefId)
Node (HirId)

48. Source: "Guide to Rustc Development"
HIR Data Structures
Crate (CrateNum)
Definition (DefId)
Node (HirId)

49. fn main() {
let numbers = vec![1, 2, 3, 4, 5];
for n in numbers {
println!("{}", n);
}
}
src/main.rs

50. for n in numbers {
}
src/main.rs

51. let result = match IntoIterator::into_iter(numbers) {
mut iter => loop {
let next;
match iter.next() {
Some(val) => next = val,
None => break,
};
(...)
},
};
src/main.rs

52. Arm {
hir_id: HirId {
owner: DefId(0:3 ~
sample_code_1[24c3]::main[0]),
local_id: 107,
},
span: src/main.rs:4:5: 6:6,
higher_level_intermediate_representation

53. Arm {
hir_id: HirId {
owner: DefId(0:3 ~
sample_code_1[24c3]::main[0]),
local_id: 107,
},
span: src/main.rs:4:5: 6:6,
higher_level_intermediate_representation
Represents an arm of
the match expression

54. Arm {
hir_id: HirId {
owner: DefId(0:3 ~
sample_code_1[24c3]::main[0]),
local_id: 107,
},
span: src/main.rs:4:5: 6:6,
higher_level_intermediate_representation
Node

55. Arm {
hir_id: HirId {
owner: DefId(0:3 ~
sample_code_1[24c3]::main[0]),
local_id: 107,
},
span: src/main.rs:4:5: 6:6,
higher_level_intermediate_representation
Owned
by a
definition

56. Arm {
hir_id: HirId {
owner: DefId(0:3 ~
sample_code_1[24c3]::main[0]),
local_id: 107,
},
span: src/main.rs:4:5: 6:6,
higher_level_intermediate_representation
Owned by
a crate

57. Arm {
hir_id: HirId {
owner: DefId(0:3 ~
sample_code_1[24c3]::main[0]),
local_id: 107,
},
span: src/main.rs:4:5: 6:6,
higher_level_intermediate_representation

58. Rust
Compiler
Source: "Guide to Rustc Development"
Lowering to MIR

59. Rust
Compiler
HIR
Compiler then lowers the HIR
Source: "Guide to Rustc Development"
Lowering to MIR

60. Rust
Compiler
HIR MIR
Compiler then lowers the HIR into Mid-level Intermediate
Representation (MIR)
Source: "Guide to Rustc Development"
Lowering to MIR

61. Source: "Guide to Rustc Development"
MIR Data Structures
Control Flow Graph

62. Source: "Guide to Rustc Development"
MIR Data Structures
Definition (DefId)
Basic Block (bb1)Basic Block (bb0)
Control Flow Graph

63. MIR Data Structures
Definition (DefId)
Basic Block (bb0) Basic Block (bb1)
Statement
Terminator
Statement
Statement Terminator
Source: "Guide to Rustc Development"
Control Flow Graph

64. if condition {
do_one_thing
} else {
do_something_else
}
example.rs

65. MIR Data Structures
Definition (DefId)
Basic Block (bb0)
Statement
Terminator
Definition (DefId)
Basic Block (bb1)
Statement
Terminator
Definition (DefId)
Basic Block (bb2)
Statement
Terminator
Source: "Guide to Rustc Development"
Control Flow Graph

66. Are there more MIR data structures?

67. Check out the "Guide
to Rustc Development"
for more info!
https://rustc-dev-guide.rust-lang.org/

68. let result = match IntoIterator::into_iter(numbers) {
mut iter => loop {
let next;
match iter.next() {
Some(val) => next = val,
None => break,
};
(...)
},
};
src/main.rs

69. let result = match IntoIterator::into_iter(numbers) {
};
src/main.rs

70. bb2: {
_5 = const <std::vec::Vec<i32> as
std::iter::IntoIterator>::into_iter(move _6)
span: src/main.rs:4:14: 4:21
mid_level_intermediate_representation

71. bb2: {
_5 = const <std::vec::Vec<i32> as
std::iter::IntoIterator>::into_iter(move _6)
span: src/main.rs:4:14: 4:21
mid_level_intermediate_representation
Basic Block

72. bb2: {
_5 = const <std::vec::Vec<i32> as
std::iter::IntoIterator>::into_iter(move _6)
span: src/main.rs:4:14: 4:21
mid_level_intermediate_representation
Local
(result)

73. bb2: {
_5 = const <std::vec::Vec<i32> as
std::iter::IntoIterator>::into_iter(move _6)
span: src/main.rs:4:14: 4:21
mid_level_intermediate_representation
Span

74. Stages of Compilation
● Lexical Analysis
● Parsing
● Semantic Analysis
● Optimization
● Code Generation
Source: "The Imposter's Handbook" by Rob Conery

75. Source: "The Imposter's Handbook" by Rob Conery
Semantic Analysis
Compiler tries to figure out what the programmer is trying to do in a way
the compiler can understand it

76. Semantic Analysis
At this point the Rust compiler will run several checks - including the
borrow checker
Source: "Guide to Rustc Development"

77. We'll come back to the borrow
checker shortly...

78. Stages of Compilation
● Lexical Analysis
● Parsing
● Semantic Analysis
● Optimization
● Code Generation
Source: "Guide to Rustc Development"

79. Optimization & Code Generation
Source: "Guide to Rustc Development"
This is where the code is transformed into an executable binary

80. Source: https://llvm.org
LLVM is a collection of modular and re-usable compiler and toolchain
technologies
LLVM

81. Rust
Compiler
Source: "Guide to Rustc Development"
Optimization & Code Generation

82. Rust
Compiler
MIR
Compiler lowers the MIR
Source: "Guide to Rustc Development"
Optimization & Code Generation

83. Rust
Compiler
Compiler lowers the MIR into LLVM Intermediate Representation
Source: "Guide to Rustc Development"
LLVM IR
Optimization & Code Generation
MIR

84. bb2:
; preds = %bb1
ret i32 %1, !dbg !194
}
llvm_intermediate_representation

85. bb2:
; preds = %bb1
ret i32 %1, !dbg !194
}
llvm_intermediate_representation
Basic Block

86. LLVM
Source: "Guide to Rustc Development"
Optimization & Code Generation

87. LLVM
And LLVM takes the LLVM IR
Source: "Guide to Rustc Development"
LLVM IR
Optimization & Code Generation

88. LLVM
And LLVM takes the LLVM IR, which runs more optimizations on it and
emits machine code
Source: "Guide to Rustc Development"
LLVM IR
Machine
Code
Optimization & Code Generation

89. LLVM then links the machine code files together to produce the final
binary
Source: "Guide to Rustc Development"
Machine
Code
Optimization & Code Generation

90. $ cargo run

91. $ cargo run
1
2
3
4
5

92. Yay!!!
$ cargo run
1
2
3
4
5

93. Back to the borrow checker!

94. src/main.rs
fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}

95. src/main.rs
fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}

96. src/main.rs
fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}

97. src/main.rs
fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}

98. src/main.rs
fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}

99. $ cargo build

100. $ cargo build
error[E0382]: borrow of moved value: `x`
--> src/main.rs:7:20
5 | let y = x;
| - value moved here
6 |
7 | println!("{}", x);
| ^ value borrowed here after move
error: aborting due to previous error
For more information about this error, try `rustc --explain
E0382`.

101. Borrow Checker
Does several things including:
● Tracking initializations and moves
Source: "Guide to Rustc Development"

102. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}
src/main.rs

103. let x: String; // x is not initialized here
src/main.rs

104. let x: String; // x is not initialized here
src/main.rs
mid_level_intermediate_representation
debug x => _1;
let _1: std::string::String

105. let x: String; // x is not initialized here
src/main.rs
mid_level_intermediate_representation
debug x => _1;
let _1: std::string::String
x

106. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}
src/main.rs

107. x = String::from("Hi Rusty Days!"); // x is initialized here
src/main.rs

108. x = String::from("Hi Rusty Days!"); // x is initialized here
src/main.rs
mid_level_intermediate_representation
_2 = const <String as
From<&str>>::from(const "Hi Rusty Days!")
_1 = move _2;

109. x = String::from("Hi Rusty Days!"); // x is initialized here
src/main.rs
mid_level_intermediate_representation
_2 = const <String as
From<&str>>::from(const "Hi Rusty Days!")
_1 = move _2;
x

110. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}
src/main.rs

111. let y = x; // x is moved here
src/main.rs

112. let y = x; // x is moved here
src/main.rs
mid_level_intermediate_representation
debug y => _3;
let _3: &std::string::String;
_3 = move _1;

113. let y = x; // x is moved here
src/main.rs
mid_level_intermediate_representation
debug y => _3;
let _3: &std::string::String;
_3 = move _1;

114. let y = x; // x is moved here
src/main.rs
mid_level_intermediate_representation
debug y => _3;
let _3: &std::string::String;
_3 = move _1;
xy

115. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}
src/main.rs

116. println!("{}", x); // x is NOT initialized here
src/main.rs

117. $ cargo build
error[E0382]: borrow of moved value: `x`
--> src/main.rs:7:20
5 | let y = x;
| - value moved here
6 |
7 | println!("{}", x);
| ^ value borrowed here after move
error: aborting due to previous error
For more information about this error,
try `rustc --explain E0382`.

118. $ cargo build
error[E0382]: borrow of moved value: `x`
--> src/main.rs:7:20
5 | let y = x;
| - value moved here
6 |
7 | println!("{}", x);
| ^ value borrowed here after move
error: aborting due to previous error
For more information about this error,
try `rustc --explain E0382`.
Span

119. $ cargo build
error[E0382]: borrow of moved value: `x`
--> src/main.rs:7:20
5 | let y = x;
| - value moved here
6 |
7 | println!("{}", x);
| ^ value borrowed here after move
error: aborting due to previous error
For more information about this error,
try `rustc --explain E0382`.

120. $ rustc --explain E0382
A variable was used after its contents have been moved elsewhere.
fn main() {
let mut x = MyStruct{ s: 5u32 };
let y = x;
x.s = 6;
println!("{}", x.s);
}
Since `MyStruct` is a type that is not marked `Copy`, the data gets moved
out
of `x` when we set `y`.

121. $ rustc --explain E0382
Sometimes we don't need to move the value. Using a reference, we can
let another function borrow the value without changing its ownership.

122. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
println!("{}", x);
println!("{}", y);
}
src/main.rs

123. $ cargo run

124. $ cargo run
Hi Rusty Days!
Hi Rusty Days!

125. Borrow Checker
Does several things including:
● Tracking initializations and moves
● Lifetime inference
Source: "Guide to Rustc Development"

126. Rust uses "lifetime" in two distinct ways
● Lifetime of a variable
○ Span of time before the value of the variable gets freed
○ Also known as "scope"
Source: "Rust RFC #2094: NLL"

127. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}
src/main.rs
x is initialized

128. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}
src/main.rs
x is moved

129. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = x;
println!("{}", x);
println!("{}", y);
}
src/main.rs
x is not
initialized here

130. Rust uses "lifetime" in two distinct ways
● Lifetime of a variable
○ Span of time before the value of the variable gets freed
○ Also known as "scope"
● Lifetime of a reference
○ Span of time in which the reference can be used
Source: "Rust RFC #2094: NLL"

131. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
println!("{}", x);
println!("{}", y);
}
src/main.rs

132. let y = &x;
src/main.rs

133. let y = &x;
src/main.rs
mid_level_intermediate_representation
debug x => _1;
debug y => _3;
_3 = &_1;

134. let y = &x;
src/main.rs
mid_level_intermediate_representation
debug x => _1;
debug y => _3;
_3 = &_1;

135. let y = &x;
src/main.rs
mid_level_intermediate_representation
debug x => _1;
debug y => _3;
_3 = &_1;
xy

136. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
println!("{}", x);
println!("{}", y);
}
src/main.rs

137. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
drop(x);
println!("{}", y);
}
src/main.rs

138. $ cargo build

139. $ cargo build
error[E0505]: cannot move out of `x` because it is borrowed
--> src/main.rs:7:10
|
5 | let y = &x;
| -- borrow of `x` occurs here
6 |
7 | drop(x);
| ^ move out of `x` occurs here
8 |
9 | println!("{}", y);
| - borrow later used here

140. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
drop(x);
println!("{}", y);
}
src/main.rs
Lifetime
of x

141. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
drop(x);
println!("{}", y);
}
src/main.rs
Needed
lifetime
of y

142. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
drop(x);
println!("{}", y);
}
src/main.rs
y can no
longer
reference x

143. fn main() {
let x: String;
x = String::from("Hi Rusty Days!");
let y = &x;
drop(x);
println!("{}", y);
}
src/main.rs
y is dead
here

144. Rust uses "lifetime" in two distinct ways
● Lifetime of a variable
○ Span of time before the value of the variable gets freed
○ Also known as "scope"
● Lifetime of a reference
○ Span of time in which the reference can be used
● If you make a reference to a value, the lifetime of that reference
cannot outlive the scope of the value
Source: "Rust RFC #2094: NLL"

145. There is SO much more to both
the compiler and the borrow checker!

146. Check out the "Guide
to Rustc Development"
for more info!
https://rustc-dev-guide.rust-lang.org/

147. Is the Borrow Checker a friend or a foe?

148. It is a friend...though a strict one

149. but one that will not only tell you when
something is wrong...

150. ...it will also tell you how to fix it.

151. Nell Shamrell-Harrington
● Sr. Staff Research Engineer at Mozilla
● Lead Editor of This Week in Rust
● Host of the This Week in Rust podcast on the
Rustacean Station
● @nellshamrell