Thursday, June 11, 2020

How to cross compile Rust on MacOs and target Linux using Docker container

I use a Mac for development, and recently I had to compile a Rust project to be run on a Linux server. This post captures how I managed to get this done. Note that the method highlighted in this post can also be used if you have a Windows (or even a Linux) dev environment.

The first thing I attempted was to install the necessary toolchain that should, in theory, allow me to cross-compile the Rust project on my Mac and target Linux. This did not go as smoothly as I would have loved. The first issue I encountered was with OpenSSL. After fixing that, the next issue was with backtrace-sys with the build failing with `error: failed to run custom build command for backtrace-sys v0.1.34`

This looks like I probably do not have the right environment/toolchain/dependencies installed.

I had the option of trying to identify all the required dependencies, but I was really running out of time and frankly, hunting around for the correct dependencies needed to set up a working compiler toolchain is also not fun. So I ended up making use of Docker to help with the build processes.

What I did was to create an image based on alpine Linux with the required toolchain. I then use the container spurn based on the image for the build processes. The procedure is as follows:

The Dockerfile:

FROM alpine:3.11 AS build
LABEL maintainer="Dadepo Aderemi"

# -- Install Rust build toolchain
ENV RUSTFLAGS="-C target-feature=+crt-static"
RUN apk add rust cargo openssl-dev

The +crt-static ensures statically linking of runtime dependencies. For more on this see Static and dynamic C runtimes

Now make sure to be in the same directory the docker file is located and create the image by running:

docker build -t rust-linux-builder .

Once built, run and connect into the container with the source code of the project mounted as volume by running:

docker run -it --rm --net=host -v $(pwd):/build rust-linux-builder

Note using $(pwd) assumes your current directory is where the source of the project is located. Also the use of the --rm flag ensures the container is removed automatically on exit. No need to keep the container around after finishing with the build.

Once in the container, switch to /build then run the build command:

/ # cd /build/
/build # cargo build --target x86_64-alpine-linux-musl --release

Once the build completes, exit the container and the binary would be found within the /target directory. Which can then be copied and installed on the linux server where it would be executed.

As you can see, there is nothing complicated in the Dockerfile and you can easily create your own, with your modification if need be. But in case you do not want to do that, I pushed the Docker image to Docker hub. This means you can easily get your rust project, targeting Linux by running:

docker run -it --rm --net=host -v $(pwd):/build dadepo/rust-linux-builder

...and follow the steps outlined above.

Sunday, March 01, 2020

Learning Rust - Day 10 - Smart Pointers

This is the 10th entry of my learning Rust journal...

It captures some of the learning points while going through chapter 15 of the Rust Book. You can read other posts in this series by following the label learning rust.

I enjoyed reading this chapter. I found it particularly interesting because it was about concepts I usually do not need to think about when working with the other programming languages I have used before now. Apart from that, it also allowed me to invalidate some wrong assumptions I had picked up along the way and also consolidates some of the concepts I have been learning.

One of the assumptions I had, which I found out was wrong while going through this chapter has to do with Stack vs Heap. For some strange reason, I had thought that Structs and Enums are always on the Heap. I suspect my familiarity with Java is to blame for this wrong assumption, since if you squint hard enough a struct looks like an Object in Java and usage of new keyword always means allocating memory on the heap.  But this is not the case in Rust. A struct or an enum does not automatically mean heap memory allocation.

Saturday, February 29, 2020

Rust Ownership Rules

If you have been following this blog, then it would have been obvious that at the beginning of this year, I started learning Rust. This blogpost is a breakaway from the journal style of capturing the main points I encountered while reading the Rust book. It instead captures my understanding thus far of Rust ownership rules.

One of Rust's main differentiator is that it provides memory safety. This it does by providing compile-time guarantees that flag code that potentially could lead to memory bugs as a compile-time error. The compile-time guarantees enforce what is normally referred to as ownership rules. In this post, I took the opportunity to re-summarise what I consider to be the essence of this ownership rules in Rust. The key points can be outlined as follows:
  • Values are owned by variables. 
  • When the owning variables go out of scope, the memory the value is occupying will be deallocated.
  • The values can be used by other variables, they just need to adhere to certain rules that are enforced by the compiler.
The ways that other variables make use of value can be grouped into 4 categories and these ways of usage will dictate the rules to be adhered to:
  • Clone: Here the value is copied to the other variable. The other variable gets its own ownership of the copied value, while the original variable keeps the ownership of its value.
  • Move. Here the ownership is handed over to the other variable that wants to make use of the value. The original variable no longer has ownership.
  • Immutable Borrow. Here no ownership transfer occurs but the value can be accessed for reading by another variable. The memory is not de-allocated if the borrowing variable goes out of scope, since the borrowing variable does not have ownership.
  • Mutable Borrow. Here the value can be accessed for both reading and writing by the other variable. The memory is also not de-allocated if this borrowing variable goes out of scope since the borrowing variable does not have ownership.

Tuesday, February 04, 2020

Learning Rust - Day 9 - Closures and Iterators

This is the 9th entry of my learning Rust journal...

It captures some of the learning points while going through chapter 13 of the Rust Book. You can read other posts in this series by following the label learning rust.

This chapter did not present any new or mind bending concepts. Most modern languages nowadays have the concept of functions as first class citizens, closures and iterators. So the chapter was about taking note of how these concepts are encoded in Rust.

I did not really enjoy how this particular chapter was written, especially section 13.01. I think the pedagogy can be improved. The section spends way too much time in motivating an example, that in my opinion, clouds the essence of what is being explained. It so happened that I found another book on Rust Introduction to Rust which ended up being really well written: concise and well explained. I personally enjoyed the chapter on closure from this book than I did reading the Rust Book itself.

So to the content of this chapter, here are some of the things that stood out:


Functions as Values

// normal function definition
fn add(a:u32, b:u32) -> u32 {
    a + b

// storing defined function into a variable
// keyword fn is used to define the type of a function
let adder_func : fn(u32, u32) -> (u32) = add;

assert_eq!(adder_func(2,3), 5)

Defining Closures

let max = |x: i32, y:i32| {
    if x > y {
    } else {

assert_eq!(max(100,10), 100)

Closures do not need to have the type annotations specified, and if the body is a single expression, can be written in one line without the curly braces. Hence the above can also be written as:

let max = |x, y| if x > y { x } else { y };

assert_eq!(max(100,10), 100)

Defining Iterator for own types

A custom type needs to implement the Iterator trait, which is defined as:

pub trait Iterator {
    type Item;

    fn next(&mut self) -> Option<Self::Item>;

For example, if I have a custom datatype of ranges with even numbers, the definition of such a data type, iterator implementation and usage will look like this: 

// Custom type
pub struct EvenRanges {
    pub first: i32,
    pub last: i32}

// Constructor for custom type
impl EvenRanges {
    pub fn new(first: i32, last: i32) -> EvenRanges {
        return EvenRanges { first, last}

// implementation of Iterator
impl Iterator for EvenRanges {
    // This is a new syntax and has to do with associative types    
   // this will be explained in chapter 19    
   type Item = i32;

    fn next(&mut self) -> Option<Self::Item> {
        let next =  if (self.first + 1) % 2 == 0 {
            self.first + 1        
          } else {
            self.first + 2 

        if next < self.last {
            self.first = next;
        } else {

fn main() {
    let ranges = EvenRanges::new(2, 10);
    // This is possible because 
   // EvenRanges implements EvenRanges    
   for i in ranges {
        println!("{:}", i)


Closures can capture values from their environment in three ways, which directly map to the three ways ownership and references work in Rust:
  1. Taking ownership
  2. Immutable borrow (shared reference)
  3. Mutable borrow
These three ways corresponds to 3 traits.
  1. FnOnce
  2. Fn
  3. FnMut
Talking about traits, it looks like the implementation of closures in Rust can be approximated to having a backing struct with these traits implemented for it. I say approximated as I believe this is not 100% exactly how the compiler works. Using the approximation, if we have the following definition:

let mut a = vec![1, 2];
let mut b = vec![3, 4];
let mut c = vec![5, 6];
let my_closure = || {
    // Takes `a` by shared reference    
    assert_eq!(a[0], 1);
    // Takes `b` by mutable reference    
b[0] = 1;
    // Moves `c` into the closure    
    let d = c;


The above would be transformed into something like:

// Struct to represent the type of the closure
struct StructForMyClosure {
    a: &Vec<i32>,
    b: &mut Vec<i32>,
    c: Vec<i32>,

// Capture the variables in the environment
let x = StructForMyClosure{ a: &a, b: &mut b, c: c };

// Implements a trait..
impl FnOnce for StructForMyClosure {
    fn call_once(self) {
        assert_eq!(self.a[0], 1);
        self.b[0] = 1;
        let d = self.c;

// my_closure() would internally call the trait function

.iter() vs .iter_mut() vs .into_iter()

Turning into iterators, one of the few things that stood out to me was the difference between .iter(), .iter_mut() and .into_iter(). Again, unsurprisingly, the differences is related to how Rust handle borrowing/references.


This allows having an iterator that have a shared reference with the vector. As seen in the snippet below:

let mut my_vector = vec![1, 2, 3, 4];

let mut iter = my_vector.iter();
// note &1 in Some(&1) indicating shared reference
assert_eq!(, Some(&1));

// can still use my_vector// since it was borrowed immutably


This allows for having an iterator that has a mutable borrow of the vector. As seen in the snippet below:

let mut my_vector = vec![1, 2, 3, 4];

let mut iter = my_vector.iter_mut();
// note &mut 1 in Some(&mut 1) indicating mutable borrow
assert_eq!(, Some(&mut 1));
assert_eq!(, Some(&mut 2));

// can still use my_vector
// since the mutable borrow is out of scope

Due to the mutable borrow the following won't compile, because the vector is being accessed when the mutable borrow is still in scope, violating the restriction that mutable borrow should be exclusive:

let mut my_vector = vec![1, 2, 3, 4];

let mut iter = my_vector.iter_mut();
// note &mut 1 in Some(&mut 1) indicating mutable borrow
assert_eq!(, Some(&mut 1));

// leads to compilation error

assert_eq!(, Some(&mut 2));



This methods leads to an iterator that takes ownership of the vector. This means after the iterator is used, the original vector cannot be used anymore. Hence below snippet will compile:

let mut my_vector = vec![1, 2, 3, 4];

let mut iter = my_vector.into_iter();
assert_eq!(, Some(1));

But this wont:

let mut my_vector = vec![1, 2, 3, 4];

let mut iter = my_vector.into_iter();
assert_eq!(, Some(1));

Note that on a normal day, iterators would not be constructed and consumed by direct calling the next() method, instead they would be used with language construct like for in, eg:

A more real life of .iter() method would look like: 

let mut my_vector = vec![1, 2, 3, 4];

for x in my_vector.iter() {
    println!("{}", x)

// prints [1, 2, 3, 4]
println!("{:?}", my_vector)

While that of iter_mut() would look like:

let mut my_vector = vec![1, 2, 3, 4];

for x in my_vector.iter_mut() { 
   // since we have iter_mut
   // we can mutate the contents of the vector  
*x = *x + 1;
println!("{}", x)

// prints mutated value [2, 3, 4, 5]
println!("{:?}", my_vector)

and usage of into_iter() would look like:

let mut my_vector = vec![1, 2, 3, 4];

for x in my_vector.into_iter() {
    println!("{}", x)

// uncommenting below would lead to compilation error
// since ownership got moved due to into_iter()
// println!("{:?}", my_vector)

Also, even though this was not mentioned in the chapter, I was interested in knowing how to get the index in for for in. Found this can be achieved by calling the enumerate() method on the iterator.

For example:

let mut my_vector = vec![1, 2, 3, 4];

for (k, v) in my_vector.iter().enumerate() {
    /** Below prints:    
     Index: 0, value: 1    
     Index: 1, value: 2    
     Index: 2, value: 3    
     Index: 3, value: 4    
   println!("Index: {}, value: {}", k,v)

Consuming adaptors and Iterator adaptors

The concept of having a lazy description of transformation over data structures and the action that executes the description is also part of iterators in Rust. In Rust, the methods that leads to lazy transformations are called Iterator adaptors, while the once that perform the transformation and results a value are called Consuming adaptors. Same thang different day you might say!

Example of the Iterator adaptors include methods like filter(),  map() while examples of consuming adaptors are sum(), count() etc.

A new syntax was also dropped in this chapter. It was referred to as associated types, but I won't be knowing what these are until chapter 19 :)

That is it for now, next chapter would be about Cargo and so I expect that to be a breeze!

Sunday, January 26, 2020

Learning Rust - Day 8 - An I/O Project: Building a Command Line Program

This is the 8th journal entry of my learning Rust journey. It captures some of the learning points while going through chapter 12 of the Rust Book. You can read other posts in this series by following the label learning rust.

Not much new concepts was introduced in this chapter. The aim was to apply the material presented in the book up till that point in building a trivial command line tool.

While working through the chapter though, I observed a couple of different strategies for dealing with errors via the Result<T,E> type.

Turn error to boolean via is_err
This seems handy when you want to convert the result to boolean based on whether the result was a success of not:

let x: Result<i32, &str> = Ok(-3);
assert_eq!(x.is_err(), false);

let x: Result<i32, &str> = Err("Some error message");
assert_eq!(x.is_err(), true);

Get success or run some logic via unwrap_or_else
This allows getting the success value, and in case of error, allows passing a callback function to process the error.

fn count(x: &str) -> usize { x.len() }

assert_eq!(Ok(2).unwrap_or_else(count), 2);
assert_eq!(Err("foo").unwrap_or_else(count), 3);

Ignore success and only run some code on failure via if let syntax
This seems to be useful in cases where you only want to do something in case calling a function returns an error.

if let Err(e) = run(config) {
    eprintln!("Application error: {}", e);

These are by far not all the available error handling strategies when dealing with Result in Rust. These were only the ones that I picked on while reading through chapter 12.

Another thing worth nothing, which is more or less like a culture shock, is the practice of putting unit tests in the same file as the code being testes. All languages I have used before now had the practice of having the tests external to the code being tested; but it seems in idiomatic Rusts, the tests go together with the implementation. This would require some getting used to!

That was it for this chapter. I now look forward to exploring the Functional Language Features in Rust as I proceed to chapter 13..

Friday, January 24, 2020

Learning Rust - Day 7 - Writing Automated Tests

This is the 7th journal entry of my learning Rust journey. You can read other posts in this series by following the label learning rust.

In this session, I read through Chapter 11 of The Rust Book. It was about writing automated tests in Rust. This chapter was a breeze; for obvious reasons. In fact, the most interesting things I learnt was not even about testing in Rust, but about another feature in the language: Attributes.

I have always noticed things like #[derive(Debug)], #![allow(unused_variables)], etc being used in the language, but I never stopped to actually read up on what they are. I know they were a facility of providing some form of metadata in the language; sort of like @annotations in Java, I just never took the time to find out what they are official called in Rust.

It ended up being that the testing mechanism in Rust revolves around the use of this language feature: notably #[cfg(test)] and #[test], so I took the opportunity to find out what exactly these things were.

They are Attributes and they are:
...a general, free-form metadatum that is interpreted according to name, convention, language, and compiler version...
They can exist in two forms. Outer attributes: the ones that starts with #! and Inner attributes: the ones that starts with  only #. The outer attribute is placed outside something - i.e. before a struct definition, function definition,  module definition etc - it applies to the thing that follows the attribute. The inner attribute is placed inside something. e.g when placed in the (root of a) crate in other to apply an attribute to the crate - it applies to the item that the attribute is declared within.

Attributes can also be classified into the following kinds: Built-in attributesMacro attributesDerive macro helper attributes, and Tool attributes. So far, so good, I find the Built-in attributes to be the most interesting ones, because, based on my knowledge of Rust, they are the ones I have mostly encountered. A list of these Built-in attributes can be found here.

Apart from learning more about Attributes, there were a couple of things I picked up about testing in Rust that are worth noting:

  • Use #[cfg(test)] attribute on module that contain test functions. Use #[test] attribute on test functions within the test module.
  • assert!assert_eq! and assert_ne! are macros that can be used for asserting test conditions.
  • favour assert_eq! and assert_ne! over assert! because they provide more useful messages in case of test failures and allow specifying of custom error messages on test failures. 
  • cargo test --help displays options that can be used with cargo test while
    cargo test -- --help displays the options you can use after the separator --. It looks like that latter can only be ran in the root of a rust project.
  • You can’t use the #[should_panic] annotation on tests that use Result<T, E>. An Err value would need to be returned to signify an expectation of error.
  • Run the tests with cargo test -- --test-threads=1 to prevent concurrent execution of tests 
  • Use #[ignore] attribute to ignoring some tests unless specifically requested.
  • To specifically run a single test, pass the name of the test function to cargo test. That is
    cargo test name_of_test_function 
  • Use cargo test -- --nocapture to also see the outputs (if any) from succeeding tests. By default outputs are shown only for failed tests
  • If there is setup code to be shared across different tests, then make sure to put them inside of tests/common/ instead of tests/ Not doing this would make the setup code appear in the test results.
  • Unit tests are placed in the same file as the module, while integration tests go into tests/ and they do not need #[cfg(test)] attribute; only the #[test] attribute is needed.
That was it for learning about writing automated tests in Rusts. I would be taking on Chapter 12 next. Which is: An I/O Project: Building a Command Line Program. It Looks like a chapter that would help solidify some of the concepts presented in the book thus far. Looking forward!

Thursday, January 23, 2020

Learning Rust - Day 6 - Generic Types, Traits, and Lifetimes

This is the 6th journal entry of my learning Rust journey. You can read other posts in this series by following the label learning rust.

In this study session I went through chapter 10 of the Rust book, which covers Generic Types, Traits, and Lifetimes. Generic types and Traits were easy to digest, as they are concepts I am already familiar with from other languages. It was Lifetime that proved to be the difficult nut to crack. Just like when going over the section about Modules, I had to consult other sources outside the book in other to be able to wrap my head around the concepts. I would not say I have it 100% locked down, but I think I now know enough of the general gist to proceed with the rest of the book.

Generics and Traits

Going over Generic Types and Traits was more or less about learning how these concepts are encoded in rust. It did introduced a lot of syntax, which I outline below:

Generic related syntax

// Function that defines the generic type    
fn generic_function<T>(input: T) -> T {

// Generic Struct and Enum    
struct Point<T> {
         x: T,

enum Color<T> {

// Generic methods    
impl<T> Color<T> {
    fn get_hue<U>(&self) -> T {

Trait related syntax

// defining a trait   
pub trait Summary {
   fn summarize(&self) -> String;

// defining a trait with default implementation    
pub trait DefaultSummary {
   fn summarize(&self) -> String {

pub struct Tweet {
     pub username: String,
     pub content: String,
     pub reply: bool,
     pub retweet: bool,

// defining an instance of a trait for a type
impl Summary /* <- trait name */ for Tweet /**/ {
    fn summarize(&self) -> String {

// specifying function parameters as accepting traits    
// uses the impl trait syntax    
fn notify(text: impl Summary) -> String {

// specifying function parameters as accepting traits    
// uses the trait bound syntax    
fn another_notify<T: Summary>(text: T) -> String {

// specifying multiple traits for a type    
fn multiple_notify(text: impl Summary + Display) ->String {

fn another_multiple_notifify<T: Summary + Display>(text:T) -> String {

// specifying multiple traits with where clause    
// instead of    
fn some_function<T: Display + Clone, U: Clone + Debug>(t: T, u: U) -> i32 {
// we have    
fn some_other_fn<T, U>(t:T, u: U) -> i32 where T: Display + Clone, U: Clone + Debug {
// which looks better with new line    
fn some_other_f<T, U>(t:T, u:U) -> i32        
        where T: Display + Clone,
              U: Clone + Debug {

// Returning Types that Implement Traits    
fn return_trait() -> impl Summary {
   Tweet {
     username: "".to_string(),
     content: "".to_string(),
     reply: false,
     retweet: false        

// implementing methods on a generic struct, 
// if the type parameter implements some traits    
struct Pair<T> {
        x: T,
        y: T,

// new method would be available, regardlass of T    
impl<T> Pair<T> {
     fn new(x: T, y: T) -> Self {
         Self {

// cmp_display would be available only if T has instance for Display 
// and PartialOrd    
impl<T: Display + PartialOrd> Pair<T> {
  fn cmp_display(&self) {
      if self.x >= self.y {
         println!("The largest member is x = {}", self.x);
      } else {
         println!("The largest member is y = {}", self.y);

// blanket implementations: Can implement ToString for T    
// only if T already implements Display    
impl<T: Display> Summary for T {
   fn summarize(&self) -> String {

fn re(x: &str) -> &str {

Some noteworthy learning around generics include:

Given a generic type T, T can be a type that can be on the heap or stack. It can't be determined from just the generic type signature. I think this have ramification when it comes to borrowing.

Some noteworthy learning around traits include:

It is only possible to implement a trait on a type only if either the trait or the type is local to my crate.

Basically either of the following scenario:
  • I have my local type, I have an external Trait. I can import the external trait and implement it for my local type.
  • I have a local Trait, I have an external type. I can import the external type and implement my trait for it.
The Trait bound syntax enforces that multiple function parameter implements same traits and are of the same type. This is not the case with impl trait syntax.

// first and second has to be the same type
// they also must have an implementation for trait Summary
fn multiple_function_same_type<T: Summary>(first: T, second: T) -> i32 {

// fist and second can be a different type
// but they must have an implementation for trait Summary
fn multiple_function(first: impl Summary, second: impl Summary) -> i32 {

If i have a function whose return type is represented with a trait, It is not possible from that function to have an implementation that could return any of the available implementations; i.e. via if else. The implementation should only be returning one type that implements that trait. Even if the types implements same traits. This restriction can be circumvented though, using Traits Objects. But I won't be learning about that, not until Chapter 17.


The bulk of the time in this study session was spent grokking (or trying to?) lifetimes. Even though I do not have the concept 100% locked down, some points I think are worthy of note. I list these below.

I think the big idea about lifetime is that they ensure all borrows are valid. That means ensuring that every use of & is valid. The borrow checker can be seen as the enforcer that is saddled with this task, and in other to perform it, it uses the concept of lifetime.

For example, given the following non compiling function definition:

fn longest(x: &str, y: &str) -> &str {
    if x.len() > y.len() {
    } else {

The use of & can either be a valid or invalid borrow.

The use of in the argument can be assumed to always be valid, since the value needs to exist in other for it to be borrowed into the function to be used. The same can't be said of the use of & in the return value.

The value the borrow in the return refers, can only be from two places. From within the function or from the input to the function.  If the borrow is to a value created within the function, then you have an automatically invalid situation, because once the function call is over the value would be cleared, leading to dangling pointer situation. 

In the case where the borrow in return is based off the input, then it becomes impossible to immediately tell if the value that was borrowed into the function would be valid for as long as the variable that ends up holding the return value. Hence why the above code snippet won't compile.

In other to make the above to compile, extra information needs to be provided that helps the borrow checker ensure that the returned borrow would indeed continue to be valid. This is done using lifetime annotations.

The updated function with lifetime annotation provided would look like this:

fn longest<'a>(x: &'a str, y: &'a str) -> &'a str {
    if x.len() > y.len() {
    } else {

So what does this buy us? 

My interpretation is that we are telling Rust, that whenever this function is used, the inputs and the return value must be covered by the same lifetime. 

This means that for the return borrow, the scope should be shorter, or at-least as long as  those of the inputs. In case where the lifetime/scope of the return value is longer than any of the input, then the constraint of the lifetime annotation is being violated and Rust won't compile the code.

To make the above concrete, we take a look at two scenarios where the function above is used. One where the lifetime constraints are respected and another where they are violated

fn main() {
  let string1 = String::from("long string is long");

    let string2 = String::from("xyz");
    let result = longest(string1.as_str(), string2.as_str());
    println!("The longest string is {}", result);
    // the lifetime of result ends here    
    // the lifetime of string2 ends here    
    // the lifetime of string1 continues    
    /* if result refers to string2, then it is fine, 
       because they have same lifetime */    
    /* if result refers to string1, it is fine, 
       because result has a shorter lifetime */  

The above respects the lifetime annotations.

fn main() {
  let string1 = String::from("long string is long");
  let result;
    let string2 = String::from("xyz");
    result = longest(string1.as_str(), string2.as_str());
    // the lifetime of result continues after here    
    // the lifetime of string2 ends here    
    // the lifetime of string1 continues after here    
    /* if result refers to string1 then it is fine,       
       because lifetime of result and string1 have same lifetime */
    /* but if result refers to string2, then it is not fine,      
       because lifetime of result is longer than string2,       
       hence violating the lifetime annotation */    
    println!("The longest string is {}", result);

So the function signature:

fn longest<'a>(x: &'a str, y: &'a str) -> &'a str

...could be interpreted as the return value should never have a longer lifetime than any of the inputs. And I think this is inline with the general rule of borrow: The subject of the reference should live as long or longer as the variable reference it..

In the first entry in this series, I noted that The plan is that, after I become proficient in Rust, I can return to these series of posts and cringe at my ignorance! I think this post, out of the others in this series is that one where that statement is truest for the most! 😂

Anyways, I think these were the main points from reading through chapter 10. This chapter took longer that expected so I am really itching to continue with the rest of the book, hopefully no more stumps! 

Sunday, January 12, 2020

Learning Rust - Day 5 - Common Collections and Error Handling

This entry captures the highlights of reading through chapter 8 (Common Collections) and chapter 9 (Error Handling) of The Book. It marks my fourth entry in the Learning Rust journey. For more entries like this, see the posts with learning rust label.

I hang out in beginners section of Rust channel on Discord, and every now and then I see folks come around and say something like "I have been fighting with the borrow checker". I think that phrase came to life for me in this learning session. The last section of chapter 8 has 3 exercises, and while trying to work on those exercises, I found myself in situation where it feels like I am wrestling with the borrow checker! - I know, I know, the borrow checker is just trying to save me from my own stupidity - but right now it feel more like a struggle! 😂 And I guess these early encounters with the borrow checker signifies the beginning of my initiation rights towards becoming a Rusteacean?

Anyways, in these post, as I do, I pen down some of the things that stood out for me.

First thing was that + has a wired API when used for concatenating strings. It requires the second operand to be a borrow, while basically making the first string unusable after concatenation.

let hello = String::from("Hello");
let world = String::from(" world");
let hello_world = hello + &world; // world has to be a borrow
println!("{}", hello_world);
println!("{}", hello); // hello can't be used anymore due to move

The + has to be the most natural API for concatenating strings in previous languages I have worked with. Not sure the explanation for this unique incarnation in Rust. It seems, in other to get the more familiar behaviour, the format! macros should be used, and that basically provides string interpolation functionality:

let hello = String::from("Hello");
let world = String::from(" world");
let hello_world = format!("{}-{}",hello, world);
println!("{}", hello_world);
println!("{}", hello); // still usable
println!("{}", world); // still usable

I also learnt that strings in Rust does not support indexing. Which means one cannot access an individual characters in a string by referencing it by index. That is, this would lead to compile time error:

let hello = String::from("Hello world");
println!("{}", hello[0])

This is due to the fact that Rust stores string as an array (vector) of bytes, plus it supports UTF-8 by default. This means that a character could actually be represented by more than 1 byte, and in such a case, indexing that only returns a byte would make no sense. As the example that was given in the book shows:

print!("{}", String::from("Здравствуйте").len());

The above might look like being composed of 12 characters, and hence be represented by Rust with a vector with 12 slots. In reality this is not the case. The above is backed by a 24 long vector. Hence providing a facility for allowing indexing into a string to return a character does not makes sense.

What Rust provides, instead is ability to retrieve slices from a String. For example:

println!("{}", &hello[0..1])

And also the ability to turn a String into a unicode scalar values, or bytes, which can then be iterated over.

Also learnt Rust has a thing called deref coercion although this was not covered in this chapters. It would be in chapter 15.

While reading through the section on Hash Map, the or_insert caught my attention. The form it appeared in the book is similar to the following:

let mut map: HashMap<&str, i32> = HashMap::new();
let r = map.entry("1").or_insert(0);
*r += 10;
*r += 10;
println!("{}", map.get("1").unwrap()) // prints 10

I do not know enough Rust to know what is idiomatic or not, but the I find the UX around the API a little odd. The fact that an operation that inserts a value to a key, when that key is absent, also returns an handle to the value which can be used to mutate the value, feels overloaded. Also, one can assume if map.entry("1").or_insert(0) returns the value which can be later mutated, then map.entry("1") should also do the same, with the step of trying to create it if it is absent removed. But no, the API does not have that level of symmetry.

Also, I know mutation in Rust is safe because of the borrow checker, I am still not sure if that is a license to use mutation over the place. It might be easier for the compiler to ensure that mutation does not lead to bugs, not sure if that level of mutation would also ease a human in understanding the logic in the code.  I personally still have preference for the Functional programming approach of treating everything as immutable values, and deal with effects instead of mutation and side effects, as I think it also add to the simplicity and elegance of the code. Fingers crossed, and mind always open, as I learn more and use Rust, I get to understand its idiomatic use and philosophy. 

While going through the chapters for this study session, I had to revisit some of the information provided in Chapter 4 regarding ownership, slice types etc. I realised there are some cogent points I did not include in my Learning Rust - Day 2 - Getting Acquainted with Ownership post, about ownership/borrowing etc, so I am including them here. It is just some laws that governs how ownership et al. works. The items in bold are a verbatim copy from the Rust book.

  • Each value in Rust has a variable that’s called its owner.
  • This variable can either be a stack variable, that is, it points to data that resides in the stack or a heap variable, that is, it points to data on the heap.
  • When it points to the heap, then you can see the variable as containing the pointer, length, capacity etc: basically things you can see as ownership related data
  • There can only be one owner at a time.
  • When the owner goes out of scope, the value will be dropped.
  • When passing data on the heap to functions, instead of making a copy of the data, to pass the function, a pointer to the location of the data is passed instead. This is a reference. This act is called borrowing.
  • At any given time, you can have either one mutable reference or any number of immutable references. 

Turning to the Error handling chapter, I learnt about the panic = 'abort' that can be added to the Cargo.toml file. It has the effect of switching off the memory clean up Rust does when panic occurs. The memory would then have to be freed up by the operating system. Doing this has the added effect of reducing the size of the generated Rust binary. And by the way, it seems panic, is Rust lingo for exception.

I also learnt about the RUST_BACKTRACE environment variable that can be used to list the full path and actual portion of the user code that triggered the panic instead. You can see this as just having proper stack trace. interesting enough, this week I learnt this would no longer be needed, as recent changes to the Rust compiler now makes the panic messages to now pointing to the location where they were called, rather than core's internals 

Up until now, I never encountered the return keyword. But I now know that Rust has a return keyword, and that it is mostly use for explicit signifying of a return out of a function, especially where there could be multiple return path. Its usage is closely related to the elvis operator: ? which seems to embody the spirit of the do notation in languages like Haskell and PureScript. So far, I have only seen how to use it with Result type, but not the Option type.

As I mentioned in the beginning I had to fight with the borrow checker while implementing the exercises listed at the end of chapter 8. But what I found astonishing was amount of these errors were lifetime related!

Here is a list of some of the errors I remembered to capture:
  • these two types are declared with different lifetimes...
  • ^^^ borrowed value does not live long enough
  • ^ expected lifetime parameter
  • help: consider giving it a 'static lifetime: `&'static`
I guess I would be learning a ton when I start with the next chapter, which is about Generic Types, Traits... and you guess it...Lifetimes!!! I can't wait!

Wednesday, January 08, 2020

Learning Rust - Day 4 - Understanding Modules

I think I now get Modules in Rust. Well, sort of…I can’t be completely sure, but I think I have developed some form of logical heuristics that I can now use to think about the concept.

This is the 4th journal entry of my learning Rust journey. You can read other posts in this series by following the label learning rust.

Modules in Rust are a really slippery and confusing concept, so it took more than reading the 7th chapter of the book to arrive at some form of understanding. I had to scour through various blog posts, reddit posts, issues on Github, chats on Rust channel on Discord etc.

For such a mundane concept as Modules, I think it took more effort that it should, but then again, this is coming at no surprise, as I was already aware that Rust's module system is one of the most confusing parts of the language.

I won’t attempt a Rust module tutorial here; since this is a journal entry and also since I am not 100% sure if I have all the details locked down yet. But I would quickly pen down what I think is the core idea that helped me in getting the handle of the topic. If you know more about how the Rust module system works and fit together, and you spot a mistake in the things mentioned below, please do leave a comment! It would really be appreciated!

As anybody who have attempted to understand the module system in Rust will know, one of the confusing part is mapping the definition of the module, to the file system. The things that finally helped me was to think about the whole thing in the following ways:

  1. In Rust you have packages, which holds crates. And crates holds modules, where modules can be thought of as some sort of grouping mechanism for things like functions, traits, etc and helps in controlling the visibility of these things.
  2. Rust compilation unit forms a tree, where the root of the tree is the src/ file and/or the src/ file. Anything that should be included in compilation must be reachable somehow from these two files. The module system also forms a tree, with the root starting from these two files.
  3. If a module definition is seen in a file, i.e mod modulename, this should be seen as the creation of a submodule by the parent module (which is the current file that contains the module definition) - I think understanding this is quite key!
  4. To define a module starting from the root (src/ file and/or the src/ or in a file that is already part of the compilation unit, you write mod mymodule; question now is, where do you place the content of this module. I think there are two ways: Inline or filesystem.
    1. Inline. Where the content is placed in {} after the module definition.
      mod mymodule {...content...}
    2. Filesystem. The content of the module can be found on the file system. There are two variants for this:
      1. modules defined in or The file would be placed also in the src directory. So if you have mod mymodule; defined in ./src/ or ./src/, then the content of that module can be found in a file in the ./src directory.
      2. modules defined in other files apart from or If within there is another module definition, then remember this means is defining a sub module. The question now is, where would the content of this submodule be? Let us say the module definition is:
        mod mysubmodule;. It turns out the content would be found within an associated directory named /mysubmodule/. It seems apart from and, any file that defines a sub module (ie contains a module definition) also has an associated directory where the content of the defined module would be placed. And the associated directory carries the same name. So in this case if within we have mod mysubmodule; then there must be a directory named /mymodule/ where the content of the modules defined in would be placed. There are now two ways of populating this directory:
        1. module name pattern. In this approach, the content of the module is placed in a file named just as the module name. In this case, within /mymodule/ there would be a file named So the full path would be /src/mymodule/ I personally find this approach the simplest
        2. pattern. In this approach, the content of the module would be placed in a file named, that would then be placed in a directory matching the name of the defined module. In this case, the full path would be src/mymodule/mysubmodule/ I personally find this approach more confusing.
  5. All the definition of modules, submodules etc leads to the idea of a path (similar to what you have in a file system) where modules that are part of the tree can be reached by specifying their path. The path separator in this case is :: and the root is crate, hence you can have something like crate::mymodule::mysubmodule as the path that allows reaching mysubmodule
  6. When there is a need to use a definition within a module, It is then best, not to think of it as importing the module, but using the path to reach the module. Having the path that reaches the module from where it exist in the module tree then gives access to its content (subject to the visibility rules of course). This is what the use keyword allows. So at the top of a file, having use crate::mymodule::mysubmodule means the path to reach mysubmodule is in scope and this can be used to access the module.
Perhaps after I have spent more time with the language, and I am more certain about how the module system fits together,  I would be able to write a more extensive posts with step by step examples. But now, I would only mention that Rust's module system is atypical, hence the intuition from other programming languages does not help in understanding how modules work in Rust. In fact, what you know about modules (or related concepts from your favourite language) is probably going to be a stumbling block.

Here are some of the links I ran into, that helped in shedding more light to the topic.

Rust modules vs files
The Rust module system is too confusing
Inline module syntax is confusing when learning modules (GitHub issue)
Data point about the new module system learnability and musings about language stability (Rust Internals)

Apart from getting a handle of a module definition and its mapping to the file system, there are other points, I got from reading the 7th chapter that is worth mentioning:

  1.  Visibility is defined using the pub keyword.
  2. Structs can have parts of their defined field public while others non public. This is not the case with enums.

Even though this chapter took more effort than the previous chapters, I am still quite happy with the results. As mentioned in the previous post, I was a little bit anxious about approaching modules in Rust, so even though it took some time, I think I was able to get some good understanding in a reasonable amount of time.

So looking forward to continuing with the rest of the book!