Iterators

# Iterators

---
## What is Iterating?

> iterate (verb): to repeat a process, especially as part of a computer program ([Cambridge English Dictionary](https://dictionary.cambridge.org/dictionary/english/iterate?q=Iterate))

To *iterate* in Rust is to produce a sequence of items, one at a time.

---
## How do you Iterate?

* With an *Iterator*
* Commonly `.into_iter()`, `.iter_mut()` or `.iter()` on some collection
* There's also an `IntoIterator` trait for automatically creating an Iterator

---
## What is an Iterator?

* An object with a `.next()` method
  * The method provides `Some(data)`, or `None` once the data has run out
  * The object holds the iterator's *state*
* Some *Iterators* will take data from a collection (e.g. a Slice)
* Some *Iterators* will calculate each item on-the-fly
* Some *Iterators* will take data from another iterator, and then calculate something new

Note:

Technically, all iterators calculate things on-the-fly. Some own another iterator and use that as input to their calculation, and some have an internal state that they can use for calculation. `fn next(&mut self) -> Self::Item` can only access `Self` so it is about what `Self` contains.

* `struct SomeIter<T> where T: Iterator { inner: T }`
* `struct SomeOtherIter { random_seed: u32 }`

---
## Important to note

* Iterators are lazy
* Iterators are used all over the Rust Standard Library
* Iterators have hidden complexity that you can mostly ignore
* Iterators cannot be invalidated (unlike, say, C++)
* Some Iterators can wrap other Iterators

---
## Basic usage

1. You need to make an iterator
2. You need to pump it in a loop

```rust []
fn main() {
    let data = vec![1, 2, 3, 4, 5];
    let mut iterator = data.iter();
    loop {
        if let Some(item) = iterator.next() {
            println!("Got {}", item);
        } else {
            break;
        }
    }
}
```

---
## Basic usage

Same thing, but with `while let`.

```rust []
fn main() {
    let data = vec![1, 2, 3, 4, 5];
    let mut iterator = data.iter();
    while let Some(item) = iterator.next() {
        println!("Got {}", item);
    }
}
```

---
## Basic usage

Same thing, but with `for`

```rust []
fn main() {
    let data = vec![1, 2, 3, 4, 5];
    // for <variable> in <iterator>
    for item in data.iter() {
        println!("Got {}", item);
    }
}
```

---
## Basic usage

Same thing, but we let `for` call `.into_iter()` for us.

```rust []
fn main() {
    let data = vec![1, 2, 3, 4, 5];
    // for <variable> in <implements IntoIterator>
    for item in &data {
        println!("Got {}", item);
    }
}
```

---
## Three kinds of Iterator

<figure>
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
 "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">


<svg width="368pt" height="312pt"
 viewBox="0.00 0.00 367.84 311.60" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 307.6)">
<title>%3</title>
<polygon fill="#ffffff" stroke="transparent" points="-4,4 -4,-307.6 363.8388,-307.6 363.8388,4 -4,4"/>

<g id="node1" class="node">
<title>Vec</title>
<polygon fill="none" stroke="#000000" points="76.9801,-145.5 76.9801,-303.1 306.1343,-303.1 306.1343,-145.5 76.9801,-145.5"/>
<text text-anchor="middle" x="191.5572" y="-286.5" font-family="Times,serif" font-size="14.00" fill="#000000">struct Vec</text>
<polyline fill="none" stroke="#000000" points="76.9801,-278.3 306.1343,-278.3 "/>
<text text-anchor="middle" x="191.5572" y="-261.7" font-family="Times,serif" font-size="14.00" fill="#000000">start: *mut T</text>
<text text-anchor="middle" x="191.5572" y="-244.9" font-family="Times,serif" font-size="14.00" fill="#000000">len: usize</text>
<text text-anchor="middle" x="191.5572" y="-228.1" font-family="Times,serif" font-size="14.00" fill="#000000">cap: usize</text>
<polyline fill="none" stroke="#000000" points="76.9801,-219.9 306.1343,-219.9 "/>
<text text-anchor="middle" x="191.5572" y="-203.3" font-family="Times,serif" font-size="14.00" fill="#000000">fn iter(&self) -> VecIter</text>
<polyline fill="none" stroke="#000000" points="76.9801,-195.1 306.1343,-195.1 "/>
<text text-anchor="middle" x="191.5572" y="-178.5" font-family="Times,serif" font-size="14.00" fill="#000000">fn iter_mut(&mut self) -> VecIterMut</text>
<polyline fill="none" stroke="#000000" points="76.9801,-170.3 306.1343,-170.3 "/>
<text text-anchor="middle" x="191.5572" y="-153.7" font-family="Times,serif" font-size="14.00" fill="#000000">fn into_iter(self) -> VecIter</text>
</g>

<g id="node2" class="node">
<title>VecIntoIter</title>
<polygon fill="none" stroke="#000000" points="0,-.5 0,-108.5 115.1144,-108.5 115.1144,-.5 0,-.5"/>
<text text-anchor="middle" x="57.5572" y="-91.9" font-family="Times,serif" font-size="14.00" fill="#000000">struct VecIntoIter</text>
<polyline fill="none" stroke="#000000" points="0,-83.7 115.1144,-83.7 "/>
<text text-anchor="middle" x="57.5572" y="-67.1" font-family="Times,serif" font-size="14.00" fill="#000000">start: *mut T</text>
<text text-anchor="middle" x="57.5572" y="-50.3" font-family="Times,serif" font-size="14.00" fill="#000000">len: usize</text>
<text text-anchor="middle" x="57.5572" y="-33.5" font-family="Times,serif" font-size="14.00" fill="#000000">cap: usize</text>
<polyline fill="none" stroke="#000000" points="0,-25.3 115.1144,-25.3 "/>
<text text-anchor="middle" x="57.5572" y="-8.7" font-family="Times,serif" font-size="14.00" fill="#000000">Item=T</text>
</g>

<g id="edge1" class="edge">
<title>Vec:ii->VecIntoIter</title>
<path fill="none" stroke="#000000" d="M76.5572,-158.3C56.8669,-158.3 50.361,-140.0038 49.4397,-118.7743"/>
<polygon fill="#000000" stroke="#000000" points="52.9386,-118.5117 49.3919,-108.5282 45.9387,-118.5444 52.9386,-118.5117"/>
</g>

<g id="node3" class="node">
<title>VecIterMut</title>
<polygon fill="none" stroke="#000000" points="133.608,-17.3 133.608,-91.7 249.5064,-91.7 249.5064,-17.3 133.608,-17.3"/>
<text text-anchor="middle" x="191.5572" y="-75.1" font-family="Times,serif" font-size="14.00" fill="#000000">struct VecIterMut</text>
<polyline fill="none" stroke="#000000" points="133.608,-66.9 249.5064,-66.9 "/>
<text text-anchor="middle" x="191.5572" y="-50.3" font-family="Times,serif" font-size="14.00" fill="#000000">parent: &mut [T]</text>
<polyline fill="none" stroke="#000000" points="133.608,-42.1 249.5064,-42.1 "/>
<text text-anchor="middle" x="191.5572" y="-25.5" font-family="Times,serif" font-size="14.00" fill="#000000">Item=&mut T</text>
</g>

<g id="edge2" class="edge">
<title>Vec:im->VecIterMut</title>
<path fill="none" stroke="#000000" d="M306.5572,-182.3C323.135,-182.3 313.971,-159.8276 306.5572,-145 294.6315,-121.1486 279.5714,-125.4169 258.5572,-109 254.2001,-105.5961 249.708,-102.042 245.2103,-98.4516"/>
<polygon fill="#000000" stroke="#000000" points="247.1267,-95.5019 237.1363,-91.9744 242.7464,-100.9621 247.1267,-95.5019"/>
</g>

<g id="node4" class="node">
<title>VecIter</title>
<polygon fill="none" stroke="#000000" points="267.2756,-17.3 267.2756,-91.7 359.8388,-91.7 359.8388,-17.3 267.2756,-17.3"/>
<text text-anchor="middle" x="313.5572" y="-75.1" font-family="Times,serif" font-size="14.00" fill="#000000">struct VecIter</text>
<polyline fill="none" stroke="#000000" points="267.2756,-66.9 359.8388,-66.9 "/>
<text text-anchor="middle" x="313.5572" y="-50.3" font-family="Times,serif" font-size="14.00" fill="#000000">parent: &[T]</text>
<polyline fill="none" stroke="#000000" points="267.2756,-42.1 359.8388,-42.1 "/>
<text text-anchor="middle" x="313.5572" y="-25.5" font-family="Times,serif" font-size="14.00" fill="#000000">Item=&T</text>
</g>

<g id="edge3" class="edge">
<title>Vec:i->VecIter</title>
<path fill="none" stroke="#000000" d="M306.5572,-207.3C349.3916,-207.3 341.0267,-146.6381 328.9766,-101.7581"/>
<polygon fill="#000000" stroke="#000000" points="332.2753,-100.5589 326.1999,-91.8791 325.5364,-102.4531 332.2753,-100.5589"/>
</g>
</g>
</svg>
</figure>

---
## Three kinds of Iterator

* Borrowed (`data.iter()`)
* Mutably Borrowed (`data.iter_mut()`)
* Owned (`data.into_iter()`)

---
## But how did that for-loop work?

If a `for` loop calls `.into_iter()` how did we get a borrowed iterator?

```rust []
fn main() {
    let data = vec![1, 2, 3, 4, 5];
    for item in &data {
        // item is a &i32
        println!("Got {}", item);
    }
}
```

---
## But how did that for-loop work?

The `&` is load-bearing...

```rust [1-9|2|3|4-5]
fn main() {
    let data = vec![1, 2, 3, 4, 5];
    let temp = &data;
    // This is .into_iter() on a `&Vec` not a `Vec`!
    let iter = temp.into_iter();
    for item in iter {
        println!("Got {}", item);
    }
}
```

Note:

* `IntoIterator` is actually dependent on the context. Depending on the context it will produce an iterator with owned elements, with references to elements, with mutable references to elements.
* e.g. `impl<T, A> IntoIterator for Vec<T, A>` for owned
* `impl<'a, T, A> IntoIterator for &'a Vec<T, A>` for refs
* `impl<'a, T, A> IntoIterator for &'a mut Vec<T, A>` for mut refs

---
## Things you can make iterators from

* [Ranges](https://doc.rust-lang.org/std/ops/struct.Range.html) (`0..10` or `0..=9`)
* [Slices](https://doc.rust-lang.org/std/primitive.slice.html) (`&[T]`)
* Things that deref to a slice (like [`Vec<T>`](https://doc.rust-lang.org/std/vec/struct.Vec.html))
* A [`HashMap`](https://doc.rust-lang.org/std/collections/struct.HashMap.html) or [`BTreeMap`](https://doc.rust-lang.org/std/collections/struct.BTreeMap.html)
* A [String slice](https://doc.rust-lang.org/std/primitive.str.html) (to get [chars](https://doc.rust-lang.org/std/primitive.str.html#method.chars), or [bytes](https://doc.rust-lang.org/std/primitive.str.html#method.bytes), or [lines](https://doc.rust-lang.org/std/primitive.str.html#method.lines), or [words](https://doc.rust-lang.org/std/primitive.str.html#method.split_whitespace)...)
* A [Buffered I/O Reader](https://doc.rust-lang.org/std/io/trait.BufRead.html), to get [Lines of text](https://doc.rust-lang.org/std/io/trait.BufRead.html#method.lines)
* A [TCP Listener](https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming), to get [TCP Streams](https://doc.rust-lang.org/std/net/struct.TcpStream.html)
* Much more!

Note:

Technically a Range *is* an Iterator. Some people consider this to be a mistake.
Especially as `Range<T> where T: Copy` is not itself `Copy`.

---
## How does this work?

* Rust has some `traits` which describe how iterators work.
* We'll talk more about traits later!

---
# You can still enjoy it without knowing how it works

---
## Useful Iterator methods (1)

These consume the old Iterator and return a new Iterator:

* `skip(N)`
* `take(N)`
* `cloned()`
* `map(func)`
* `filter(func_returns_bool)`
* `filter_map(func_returns_option)`
* `zip(second_iterator)`

Note:

* `skip(N)` will skip the first N items from the underlying iterator, then just pass every other item through
* `take(N)` will take the first N items from the underlying iterator, then just tell you there is nothing left
* `cloned` takes an iterator that gives you references, and calls `.clone()` on each reference to create a new object
* `map(func)` will give you a new iterator that fetches an item from the underlying iterator, calls `func` with it, and gives you the result
* `filter(func)` will give you a new iterator that fetches an item from the underlying iterator, calls `func` with it, and if it's not true, refuses to give it to you and tries the next item instead
* `filter_map(func)` is both a filter and a map - the `func` should return an `Option<T>` and anything `None` is filtered out
* `zip` will take this iterator, and the given iterator, and produce a new iterator that produces two-tuples (`(itemA, itemB)`)

---
## Useful Iterator methods (2)

These actively fetch every item from the old Iterator and produce a single value:

* `sum()`
* `count()`
* `collect()`
* `max()` and `min()`
* `fold(initial, func)`
* `partition(func)`

Note:

* `sum` will add up every item, assuming they are numeric
* `count` will tell you how many items the iterator produced
* `collect` will take every item from the iterator and stuff it into a new collection (e.g. a `Vec<T>`)
* `max` and `min` find the largest/smallest item
* `fold` will maintain an accumulator, and call `func` with each item and the current value of the accumulator
* `partition` will create two new collections by taking every item from the iterator and stuffing it into one of two new collections

---
## Call chaining (1)

This style of code is idiomatic in Rust:

```rust [1-13|1-8|3|4|5|6|7]
/// Sum the squares of the even numbers given
fn process_data(data: &[u32]) -> u32 {
    data.iter()
        .cloned()
        .filter(|n| n % 2 == 0)
        .map(|n| n * n)
        .sum()
}

fn main() {
    let data = [1, 2, 3, 4];
    println!("result = {}", process_data(&data));
}
```

Note:

* Point out the type inference where Rust figures out `data` is an array of `u32` and not the default `i32`s.

---
## Call chaining (2)

What really happened:

```rust [1-13|1-8|3|4|5|6|7]
/// Sum the squares of the even numbers given
fn process_data(data: &[u32]) -> u32 {
    let ref_iter = data.iter();
    let value_iter = ref_iter.cloned();
    let evens_iter = value_iter.filter(|n| n % 2 == 0);
    let squares_iter = evens_iter.map(|n| n * n);
    squares_iter.sum()
}

fn main() {
    let data = [1, 2, 3, 4];
    println!("result = {}", process_data(&data));
}
```

Note:

For the more advanced students, this mini quiz is a good one:
https://dtolnay.github.io/rust-quiz/26