Capstone: A Word-Frequency CLI Tool

This is the last lesson of the course, and instead of introducing one more concept, it asks you to build something — a small command-line tool that reads text, counts how often each word appears, and prints the words used most. This exact shape, 'read data, transform it, summarize it,' underlies an enormous number of real CLI tools: log analyzers, `wc`, spell checkers, even the ranking step behind a search engine. By the end of this lesson you'll have written a complete, compilable program that leans on nearly everything you've learned — ownership, `Result`, `HashMap`, and iterator chains chief among them — and you'll see exactly where each one earns its place, instead of being told to trust that it will someday.

One practical note before the code: a real CLI would read its input from a file (`std::fs::read_to_string`) or from piped standard input (`std::io::stdin().lines()`), but this lesson hardcodes a sample string instead, so every example here compiles and runs completely on its own, with nothing external to wire up. That's not a simplification of the interesting part — the interesting part is everything that happens *after* you have a `String`, and that logic is identical whether the string came from a hardcoded literal, a file, or a network request. Swap the source, keep the rest, and you have a real tool; that's the whole point of separating 'how did this String arrive' from 'what do I do with this String' from the start.

Stage one is turning raw text into a clean list of words. `split_whitespace()` gets you most of the way, but it isn't quite enough: "fox.", "fox,", and "Fox" would all count as different words even though a human reading the text would call them the same one. So each word also needs its punctuation stripped and its case flattened before it's a fair unit to count. This is exactly the iterator-chaining style from earlier in this course — `.map()` to transform each word, `.filter()` to drop anything that becomes empty after stripping punctuation, `.collect()` to materialize the result — chosen over a hand-written loop because each step names exactly one transformation, in the order it happens.

Stage two is counting, and `HashMap<String, u32>` is the natural structure for it: a word maps to how many times it's been seen. The idiomatic way to update a count is the entry API — `*counts.entry(word).or_insert(0) += 1` — which handles both cases ('I've never seen this word' and 'I've seen this word before') in a single expression instead of an `if let`/`else` pair. Notice that building the word list and building the count map are two separate steps operating on owned `String`s rather than borrowed `&str` slices; the `HashMap` needs to own its keys since it will outlive any single reference into the original text, which is ownership from early in this course showing up again in a data structure's design, not just in function signatures.

Stage three is sorting, and it exposes something worth knowing about `HashMap`: it has no defined iteration order, and that order can even change between runs of the same program. To rank words by frequency you first collect the map into a `Vec<(String, u32)>` with `.into_iter().collect()`, which gives you something with a real, controllable order, and then call `.sort_by()` with a comparator. Sorting by count alone isn't quite deterministic either, since ties (two words appearing the same number of times) would come out in whatever order the `HashMap` happened to produce — so the comparator here breaks ties alphabetically, which is a small detail but the difference between a tool whose output you can write a test against and one you can't.

The last design decision is what the core logic function returns. It would be simpler for `top_n_words` to just return a `Vec<(String, u32)>` and let an empty input silently produce an empty vector — but that hides a real distinction: 'there were legitimately no interesting words' and 'something was wrong with the input' are different situations, and a caller might want to react to them differently. Returning `Result<Vec<(String, u32)>, EmptyInputError>` makes that distinction explicit in the type signature itself, exactly like the `Result`-returning functions from earlier in this course. `main` is then the one place that actually decides what to do about failure — print an error and exit gracefully via a `match`, rather than letting the whole program panic over what might just be an empty file.

It's worth noticing what `main` does *not* contain: none of `tokenize`, `count_words`, or `top_n_words` reads a file, prints anything, or touches the outside world at all — they're pure functions, taking data in and returning data out. That split matters more than it looks like it does, because pure functions are exactly the kind you can drop straight into a `#[cfg(test)] mod tests` block from the last lesson and assert against, with no stdin to fake and no filesystem to mock. `main`'s only job becomes gathering input, calling the logic, and reporting the result — a shape sometimes called 'thin main, thick library,' and it's the same instinct that put your business logic in `lib.rs` and your entry point in `main.rs` back in the modules and crates lesson.

It's also worth seeing how the rest of this course would extend this exact program, even though we won't build it here. Counting several files at once is an easy fit for `thread::spawn` and a `mpsc` channel from the concurrency lesson — spawn one thread per file, have each one send back its own word counts, and merge them in the main thread. If several parts of the program needed to share and update one running tally at the same time instead, that's precisely the situation `Arc<Mutex<HashMap<String, u32>>>` exists for. None of that changes a single line of `tokenize` or `count_words` — it only changes how their results are gathered, which is exactly the payoff of keeping the core logic small and pure in the first place.

That's the course. You started with variables and ownership, moved through structs, enums, and pattern matching, learned to handle absence and failure explicitly with `Option` and `Result`, generalized your code with generics and traits, and finished with the tools — smart pointers, threads, tests — that turn 'a program that compiles' into 'a program you can trust in production.' The word-frequency counter you just built is small on purpose, but every idea in it scales: swap the hardcoded string for a file, add `clap` from crates.io for real argument parsing, and you have a genuine command-line tool. From here, the best next step isn't another lesson — it's a small project of your own, built the same way this one was: one compiling stage at a time.

fn tokenize(text: &str) -> Vec<String> {
    text.split_whitespace()
        .map(|word| {
            word.chars()
                .filter(|c| c.is_alphanumeric())
                .collect::<String>()
                .to_lowercase()
        })
        .filter(|word| !word.is_empty())
        .collect()
}

fn main() {
    let sample = "The quick brown fox jumps over the lazy dog. The dog barks, but the fox runs away!";
    let words = tokenize(sample);
    println!("{:?}", words);
}

use std::collections::HashMap;

fn tokenize(text: &str) -> Vec<String> {
    text.split_whitespace()
        .map(|word| {
            word.chars()
                .filter(|c| c.is_alphanumeric())
                .collect::<String>()
                .to_lowercase()
        })
        .filter(|word| !word.is_empty())
        .collect()
}

fn count_words(words: &[String]) -> HashMap<String, u32> {
    let mut counts = HashMap::new();
    for word in words {
        *counts.entry(word.clone()).or_insert(0) += 1;
    }
    counts
}

fn main() {
    let sample = "the quick brown fox the lazy dog the fox";
    let words = tokenize(sample);
    let counts = count_words(&words);
    println!("{:?}", counts);
}

use std::collections::HashMap;
use std::fmt;

#[derive(Debug)]
struct EmptyInputError;

impl fmt::Display for EmptyInputError {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "input text was empty after tokenizing")
    }
}

impl std::error::Error for EmptyInputError {}

fn tokenize(text: &str) -> Vec<String> {
    text.split_whitespace()
        .map(|word| {
            word.chars()
                .filter(|c| c.is_alphanumeric())
                .collect::<String>()
                .to_lowercase()
        })
        .filter(|word| !word.is_empty())
        .collect()
}

fn count_words(words: &[String]) -> HashMap<String, u32> {
    let mut counts = HashMap::new();
    for word in words {
        *counts.entry(word.clone()).or_insert(0) += 1;
    }
    counts
}

fn top_n_words(text: &str, n: usize) -> Result<Vec<(String, u32)>, EmptyInputError> {
    let words = tokenize(text);
    if words.is_empty() {
        return Err(EmptyInputError);
    }
    let counts = count_words(&words);
    let mut ranked: Vec<(String, u32)> = counts.into_iter().collect();
    ranked.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
    ranked.truncate(n);
    Ok(ranked)
}

fn main() {
    let sample = "The quick brown fox jumps over the lazy dog. The dog barks, but the fox runs away! The fox is quick.";

    match top_n_words(sample, 3) {
        Ok(top) => {
            println!("Top words:");
            for (word, count) in top {
                println!("  {word}: {count}");
            }
        }
        Err(e) => eprintln!("Error: {e}"),
    }
}