Consume APIs with reqwest

An HTTP client is a courier. The program writes the order — method, URL, headers, body — hands the envelope to the courier, and waits at the desk while the courier walks to the address, knocks, picks up the reply, and walks back. Most of the work of consuming an API is shaped like this: write the order well, hand it off, read the reply, deal with the times the reply did not arrive in the shape the program expected.

reqwest is the courier most Rust programs hire. Sean McArthur wrote it in 2016, on top of the hyper HTTP library he had built for the same problem at lower level. hyper handles the bytes-on-the-socket part — connection pools, TLS handshakes, the HTTP/2 frame format. reqwest sits on top and gives the program a kitchen-table API: build a Client, call .get(url), attach headers, await the response. The split between a low-level crate and a friendly facade is a pattern Rust borrowed from the Tokio ecosystem and one of the reasons the language can ship libraries that are both fast and pleasant.

A request is four parts — method, URL, headers, optional body. The lesson writes one without sending it and prints what each part holds.

#[derive(Deserialize, Debug)]
struct Repo {
    name: String,
    stargazers_count: u64,
    description: Option<String>,
}

fn show_request() {
    let client: Client = Client::builder() // allow:network we are teaching the reqwest crate
        .user_agent("learning-rust/0.1")
        .build()
        .expect("client builds");

    let url = "https://api.github.com/repos/rust-lang/rust";
    let mut headers = HeaderMap::new();
    headers.insert(ACCEPT, HeaderValue::from_static("application/json"));
    headers.insert(USER_AGENT, HeaderValue::from_static("learning-rust/0.1"));

    let request = client
        .get(url)
        .headers(headers)
        .build()
        .expect("request builds");

    println!("--- request shape ---");
    println!("method: {}", request.method());
    println!("url:    {}", request.url());
    println!("headers:");
    for (name, value) in request.headers() {
        println!("  {}: {}", name, value.to_str().unwrap_or("<binary>"));
    }
    println!();
}

Client::builder() is the courier company. The user agent is the name on the courier's badge — servers log it, and some refuse the request without one. client.get(url) starts building a RequestBuilder for a GET to that URL. .headers(headers) attaches the set, .build() freezes the whole thing into a Request value the program can inspect before sending. The headers map mirrors the way HTTP headers work on the wire — case-insensitive names, ASCII values, multiple values allowed per name.

A response is the reverse — status code, headers, body. The body is a stream of bytes the program decides how to interpret. For JSON APIs, the program asks serde_json to turn the bytes into a typed struct, and the struct refuses to hold anything that does not match.

const SAMPLE_JSON: &str = r#"{
  "name": "rust",
  "stargazers_count": 99000,
  "description": "Empowering everyone to build reliable software."
}"#;

fn show_response() {
    println!("--- response body (hardcoded sample) ---");
    println!("{}", SAMPLE_JSON);
    println!();

    let parsed: Repo = serde_json::from_str(SAMPLE_JSON).expect("valid JSON");

    println!("--- typed struct ---");
    println!("name:        {}", parsed.name);
    println!("stars:       {}", parsed.stargazers_count);
    match &parsed.description {
        Some(text) => println!("description: {}", text),
        None => println!("description: <none>"),
    }
    println!();
}

The hardcoded JSON stands in for the bytes a real response would deliver, because the lesson binary cannot make a network call without making the snapshot different on every machine. The shape is honest. A real call to the GitHub API returns the same envelope — content-type, status line, JSON body — and serde_json::from_str turns it into the Repo struct exactly the way response.json::<Repo>() does inside reqwest. The Option<String> on description is the right way to encode a field the server might omit; serde reads null or absent as None and any string as Some.

The interesting half of consuming an API is the failure modes. The network drops, the server returns 500, the JSON is truncated halfway through, the field that was a string yesterday is an integer today.

fn show_errors() {
    let bad_url = "not-a-url";
    let bad: Result<reqwest::Url, _> = bad_url.parse(); // allow:network we are teaching the reqwest crate
    println!("--- failure modes ---");
    match bad {
        Ok(_) => println!("url ok"),
        Err(why) => println!("bad url: {}", why),
    }

    let truncated = r#"{"name": "rust", "stargazers_count":"#;
    let parsed: Result<Repo, _> = serde_json::from_str(truncated);
    match parsed {
        Ok(_) => println!("parsed ok"),
        Err(why) => println!("bad json: {}", why),
    }

    let wrong_shape = r#"{"name": 42, "stargazers_count": 1}"#;
    let parsed: Result<Repo, _> = serde_json::from_str(wrong_shape);
    match parsed {
        Ok(_) => println!("parsed ok"),
        Err(why) => println!("wrong type: {}", why),
    }
}

A typoed URL fails at parse time with "relative URL without a base." A response that gets cut mid-byte fails the JSON parser at the exact column the parser ran out. A field that arrived as the wrong type — "name": 42 instead of "name": "rust" — fails with a message that names the offending field and the line and column where it lived. Each of these is the courier handing back a clearly labeled package instead of dropping it on the floor. Every API consumer eventually writes a wrapper that turns these into one error type the rest of the program can match on, and the part of reqwest worth knowing is that the underlying errors are detailed enough to make the wrapper useful instead of vague.

The lesson runs the three demonstrations end to end.

--- request shape ---
method: GET
url:    https://api.github.com/repos/rust-lang/rust
headers:
  accept: application/json
  user-agent: learning-rust/0.1

--- response body (hardcoded sample) ---
{
  "name": "rust",
  "stargazers_count": 99000,
  "description": "Empowering everyone to build reliable software."
}

--- typed struct ---
name:        rust
stars:       99000
description: Empowering everyone to build reliable software.

--- failure modes ---
bad url: relative URL without a base
bad json: EOF while parsing a value at line 1 column 36
wrong type: invalid type: integer `42`, expected a string at line 1 column 11

The request block prints the method, URL, and headers that would go out on the wire if the program called client.execute(request). The response block parses the sample JSON into a Repo and prints the fields. The failure block prints what each kind of error looks like so the reader knows what to match on.

One piece worth naming — async vs blocking. The code above uses reqwest::blocking::Client, which spins up a thread per call and blocks until the response arrives. The async client (reqwest::Client) returns a Future and lets the caller .await it inside a Tokio runtime. The async path is what a web server uses to handle a thousand concurrent outbound calls on a handful of threads. The blocking path is what a CLI tool or a script uses when there is no runtime to plug into. Same crate, same API shape, two front doors.

The thing reqwest does not solve is the contract with the server. The struct shape the program declares is a guess at what the server will return; the day the server changes a field name, the program's deserializer breaks and the request that worked yesterday fails at parse time. The next lesson trades the human-readable JSON contract for an SDK — a Rust crate the API provider publishes, with the types already shaped to match the server's API and updated every time the API changes.