I/O

I/O#

Rust’s I/O surface is in std::io and std::fs. Two traits, Read and Write, abstract every readable / writable thing; BufReader and BufWriter add buffering; Seek adds positional access. serde (with serde_json, toml, bincode, etc.) handles serialisation.

For networking I/O, see Networking. For CLI argument handling, see CLI.

Files#

std::fs has the one-shot helpers; File plus buffered wrappers do the streaming.

use std::fs;

let text = fs::read_to_string("config.toml")?;             // owned String
let bytes: Vec<u8> = fs::read("logo.png")?;
fs::write("out.txt", "hello\n")?;
fs::create_dir_all("dir/a/b")?;

use std::fs::File;
use std::io::{BufReader, BufWriter, Read, Write};

let mut r = BufReader::new(File::open("input.log")?);
let mut buf = String::new();
r.read_to_string(&mut buf)?;

let mut w = BufWriter::new(File::create("out.log")?);
writeln!(w, "count={}", 42)?;
w.flush()?;                                                  // not Drop-flushed silently

BufReader’s Drop flushes its underlying Write only if it is also wrapping a Write; for BufWriter always call flush (errors during drop are silently swallowed otherwise).

Lines and tokens#

BufRead::lines returns an iterator over Result<String, io::Error>.

use std::io::{BufRead, BufReader};

let r = BufReader::new(File::open("/var/log/syslog")?);
for line in r.lines() {
    let line = line?;
    if line.contains("error") { println!("{line}"); }
}

For binary frames, read_exact / read_to_end give known-length / unbounded reads.

let mut header = [0u8; 16];
r.read_exact(&mut header)?;

Standard streams#

std::io::stdin, stdout, stderr return handles the operator wraps with BufReader / BufWriter when performance matters.

use std::io::{self, BufRead, Write};

let stdin = io::stdin();
let stdout = io::stdout();
let mut out = stdout.lock();                         // amortise locking

for line in stdin.lock().lines() {
    let line = line?;
    writeln!(out, "{}", line.to_uppercase())?;
}

For diagnostics, eprintln!.

eprintln!("warning: {msg}");

Formatting#

println! / write! / format! use the format string syntax in std::fmt. The operator’s verbs.

Verb

Effect

{}

Display

{:?}

Debug

{:#?}

Debug pretty-print

{x}

capture variable x (Rust 2021+)

{x:>10}

right-align width 10

{:.2}

2 decimals

{:08x}

8-char hex, zero-padded

 
let port = 8080;
println!("port={port}");
println!("hex={port:#08x}");                 // hex=0x001f90
println!("{user:#?}");                       // pretty Debug

For derived Debug, #[derive(Debug)] on the type.

JSON and other formats#

serde is the operator’s serialisation framework; data-format crates plug into it (serde_json, toml, bincode, ron, serde_yaml).

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
struct Config { host: String, port: u16 }

let cfg: Config = serde_json::from_str(&text)?;
let text = serde_json::to_string_pretty(&cfg)?;

For streaming, the deserializer reads from any Read.

let r = BufReader::new(File::open("events.ndjson")?);
for line in r.lines() {
    let event: Event = serde_json::from_str(&line?)?;
    handle(event);
}

Binary#

For raw little- / big-endian binary, bincode or byteorder (re-exported by bytes).

use byteorder::{BigEndian, ReadBytesExt, WriteBytesExt};

let port = r.read_u16::<BigEndian>()?;
w.write_u32::<BigEndian>(0xdeadbeef)?;

For arbitrary byte slices, &[u8] implements Read; Vec<u8> implements Write. The operator unit-tests parsers without files using these.

Memory-mapped#

The memmap2 crate maps a file into memory. Useful for search through large read-only inputs without copying.

use memmap2::Mmap;

let file = File::open("large.bin")?;
let map = unsafe { Mmap::map(&file)? };       // unsafe: backing file mutation
if let Some(pos) = memmem::find(&map, b"needle") { println!("at {pos}"); }

The unsafe here is the borrow-checker’s escape: another process could mutate the file under our mapping; the operator accepts this assumption when using mmap.

Atomic writes#

Write to a temp file, sync_data, then rename over the target.

fn write_atomic(path: &Path, data: &[u8]) -> std::io::Result<()> {
    let tmp = path.with_extension("tmp");
    {
        let mut f = File::create(&tmp)?;
        f.write_all(data)?;
        f.sync_data()?;
    }
    std::fs::rename(&tmp, path)
}

References#

Contents