I/O

I/O#

Go’s I/O surface is built around two interfaces: io.Reader and io.Writer. Files, network sockets, byte buffers, compressed streams, and HTTP request bodies all satisfy them, so the same helpers (io.Copy, io.ReadAll, bufio.Scanner) work everywhere. os covers the process surface; encoding/json, encoding/binary, and fmt cover serialisation and formatting.

For networking I/O, see Networking. For CLI argument handling, see CLI.

Files#

os.ReadFile and os.WriteFile are the one-shot path. os.Open and os.Create give a *os.File for streaming.

import "os"

data, err := os.ReadFile("config.json")
if err != nil { return err }

err = os.WriteFile("out.txt", []byte("hello\n"), 0o644)
if err != nil { return err }

f, err := os.Open("input.log")
if err != nil { return err }
defer f.Close()

out, err := os.Create("output.log")
if err != nil { return err }
defer out.Close()

Readers and writers#

io.Reader, io.Writer, io.Closer, and their combinations (io.ReadWriteCloser) are the operator’s main abstraction.

import "io"

n, err := io.Copy(dst, src)              // copy any reader to any writer
data, err := io.ReadAll(r)               // drain a reader

r := io.LimitReader(src, 1024)           // cap at 1 KB
tee := io.TeeReader(src, log)            // split a stream

A function should accept the smallest interface it needs; io.Reader covers most read-only callers.

func summarize(r io.Reader) (int, error) {
    data, err := io.ReadAll(r)
    if err != nil { return 0, err }
    return len(data), nil
}

Buffered I/O#

bufio wraps a reader / writer with a buffer. bufio.Scanner gives line- or token-iteration; bufio.Reader exposes the ReadString / ReadBytes family.

import "bufio"

f, _ := os.Open("input.log")
defer f.Close()

sc := bufio.NewScanner(f)
sc.Buffer(make([]byte, 0, 1<<20), 1<<20)   // raise the line cap
for sc.Scan() {
    line := sc.Text()
    if strings.Contains(line, "error") {
        fmt.Println(line)
    }
}
if err := sc.Err(); err != nil { return err }

For writes, bufio.Writer plus defer w.Flush().

w := bufio.NewWriter(out)
defer w.Flush()
for _, item := range items {
    fmt.Fprintln(w, item)
}

Standard streams#

os.Stdin, os.Stdout, os.Stderr are *os.File values. fmt.Println writes to stdout; fmt.Fprintln lets the operator pick the writer.

fmt.Fprintln(os.Stderr, "warning:", msg)

sc := bufio.NewScanner(os.Stdin)
for sc.Scan() {
    fmt.Println(strings.ToUpper(sc.Text()))
}

JSON#

encoding/json covers the typical case. The operator marshals to a struct (with tags) for shape-driven serialisation; map[string]any for dynamic data.

import "encoding/json"

type Config struct {
    Host string `json:"host"`
    Port int    `json:"port"`
}

var c Config
if err := json.Unmarshal(data, &c); err != nil { return err }

out, err := json.MarshalIndent(c, "", "  ")
if err != nil { return err }

For streaming, json.Decoder and json.Encoder.

dec := json.NewDecoder(r)
for {
    var item Item
    if err := dec.Decode(&item); err == io.EOF { break } else if err != nil { return err }
    handle(item)
}

enc := json.NewEncoder(w)
enc.SetIndent("", "  ")
for _, item := range items {
    if err := enc.Encode(item); err != nil { return err }
}

Binary#

encoding/binary handles fixed-format binary payloads with explicit endianness.

import "encoding/binary"

var port uint16
if err := binary.Read(r, binary.BigEndian, &port); err != nil { return err }

if err := binary.Write(w, binary.LittleEndian, header); err != nil { return err }

Strings, bytes, builders#

strings.Reader and bytes.Buffer adapt strings / byte slices to the reader / writer interfaces. strings.Builder is the operator’s quadratic-free string concatenator.

r := strings.NewReader("hello")
io.Copy(os.Stdout, r)

var b bytes.Buffer
b.Write(header); b.Write(payload)
data := b.Bytes()

var sb strings.Builder
for i := 0; i < 100; i++ { fmt.Fprintf(&sb, "%d ", i) }
out := sb.String()

Formatting (fmt)#

fmt is the operator’s printf. Verbs: %v (default), %+v (struct with field names), %#v (Go syntax), %s / %q, %d / %x, %f / %g, %w (wrap an error), %T (type), %t (bool).

fmt.Printf("%-10s %5d %.2f%%\n", "cpu", 12, 12.345)
fmt.Sprintf("%+v", user)                 // {Name:rk Age:30}
fmt.Errorf("loadConfig %q: %w", path, err)

Paths#

path/filepath for filesystem paths (handles OS separator); path for forward-slash paths (URL-style).

import "path/filepath"

p := filepath.Join("/etc", "app", "config.json")
base := filepath.Base(p)                   // "config.json"
dir  := filepath.Dir(p)                    // "/etc/app"
abs, err := filepath.Abs("relative/path")

Process and env#

os.Getenv, os.Setenv, os.Getwd, os.Exit. os/exec for running subprocesses.

home := os.Getenv("HOME")

cmd := exec.Command("ip", "-j", "addr")
out, err := cmd.Output()                   // captures stdout
if err != nil { return err }

cmd2 := exec.CommandContext(ctx, "long-job")
cmd2.Stdout, cmd2.Stderr = os.Stdout, os.Stderr
err = cmd2.Run()

References#

Contents