I/O and Pipelines

I/O and Pipelines#

PowerShell carries six output streams, not the three of a Unix shell. The pipeline only transports the Success stream; the others (Error, Warning, Verbose, Debug, Information) surface to the host or to redirection targets. Pipelines pass typed objects, so downstream cmdlets reach for properties by name instead of column position. Parallel execution comes in two flavors: runspace pools through ForEach-Object -Parallel (7+) and jobs through Start-Job / Start-ThreadJob.

Streams#

Every cmdlet emits into one of six named streams. The pipeline carries only the Success stream; the others surface separately.

#

Stream

Cmdlet

Used for

1

Success

Write-Output

Pipeline traffic; the result the operator wanted

2

Error

Write-Error

Errors and exceptions

3

Warning

Write-Warning

Warnings

4

Verbose

Write-Verbose

Detail visible with -Verbose

5

Debug

Write-Debug

Developer-only output

6

Information

Write-Information

General-purpose, structured info (7+)

 
        flowchart LR
  P[cmdlet] --> S[(Success · 1)]
  P --> E[(Error · 2)]
  P --> W[(Warning · 3)]
  P --> V[(Verbose · 4)]
  P --> D[(Debug · 5)]
  P --> I[(Information · 6)]
  S --> NEXT[downstream cmdlet]

The Success stream is for the answer; everything else is for the commentary. The discipline matters because the pipeline only consumes Success; a cmdlet that Write-Output-s a log message pollutes whatever reads it. Use Write-Verbose or Write-Information instead.

Write-Output 'answer'
Write-Verbose 'log message' -Verbose
Write-Warning 'about to retry'

Redirection#

Redirection rewires a stream for one command before the command runs. Operators are stream-number prefixed.

Operator

Effect

> file

Success → file, truncate

>> file

Success → file, append

2> file

Error → file

2>&1

Error → Success (merge)

3> file, 3>&1

Warning → file or Success

4> file, 4>&1

Verbose → file or Success

5> file, 5>&1

Debug → file or Success

6> file, 6>&1

Information → file or Success

*> file

All streams to file

2>$null

Discard the error stream

*>$null

Discard every stream

 
Get-Process > processes.txt
./build.ps1 *> build.log
Get-ChildItem C:\missing 2>$null

Order matters. cmd > all.log 2>&1 sends both streams to all.log. cmd 2>&1 > all.log sends error to the terminal and only success to the file, because PowerShell processes redirections left to right.

$null and Out-Null#

Two ways to discard pipeline output. Both produce the same effect; $null = expr is faster than | Out-Null because Out-Null keeps the pipeline running.

$null = $list.Add('x')                  # discard the int return
$list.Add('x') | Out-Null
$list.Add('x') > $null                  # same idea via redirection

Object Pipeline#

A pipeline chains cmdlets so each step’s Success output becomes the next step’s input. Unlike bash, the data is typed .NET objects, not bytes; downstream cmdlets reach for properties by name. The pipeline is streaming; an advanced function with a process block emits each object as it arrives, so memory stays bounded even on huge inputs.

Get-ChildItem -Recurse -Filter *.log |
    Where-Object Length -gt 1KB |
    Sort-Object Length -Descending |
    Select-Object -First 10 Name, Length

        flowchart LR
  A[Get-ChildItem] -->|FileInfo| B[Where-Object]
  B -->|FileInfo| C[Sort-Object]
  C -->|FileInfo| D[Select-Object]
  D --> OUT[(host)]

The five most-reached-for pipeline cmdlets.

Cmdlet

Alias

Effect

Where-Object

? / where

Filter input by a predicate

ForEach-Object

% / foreach

Transform each input

Select-Object

select

Project columns, -First, -Last, -Unique

Sort-Object

sort

Sort by property

Group-Object

group

Group by property, return { Name, Count, Group }

 
Get-Process |
    Where-Object WorkingSet -gt 100MB |
    Sort-Object WorkingSet -Descending |
    Select-Object -First 5 Name, WorkingSet

Parallel Pipelines#

Two ways to run a pipeline in parallel.

ForEach-Object -Parallel (PowerShell 7+) uses runspace pools to run a script block against each input on a worker thread. -ThrottleLimit caps concurrency.

1..20 | ForEach-Object -Parallel {
    Start-Sleep $_
    "done $_"
} -ThrottleLimit 4

Inside the parallel script block, $_ is the current input. Reach outer variables with $using:var.

$apiBase = 'https://target.example.com'
$endpoints | ForEach-Object -Parallel {
    Invoke-RestMethod "$using:apiBase/$_"
} -ThrottleLimit 8

Jobs#

Jobs run in background processes (Start-Job) or background threads (Start-ThreadJob). Wait-Job blocks; Receive-Job collects output; Remove-Job cleans up.

$job = Start-Job -ScriptBlock {
    Invoke-RestMethod https://target/healthz
}
$job | Wait-Job -Timeout 30 | Receive-Job
$job | Remove-Job

# thread job (lighter than Start-Job; comparable to -Parallel)
$hosts | ForEach-Object {
    Start-ThreadJob -ScriptBlock { ssh $using:_ uptime }
} | Wait-Job | Receive-Job

External Native Commands#

Calling a native (non-cmdlet) program returns a string per line on the Success stream. PowerShell does not parse the output; 2>&1 works, but the operator usually wants to inspect $LASTEXITCODE afterwards to detect failure.

$out = & git status --porcelain 2>&1
if ($LASTEXITCODE -ne 0) { throw "git failed" }

./somebinary --flag value
$LASTEXITCODE                              # exit code of the last native call

The & call operator dispatches to a command stored in a variable; the . operator dot-sources a script into the current scope. Native programs do not honour the object pipeline; piping a native program’s output to a cmdlet still passes strings.

Common Tasks#

Tee output to a file while keeping it on the pipeline.

Get-Process | Tee-Object processes.txt | Format-Table

Capture both Success and Error to one file.

./script.ps1 *> run.log

Silence Success, keep Error.

./script.ps1 > $null

Silence everything.

./script.ps1 *> $null

Send Success to one file, Error to another.

./script.ps1 > out.log 2> err.log

Read a file line by line, streaming.

Get-Content input.txt | ForEach-Object { $_.ToUpper() }

Read a file as an array.

$lines = Get-Content input.txt

Compare two sorted streams.

Compare-Object (Get-Content a.txt) (Get-Content b.txt)

Run N items in parallel with a throttle.

$hosts | ForEach-Object -Parallel {
    ssh $_ uptime
} -ThrottleLimit 8

Capture both stdout and exit code of a native command.

$out = & ./bin --flag 2>&1
$rc  = $LASTEXITCODE

References#

  • Get-Help about_Pipelines, Get-Help about_Redirection, Get-Help about_Pipelines_Parallel, Get-Help about_Jobs.

  • Overview for the language-level reading and writing.

  • Patterns for $ErrorActionPreference and the advanced-function template that hooks into these streams.

  • Standard I/O for the OS-level view of standard streams; PowerShell’s success and error map onto stdout and stderr when talking to native programs.

  • about_Output_Streams

Contents