-
Most io.Reader streams are consumable once; reading advances state. Do not assume a reader can be re-read without special handling
-
If you must read data multiple times, buffer it once and recreate readers on demand:
- Use
io.ReadAll (or a limited read) to obtain []byte, then create fresh readers via bytes.NewReader(buf) or bytes.NewBuffer(buf) for each reuse
- For strings, use
strings.NewReader(s); you can Seek(0, io.SeekStart) on *bytes.Reader to rewind
-
For HTTP requests, do not reuse a consumed req.Body. Instead:
- Keep the original payload as
[]byte and set req.Body = io.NopCloser(bytes.NewReader(buf)) before each send
- Prefer configuring
req.GetBody so the transport can recreate the body for redirects/retries: req.GetBody = func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(buf)), nil }
-
To duplicate a stream while reading, use io.TeeReader (copy to a buffer while passing through) or write to multiple sinks with io.MultiWriter
-
Reusing buffered readers: call (*bufio.Reader).Reset(r) to attach to a new underlying reader; do not expect it to “rewind” unless the source supports seeking
-
For large payloads, avoid unbounded buffering; consider streaming, io.LimitReader, or on-disk temporary storage to control memory
-
Use io.Pipe to stream without buffering the whole payload:
- Write to
*io.PipeWriter in a separate goroutine while the reader consumes
- Always close the writer; use
CloseWithError(err) on failures
io.Pipe is for streaming, not rewinding or making readers reusable
-
Warning: When using io.Pipe (especially with multipart writers), all writes must be performed in strict, sequential order. Do not write concurrently or out of order—multipart boundaries and chunk order must be preserved. Out-of-order or parallel writes can corrupt the stream and result in errors.
-
Streaming multipart/form-data with io.Pipe:
pr, pw := io.Pipe(); mw := multipart.NewWriter(pw); use pr as the HTTP request body- Set
Content-Type to mw.FormDataContentType()
- In a goroutine: write all parts to
mw in the correct order; on error pw.CloseWithError(err); on success mw.Close() then pw.Close()
- Do not store request/in-flight form state on a long-lived client; build per call
- Streamed bodies are not rewindable; for retries/redirects, buffer small payloads or provide
GetBody