In Rust, I want to read (using BufReader) into a buffer array to fill it. I want to read the exact number of bytes, except in case of EOF, where reading less is fine. I have implemented a helper method myself, as follows, and that works. However I wonder if there is no convenience method that already does exactly that.
pub fn read_fully<R: Read>(read: &mut R, buffer: &mut [u8]) -> Result<usize, Error> {
let mut count: usize = 0;
loop {
let r = read.read(&mut buffer[count..])?;
if r == 0 {
return Ok(count);
}
count += r;
}
}
With each existing method I found there seems to be a problem:
- “read” may read less (e.g. for stdin).
- “read_exact” doesn’t work well with EOF.
- “read_to_end” expects a Vec and not an array, and will read everything, possibly expanding the vector.
Here’s an idea, which I’m not sure is better than yours, but it also works:
pub fn read_fully<R: Read>(read: &mut R, buffer: &mut [u8]) -> Result<usize, Error> {
let mut data = Vec::with_capacity(buffer.len());
read.take(buffer.len().try_into().unwrap())
.read_to_end(&mut data)?;
buffer[..data.len()].copy_from_slice(&data);
Ok(data.len())
}
It’ll probably be much less efficient. Whether it’s clearer is up to you to decide.
It seems that ErrorKind::Interrupted should be ignored. So the solution would become:
pub fn read_fully<R: Read>(mut read: R, buffer: &mut [u8]) -> Result<usize, Error> {
let mut count: usize = 0;
loop {
let r = read.read(&mut buffer[count..]);
match r {
Ok(0) => {
return Ok(count)
},
Ok(x) => {
count += x
},
Err(e) => {
if e.kind() == ErrorKind::Interrupted {
// retry
} else {
return Err(e);
}
}
};
}
}
Here’s how you can do it shorter and cleaner, but using an experimental API:
#![feature(read_buf, core_io_borrowed_buf)]
use std::io::{BorrowedBuf, Error, ErrorKind, Read};
pub fn read_fully<R: Read>(read: &mut R, buffer: &mut [u8]) -> Result<usize, Error> {
let mut buffer = BorrowedBuf::from(buffer);
match read.read_buf_exact(buffer.unfilled()) {
Ok(()) => {}
Err(err) if err.kind() == ErrorKind::UnexpectedEof => {}
Err(err) => return Err(err),
}
Ok(buffer.len())
}
At this point, you may want to just have a BorrowedCursor
as a parameter:
#![feature(read_buf, core_io_borrowed_buf)]
use std::io::{BorrowedCursor, Error, ErrorKind, Read};
pub fn read_fully<R: Read>(read: &mut R, buffer: BorrowedCursor<'_>) -> Result<(), Error> {
match read.read_buf_exact(buffer) {
Err(err) if err.kind() == ErrorKind::UnexpectedEof => Ok(()),
v => v,
}
}
This allows neat things like using an uninitialized buffer.
read_to_end()
is not what you want. It will read more bytes than wanted until the end of the file. I don’t think there is another method you can use, your custom code is fine.One thing you may want to do is to ignore
ErrorKind::Interrupted
and just repeat the read in this case.Yes I tried read_to_end() and found it can read more bytes… I changed the question. Thanks about the tip for ErrorKind::Interrupted: when reading about it, it seems the loop should ignore this and continue. This makes it more complicated, but it seems to make sense.
Quick note. You should take
R: Read
directly instead of&mut R
per API guidelines@ThomasMueller Mark it as
mut
(mut read: R
).Show 2 more comments