Skip to content

Commit d9401bb

Browse files
authored
Rollup merge of rust-lang#53981 - fbernier:patch-1, r=sfackler
Implement initializer() for FileDesc Here was my initial issue: ```rust use std::process::{Command}; fn main() { let output = Command::new("curl").arg("-s").arg("http://ovh.net/files/100Mio.dat").output(); println!("{:?}", output.unwrap().stdout.len()); } ``` ``` ~/stuff ❯❯❯ time ./dwl 104857600 ./dwl 16.22s user 1.80s system 23% cpu 1:15.24 total ``` ```rust use std::process::{Command, Stdio}; fn main() { let child = Command::new("curl").arg("-s").arg("http://ovh.net/files/100Mio.dat").stdout(Stdio::piped()).spawn(); let output = child.unwrap().wait_with_output().unwrap(); println!("{:?}", output.stdout.len()); } ``` ``` ~/stuff ❯❯❯ time ./dwl2 104857600 ./dwl2 0.64s user 2.18s system 5% cpu 53.072 total ``` As you can see the first version is spending much more time in userland and also uses more cpu. With the help of @programble, @talchas and @habnabit on the rust IRC, we discovered that the slow version uses two pipes, one for `stdin` and one for `stderr` and in that case it polls when going through [this function](https://github.com/rust-lang/rust/blob/master/src/libstd/sys/unix/pipe.rs#L82). The polling calls `read_to_end` on the pipes repetitively and this results in zeroing its internal buffer each time. To avoid this zeroing, `FileDesc` needs to implement `initializer`. We see no reason why it [wouldn't work with uninitialized memory](https://doc.rust-lang.org/1.26.1/src/std/io/mod.rs.html#534) so this PR fixes that. Here is some tracing of the slow program: ![image](https://user-images.githubusercontent.com/147585/45133180-ed8a2d80-b161-11e8-9ec7-09979ec96145.png) versus the fast program: ![image](https://user-images.githubusercontent.com/147585/45133216-0c88bf80-b162-11e8-908e-ff81d59239fb.png) I have not tested the change yet but will try to build it tomorrow.
2 parents afe35b6 + 28745a6 commit d9401bb

File tree

1 file changed

+6
-1
lines changed

1 file changed

+6
-1
lines changed

src/libstd/sys/unix/fd.rs

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
#![unstable(reason = "not public", issue = "0", feature = "fd")]
1212

1313
use cmp;
14-
use io::{self, Read};
14+
use io::{self, Read, Initializer};
1515
use libc::{self, c_int, c_void, ssize_t};
1616
use mem;
1717
use sync::atomic::{AtomicBool, Ordering};
@@ -270,6 +270,11 @@ impl<'a> Read for &'a FileDesc {
270270
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
271271
(**self).read(buf)
272272
}
273+
274+
#[inline]
275+
unsafe fn initializer(&self) -> Initializer {
276+
Initializer::nop()
277+
}
273278
}
274279

275280
impl AsInner<c_int> for FileDesc {

0 commit comments

Comments
 (0)