unistd: avoid infinite loop caused by reserve_double_buffer_size

Functions such as Group::from_anything use reserve_double_buffer_size
in a loop, expecting it to return ERANGE if the passed limit is
reached.

However, the returned vector is passed as pointer to a libc function
that writes data into memory and doesn't update the length of the
Vec. Because of this, the previous code would never return ERANGE and
the calling loops would never exit if they hit a case where the
required buffer was larger than the maximum buffer.

This fixes the problem by checking the capacity rather than the
length.

Signed-off-by: Steven Danna <[email protected]>

Author: stevendanna

CHANGELOG.md

@@ -22,6 +22,11 @@ This project adheres to [Semantic Versioning](http://semver.org/).
   (#[1198](https://github.com/nix-rust/nix/pull/1198))
 
 ### Fixed
+
+- Fixed a bug in nix::unistd that would result in an infinite loop
+  when a group or user lookup required a buffer larger than
+  16KB. (#[1198](https://github.com/nix-rust/nix/pull/1198))
+
 ### Removed
 
 ## [0.17.0] - 3 February 2020

src/unistd.rs

@@ -507,13 +507,13 @@ pub fn mkfifo<P: ?Sized + NixPath>(path: &P, mode: Mode) -> Result<()> {
 }
 
 /// Creates new fifo special file (named pipe) with path `path` and access rights `mode`.
-/// 
+///
 /// If `dirfd` has a value, then `path` is relative to directory associated with the file descriptor.
-/// 
-/// If `dirfd` is `None`, then `path` is relative to the current working directory. 
-/// 
+///
+/// If `dirfd` is `None`, then `path` is relative to the current working directory.
+///
 /// # References
-/// 
+///
 /// [mkfifoat(2)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/mkfifoat.html).
 // mkfifoat is not implemented in OSX or android
 #[inline]
@@ -559,7 +559,7 @@ pub fn symlinkat<P1: ?Sized + NixPath, P2: ?Sized + NixPath>(
 fn reserve_double_buffer_size<T>(buf: &mut Vec<T>, limit: usize) -> Result<()> {
     use std::cmp::min;
 
-    if buf.len() >= limit {
+    if buf.capacity() >= limit {
         return Err(Error::Sys(Errno::ERANGE))
     }