Table of Contents

Name

qalloc_makestatmap, qalloc_makedynmap, qalloc_loadmap - creates or loads a map file

Synopsis

#include <qthread/qalloc.h>

void *
qalloc_makestatmap (const off_t filesize, void *addr, const char *filename, size_t itemsize, const size_t streams);

void *
qalloc_makedynmap (const off_t filesize, void *addr, const char *filename, const size_t streams);

void *
qalloc_loadmap (const char *filename);

Description

These functions are for creating and/or loading memory maps. The concept is simple: create a giant mmap()’d file on disk, and allocate space out of it much like a malloc(). Of course, for speed and simplicity reasons, it doesn’t work exactly like a malloc().

The qalloc_makestatmap() and qalloc_makedynmap() functions both will either create a file with the specified filename, or, if such a file already exists, will load it. The qalloc_loadmap() function expects an existing file named filename, will read the necessary information out of it, and attempt to load it.

The files created by qalloc_makestatmap() and qalloc_makedynmap() will be filesize bytes long, prepared to serve streams concurrent requests, and will be loaded at the address addr. addr may be NULL to indicate no preference. The qalloc_makestatmap() function also takes an itemsize argument, which specifies the size of allocations that need to be made from the file.

Concept

There are two kinds of on-disk maps: dynamic maps and static maps. Static maps have very lwo overhead, but have the restriction that all allocations in them have the same size (also, they’re slower to create, initially). Dynamic maps allow you to decide how much memory to use for each allocation at runtime, but have more overhead both in terms of space and time (also, they’re very fast to create, initially).

The mmap()’d files contain mutexes to protect the allocation and deallocation operations, so the files can actually be shared by multiple threads or multiple processes. Each thread is mapped to a given "stream" (or set of mutexes) by its thread id, obtained from pthread_self().

Because mmap() does not, and cannot, guarantee that the address specified is the one that will be used, this may present a problem. Pointers stored in this memory will be inaccurate if the file is loaded at a different location than it was saved with. Conveniently, each file stores the address it was first created for inside it, so that if the mmap() loads the file at a different address, it is detected by the library. The question remains, however, how to address the problem once it has been detected. There are two ways of addressing this problem.

From the known-original location stored in the file, and the known-current location, a "correction factor" or "offset" can be calculated. Addresses can be modified when reading and writing from the file, such that they are made always correct. This, however, involves a lot of overhead, particularly since the common-case is expected to be that the file is always mmap()’d into the exact same location.

The second option, which is what is currently done, is to simply assert that the file MUST be loaded at the same location, and to throw an error if it is not. The only reason that mmap() would refuse to load the file into the specified location would be if there is some conflict and something else is already occupying that memory. This is considered to be the programmer’s responsibility to ensure that this does not happen. Additionally, if the system can use a 64-bit address space, there should be more than sufficient space in that address space to find a location that never conflicts.

See Also

qalloc_checkpoint(3) , qalloc_cleanup(3) , qalloc_free(3) , qalloc_malloc(3)

Table of Contents