I ended up in this situation like I end up in most, by just learning as I go.

With this I learned that the libc allocator has a max amount that it'll add to the heap, and how mmap works, and that doing frequent munmap is slow.

Now I learned why it is slow!