Products Resources Support About Us

Handling very large git repositories on z/OS

If you attempt to clone or checkout a very large git repository (one with a great deal of history), you may get this message:

bash-2.03$ git checkout --track -b v0.12-zos-ascii origin/v0.12-zos-ascii
fatal: Out of memory? mmap failed: EDC5124I Too many open files. (errno2=0x07360344)
bash-2.03$

The problem is that git, by default, attempts to mmap the entire pack file (in the .git/objects/pack directory) into memory, and in this case that’s a very large file (about 235MB):

bash-2.03$ ls -l .git/objects/pack
total 494592
-r--r--r--   1 jcallen  ussuser  7578432 Jan 26 11:23 pack-13c65cae0f429287bcff36df4d582d30125aa766.idx
-r--r--r--   1 jcallen  ussuser  245499367 Jan 26 11:23 pack-13c65cae0f429287bcff36df4d582d30125aa766.pack
bash-2.03$

The z/OS port is 31-bit program, so that represents a sizable chunk of the entire 2GB address space.
Fortunately, git provides some knobs to control memory use. A stackoverflow item provided the necessary clues. I used these commands to crank down the memory usage, and was then able to get the checkout to run:

git config pack.packSizeLimit 20m
git config core.packedGitWindowSize 16m
git config core.packedGitLimit 32m
git config pack.windowMemory 32m
git config pack.thread 1
git config pack.deltaCacheSize 1m
1 Like

Are you planning a 64-bit git release?