How can I get more than 4GB for my macOS application?

by

I am coding in C using Xcode on a MacBook.

The following program runs fine with n=1073741823 and gives EXC_BAD_ACCESS (code=1, address=0x200007ffc), if I increase the value of n by 1.

#include <stdio.h>

#define n 1073741823

int array[n];

int main(int argc, const char * argv[]) 
{
   printf("\n Size of array = %lu\n", (unsigned long) n*sizeof(int));
   array[n-1] = 1;
   return 0;
}

With the lower value of n, the size of the array is just below 4GB, and incrementing n by 1 pushes the size over 4GB and causes the runtime error.

I am using Xcode 15.0.1 on a 64GB MacBook Pro with Sonoma 14.0.

I cannot imagine that Apple wants to limit me to 4GB, and I cannot find any reference to such a limit (or to an option to increase memory size).

What must I do to be able to use 20GB or more in my code?

  • 4

    What happens if you dynamically allocate the array?

    – 

  • 1

    “I cannot imagine that Apple wants to limit me to 4GB” — you are evidently lacking in imagination, then. I don’t know whether it is an intentional or acknowledged design constraint that is getting in your way here, but I have no trouble at all imagining that it could be. Note also that there is a major difference between limiting you to 4GB total and limiting you to 4GB for any single object.

    – 

  • 1

    That’s a lot of data to store in the data segment of your executable.

    – 

  • Possibly a ‘policy’ to achieve better memory management. Dynamic allocation or a memory mapped file might allow the OS to better manage the memory demands of your application and that if others perhaps.

    – 




  • 1

    To OP: Again, as Mark (and I) suggested, try using malloc. The overhead of the [indirect] pointer is negligible with a 20GB array compared to the overhead of cache misses, etc. Also, it’s more flexible as you can grow/shrink the size and you’re not hamstrung as you would be with a fixed size (e.g. you specify 20GB but for a single given invocation, you need 21GB).

    – 

This may be a clang thing and not an Apple thing. On a BSD system with clang 13.0.0, I get:

ld: error: Insanely large symbol size for object (in GOT) array

gcc seems to have no issue with 4GB and larger BSS arrays. Apple’s clang (which is a more recent version of clang, 15.0.0) apparently is not issuing error messages in this regard when it should. Apple may have it’s own modified ld.

You should be using malloc() for large arrays. Even much smaller large arrays. Putting that array in the BSS requires, per the C standard, that it all be initialized to zeros, or at least logically appear to be zeros (by initializing only on demand), which you likely don’t need. malloc() doesn’t initialize the memory.

Leave a Comment