Hi, I apologize in advance because I don't really understand computer hardware, and may get some terms mixed up.
I'm running a process that is multi-threaded, where very large variables are sent to the different threads. I've been running this process for a couple months now on a machine with 4 processors and 8 GB of RAM. When I do this with 4 threads, my computer has no problem, and uses about 3.3 GB of RAM.
The process can take more than 3 hours to run with 4 threads, so we've been trying to experiment with our IT department to see if more computing power will speed up this process. We've been given a dedicated server with 16 processors and 16 GB of RAM to test this out. I've been trying different numbers of threads, and it never works with 16 threads. It sometimes works with 10, but other times, even though the size of the variables sent to the threads will be about the same, it will bomb citing not enough memory after as little as 5 threads are started. I'm not understanding why there is so much volatility in my testing, as no one else is using this server. When I've been running these tests, the most memory used by my computer before it crashes is about 4 GB, and CPU usage is around 50%.
I'm wondering if APL Next Supervisor is equipped to handle variables needing this much memory, or if anyone has any suggestions of what to check, or what needs to be improved? Again, I am new to this, and any help will be greatly appreciated.