SYMPTOM
When running many processes, each with many threads, as the AtScale or runtime user, the error "java.lang.OutOfMemoryError:unable to create new native thread" may be seen under heavy load.
This error indicates the user has hit the thread limit set at the system level.
RESOLUTION
It is recommended to check the thread limit as the AtScale or runtime user using: # ulimit -u
This issue can be resolved by creating a new file for the runtime user (in the below example, atscale) to increase the thread limit. Using the root user, do as follows:
# vi /etc/security/limits.d/atscale.conf atscale - nofile 65536 atscale - nproc 65536
This requires a restart of all AtScale services, including the supervisor process.
-- To stop the services: # /opt/atscale/current/bin/atscale_stop -- To start the services: # /opt/atscale/current/bin/atscale_start