How to set ulimit max number open files and other limits? Help!
You can use this reference to learn how to update any of the kernel / userspace limits.
Increase max number of open files nofile
ubuntu (and other os's)
Fun fact:
fs.file-max
is probably not what you want to change, you probably want to changeulimit
. But why? The deaultfs.file-max
for the kernel on ubuntu is already pretty high: (9223372036854775807
) You can check for yourself by runningsysctl -a | grep 'fs.file-max'
. But that's set at the kernel level.file-max
"sets the maximim number of file-handles that the Linux Kernel will allocate". What you probably want to change is theulimit
for the userspace level (does theu
stand for 'user space'? Probably. In operating systems there is a disinction between user-space, and kernel space- note this is userspace not 'a user').
So how to change the ulimit on Ubuntu in user space?
Did you know there's two vaues for each? A 'hard' value and
a 'soft' value. Often I make the mistake of changing the 'hard' value, then checking the 'soft' value which hasn't changed.
Check current value for hard and soft value
Display current soft limit of open files:
ulimit -Sn
Where -S
means 'show me the soft value',
and n
means 'nofiles' , which means 'number of files'
How do I know that? (See help ulimit
):
help ulimit
... there's more...
-l the maximum size a process may lock into memory
-m the maximum resident set size
-n the maximum number of open file descriptors
-p the pipe buffer size
-q the maximum number of bytes in POSIX message queues
... there's more...
Display current hard limit of open files:
ulimit -Hn
Example checking both:
$ ulimit -Sn
1024
$ ulimit -Hn
1048576
How to increase the limit
vi /etc/security/limits.conf
(read the note at the top of the file):
If root is the user you want to increase limit for, add:
root hard nofile 9999
You might want to put both a hard and soft limit:
root hard nofile 9999
root soft nofile 9999
I would like to draw your attention to read all of the documentation at the top of /etc/security/limits.conf
.
Especially:
- NOTE: group and wildcard limits are not applied to root. To apply a limit to the root user, <domain> must be the literal username root.
Do I need to reboot after this?
No. But you do need to kill your current proccess, which means
if you have an ssh
connection and shell open, then you need to exit
and re-connect. Why? Because only new processes
will get the new limit. Think of the process tree (type pstree
), child processes inherit permissions and rules of their parent (usually) , so you won't see your change until you have a brand new parent process.
Someone on the internet said I need to edit /etc/pam.d/commons-session
. Do I really need to do that?
No. That information appears to be out of date. This information will also be out of date eventually :)
Why is this hard? How to make it easier
Hopefully this document helped. For me the disinction between 'kernel space' and 'user space' helped a lot. But also:
The manual is hard to find if you're like me and use man
all the time
Where is the manual?
I tried man ulimit
- does not work
Instead: help ulimit
- does work
Why does help ulimit
work?
ulimit and disown are Bash built in functions and those don't have a manpage of its own but are described in the Bash manpage. src
The command options, and names are short, so it's impossible to instantly know which is which
But you can find out what all the single letter options mean by typing help ulimit
What is the difference between /etc/sysctl.conf and /etc/security/limits.conf I'm confused they seem the same to me?
You're not the only person to ask this question:
Here you can find posts spanning 5 years with two authors copy-pasting the same views without any reference to the source:
- https://unix.stackexchange.com/questions/379336/whats-the-difference-between-setting-open-file-limits-in-etc-sysctl-conf-vs-e
- https://unix.stackexchange.com/questions/585785/confused-about-the-diff-between-etc-sysctl-conf-vs-etc-security-limits-conf
Stephen provides most useful answer because it explains these setting affect the linux kernel, and links to the reference: https://unix.stackexchange.com/a/585842.
In summary, this is the official Documentation for /proc/sys/fs/ and edits to the file /etc/sysctl.conf
(once reloaded) will write to /proc/sys/fs/
, for example /proc/sys/fs/file-max
. Which provides an interface to kernel data structures (such as the maximum number of files the kernel will allow to be open).
/proc
(see man proc
, is a virtual in-memory filesystem
It's not you it's me
This is a lower level tuning part of the operating system. :) stick at it
From the help ulimit
Not all options are available on all platforms, which makes sense, you might be running on hardware which does not have a network card for example, so
/proc/net
will probably have less valid options to choose from.
Good luck!
If you missed it, the heading (near the top) if you want to just-get-it-done is "How to increase the limit".