Category: Linux OS-level Issues

Speed in Lower RAM on Manjaro and Arch Linux, With a Custom Kernel
article #1294, updated 38 days ago

Background and synopsis

The modern web is becoming more and more demanding of web browsers, for a larger and larger proportion of web sites, every year. A lot of people have solid working hardware which cannot accept more than 8 gigabytes of RAM, and often much less. I’m fairly certain that a completely full-blown desktop/laptop of now, with full and smooth online+offline video playing, USB stick automounting, camera/phone photo autoslurping, LibreOffice or Softmaker Office, Thunderbird for email and calendaring, et cetera, cannot be had in less than 8 gigabytes of RAM. If you know differently please do email me! I use Manjaro, XFCE build, which gives me a great balance of:

  • nonreinvention of wheels at install,
  • amazing buglessness (congrats folks!!!),
  • full desktop functionality available,
  • ability to easily turn off any bits of desktop functionality which I don’t want, while
  • keeping everything familiar and easy to find.

But even in its max of 8G RAM, the aged 2.2 GHz machine I’m typing into right now has had lots of trouble keeping up with my massive multitab web habit, even with sysctl settings, unless I do what is outlined below.

The gist of it is, we compile a kernel, in a very special way. The kernel is the core of any Linux machine – in fact, the kernel is really what Linux is, everything else is other projects which are combined with the Linux kernel to produce a complete machine. This is why one can still find some grammatical purists out there complaining that there is no such thing as a “Linux computer”, that computers are “GNU/Linux”, combinations of GNU software and the Linux kernel. This is helpful to recognize grammatical purists, but certainly no one says “EveryoneElse/Windows” or “EveryoneThatAppleWillTolerate/Apple”, and so I don’t think we need to listen to grammatical purists very much :-) But I digress. The point for this exercise, is that probably every distribution of Linux ships with a kernel compiled in a generic fashion – it will run on the very minimum hardware stated by the distro documentation, and it will run acceptably on some really high maximums too.

But Linux is Linux: we can compile a kernel for our own particular needs, for our particular hardware. And believe you me, there is major advantage in doing so. Sometime, if you find yourself caring enough, dig into the differences between actual models of Intel CPUs from the last twenty years. Twenty years is good because that’s about the maximum range distro kernels have to handle. And the differences are tremendous. Newer ones have far more registers, far more internal simultaneous operations they can do, far more lots and lots of things, things I wouldn’t really understand without possibly hours of study. And I don’t have to know about them; I just find it wonderful that if I compile my kernel for my very own CPU, my entire system suddenly starts to use more of what I have, and things run much better.

Granted, one can find lots of people on the web claiming otherwise. One can find lots of different people on the web. :-)

But simply running CPU-optimized does not address my own problem. I like to read news by opening between six or a dozen browser tabs with right-clicks from front pages. 8 gigabytes of RAM was eaten by this to crash quite easily…until I added a setting to my kernel compilation. I’m still doing CPU-optimization. But I’m also choosing to optimize for size, not performance. Default is performance. Size pulls the miracle :-) I do other memory-intensive things too sometimes, but massive multitabulation seems to be a terrific stress-test.

Sorry for the extended verbiage, but I’m fairly certain some readers won’t be familiar with some of the above, and it forms a good conceptual basis! So here we go.

Here we go!

On modern Manjaro/Arch, there is a new AUR-inclusive package management tool, called ‘yay’. ‘yay’ is in the Manjaro community repo, and this is one small reason among a great many why I like Manjaro! So starting with a well-behaved Manjaro, we install yay:

sudo pacman -S yay

And then we set it up for PKGBUILD editing:

yay --save --editor /usr/bin/nano --editmenu --noansweredit

What is a PKGBUILD, you ask! Well, a PKGBUILD is an Arch-standard file which defines how a package is compiled and installed. Every package, AUR or repo, has one. We have just set up to make it very easy to edit PKGBUILD files for every AUR package we take in by ‘yay’. The full-screen text editor ‘nano’ will be used for editing, and yay will ask us every time if we want to edit. ‘yay’ is very nicely configurable, another set of congrats. It is worthwhile to mention, that we do edit PKGBUILD files only most carefully, packages break trivially with bad edits, and worse things can occur, albeit relatively rarely.

But the next step, is to do a little preparing of the environment. Edit /etc/makepkg.conf, and find these lines:

CFLAGS=”
CXXFLAGS=”

Both lines will be quite long, with close-quotes, containing several items. One of the items in both is -march=; this needs to be changed from whatever it is to -march=native. We also need an item added or changed if it exists: we need -mtune=native. This will make everything we compile, run by far the best on the very make and model CPU we have in this machine. It will also make the packages not run well on anything else, and not run at all on some machines! Fair warning :-)

Also in this file, find a line starting with this:

#MAKEFLAGS="-j

There will be a number to the right and a close quote. Find out how many CPU cores your machine has, and add one; so if you have a dual core, you’ll add this line just below the original:

MAKEFLAGS="-j3"

This speeds up package compilation a lot, even with just two cores, and enormously with 4 or more.

There is one more item to prepare. In this file (“~” means your home directory):

~/.gnupg/gpg.conf

you’ll want to add the following:

keyserver-options auto-key-retrieve
auto-key-locate hkp://pool.sks-keyservers.net

This eliminates the need to manually receive and approve GPG signing keys for various source files as they are downloaded.

So once we have the above done, we need to choose our kernel package. Most recently I have been using “linux-uksm”, one of many Linux kernels available through AUR with sets of performance patches. All of them have different strengths, and many of them do not make it easy to customize. Thus far linux-uksm has done extremely well. So we begin:

yay -S linux-uksm

We give the default responses by pressing Enter/Return, until it asks if we want to edit the PKGBUILD:

:: Parsing SRCINFO (1/1): linux-uksm
  1 linux-uksm                               (Installed) (Build Files Exist)
==> PKGBUILDs to edit?
==> [N]one [A]ll [Ab]ort [I]nstalled [No]tInstalled or (1 2 3, 1-3, ^4)
==>  

For this, type just the number 1 (that’s a package count number, in case you were running ‘yay’ on multiple packages at once), and press Enter/Return. The PKGBUILD will come up in the text editor ‘nano’. As of this writing, it looks like this:

# Maintainer: Piotr Gorski <lucjan.lucjanov@gmail.com>
# Contributor: Jan Alexander Steffens (heftig) <jan.steffens@gmail.com>
# Contributor: Tobias Powalowski <tpowa@archlinux.org>
# Contributor: Thomas Baechler <thomas@archlinux.org>

### BUILD OPTIONS
# Set these variables to ANYTHING that is not null to enable them

### Tweak kernel options prior to a build via nconfig
_makenconfig=

### Tweak kernel options prior to a build via menuconfig
_makemenuconfig=

### Tweak kernel options prior to a build via xconfig
_makexconfig=

### Tweak kernel options prior to a build via gconfig
_makegconfig=

                           [ PKGBUILD -- 386 lines ]
^G Get Help  ^O Write Out ^W Where Is  ^K Cut Text  ^J Justify   ^C Cur Pos
^X Close     ^R Read File ^\ Replace   ^U Paste Text^T To Spell  ^_ Go To Line

Using the cursor buttons on your keyboard, move the text cursor to the position just to the right of the line _makexconfig=. Press the Y letter key once. You have now instructed the PKGBUILD engine to build and run a kernel compilation configurator application, called ‘xconfig’, as the next step. Press Ctrl-X to initate closure of nano, press Y to confirm saving, press Return/Enter to save and close the file. Because a kernel PKGBUILD is a large and multifaceted thing, yay may bring up one or more files; keep on pressing Ctrl-X until you’re back at the ‘yay’ prompt (at this writing, just one additional Ctrl-X):

==> PKGBUILDs to edit?
==> [N]one [A]ll [Ab]ort [I]nstalled [No]tInstalled or (1 2 3, 1-3, ^4)
==> 1

==> Proceed with install? [Y/n] 

Press the Y key, and Return/Enter. The source code for the kernel will download, and then xconfig will be compiled, and will be run. It’s GUI, and the big advantage for us right now, is the search works very well. There are a truly enormous number of changes that can be made, almost all of them can result in an unusable kernel (or sometimes even a kernel that can do harm over time); we want exactly two.

So click the edit menu, choose Find, and type in the word “native”, no quotes. A nice GUI chooser circle will appear to the left of “Native optimizations autodetected by GCC”. Click that GUI chooser to fill the circle. This is the step which will engage all, or nearly all (depending on version of the GNU compiler and how new your CPU is!), of the features of the very CPU you are using, in the kernel which will shortly be compiled.

Then, click in the Find box field, and type the word “size”, no quotes. Another nice GUI chooser circle will appear, to the left of “Optimize for size”. Click that GUI chooser to fill the circle. This is the step which will optimize for size (memory-usage efficiency), not performance. Default is performance.

Once the above is done, close the Find window using basic GUI, usually the extreme upper-right corner of the window. Click the File menu, choose Save. Then File menu, Quit. Kernel compilation will begin.

Depending on how much RAM and how many CPU cores you have, compilation can take a very long time. If just one core and (say) 1 gigabyte of RAM, this is probably several hours. It will complete the kernel install, after it’s done. However, at least as of this writing, you’ll have to do the following to get the kernel headers (essential) and docs (admittedly optional) in:

cd ~/.cache/yay/linux-uksm
sudo pacman -U linux-uksm*pkg.tar.xz

Also be aware, if you have compiled more than one linux-uksm kernel, the above will attempt to reinstall them all, you may want to clean house first :-)

After you reboot, your new kernel will probably be active. You can use uname -a to verify this with certainty, though if you’ve never done this before you’ll probably notice a performance jump immediately. If you would like more control of the kernel you run at any time, grub-customizer is highly recommended.

Categories:      

==============

Recompiling for Performance on Arch Linux and Derivatives
article #1196, updated 39 days ago

At the core, any current desktop OS is running binary code; and the vast majority of it is binary code which uses only a subset of the CPU at hand. This is because there are so many different CPUs which need to run the same code. Intel and AMD constantly add things to the CPUs they put out, but code of general distribution lags very far behind, because when one downloads and installs, that code set has to run on everything, whether it be ten years old or three months old. On an excellent Linux, one can recompile any package in such a way that the binary code resultant uses the entire CPU which one is using. Performance gains are thus available.

Most of the advice I have been given and found, for recompiling certain packages for Arch Linux and derivatives, has made things very very complicated, and often includes statements that it’s not worth it. Well, I am a witness that it is well worth it, one can increase performance quite a lot, and it’s not too complicated as of this writing.

My Arch derivative of choice is Manjaro, it does a lot of things for us. But all of these methods are pure Arch, all you have to do is get the prerequisites.

Prepare the environment

Before we do anything, we update the system and reboot. This is partly because operations further down will get new packages.

Then we install yay. Manjaro has it in its standard repos and can be installed just with pacman -S yay; it’ll be a bit more difficult under pure Arch. Once yay is in, we edit /etc/makepkg.conf, and find these lines:

CFLAGS=”
CXXFLAGS=”

Both lines will be quite long, with close-quotes, containing several items. One of the items in both is -march=; this needs to be changed from whatever it is to -march=native. We also need an item added or changed if it exists: we need -mtune=native. This will make everything we compile, run by far the best on the very make and model CPU we have in this machine. It will also make the packages not run well on anything else, fair warning :-)

In addition in this file, find a line starting with this:

#MAKEFLAGS="-j

There will be a number to the right and a close quote. Find out how many CPU cores your machine has, and add one; so if you have a dual core, you’ll add this line just below the original:

MAKEFLAGS="-j3"

This speeds up package compilation a lot, even with just two cores, and enormously more with 4 or more.

There is one more item to prepare. In this file (“~” means your home directory):

~/.gnupg/gpg.conf

you’ll want to add the following:

keyserver-options auto-key-retrieve
auto-key-locate hkp://pool.sks-keyservers.net

This eliminates the need to manually receive and approve GPG signing keys for various source files as they are downloaded.

Install an Optimized Kernel

So. Once the above is done, it’s not hard to use yay to build and install the Xanmod kernel, an excellent desktop-optimized kernel:

yay -S linux-xanmod

Yay will bring in the PKGBUILD, the file defining how the kernel source is download and package built. It quickly gives the option to edit it, and doing so is part of our procedure. As of this writing, you’ll look for one line:

_microarchitecture=0

and change this to:

_microarchitecture=22

This is according to documentation in the file itself; in this version at least, 22 equals “native”, which means the kernel will be optimized for the very specific CPU make and model in your machine. You can then save and choose the defaults for the rest of the process. It will take a while, 30 minutes often and much more on slower machines. Once the rebuild and install is done, you will notice a performance boost after booting the new kernel. Do be aware that automatic updates may override this kernel down the road; you can use grub-customizer (also available via yay) to specify which kernel you will boot.

Build glibc

After the kernel itself, by far the most used boulder of code in a Linux machine is the GNU C Library, glibc for short. So we rebuild this next.

We pull the PKGBUILD and related build scripts with yay:

yay -G glibc

And then we cd into the directory created, and light off makepkg and watch it go:

cd glibc
makepkg -s

If packages are needed for the build, install will commence, and then compilation. Compilation will take quite a while, longer even than the kernel. After it’s done, install:

sudo pacman -U *.pkg.tar.xz

and reboot to fully engage, though you may see improvement as soon as you start running or restarting programs.

Issues with many packages

There are issues which can show up with many packages.

First of all, compilation may fail. glibc is a huge package with a very large number of complications, and sometimes those complications have to do with specific versions of gcc and other items — which means if your machine is updated past those versions, you won’t compile successfully. You can either dig deep in code and/or forums to see what is going on, or just wait until the (very knowledgeable and capable, much more so than I) primary developers resolve it for all of us. Even something like the Xanmod kernel compilation may fail occasionally for the same reasons; there are quite a few more kernels available to try from yay, though each of them have different methods of setting CPU optimization, do watch for this.

Secondly, getting the versions you need. You probably should want the standard version, not the AUR (bleeding edge sometimes, archival and out of date sometimes too!) version. yay -G will tell you what it’s doing, but do be careful to not try to use outdated versions, that can break your OS if you go off the beaten path.

And thirdly, when you automatically update using pacman or GUIs, newer un-optimized versions will be autoinstalled over your optimal one. There may be ways to override this, but override is very questionable, because a very outdated package of many sorts is likely to produce crashes, especially something core like glibc or xorg-server. Better to just recompile after the update is installed. It is also helpful to choose such packages for the rarity of their updates, and glibc is one such.

Other packages to CPU-optimize

There are many other packages worth recompiling. I choose these regularly and differently according to high result/effort ratio! Here is a list, there are doubtless many more. These are all for the yay -G and makepkg method used for glibc, not yay -S. There may well be others which will help more, certainly for particular use purposes, e.g., audio and video.

gtk3
gtkmm3
gtk2
gtkmm
cairo
cairomm
qt4
qt5-base
pth
libxml2
glade
libglade
libglademm

Categories:      

==============

Headless Running X in Manjaro Linux
article #678, updated 69 days ago

  1. Install normally, using monitor and keyboard. Get the install solid with updates and testing.
  2. Switch to a virtual console with Ctrl-Alt-F2 and log in.
  3. Run this to turn off X: sudo systemctl isolate multi-user
  4. Run this: X -configure
  5. A file is created, /root/xorg.conf.new.
  6. chdir to /etc/X11/xorg.conf.d
  7. Move /root/xorg.conf.new to this new name at your current location: ./10-monitor.conf
  8. Reboot and test. If you change video cards, you may well have to do it again, and you’ll probably need to boot from install media in order to remove the file.

Categories:      

==============

Allow a group all commands with sudo
article #1282, updated 79 days ago

In /etc/sudoers, or a file last in the list in /etc/sudoers.d, you’ll need this:

%wheel ALL=(ALL) NOPASSWD:ALL

Make sure your user is in the group ‘wheel’ (default in most full desktop Linuxes), and you’re in.

Categories:      

==============

A stop job is running for session c1 of user...
article #1226, updated 80 days ago

This notice can be seen at shutdown of recent Linux. The only thorough solution this writer has seen, is to use the ‘watchdog’ service to get rid of hanging processes. You’ll need to compile, perhaps using ‘yay’, on Arch and derivatives:

yay -S watchdog
sudo systemctl enable watchdog.service
sudo systemctl start watchdog.service

Categories:      

==============

Cross-Linux Packaging Systems, esp. Flatpak
article #1213, updated 324 days ago

Flatpak appears currently the best supported. Many apps and tools can be loaded through it. The Snap system is not at this writing satisfactorily supported under Arch Linux, but it alone appears to contain PowerShell.

To use Flatpak under Arch, first you’ll need to install the Flatpak system:

pacman -S flatpak

and then you’ll want to download a .flatpakref file from flathub.org. Once you have that, run this:

flatpak install *.flatpakref

and the install will commence, it will install runtime prerequisites as needed.

Categories:      

==============

Run Linux on Windows 10
article #1175, updated 427 days ago

From the extraordinary Mike Hunsinger:

Today I learned something fascinating. You can run linux as a Windows Subsystem w/o using a vm or dual-boot. All you have to do is run PS as admin, execute this:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

reboot when prompted and then you can use the windows store to install just about any Linux distro you want.

This article describes folks running Linux GUI apps talking to Windows-native X Windows. Ponderworthy hasn’t tested this yet, but the following X server looks like a very worthy candidate:

sourceforge.net/projects/vcxsrv/

Categories:      

==============

Linux Speed, Responsiveness, and Latency Reduction with 'sysctl' Settings
article #892, updated 438 days ago

These items help a lot in any application, including desktop, web server, or terminal server. The end of this post has two large compilations of these settings, one for wired (“non-lossy”) networking, one for wireless (“lossy”).

On the vast majority of Linux distributions, one can just add these changes to /etc/sysctl.conf, and then run sysctl -p to apply them without reboot. However, recent additions to standards have enabled us to place custom settings in our own configuration files, so that we don’t take /etc/sysctl.conf out of distro control.

On recent Debian and Ubuntu, we may best put them in /etc/sysctl.d/60-custom.conf (or replace the word “custom” to your liking), and then run sysctl --system to load both /etc/sysctl.conf and everything under /etc/sysctl.d.

On some other recent distros, it’s /etc/sysctl.d/custom.conf (the word “custom” is still arbitrary), and then run systemctl restart systemd-sysctl.

You can check your results with sysctl -A.

The first selection is for wired networking performance:

net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_workaround_signed_windows=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_fack=1
net.ipv4.tcp_low_latency=1
net.ipv4.ip_no_pmtu_disc=0
net.ipv4.tcp_mtu_probing=1
net.ipv4.tcp_frto=2
net.ipv4.tcp_frto_response=2
net.ipv4.tcp_congestion_control=illinois

A bit different first group for networking performance, is recommendable for anything involving wireless, i.e., “lossy” networks:

net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_workaround_signed_windows=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_fack=1
net.ipv4.tcp_low_latency=1
net.ipv4.ip_no_pmtu_disc=0
net.ipv4.tcp_mtu_probing=1
net.ipv4.tcp_frto=2
net.ipv4.tcp_frto_response=2
net.ipv4.tcp_congestion_control = hybla
net.ipv4.tcp_allowed_congestion_control = hybla cubic

And then some general networking performance items:

net.core.rmem_default = 31457280
net.core.rmem_max = 12582912
net.core.wmem_default = 31457280
net.core.wmem_max = 12582912
net.core.somaxconn = 4096
net.core.netdev_max_backlog = 65536
net.core.optmem_max = 25165824
net.ipv4.tcp_mem = 8388608 8388608 8388608
net.ipv4.udp_mem = 8388608 8388608 8388608
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem_min = 16384
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem_min = 16384
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.route.flush=1

And some for network security enhancement:

net.ipv4.tcp_synack_retries = 2
net.ipv4.ip_local_port_range = 2000 65535
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15

And now a few to keep virtual memory usage under good control:

vm.swappiness=20
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2

And one to increase the maximum number of open and watched files, very helpful indeed for servers, file synchronization of all sorts, and many other functions:

fs.file-max = 2097152
fs.inotify.max_user_watches = 524288

The above was compiled from these two excellent articles:

http://www.networkworld.com/article/2227856/opensource-subnet/best-networking-tweaks-for-linux.html
https://easyengine.io/tutorials/linux/sysctl-conf/

and other sources. Here is the whole set for wired (non-lossy) networking:

net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_workaround_signed_windows=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_fack=1
net.ipv4.tcp_low_latency=1
net.ipv4.ip_no_pmtu_disc=0
net.ipv4.tcp_mtu_probing=1
net.ipv4.tcp_frto=2
net.ipv4.tcp_frto_response=2
net.ipv4.tcp_congestion_control=illinois
pre. net.core.rmem_default = 31457280
net.core.rmem_max = 12582912
net.core.wmem_default = 31457280
net.core.wmem_max = 12582912
net.core.somaxconn = 4096
net.core.netdev_max_backlog = 65536
net.core.optmem_max = 25165824
net.ipv4.tcp_mem = 8388608 8388608 8388608
net.ipv4.udp_mem = 8388608 8388608 8388608
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem_min = 16384
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem_min = 16384
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.route.flush=1
vm.swappiness=20
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2
fs.file-max = 2097152
fs.inotify.max_user_watches = 524288

and another full set for wireless / lossy networking:

net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_workaround_signed_windows=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_fack=1
net.ipv4.tcp_low_latency=1
net.ipv4.ip_no_pmtu_disc=0
net.ipv4.tcp_mtu_probing=1
net.ipv4.tcp_frto=2
net.ipv4.tcp_frto_response=2
net.ipv4.tcp_congestion_control = hybla
net.ipv4.tcp_allowed_congestion_control = hybla cubic
pre. net.core.rmem_default = 31457280
net.core.rmem_max = 12582912
net.core.wmem_default = 31457280
net.core.wmem_max = 12582912
net.core.somaxconn = 4096
net.core.netdev_max_backlog = 65536
net.core.optmem_max = 25165824
net.ipv4.tcp_mem = 8388608 8388608 8388608
net.ipv4.udp_mem = 8388608 8388608 8388608
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem_min = 16384
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem_min = 16384
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.route.flush=1
vm.swappiness=20
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2
fs.file-max = 2097152
fs.inotify.max_user_watches = 524288

Categories:      

==============

Package key signing issues in Manjaro Linux
article #1157, updated 473 days ago

Per the Forum as of this writing, try each of these twice if needed:

sudo pacman -Sy archlinux-keyring manjaro-keyring
sudo pacman-key --populate archlinux manjaro
sudo pacman-key --refresh-keys

Categories:      

==============

Mount NFS Shares in Linux
article #1045, updated 646 days ago

So you have an NFS server share known to be working, and you want to mount its share(s) on your Linux client machine. Here’s a summary. We’re using NFS4.

  1. So we start out thinking that we have an NFS server 10.11.11.10. We verify it’s ready for connection, if we run this command from the client:
    showmount -e 10.11.11.10
    and receive one or more NFS folder shares, e.g.:
    /Media 10.11.11.0/255.255.255.0
  2. We need a place to mount the share on this machine. So:
    mkdir /Media
  3. Next we try it manually:
    mount -t nfs 10.11.11.10:/Media /Media
  1. Now let’s make sure all of our NFS4-related services are enabled and running on the client.
sudo systemctl enable nfslockd.service
sudo systemctl start nfslockd.service
sudo systemctl enable nfs-idmapd.service
sudo systemctl start nfs-idmapd.service
  1. We now make sure that permissions allow users of the client read/write access to the share. There are two problems here. The first is to make the shared folder and all contents read/writable by all designated users. The second is to make this happen across systems, i.e., from server to clients.

So first we set the shared folder and all contents chmoded g+rwXs, chgrped to a sharing-designated group, on the server side. We then create a new user group on both server and clients, perhaps named “sharedfiles”, add all selected users to that group on clients and server, make sure the GID is the same for the group name on all of the machines, and then reboot them all.

The GID is a numerical indicator for the group, which is more important to the local OS than the name of it. As part of the above we do need to make sure the GID is the same for our new user group, across all machines, or else the OSes will not recognize the equivalency. On each of them then, after the group is created, we do this:

groupmod -g 20000 sharedfiles

A list of current groups and GIDs is available here:

cat /etc/group

You’ll notice that 20000 places the new group long after all of the others in general. This is intentional, to avoid conflict with existing configurations.

  1. Then we set up automount. We do this by adding the following line to the end of /etc/fstab:
10.11.11.10:/Media /Media nfs noauto,x-systemd.automount,intr,rsize=4194304,wsize=4194304,x-systemd.device-timeout=10,timeo=14,x-systemd.idle-timeout=1min 0 0

Items:

  • NFS by itself is normally a very strong connection at a very low level, which means that unless an NFS mount is handled with care by other facilities, a client machine can freeze up very hard if its server becomes unreachable, rather different than SMB on Windows. There are at least three different methods to do this well. The above uses a facility within systemd, which this author found much easier to handle than the other two he found. If your Linux system does not use systemd, you should use one of the others, e.g., autofs, which is a layer unto itself.
  • The option “intr” is instead of “hard” or “soft”. It makes NFS transactions explicitly interruptible, which helps prevent corruption if the server goes down.
  • rsize and wsize can vary a lot. The number is in bytes. The above is a recent report on gigabit; if you’re on a lower-speed network you should use a correspondingly full order of magnitude smaller pair of numbers, e.g., the commonly reported rsize=8192,wsize8192.

Categories: