ziima
November 1, 2022, 8:35pm
1
Since the switch to Turris OS 6.0, the NFS on the USB drive connected to Turris 1.0 hangs.
I have external drive connected to USB and mounted to /mnt/my_drive. I can see it’s contents from turris shell./etc/exports:
/mnt 192.168.1.0/255.255.255.0(ro,subtree_check,insecure,crossmnt)
I also created a test file /mnt/nfs-test.
I mount the NFS as
sudo mount 192.168.1.1:/mnt /media/turris/
I can see the file /media/turris/nfs-test and an empty directory /media/turris/my_drive. But when I try to access the /media/turris/my_drive all commands, e.g. ls /media/turris/my_drive hang. I found nothing in any log file.
It definitely worked before the upgrade. I tried several options to set up, but with no success. I also couldn’t quite anything useful to debug the problem.
Maybe you should try mounting as
# mount -F nfs [-o mount-options] server:/directory /mount-point
ziima
November 8, 2022, 7:32pm
3
Do you have any particular options in mind?
I might have a similar issue after update to 6.0. Could you try mounting as NFSv3?
sudo mount -t nfs -o vers=3 192.168.1.1:/mnt /media/turris/
also verbose output from the mount command might help debugging:
sudo mount -v 192.168.1.1:/mnt /media/turris/
ziima
November 11, 2022, 5:03pm
5
No luck. verbose doesn’t print anything useful. Forcing NFSv3 doesn’t change the behavior. I tried also -o retry=1, but it seems to have no effect either.
ziima
November 22, 2022, 8:14pm
6
It works now. No idea why, but I found
• Updated package kmod-fs-nfs-common from version 5.10.148-1-42186e8efb70da7a8430b969416854a2 to version 5.10.154-1-42186e8efb70da7a8430b969416854a2
• Updated package kmod-fs-nfs from version 5.10.148-1-42186e8efb70da7a8430b969416854a2 to version 5.10.154-1-42186e8efb70da7a8430b969416854a2
• Updated package kmod-fs-nfs-common-rpcsec from version 5.10.148-1-42186e8efb70da7a8430b969416854a2 to version 5.10.154-1-42186e8efb70da7a8430b969416854a2
• Updated package kmod-fs-nfsd from version 5.10.148-1-42186e8efb70da7a8430b969416854a2 to version 5.10.154-1-42186e8efb70da7a8430b969416854a2
• Updated package kmod-fs-nfs-v4 from version 5.10.148-1-42186e8efb70da7a8430b969416854a2 to version 5.10.154-1-42186e8efb70da7a8430b969416854a2
in update notifications, so it may be fixed by the new versions.
For me it’s still broken with an Omnia 2020. It seems to be this issue:
opened 12:00PM - 29 Nov 21 UTC
Maintainer: @tripolar
Environment: `arm_cortex-a15_neon-vfpv4`, `ipq806x`, TP-… Link Archer C2600, OpenWrt 21.02.1 stable
## Description:
I'm unable to get my NFS server config to work after upgrading from OpenWrt 19.07 to 21.02. When I try to mount an NFS share on my Linux client (Ubuntu 20.04), **the mount command always times out without success – or hangs indefinitively in case of no timeout**.
- I tried various different mount settings, both `nfsvers=3` and `nfsvers=4` (and `nfsvers=4.2`), all without luck. Using the `mount --verbose` flag doesn't output anything suspicious (just infinite wait at the point where it should actually have succeeded).
- [`showmount -a`](https://man.cx/showmount) on the router shows a connected client (although the mount never succeeded). Same command on 19.07 doesn't show any client regardless of successful mounts.
- `showmount -e` on the router shows all my NFS shares fine, same as with 19.07.
- `showmount -e 192.168.1.1` on the client also shows all my NFS shares, same as with 19.07.
- `rpcinfo -p 192.168.1.1` on the client gives the same output as with 19.07:
<details>
```
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
100003 3 tcp 2049 nfs
100003 4 tcp 2049 nfs
100021 1 udp 32777 nlockmgr
100021 3 udp 32777 nlockmgr
100021 4 udp 32777 nlockmgr
100021 1 tcp 32777 nlockmgr
100021 3 tcp 32777 nlockmgr
100021 4 tcp 32777 nlockmgr
100005 1 udp 32780 mountd
100005 1 tcp 32780 mountd
100005 2 udp 32780 mountd
100005 2 tcp 32780 mountd
100005 3 udp 32780 mountd
100005 3 tcp 32780 mountd
100024 1 udp 32778 status
100024 1 tcp 32778 status
```
</details>
- `netstat -nlpu | grep rpcbind` on the router gives
```
udp 0 0 0.0.0.0:64795 0.0.0.0:* 1800/rpcbind
udp 0 0 0.0.0.0:111 0.0.0.0:* 1800/rpcbind
udp 0 0 :::111 :::* 1800/rpcbind
udp 0 0 :::58014 :::* 1800/rpcbind
```
while on 19.07 it's
```
udp 0 0 0.0.0.0:59643 0.0.0.0:* 1085/rpcbind
udp 0 0 0.0.0.0:111 0.0.0.0:* 1085/rpcbind
udp 0 0 :::64081 :::* 1085/rpcbind
udp 0 0 :::111 :::* 1085/rpcbind
```
not sure if the differences matter...
- The ["NFS Service Setup" mentioned in the docs](https://openwrt.org/docs/guide-user/services/nas/nfs.server#nfs_service_setup) that exports a directory under `/tmp/` to a different directory under `/tmp/`, i.e. both on the server device, works fine. I know I [cannot use mountpoints on the OverlayFS](https://github.com/openwrt/packages/issues/15633) and avoided that from the beginning.
- `cat /proc/fs/nfsd/versions` gives me `-2 +3 +4 +4.1 +4.2`, same as on 19.07.
- [`rpc.statd`, `rpc.mountd`](https://openwrt.org/docs/guide-user/services/nas/nfs_configuration#start_on_boot) and `nfsd` processes are all up and running.
- I did not spot any useful messages in OpenWrt's system or kernel log.
- `nfs-kernel-server` version on 21.02 stable is `2.5.2-1` while on 19.07 stable it's `2.3.4-3`.
- There is [a similar report on the official OpenWrt forum](https://forum.openwrt.org/t/getting-nfs-to-work-on-openwrt-21-02-rc4/104976) (with no solution) as well as [one that might be related](https://forum.openwrt.org/t/difficulties-getting-nfs-server-to-work/109847) (I can't really tell; again, no solution).
Unfortunately, I lack the knowledge on how to further debug this issue. Would be glad if someone more proficient could provide some guidance. Or maybe this is even a known issue?
## Workarounds
- *ADDED on 2022-01-02 thanks to a [hint from @hb9xcg](https://github.com/openwrt/packages/issues/17234#issuecomment-1003582527)*: Everything works fine with `nfs-kernel-server 2.5.2-1` from OpenWrt 21.02 **iff** you explicitly specify a unique integer between 0-255 as `fsid` for *every* NFS export (i.e. every line in `/etc/exports`), not just `fsid=0` (or `fsid=root`) for the NFS exports root. Specifying UUIDs instead doesn't work, i.e. results in the same problem as not specifying an `fsid` at all.
- Another possible workaround is to use the `nfs-kernel-server` package from the OpenWrt 19.07 stable package feed, which works just fine on 21.02:
```sh
wget https://downloads.openwrt.org/releases/19.07.8/packages/arm_cortex-a15_neon-vfpv4/packages/nfs-kernel-server_2.3.4-3_arm_cortex-a15_neon-vfpv4.ipk
opkg update
opkg install ./nfs-kernel-server_2.3.4-3_arm_cortex-a15_neon-vfpv4.ipk
rm nfs-kernel-server_2.3.4-3_arm_cortex-a15_neon-vfpv4.ipk
```
The `nfs-kernel-server` package `2.5.4-3` from the [21.02 snapshot package feed](https://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/packages/nfs-kernel-server_2.5.4-3_arm_cortex-a15_neon-vfpv4.ipk) on the other hand seems incompatible. `nfsd` fails with the following errors when trying to start (`service start nfsd`):
<details>
```
Error relocating /usr/sbin/exportfs: __lstat_time64: symbol not found
Error relocating /usr/sbin/exportfs: __localtime64: symbol not found
Error relocating /usr/sbin/exportfs: __stat_time64: symbol not found
Error relocating /usr/sbin/exportfs: __fstatat_time64: symbol not found
Error relocating /usr/sbin/exportfs: __time64: symbol not found
Error relocating /usr/sbin/rpc.nfsd: __stat_time64: symbol not found
Error relocating /usr/sbin/rpc.nfsd: __time64: symbol not found
Error relocating /usr/sbin/rpc.nfsd: __localtime64: symbol not found
/etc/rc.common: line 48: procd_add_reload_mount_trigger: not found
```
</details>
## OpenWrt builds I've tried so far
The following info is not really relevant for the above problem, I think. But nevertheless I've written down my little 21.02 build journey:
1. The point of departure is that with OpenWrt 19.07.7, the NFS server worked fine. But I wanted to upgrade to 21.02.
2. I tried [@ACwifidude's NSS build](https://forum.openwrt.org/t/ipq806x-nss-build-netgear-r7800-tp-link-c2600-linksys-ea8500/82525) (the [`ath10k-ct` build based on 21.02 stable](https://github.com/ACwifidude/openwrt/blob/openwrt-21.02-nss-qsdk10.0/bin/targets/ipq806x/generic/C2600-20211106-Stable2012NSS-sysupgrade.bin)).
The three packages `kmod-crypto-cts`, `kmod-fs-nfs-common-rpcsec` and `kmod-fs-nfsd` are not part of the image but required by `nfs-kernel-server`. Since `kmod-*` packages need to match the build's exact kernel ID[^1], @ACwifidude kindly offers [some more prebuilt packages](https://github.com/ACwifidude/openwrt/tree/openwrt-21.02-nss-qsdk10.0/bin/targets/ipq806x/generic/packages)) from where I've installed the mentioned `kmod-*` packages. Then I've installed `nfs-kernel-server` from OpenWrt's 21.02's regular package repository.
The NFS server issue described above is present.
2. I assumed the NFS server problem was caused by some remaining/obscure package/dependency incompatibilities. Thus I compiled my own image that preincluded all the required packages.
Exactly the same result, the NFS server issue described above is present.
3. Then I tried with latest OpenWrt 21.02.1 stable.
Exactly the same result, the NFS server issue described above is present.
4. Finally I reverted back to OpenWrt 19.07.08 stable since I need a working NFS setup.
Everything works fine with the *exact* same NFS configuration as I used on 21.02.
[^1]: If a package with a (transitive) dependency on the kernel from OpenWrt's official package repo is tried to install, an error `pkg_hash_check_unresolved: cannot find dependency kernel ...` is displayed since @ACwifidude's NSS build includes custom commits [which trigger an additional hash at the end of the kernel version](https://forum.openwrt.org/t/ipq806x-nss-build-netgear-r7800-tp-link-c2600-linksys-ea8500/82525/159). If such a package is a `kmod-*`, the installation fails, otherwise installation succeeds but the error ist still displayed. This was the main motivation for me to compile an own build (see 2\. above) in which such supposed incompatibilities are excluded in the first place.
I had to install the package from 5.4.4 to get it to work:
wget https://repo.turris.cz/archive/5.4.4/omnia/packages/packages/nfs-kernel-server_2.3.4-3_arm_cortex-a9_vfpv3-d16.ipk
opkg install ./nfs-kernel-server_2.3.4-3_arm_cortex-a9_vfpv3-d16.ipk
ziima
April 28, 2023, 8:42am
8
For me, it was working sometimes . In the end I switched to samba instead.
Is there a way to ‘hold on’ to an old package version so that updates do not upgrade it?
Read /etc/updater/conf.d/example.lua
1 Like
Thanks! Seems this did the trick:
Install("nfs-kernel-server", { version="=2.3.4-3", priority = 60 })