16 groups membership limit (part 2)

Part 1 is here: Group membership limit

So I am having to solve and re-visit this problem... I've tried various OSes (Solaris10/Opensolaris/MacOSX/Debian) and ngroup_max settings, some work for local filesystems but not over NFSv4.

Peter Harvey's blog

Eisler's NFS Blog: What's the deal on the 16 group id limitation in NFS?

Bug ID: 4088757 Customer would like to increase ngroups_max more than 32

Has anyone overcome this problem of being limited to 16 groups over NFS?

---------- Post updated at 03:46 PM ---------- Previous update was at 09:37 AM ----------

I've been given a hint to use AUTH_DH: Diffie-Hellman authentication over NFS to achieve more than 16 group permissions...

So I've been trying my hardest but can not get the keys and authentication set up correctly for this to work. My attempt with:

mount -F nfs -o sec=dh server:/var/tmp/test /mnt

hangs forever! Could someone explain how I set up these authentication keys on the server and client for this to work, please?

I am running NIS, on the NFS server I have in the /etc/dfs/dfstab

share -F nfs -o sec=dh,rw=client,root=client /var/tmp

On each server and client I've run newkey -h server/client and I've even done this on the NIS master and pumped the keys out using the publickey file. Nothing seems to be working... why? Am I missing out a step here? Help or hints will be appreciated!

One thought is to make absolutely sure you are mounting using NFS Version 4 by specifiying that in the mount line, .e.g:

mount -o vers=4 nfs_server:/export_path /mount_point 

Or amend /etc/default/nfs to prevent the system dropping back to NFS V3 or V2 (a bit drastic though).

The other suggestion is confirm that the kernel change has been picked up by running:

getconf -a | grep ngroups 

in order to check what the kernel reports the maximum number of groups to be. Saying that on boot you get a warning message about having more than 16 groups will break with NFS V3 which should be obvious enough.

The increasing of the number of groups is only a case of putting the line into /etc/system, e.g.:

set ngroups_max=32

and rebooting, it is not a hack but a long recognised but little used configuration change due to the NFS problem.

Many thanks for the reply and ideas. I have checked all that's suggested and they are all correct, but this still does not work, and still hangs.

I think the current problem is to do with the authentication keys... please help! Here are some log messages:

client# mount -F nfs -o vers=4,sec=dh server:/var/tmp/test /mnt

nfs mount: mount: /mnt: Invalid argument

client# tail messages
Feb 15 12:42:21 client rpcsec: [ID 270986 kern.notice] NOTICE: authdes_create: unable to get client's netname: RPC: Timed out (error 5)
Feb 15 12:49:19 client last message repeated 3 times
Feb 15 12:49:19 client nfs: [ID 120876 kern.warning] WARNING: NFS server initial call to server failed: Invalid argument

What RPC service should be running? NIS (ypbind) certainly is...

---------- Post updated at 03:53 PM ---------- Previous update was at 01:24 PM ----------

More error messages:

Feb 15 13:13:20 client rpcsec: [ID 270986 kern.notice] NOTICE: authdes_create: unable to get client's netname: RPC: Timed out (error 5)
Feb 15 13:16:51 client last message repeated 1 time
Feb 15 13:20:19 client rpcsec: [ID 270986 kern.notice] NOTICE: authdes_create: unable to get client's netname: RPC: Timed out (error 5)
Feb 15 13:20:19 client nfs: [ID 120876 kern.warning] WARNING: NFS server initial call to server failed: Invalid argument
Feb 15 13:35:07 client rpcsec: [ID 270986 kern.notice] NOTICE: authdes_create: unable to get client's netname: RPC: Timed out (error 5)
Feb 15 13:42:06 client last message repeated 3 times
Feb 15 13:42:06 client nfs: [ID 120876 kern.warning] WARNING: NFS server initial call to server failed: Invalid argument

What other services do I need to run? (ypbind is certainly running)...

Two suggestions:

  1. Make an alternative mount point to /mnt, e.g. /mount and try it.

  2. Can the NFS server ping the client by name and can the NFS client ping the server by name? If not then either put their names and IP addresses in the each ends hosts files or else put them into the NIS hosts table.

If still no success then what does running:

# rpcinfo -p server

show you when run on the client?

  1. Done this and it does not work (so mount point /mnt is not the problem).
  2. Yes, and yes. In fact a normal NFS share (without the sec=dh) shares and mounts (on /mnt) no problems. So I assume it is all to do with keys and AUTH_DH authentication and the mounting method. Anyone got any suggestions on how to do this..?
client# rpcinfo -p server
   program vers proto   port  service
    100000    4   tcp    111  rpcbind
    100000    3   tcp    111  rpcbind
    100000    2   tcp    111  rpcbind
    100000    4   udp    111  rpcbind
    100000    3   udp    111  rpcbind
    100000    2   udp    111  rpcbind
    100024    1   udp  32773  status
    100024    1   tcp  32772  status
    100133    1   udp  32773
    100133    1   tcp  32772
    100004    2   udp   1023  ypserv
    100004    1   udp   1023  ypserv
    100004    1   tcp   1017  ypserv
    100004    2   tcp  32773  ypserv
1073741824    2   udp  32774
    100021    1   udp   4045  nlockmgr
    100021    2   udp   4045  nlockmgr
    100021    3   udp   4045  nlockmgr
    100021    4   udp   4045  nlockmgr
    100021    1   tcp   4045  nlockmgr
    100021    2   tcp   4045  nlockmgr
    100021    3   tcp   4045  nlockmgr
    100021    4   tcp   4045  nlockmgr
    100007    3   udp  32781  ypbind
    100007    2   udp  32781  ypbind
    100007    1   udp  32781  ypbind
    100007    3   tcp  32776  ypbind
    100007    2   tcp  32776  ypbind
    100007    1   tcp  32776  ypbind
1073741824    1   tcp  32777
    100011    1   udp  32787  rquotad
    100005    1   udp  32790  mountd
    100005    1   tcp  32779  mountd
    100005    2   udp  32790  mountd
    100005    2   tcp  32779  mountd
    100005    3   udp  32790  mountd
    100005    3   tcp  32779  mountd
    100003    4   tcp   2049  nfs