ACL permissions setup

All,

I am building a glusterfs environment for file storage and need to set up ACL's as there are multiple users that need different types of access. I have ingested ~20TB of needed data to /toplevel dir and:

chown -R root:root /toplevel ; chmod -R 775 /toplevel

What I need from ACL as far as permissions on all files and folders is:

group1:rwx
group2:rwx
group3:r--

This it appears I can accomplish with:

setfacl -R -m g:group1:rwx,g:group2:rwx,g:group3:r /toplevel

The problem I'm having is with the defaults that need to be in place for user folder and file creation inheriting the same permissions. I ran:

setfacl -R -m default:g:group1:rwx,default:g:group2:rwx,default:g:group3:r /toplevel

However, this did not allow for what I need (possibly due to mask?). If a user in group1 created a folder, another user in group1 could not write to it.

So, in testing a couple things I ran:

setfacl -m m:rwx /toplevel
setfacl -m default:u::rwx,default:g::rwx,default:o::rx /toplevel

This did not change the behavior. Here is the getfacl on /toplevel now:

# getfacl toplevel
# file: toplevel
# owner: root
# group: root
user::rwx
group::rwx
group:group1:rwx
group:group3:r--
group:group2:rwx
mask::rwx
other::r-x
default:user::rwx
default:group::rwx
default:group:group1:rwx
default:group:group3:r--
default:group:group2:rwx
default:mask::rwx
default:other::r-x

Does anyone see here what I am doing wrong?

A related question: When configuring the default settings, is using the '-R' ONLY looking for directories to set the defaults or is it traversing files and skipping them, slowing down the process? Or would it be more efficient to do something like:

find /toplevel -type d -exec <setfacl cmd> {} \;

Thanks in advance, any guidance is greatly appreciated..

HB

---------- Post updated at 05:50 PM ---------- Previous update was at 03:00 PM ----------

Update:

When I create a folder under /toplevel from the command line as directory owner root, I get the following ACL's:

# getfacl toplevel/testing3
# file: toplevel/testing3
# owner: root
# group: root
user::rwx
group::rwx
group:group1:rwx
group:group3:r--
group:group2:rwx
mask::rwx
other::r-x
default:user::rwx
default:group::rwx
default:group:group1:rwx
default:group:group3:r--
default:group:group2:rwx
default:mask::rwx
default:other::r-x

And with these ACL's, I can write to the newly created /toplevel/testing3 with users in group1.

When I create a folder under /toplevel from a CIFS share as a user in group1, I get the following ACL's:

# getfacl toplevel/testing4
# file: toplevel/testing4
# owner: user.1
# group: domain\040users
user::rwx
group::rwx		#effective:r-x
group:group1:rwx	#effective:r-x
group:group3:r--
group:group2:rwx	#effective:r-x
mask::r-x
other::r-x
default:user::rwx
default:group::rwx
default:group:group1:rwx
default:group:group3:r--
default:group:group2:rwx
default:mask::rwx
default:other::r-x

With these ACL's users in group1 other than user.1 are unable to write to the newly created /toplevel/testing4 directory.

Does anyone have any idea of what is going on here?

Thanks,

HB

Update:

I have found that my problem was with my Samba settings (smb.conf). Specifically, I needed to add:

inherit acls = yes
inherit owner = yes
inherit permissions = yes

Now users in group1 and group3 can create folders and files that others in the groups can rw to.

Thanks,

HB