Permission set for /var/opt/morpheus/morpheus-ui

Hi Team,

We have setup the 3node HA and configured NFS shared path to the morpheus-ui location.When we tried to deploy the terraform app we are getting the below error.

2023-12-18_11:02:27.97772 '[2023-12-18 11:02:27,971] [RxCachedThreadScheduler-1173] WARN c.m.MorpheusUtils - error setting file owner on /var/opt/morpheus/morpheus-ui/workspace/terraform/cc5c9435278818ab79d826802ec160e1/app-7e88d397-4aad-44ad-a881-42371eee8251/terraform.tfstate to morpheus-app morpheus-local: {}
2023-12-18_11:02:27.97773 'java.nio.file.FileSystemException: /var/opt/morpheus/morpheus-ui/workspace/terraform/cc5c9435278818ab79d826802ec160e1/app-7e88d397-4aad-44ad-a881-42371eee8251/terraform.tfstate: Operation not permitted
2023-12-18_11:02:27.97773

We are able to change the permission manually to respective location but from the morpheus we are getting the above error? As Per my knowleadge when morpheus creating any files and folder its not able to change the user to morpheus-local.

make sure the morpheus-app and morpheus-local gid and uid are the same on all 3 appliances.

Yes all are same.

node1~# id morpheus-app
uid=997(morpheus-app) gid=999(morpheus-app) groups=999(morpheus-app),997(morpheus-local)
node1~# id morpheus-local
uid=996(morpheus-local) gid=997(morpheus-local) groups=997(morpheus-local)

node2~#id morpheus-app
uid=997(morpheus-app) gid=999(morpheus-app) groups=999(morpheus-app),997(morpheus-local)
node2~# id morpheus-local
uid=996(morpheus-local) gid=997(morpheus-local) groups=997(morpheus-local)

node3~# id morpheus-app
uid=997(morpheus-app) gid=999(morpheus-app) groups=999(morpheus-app),997(morpheus-local)
node3~# id morpheus-local
uid=996(morpheus-local) gid=997(morpheus-local) groups=997(morpheus-local)

Hi Team,

Any input on the above query?

Regards,
Ibrahim K

We are currently facing the same challenge. It works on the single node test environment, but not in the 3 node ha environment. Same error message.

Hi @ibrahim

  1. can you do getfacl on morpheus-ui folder under /var/opt/morpheus
  2. can you sudo as morpehus-app and morpehus-local user and see if you can create a file in this location?
    /var/opt/morpheus/morpheus-ui/workspace/terraform/cc5c9435278818ab79d826802ec160e1/app-7e88d397-4aad-44ad-a881-42371eee8251/
  3. what you have in /etc/fstab ?

root@cmpnode2:/var/opt/morpheus# getfacl morpheus-ui

  1. can you do getfacl on morpheus-ui folder under /var/opt/morpheus

file: morpheus-ui

owner: morpheus-app

group: morpheus-local

user::rwx
group::r-x
other::–x

root@cmpnode2:/var/opt/morpheus# getfacl morpheus-ui/workspace/

file: morpheus-ui/workspace/

owner: morpheus-app

group: morpheus-local

user::rwx
group::rwx
other::—
2. can you sudo as Morpheus-app and Morpheus-local user and see if you can create a file in this location?
Yes
3. what you have in /etc/fstab ?
nfs-ip:/var/nfs /var/opt/morpheus/morpheus-ui nfs auto,noatime,nolock,bg,intr,actimeo=1800 0 0 -o

you can try changing the fstab to these, and see if it still an issue,

nfs nfsvers=4.1,defaults,nofail 0 0

I am getting the below error:

Error: Error locking state: [{%!s(tfdiags.Severity=69) Error acquiring the state lock Error message: mkdir /var/opt/morpheus/morpheus-ui/workspace/terraform/4f27bad95953da76e7efa291281af4c3/app-be901878-1a6f-49f7-8e41-53d73055bba0/.terraform: permission denied

Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the “-lock=false”
flag, but this is not recommended. }]

Hi Ibrahim,
Please find the steps below.
in NFS
sudo groupadd -g 999 morpheus-app
sudo groupadd -g 997 morpheus-local

sudo useradd -u 997 -g 999 -G 997 -m -s /bin/bash morpheus-app
sudo useradd -u 996 -g 997 -m -s /bin/bash morpheus-local

Compare the below command O/P from NFS server and Node1
id morpheus-app
id morpheus-local
cat /etc/group | morpheus-app

chown -R morpheus-app:morpheus-local /nfs/morpheus/

in node 1

morpheus-ctl stop morpheus-ui
/var/opt/morpheus# cp -a morpheus-ui morpheus-ui.bak
mount -a
/var/opt/morpheus# cp -a morpheus-ui.bak/* morpheus-ui
morpheus-ctl start morpheus-ui

Thanks
Velan