Permission denied when docker container accesses mounted file or folder (file system and permission on Linux)

Recently, I got a task that I have to work on it on my Ubuntu VM instead of WSL.

I set up everything I have currently and try to run the project up, but got an error

lakefs                     | time="2023-11-03T07:07:31Z" level=fatal msg="Failed to create block adapter" func=cmd/lakefs/cmd.glob..func8 file="cmd/run.go:154" error="dialing: cannot read credentials file: open /credentials/provenance-386123-65c1435a7267.json: permission denied"

SELinux?

After googling it, it seems an issue comes from SELinux (Security-Enhanced Linux).

If you can simply add :z to the end of mount path. (reference)

For example:

# in docker-compose.yml

    volumes:
      - ./credentials/provenance-386123-65c1435a7267.json:/credentials/provenance-386123-65c1435a7267.json:z

The problem is it only applies to single file or folder and doesn’t affect subfolder or file in subfolder, so you must indicate the folder or file very clear.

File permission?

I went to check the permission of this file.

Let’s take a close look. What does it mean?

If you look at the file provenance-386123-65c1435a7267.json

At begining, we can see -rwxrwxrwx 1 takuro takuro

We can easily change the permission by chmod command.

For example:

chmod 770 provenance-386123-65c1435a7267.json

Then, we can check it again with command ls -la

You can see it changed from -rwxrwxrwx to -rwxrwx—

Breaking down 770, each number indicates the permission of owner, group and others.

Each permission also can represent by a number.

w (write) = 4

r (read) = 2

x (execute) = 1

7 (w+r+x) indicates the assignee can write, read and execute. 0 means the assignee cannot do anything on the file.

As you can see, I updated the .json file permission to 777. However, it didn’t work.

Set docker container execution user?

After googling and trying several approaches turns out set user is a simplest way to get this problem solved.

I was trying to run lakeFS by pulling the official image from the hub.

The .json file is owned by takuro user and takuro group, so if I set the user/group to takuro, then it should solve the problem, right?

To set user and group for container in docker-compose is adding

user: "${UID}:${GID}"

How can I know UID (user id) and GID (group id)?

You can see all users from /etc/passwd

# list all users
cat /etc/passwd

# search user
grep -w '^takuro' /etc/passwd

the first 1000 means UID, the second 1000 means GID

You can see all groups from /etc/group

# list all groups
cat /etc/group

# search group
grep -w '^takuro' /etc/group

the 1000 means GID

To know your current user and group info

# list all information about the current user
id

# display the current user group id
id -g

# display all groups that the current user belongs to
id -G

According to the information above, eventually I came out the setting like below

services:
  lakefs:
    image: treeverse/lakefs:latest
    user: "1000:1000" # or "1000" should do the work

It did gain the permission to read credentials, but another error occured.

lakefs                     | time="2023-11-03T17:10:18Z" level=fatal msg="failed to create catalog" func=cmd/lakefs/cmd.glob..func8 file="cmd/run.go:167" error="create tiered FS for committed metaranges: creating base dir: /lakefs/.local_tier/meta-range - mkdir /lakefs: permission denied"

Seems the user doesn’t have permission to create folders inside the container.

So closed to solve the issue…

While I was trying to figure out this permission issue, I had read some articles that talk about changing file/folder owner or permission via chown

# change myfile's owner to myuser
sudo chown myuser myfile

# change myfile's group to mygroup
sudo chown :mygroup myfile

# change owner and group at the same time
sudo chown myuser:mygroup myfile

# display result
sudo chown -v myuser:mygroup myfile

# change recursively
sudo chown -R myuser:mygroup myfile

Therefore, maybe I can only assign the container user to a certain group…

services:
  lakefs:
    image: treeverse/lakefs:latest
    user: ":1000"

Finally, everything worked out….

Reference:
https://www.cyberciti.biz/faq/understanding-etcpasswd-file-format/
https://blog.gtwang.org/linux/linux-chown-command-tutorial/