ayufan/pve-backup-server-dockerfiles

Add datastore, uses only 10% of the disk

Closed this issue · 9 comments

Hi,

I have a total clean install of buster lite on a RPI 4 and got all up and running (docker) and also connected to my VM server, BUT im struckling with a fresh USB mounted disk, i triede EXT4 and NTFS (both shows over 240GB free space (250GB disk), when i add storage, it only uses 28.43GB
Udklip

Any hints on what to do?

Sure... Here it is...

Mount:
sudo mount /dev/sdb1 /ssd

docker-compose.yml:

version: '2.1'

services:
  pbs:
    image: ayufan/proxmox-backup-server:latest
    network_mode: host
    mem_limit: 2G
    volumes:
      - pbs_etc:/etc/proxmox-backup
      - pbs_logs:/var/log/proxmox-backup
      - pbs_lib:/var/lib/proxmox-backup
      - backups:/backups
    tmpfs:
      - /run
    restart: unless-stopped
    stop_signal: SIGHUP

volumes:
  pbs_etc:
  pbs_logs:
  pbs_lib:
  backups:
    driver: local
    driver_opts:
      type: ''
      o: bind
      device: /ssd

Can you format that with codeblocks https://docs.github.com/en/github-ae@latest/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks?

And execute following:

mount
docker ps -a

Did you create data store in /backups? Do you see any new folders created in /ssd on host?

Did you do docker-compose up -d after updating /volumes?

Can you format that with codeblocks https://docs.github.com/en/github-ae@latest/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks?

Done, i didnt know that, thanks :-)

mount

/dev/sdb1 on /var/lib/docker/volumes/pi_backups/_data type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096)

docker ps -a

CONTAINER ID   IMAGE                                 COMMAND             CREATED          STATUS          PORTS     NAMES
16413b53eac4   ayufan/proxmox-backup-server:latest   "runsvdir /runit"   27 minutes ago   Up 27 minutes             pi_pbs_1

Did you create data store in /backups? Do you see any new folders created in /ssd on host?

I triede first with /ssd and then used backups from the yml file, no difference, same result...

Did you do docker-compose up -d after updating /volumes?

Yes i did, after every change i made

I'm seeing some partial information. Not seeing the full configuration :)

Can you?

  1. mount | grep sdb1
  2. Do docker inspect pi_pbs_1
  3. Do a screenshot of a datastore similar to included?
  4. You appear to be using fuseblk, which likely indicates the drive to be NTFS formatted. If this is correct regardless the disk size problem it will not work well with PBS.

image

  1. mount | grep sdb1

Just formated it back as EXT4, as i had at first... Still same problem...

/dev/sdb1 on /ssd type ext4 (rw,relatime)
/dev/sdb1 on /var/lib/docker/volumes/pi_backups/_data type ext4 (rw,relatime)
  1. Do docker inspect pi_pbs_1
[
    {
        "Id": "cfc63f2b849c575cd856e31396e8e9c373ef96869e74b568c00373bc1a1bf50f",
        "Created": "2022-04-15T20:08:31.719381239Z",
        "Path": "runsvdir",
        "Args": [
            "/runit"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 8023,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-04-15T20:08:32.705621722Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:1248531d9c613cc594525f26ce21110c4ab706b2ac442d4e3df03f376b431558",
        "ResolvConfPath": "/var/lib/docker/containers/cfc63f2b849c575cd856e31396e8e9c373ef96869e74b568c00373bc1a1bf50f/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/cfc63f2b849c575cd856e31396e8e9c373ef96869e74b568c00373bc1a1bf50f/hostname",
        "HostsPath": "/var/lib/docker/containers/cfc63f2b849c575cd856e31396e8e9c373ef96869e74b568c00373bc1a1bf50f/hosts",
        "LogPath": "/var/lib/docker/containers/cfc63f2b849c575cd856e31396e8e9c373ef96869e74b568c00373bc1a1bf50f/cfc63f2b849c575cd856e31396e8e9c373ef96869e74b568c00373bc1a1bf50f-json.log",
        "Name": "/pi_pbs_1",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "pi_pbs_lib:/var/lib/proxmox-backup:rw",
                "pi_pbs_logs:/var/log/proxmox-backup:rw",
                "pi_pbs_etc:/etc/proxmox-backup:rw",
                "pi_backups:/backups:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "host",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "unless-stopped",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "Tmpfs": {
                "/run": ""
            },
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": -1,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/84e9bfc8e184476b9a9cb13291f051154c80bdf58beb7bbeb4abab3ae083ddd1-init/diff:/var/lib/docker/overlay2/8f42a71e7ed6296dbeaca85d2e9ce895dd5b259b3e6bee9464dbfe31baf12ed5/diff:/var/lib/docker/overlay2/f263e5c0694ea692ef2bab46fb7112e5383751bd430f9331cb17054a61554982/diff:/var/lib/docker/overlay2/ed107af925cfee25c89ae4e475a93aa595e39a0851053a849f07907478a7cf3d/diff:/var/lib/docker/overlay2/90cd4d9074e01a10cb06c2f69c713fce16866ad1e57aeb508e2edb5a21fd77a0/diff:/var/lib/docker/overlay2/6d0043ccd7fe44fc46a3c636bd245db40549a091faa2f0474035431a63f68f4c/diff",
                "MergedDir": "/var/lib/docker/overlay2/84e9bfc8e184476b9a9cb13291f051154c80bdf58beb7bbeb4abab3ae083ddd1/merged",
                "UpperDir": "/var/lib/docker/overlay2/84e9bfc8e184476b9a9cb13291f051154c80bdf58beb7bbeb4abab3ae083ddd1/diff",
                "WorkDir": "/var/lib/docker/overlay2/84e9bfc8e184476b9a9cb13291f051154c80bdf58beb7bbeb4abab3ae083ddd1/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "pi_pbs_etc",
                "Source": "/var/lib/docker/volumes/pi_pbs_etc/_data",
                "Destination": "/etc/proxmox-backup",
                "Driver": "local",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "volume",
                "Name": "pi_backups",
                "Source": "/var/lib/docker/volumes/pi_backups/_data",
                "Destination": "/backups",
                "Driver": "local",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "volume",
                "Name": "pi_pbs_lib",
                "Source": "/var/lib/docker/volumes/pi_pbs_lib/_data",
                "Destination": "/var/lib/proxmox-backup",
                "Driver": "local",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "volume",
                "Name": "pi_pbs_logs",
                "Source": "/var/lib/docker/volumes/pi_pbs_logs/_data",
                "Destination": "/var/log/proxmox-backup",
                "Driver": "local",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "raspberrypi",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "runsvdir",
                "/runit"
            ],
            "Image": "ayufan/proxmox-backup-server:latest",
            "Volumes": {
                "/backups": {},
                "/etc/proxmox-backup": {},
                "/var/lib/proxmox-backup": {},
                "/var/log/proxmox-backup": {}
            },
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "6277f190061acf2496cd9680c66ac4ec3b9527ffb935e42aabec0da5855bc8ee",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "pi",
                "com.docker.compose.project.config_files": "docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/pi",
                "com.docker.compose.service": "pbs",
                "com.docker.compose.version": "1.25.0"
            },
            "StopSignal": "SIGHUP"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "6234b543208e97befc5502fece3c762fa9c24e5df7389660c5a7885c1919db14",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/default",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "host": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "fb1269a7ec0c255cfc5f317c764ea91c141c5b32cbe4d93b902232496db9cec6",
                    "EndpointID": "14a01827d59bf31da185419a43a430b96dab2aab0842df886d3686ec3369d24e",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]
  1. Do a screenshot of a datastore similar to included?
![Udklip](https://user-images.githubusercontent.com/11716346/163627887-c4d0a385-f03e-48e2-9762-bb14969e225a.PNG)
  1. You appear to be using fuseblk, which likely indicates the drive to be NTFS formatted. If this is correct regardless the disk size problem it will not work well with PBS.
Okay, as i wrote in the 1. i just formatted it back to ext4 :-)

So. Your datastore is created in /ssd instead of /backups which is a volume in container.

Just change volumes: - backups:/ssd and it should work?

(y)
It worked :-)

I know what i did wrong i misunderstod
I had - backups:/ssd, but in device i had /dev/sdb1...

So backups AND device, both has to be the moint point (may i suggest that to be noted in the docker-compose.yml?

Thanks a lot for your help :-)

PS: Im not so used in docker/docker-compose, so if you in your README.md at the

docker-compose up -d

add that there is a docker-compose.yml to use, else you get an error running docker-compose up -d ;-)

Cool. Happy to help out.