khenriks/mp3fs

memory

Closed this issue · 8 comments

RobMeerwijk commented

mp3fs in docker keeps a lot of memory occupied: 4.3 GB

I use mp3fs for daily converting new music to mp3 by means of rsync. After adding a few CD's the job is finished. The memory is not yet freed 12 hours later.
Screenshot 2021-12-08 at 14-48-07 nas-1 - Synology NAS

khenriks commented

Hi, can you provide more info about how you're running it, like docker command? Do you know which version it is?

khenriks commented

The first 30 lines or so of docker logs output would actually be really helpful.

RobMeerwijk commented

Hi,

I use docker-compose.yml:

# Basic Docker Compose file for mp3fs
version: '3'
services:
    mp3fs:
        image: khenriks/mp3fs:latest
        devices:
            - /dev/fuse
        cap_add:
            - SYS_ADMIN
        security_opt:
            - apparmor:unconfined
        volumes:
            - /volume1/media/Music:/music:ro
            - /volume1/mp3fs/mount:/mnt:shared
        command: "-b 128"

The command I use is docker-compose up -d

docker version:

Client:
 Version:           20.10.3
 API version:       1.41
 Go version:        go1.15.13
 Git commit:        b455053
 Built:             Thu Aug 19 07:13:24 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.3
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.15.13
  Git commit:       a3bc36f
  Built:            Thu Aug 19 07:11:25 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.3
  GitCommit:        ea3508454ff2268c32720eb4d2fc9816d6f75f88
 runc:
  Version:          v1.0.0-rc93
  GitCommit:        31cc25f16f5eba4d0f53e35374532873744f4b31
 docker-init:
  Version:          0.19.0
  GitCommit:        ed96d00

docker logs:

2021-12-09T08:33:13.320499054Z [2021-12-09 08:33:13] tid=140186708821824 DEBUG: LAME ready to initialize.
2021-12-09T08:33:13.322319578Z [2021-12-09 08:33:13] tid=140186708821824 DEBUG: mp3fs version: 1.1.1
2021-12-09T08:33:13.322435047Z LAME library version: 3.100
2021-12-09T08:33:13.322491357Z FLAC library version: 1.3.2
2021-12-09T08:33:13.322540077Z Xiph.Org libVorbis 1.3.6
2021-12-09T08:33:13.322588907Z FUSE library version: 2.9
2021-12-09T08:33:13.322635967Z 
2021-12-09T08:33:13.322674527Z [2021-12-09 08:33:13] tid=140186708821824 DEBUG: MP3FS options:
2021-12-09T08:33:13.322723476Z basepath:       /music
2021-12-09T08:33:13.322766906Z bitrate:        128
2021-12-09T08:33:13.322810286Z desttype:       mp3
2021-12-09T08:33:13.322856546Z gainmode:       1
2021-12-09T08:33:13.322897426Z gainref:        89
2021-12-09T08:33:13.322938716Z log_format:     [%T] tid=%I %L: %M
2021-12-09T08:33:13.322985205Z log_maxlevel:   DEBUG
2021-12-09T08:33:13.323027275Z log_stderr:     1
2021-12-09T08:33:13.323069105Z log_syslog:     0
2021-12-09T08:33:13.323110735Z logfile:        
2021-12-09T08:33:13.323151565Z quality:        5
2021-12-09T08:33:13.323193855Z statcachesize:  0
2021-12-09T08:33:13.323249545Z vbr:            0
2021-12-09T08:33:13.323294794Z FUSE library version: 2.9.9
2021-12-09T08:33:13.323340144Z nullpath_ok: 0
2021-12-09T08:33:13.323382634Z nopath: 0
2021-12-09T08:33:13.323448424Z utime_omit_ok: 0
2021-12-09T08:33:13.323492164Z unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
2021-12-09T08:33:13.323548654Z INIT: 7.23
2021-12-09T08:33:13.323590664Z flags=0x0003fffb
2021-12-09T08:33:13.323634913Z max_readahead=0x00030000
2021-12-09T08:33:13.323673763Z    INIT: 7.19
2021-12-09T08:33:13.323710353Z    flags=0x00000011
2021-12-09T08:33:13.323748503Z    max_readahead=0x00030000
2021-12-09T08:33:13.323799433Z    max_write=0x00020000
2021-12-09T08:33:13.323839763Z    max_background=0
2021-12-09T08:33:13.323879953Z    congestion_threshold=0
2021-12-09T08:33:13.323920312Z    unique: 1, success, outsize: 40

Actually I found a solution in docker-compose.yml that works on my docker version. But as I understand it, the options on limits are depending on the docker-compose version and not backwards/forwards compatible:

# Basic Docker Compose file for mp3fs
version: '3'
services:
    mp3fs:
        image: khenriks/mp3fs:latest
        devices:
            - /dev/fuse
        cap_add:
            - SYS_ADMIN
        security_opt:
            - apparmor:unconfined
        volumes:
            - /volume1/media/Music:/music:ro
            - /volume1/mp3fs/mount:/mnt:shared
        command: "-b 128"
        deploy:
           resources:
             limits:
                memory: 512M
khenriks commented

Do you run rsync locally or access mp3fs over the network? The main thing I can think of that would keep the memory occupied is if the file handles never get closed.

RobMeerwijk commented
RobMeerwijk commented

For me the issue is solved, because I can limit available memory by means of docker-compose.

khenriks commented

I'll close this since it's addressed for you, but I am curious about the memory usage, so I might add some options to help with tracking memory usage.

RobMeerwijk commented