pangeo-data/jupyter-earth

Can we make a pod use some allocated memory and mount it as a filesystem location to write to?

Closed this issue · 1 comments

/dev/shm is a trick one can apply to mount RAM memory to a disk location. Can we make a k8s pod take some of its allocated memory and mount for use within the pod?

This is recommended when using Ray, but can we do that in a k8s environment where we have Pods etc, and where we both have requested memory that we are guaranteed, and a memory limit, where we are not guarnateed but could potentially use?

We have done this now. Here is a snippet from the JupyterHub helm chart configuration related to this.

    singleuser:
      # /dev/shm is mounted as a filesystem path, where writing to it means to
      # write to memory.
      #
      # How to: https://stackoverflow.com/questions/46085748/define-size-for-dev-shm-on-container-engine/46434614#46434614
      # Request for this by Ellie: https://fperezgroup.slack.com/archives/C020XCEFPEH/p1658168872788389
      #
      storage:
        extraVolumes:
          - name: dev-shm
            emptyDir:
              medium: Memory
        extraVolumeMounts:
          - name: dev-shm
            mountPath: /dev/shm