Kontemplate does not work with Ubuntu 17.10 guest under VMWare Player Windows 10 host

[agearhart]

Running kontemplate in an Ubuntu 17.10 virtual machine inside a Windows 10 host consistently produces
Warning: Resource set 'general' does not exist or contains no valid templates

Running the same command, kontemplate template environment/development.yml, on the host succeeds as expected.

[tazjin]

Hey,

are you using the same version of kontemplate on both systems?

on the host succeeds as expected

on the host == on the windows machine?

[tazjin]

I run neither Ubuntu nor Windows, but experimenting in an Ubuntu 17.10 container works fine. The kontemplate release binary is actually statically linked, so the host distribution and its libraries shouldn't make much of a difference for the program.

AFAIK Windows filesystems handle paths differently than *nix-like systems, could that be related? (e.g. path specifications in "Windows format" in your configuration?)

[agearhart]

Yes, I'm running 1.3.0 on both Windows 10 and the Ubuntu VM. And yes, the host is Windows 10 and Ubuntu is a virtual machine running in Windows 10.

Windows does handle pathing differently, but I don't believe that should matter since I'm working in the VM and the templates are also in the VM on the virtual HDD. Are there logs generated that I could attach to help?

[tazjin]

Hm, it's probably related to the directory structure. Can you post the relevant include key from development.yml and maybe a tree or find output?

[agearhart]

I tried running kontemplate template environment/development.yml -i auth-api from inside the platform folder.

The include for -i general looks like

include:
  - name: general

The include for Auth-API looks like

  - name: auth-api
    values:
      version: 1.8.0
      replicaCount: 2
      redisIP: 192.168.0.1
      tokenDurationHours: 24
      inMaintenanceMode: false

Tree

kubernetes-global
├── monitoring
│   ├── elk
├── platform
│   ├── auth-api
│   │   ├── auth-api-config.yml
│   │   └── auth-api-deployment.yml
│   ├── crash-upload-api
│   │   ├── crash-upload-api-config.yml
│   │   ├── crash-upload-api-deployment.yml
│   │   ├── crash-upload-api-ingress.yml
│   │   └── crash-upload-api-service.yml
│   ├── create-once
│   │   ├── auth-api-ingress.yml
│   │   ├── auth-api-service.yml
│   │   ├── chat-api-ingress.yml
│   │   ├── chat-api-service.yml
│   │   ├── matchmaking-api-ingress.yml
│   │   ├── matchmaking-api-service.yml
│   ├── development-secrets
│   ├── environment
│   │   ├── development.yml
│   │   ├── internal.yml
│   │   ├── production.yml
│   │   └── staging.yml
│   ├── general
│   │   └── logging-config.yml
│   ├── general-secrets
│   ├── matchmaking-api
│   │   ├── matchmaking-api-config.yml
│   │   └── matchmaking-api-deployment.yml
│   ├── matchmaking-worker
│   │   ├── matchmaking-worker-config.yml
│   │   └── matchmaking-worker-deployment.yml
│   ├── staging-secrets
└── README.md

[tazjin]

(I edited your comment slightly to add code blocks, hope you don't mind)

---

This behaviour is actually expected, perhaps a bit counter-intuitively the path of a resource set (unless manually overridden by setting path) is constructed using the folder in which the context configuration is as the base for all relative URLs, not the working directory.

So say you have this simple layout:

.
├── environment
│   └── development.yml
└── general
    └── logging-config.yml

If development.yml includes general the layout kontemplate expects is something like this:

.
├── environment
    ├── development.yml
    └── general
        └── logging-config.yml

Even when run from ..

TL;DR: This should work if you move your files from environment one layer up, such that resource sets are sub folders of the folder in which your context configuration resides.

---

Longer trailing thoughts on this:

You're not the first person who got confused by this, so there is clearly some sort of friction involved. My reasoning for not constructing paths relative to the working directory is two-fold:

1. I want things to be completely deterministic regardless of where the user is currently operating, so say you're in ~/projects/foo and want to quickly run kontemplate apply ../kubernetes/some-cluster.yml that should work.

2. If you think of the "root" folder containing your kontemplate configuration in terms of types, you may have something like this:

data FolderContent = ClusterConfig ... -- ^ Represented as a _text file_
                   | ResourceSet [Resources] -- ^ Represented as a _folder_ or a text file

type RootFolder = [FolderContent]

With that conceptual model every folder must be a resource set, so what happens with the above "broken" layout if you were to type:

include:
  - name: environment

I'm thinking that the current behaviour is correct (™️), but that it should probably be documented better. I'll tag this with my usual friction-reduction tag and see if I can come up with something.

---

I hope this makes sense to you and please let me know if it doesn't!

[tazjin]

Further thoughts: I'm unsure how this can work when run from the host. What currently happens is this:

// context.go
{
  // in LoadContextFromFile
  c.BaseDir = path.Dir(filename)
  // if filename is "environment/development.yaml", BaseDir will be just "environment"
}

// templater.go
{
  // in processResourceSet
  rp := path.Join(c.BaseDir, rs.Path)

  // where rs.Path is either set to the "name" (default) or user overriden, so for
  // `include: {"name": "general"}` this would be "environment/general", which is
  // then opened as a folder and traversed.
}

If this for some reason works on Windows, something is probably wrong there and that is in fact a bug. I don't really have access to a Windows machine to test that on though, hmm

[agearhart]

That entirely makes sense and thank you for your time guiding me through it. We'll adopt your solution and move the environment yamls out to the root folder. This really does work on Windows though with the same command:

agearhart@PC-AGEARHART D:\platform\kubernetes-global\platform

kontemplate template environment\development.yml -i auth-api
Loading resources for auth-api
Rendered file auth-api/auth-api-config.yml:

---

apiVersion: v1
kind: ConfigMap
metadata:
name: auth-api-config
namespace: development
data: ....

[tazjin]

That entirely makes sense and thank you for your time guiding me through it

np! A good side-effect of explaining the reasoning behind stuff like this is that people can call me out when it's silly ;-)

This really does work on Windows though with the same command

Strange, I'll see if I find some time to debug that. Thanks for pointing it out!