Inconsistantly defined $ref for server

[jonaslagoni]

Describe the bug

In b51f0aa#diff-c42eed206b44fe4fa90f1635ff156f9804ae2450b901771b9c4d6db5188d640cR173 we released a property to the server object, to contain $ref instead of using the existing reference definition.

This results in an inconsistency in how references are defined.

Expected behavior

All references should use the same reference definition.

[jonaslagoni]

This was solved in #128