External assets
zdne opened this issue · 43 comments
Add the possibility to use Markdown links wherever an asset is expected (e.g. a payload body or schema).
I'm interested in this capability, and would ideally like it added sooner rather than later.
Our particular use case is for documenting APIs where the request and response bodies are binary structures produced by Apache Avro. For these requests/responses, we'd like to be able to have the "Schema" be a link to either the Avro schema definition file, or a generated HTML document describing said Avro schema. It might look something like this:
## POST /myresource
Creates a new myresource.
+ Request (application/vnd.example.myresource+avro)
+ Schema:
[MyResource.avsc](http://docs.example.com/avro/myresource.avsc)
+ Response 201 (application/vnd.example.myresource+avro)
+ Schema:
[MyResource.avsc](http://docs.example.com/avro/myresource.avsc)👍
@mhurne @davidmc24 Have you thought about representation of this feature in the AST? Would you like to see hyperlink in the AST or blueprint parser should be capable to fetch target url and embed its content to the AST?
My expectation is that the AST would contain the hyperlink, not the content of said hyperlink. It might be useful for the AST to "know" that it's a hyperlink as opposed to direct content, so that links can be produced in HTML documentation, for example.
In cases where you've chosen to externalize the schema from the API, it's likely because either the schema is too large to make sense to inline in the API documentation, the schema is used by multiple APIs and has its own authoritative documentation, or the schema is a format which isn't appropriate for embedding. While those examples are schema-specific, I expect similar reasons apply to other payload types.
I do like it. If it would be just the hyperlink it would be probably fairly easy to implement in the parser.
The bigger challenge however comes with the needed change to the AST and its media-types
Probably to something like this:
{
"name": "<name>",
"description": "<description>",
"headers": {
"<header>": {
"value": "<header value>"
}
},
"body": {
"content": "<body>",
"href": "<uri>"
},
"schema": {
"content": "<schema>",
"href": "<uri>"
}
}Yep, that's exactly the AST I was considering.
Additionally add additional syntax for external assets to be embedded (not just referenced-linked) by the Drafter apiaryio/snowcrash#57
:[file.json](path/to/file.json)
or
=[file.json](path/to/file.json)
Alternatively using the Mardkown implicit link name:
:[path/to/file.json]()
or
=[path/to/file.json]()
Hello,
Here's another use case for having external links: I would like to link the definitions/descriptions of attributes in the API to their respective definition in our master glossary used by all documentation. I think this opens the door to keeping all documentation up to date and in sync. Otherwise, I will have to update the API doc. separately, which leave room for definition not to be in sync.
Markdown is an uncomfortably loosely coupled family of mostly compatible extensions, so I'm not sure, but the only case I can think of where a link-like syntax actually means "embed not reference" is the image syntax (using a ! sigil): , which is part of Gruber's original specification. Z's suggestions above all violate this structure (file name in the wrong place). Wouldn't it be better to conform, just use a new sigil on the otherwise existing syntax?
Good point @jrep !
Wouldn't it be better to conform, just use a new sigil on the otherwise existing syntax?
That is my goal.
According to Gruber's following should achieve the same goal:
[label](path/to/image.png)
[alt text][label2]
[label2]: path/to/image.png
[label3][]
[label3]: path/to/image.png
Where label, alt text and label3, in theory, does not have be present but then the elements are not visible after rendering.
I guess my point with
=[path/to/file.json]()
Should be
=[path/to/file.json][]
where the lablel is the actual URL, so proper full definition (so the markdown renders correctly) would be:
=[path/to/file.json][]
[path/to/file.json]: path/to/file.json
Now it would be obviously tedious to write:
[path/to/file.json]: path/to/file.json
Question is – should this (defining the label) be implicitly assumed by the parser? Or should we just ask blueprint writers to use the [alt text](URL) syntax?
Couldn't we just use the alt/label part for actually describing what the asset is so the link is human-readable? Writing something like =[file.json](path/to/file.json) seems to be needlessly tedious, but =[Schema for notes](path/to/file.json) seems fine to me - encouraging blueprint writers to make blueprints less cryptic for readers. Actually, assets can be missing or it can be sometimes non-obvious what precisely is linked. alt attributes on images were invented exactly for this case I think.
If someone wants to bypass this and ignore the best practise of describing assets this way, she can always put a dummy text there (such as already mentioned file.json).
I used the = syntax just as an example, the same would apply for !.
I feel we should just support markdown with the extra sigil regardless of the syntax used – be it direct link, explicit or implicit label.
If we agree on this only two questions remain:
-
What sigil to use
-
Whet style we will use in examples / tutorial
Note that in my opinion label such as
Schema for notesin=[Schema for notes](path/to/file.json)
is dup as well because most often than not you will use it in a schema section of a Notes resources...
e.g.:
# Notes [/notes]
## GET
+ Response 200
+ Body
...
+ Schema
=[Schema for Notes](path/to/file.json)
Thoughts?
I'm all for supporting all of the standard variants, but this:
=[path/to/file.json][]
[path/to/file.json]: path/to/file.json
... is not how I understand Gruber's definition. Rather, I agree with honzajavorek: the stuff in the first [] is in some way a human-readable tag, not a redundant and tiresome duplicate of the link text. So, while it seems to be syntactically valid to include a path there, it's (a) weird, and (b) unhelpful.
If the intent is "any Gruber-valid link form, with a new sigil," then I like that semantic but suggest a less surprising example (of the sort honzajavorek and I have suggested).
@jrep @honzajavorek I agree.
I think its difficult to compare inclusion of assets to existing pattenrs for either images or links. Isn't it actually combination of those two together?
The problem with image syntax analogy is that in the world of images, alt texts are important, they're usually not duplicate in given context and nobody wants to actually write  - only content managament systems and a "lazy" people do this. Requiring the alt text is teaching a good habit.
But assets don't necessarily need alt texts, that's true. As @zdne mentioned, they'll be always used in such context that it'll be totally clear what's going on here. I agree with that. The problem is, the only analogy to that I can think of are links without labels:
[Google](http://www.google.com) versus...
<http://www.google.com>(Gruber)http://www.google.com(today's Markdown reality, I guess)
If we need something hybrid, let's call it "autolink syntax for images", and we want to adhere to Gruber as much as we can, we want something like following:
=[Alternative text about assets](/dev/null)← everyone is familiar with this, this should be in examples (it's like images, makes sense! popping in peoples' heads)=[Alternative text about assets][id]← for including the same asset onto multiple places=</dev/null>or<=/dev/null>← autolink syntax (...how would autolink syntax look like for images?), noone knows the Gruber's version of autolink (because links are converted automagically everywhere, even without<s and>s), but in case they really really don't need the alt text and they're annoyed by it, people can learn to use this
Again, using = just as an example. In my opinion those would be the most readable and/or at least Gruber-ish solutions. For example, I consider
[label3][]
[label3]: path/to/image.png
to be rather cryptic for me as a reader and not following the reality of existing Markdown usage. I didn't even know about such possibility, I thought pairing has to be done by the second couple of brackets. (However, I am fully aware this last paragraph is very subjective).
@zdne : as to the sigil: I don't much care, but the = we've all been using seems fine.
@zdne: I acknowledge that the most likely text =[IN HERE](schema.json) is redundant for any Blueprint usage I can think of. Just brainstorming, here, but what if we leverage that redundancy to simplify the grammar? That is, replace
# Notes [/notes]
## GET
+ Response 200
+ Body
...
+ Schema
=[Schema for Notes](path/to/file.json)
with
# Notes [/notes]
## GET
+ Response 200
+ Body
...
+ =[Schema for Notes](path/to/file.json)
@honzajavorek: are you actually suggesting that Blueprint allow/mandate the string "/dev/null" in those places? Or was that just a place-holder?
I never use "reference links," either, but I wouldn't object to Blueprint carrying them forward.
@jrep I used /dev/null just as a placeholder :)
Using links directly in section headers does not look bad at all, actually! Of course, it's just a first sight, I didn't really thought about what are the cons of such thing, but... at first sight it seems appealing to me.
@jrep I like it.
Taking it even further
# Notes [/notes]
## GET
+ Response 200
+ =[Body](path/to/body.json)
+ =[Schema](path/to/schema.json)
The ability to split the file into separate files and then import them in would be very useful for long verbose API's.
In the interm I manage my blueprint project with grunt. I use grunt-import and my index.source.apib file is made up of mostly @import "/include/section.apib";
Then all I do is run grunt watch and as soon as I save any file grunt will recompile my index.source.apib file into 'index.apib' - which is also being watched by aglio, refreshing my browser to show me the render in real time.
It takes 5 minutes to setup, but it makes life so much nicer! 👍
@alexborisov that is pretty neat! Thanks for the sharing. As I have said (elsewhere) – we have started the works on a parser harness tool that should take eventually care of this.
Also, for what its worth, here is a script that composes multiple blueprints into one and publish it using the Apiary client gem https://gist.github.com/danvine/11087404
@alexborisov, that's a really smart work-around! Do you (or anyone watching this issue) happen to know if there's a Gulp counterpart to grunt-import? The gulp ones I found all target JS and html docs.
@skawaguchi You can use gulp-file-include and create a gulp task to watch your json and build the api. The only downside is that you have to have an api.source.md to generate your api.md, but IMO it's a small price to pay.
Also for what it's worth checkout @jamesramsay 's Hercule – https://github.com/jamesramsay/hercule
Thanks @zdne for the mention. I had similar challenges when writing API documentation and we have been using Hercule internally for the last 3 months.
Hercule uses a simple {{filename.md}} pattern for inclusion.
We have a number of API contexts for different types of API consumers that have nearly identical resources and we were keen to reduce duplication between the different documents. Thus, we have also extended this syntax to also support:
{{entity.json links:role-links.json}}
{
"id": 123,
"name": "example",
"links": {
{{links}}
}
}
The {{links}} placeholder can then have the target file specified by one of it's parent files. We've found this helpful for large schemas which are nearly the same.
I am just wondering why to use {{link}} syntax instead of Markdown's very own [<placeholder>](link) ? as discussed above. For example:
=[gist](gist.apib)or
:[gist](gist.apib)it would work similarly, it would render reasonably as markdown, so you would be able to navigate through links, and you could use much richer syntax such as [placeholder][] with `[placeholder] link options...
@zdne this is a really great suggestion! Too bad I didn't see this thread 6 months ago.
The {{link}} syntax was based on MultiMarkdown 4's transclusion syntax. It did the trick and I wasn't inventing a new syntax.
At the time I also looked at Marked app (<<[link]) and a couple of other Markdown compilers.
With the advantage of being able to click through the links when viewing the source documentation in Github and improved consistency with the feel of Markdown I've updated hercule to use :[gist](gist.apib). If anyone would like to take it for a spin, it's on NPM.
Hey guys, I am using API Blueprint with aglio for our internal project, and since API Blueprint doesn't support file including currently, aglio implements one with the syntax <!-- include(filename.json) -->, thanks to this feature we can split request/resposne JSON data from API document clearly, but when we organizing the seperated JSON data files, we find it still lack of a proper way to reuse the same part, for example, we have a user_model.json:
{
"username": "alice",
"comments": [
{
"id": 1,
"content": "hello"
}
]
}and a comment_model.json:
{
"id": 1,
"content": "hello"
}the comment part in user_model.json is and should always be the same as comment_model.json, if there is a mechanism that allows JSON file to include each other, it would be very easy to maintain their consistency, so that in case the commend model is changed, we don't need to change it everywhere.
I consider it for some time and try making a syntax to support JSON file including, here is my effort,
a include sytanx with a Python implementation: https://github.com/reorx/json_include
The syntax would be like, in previous case:
{
"username": "alice",
"comments": [
{
"...": "include(commend_model.json)"
}
]
}using { "...": "include(commend_model.json)"} to represent a including to commend_model.json file.
Its not a special case for API Blueprint, but something rather general, to try to think of JSON itself with the include capability, which I think may also be used in API Blueprint.
Please take a look and give some suggestions, currently this json-include is working fine in some of my projects, any thoughts about it would be very appreciated, since I can't decide whether it is good and meaningful or not alone by myself :)
Hey @reorx ! Thanks for sharing the idea. Recently we have launched (beta) of MSON – a syntax for describing data (not only) in API Blueprint. The idea is, you describe your data once and then ask for serialization into different formats.
Using your examples it works like this:
# Data Structures
## User
- username: alice
- comments (array[Comment])
## Comment
- id: 1
- content: helloand then you can ask to render it as JSON:
# Resource [/r]
## Retrieve [GET]
+ Response (application/json)
+ Attributes (User)
What do you think? Will this help you?
Is this feature anywhere in the pipeline? It would be very nice for us.
We use Blueprint to describe methods, but have our JSON schema hosted separately. We have to include the schema as links in the method descriptions, since it isn't supported under the request/response element currently.
Actually the most ideal thing would be if we could "include" the JSON schema webpage using a link under the request/response element, but it would have to be lazily loaded, to avoid doing hundreds of includes when the documentation page is loaded.
I think that the fact you can't include external references for requests/response body examples and definition schema files is one of the very big downsides that Blueprint has in respect to other API definition languages.
+1
@zdne I'm not following.
Those two examples do not show much. What am I looking at?
@zdne: yes, as long as it allows inclusion of any type of external resource, be it .md/.apib or xml, json that's good.
Is it something Apiary will implement, or is your message an advise for us to use hercule before committing to apiary?
Back ago I have prepared a list of solutions for splitting blueprint into multiple files (and using exteral assets): https://gist.github.com/zdne/137396a1c2d45f75b306
@jamesramsay put together the awesome Hercule tool which implements the syntax I would like to use for splitting blueprint into multiple parts.
As for the external assets. Hercule works there as well albeit I am not sure if this should be the final syntax and suggested use inside JSON blobs especially through the prism of MSON.
Nevertheless if you do not rely on editing inside Apiary you can use Hercule today to get most of this functionality.
Is it something Apiary will implement?
We want to implement some transclusion based on the :[](url) syntax, yes. The catch here is the Apiary editor and implementation of a "filesystem". Until then you would have to use Hercule and push concatenated file into Apiary.
Does this clarify?
The catch here is the Apiary editor and implementation of a "filesystem"
Maybe you can circumvent that by using URLs?
For fetching the documents perhaps yes but not for editing.
Currently we have an Apiary blueprint stored in our git repository, which is connected to Apiary. The blueprint has many links to an internal server that hosts our JSON schema.
I suppose with this solution we would have an original Markdown file, then use a Git hook or something to run the transclusion and commit the result as the sync'd blueprint. Sounds like it could work; a bit of a hassle to set up the transclusion process, but it would be slick if it works, so I will look into it. Thanks!
@lpbm Hercule supports both local file (relative file paths) and remote file (URL) transclusion, and although labelled as a 'Markdown transclusion' should support any text file (.md, .apib, .json, .xml etc). One use case that might be similar to yours that some people are using Hercule for is transcluding JSON Schema generated by their application via HTTP, and also local file transclusion to increase the DRY-ness of their documentation.
@jonathancrosmer spot on - we do something similar using Jenkins on our internal CI environment. At Adslot, whenever a commit is pushed to any branch, along with unit tests we compile the API Blueprint with Hercule, and then pass it into dredd. If all is passing, as part of the deploy task, we can optionally push to Apiary. We originally pushed directly to Apiary as soon as changes were made to the API docs, but we found this sometimes caused problems when we were working on larger releases with feature switches and such, thus we now deploy alongside the application.
What about just allowing plain URIs to be specified? The renderer can decide whether to render the URI to a link or include its contents in the documentation output. This also saves us the trouble from having to deal with different Markdown flavors.
What about just allowing plain URIs to be specified?
That should be the case. [label](URI) should be used for transclustion and also render well. The difference is we need to mark if for the transclustion tool hence :[label](URI)
Hi,
Is there any news on this?
I think schema and examples should be separated from the main body of the api documentation, so they can be used by the server and/or client for validation and unit tests. Links are ok.
Cheers,
Tamas
Hello everyone, I'm sorry for getting here late and reopening and digging old stuff. If this isn't the right place please forgive me. But, doing a research to develop blueprint mocks I've found this discussion and this is the closest that I've could find until now.
I'm struggling to split my responses by model :(. Isn't it possible to referer a model into a different folder?
I would like to have inside my blueprints folder all my requests, and, for example into a separated models folder all my entities models (MSON/MD/JSON/whatever files, that whose fit) to reuse them into different responses easily. Anyone have success doing this?