Create a simple way to run this on local doc files
jenweber opened this issue · 3 comments
I was able to get it to run on local files after bypassing a lot of things by commenting them out. It would be good to have it as an options flag. I don't think it would be too hard, since there's already an option flag for individual versions. I'm wondering if I just named things perfectly and put them in place if it would work as-is.
My working approach from start to finish
- Clone and open Ember.js core
- Generate YUIdocs by running
yuidoc .
which creates adocs/
directory of files. The important file isdata.json
, which contains all the documentation and internal relationships - Clone and open ember-jsonapi-docs
- From the command line, export some AWS keys as described in the readme (I found that some scoped IAM keys I had were insufficient, something something permissions)
- In the same console tab, run
yarn start -- --project ember --version 2.11.0
or similar. Just pick a version. This will download some stuff from s3, generate a big filetree under thetmp/
directory, and process the specified version to turn it into JSONAPI. You'll use that tree as the skeleton for operating on your local yuidoc files. - Grab that
data.json
file that was generated in step 2. Paste it intotmp/s3-docs/v2.11.0
- Rename the
v2.11.0
directory to a bogus version that doesn't exist, like2.11.18
. Whatever you rename it to, change the version in data.json to match:
{
"project": {
"name": "The Ember API",
"description": "The Ember API: a framework for building ambitious web applications",
"url": "https://emberjs.com/",
"version": "2.11.18" <<<<<<<<< Change version
},
- Delete
tmp/v2.11.0/ember-docs.json
. Renamedata.json
toember-docs.json
- Run the jsonapi generator again, but specify the made up version
yarn start -- --project ember --version 2.11.18
- It will look like things are downloading. Your json file from yuidocs will be transformed into jsonapi. You should see the results in
tmp/rev-index/ember-2.11.18.json
andtmp/json-docs/ember/2.11.8
- Copy
json-docs
andrev-index
into thepublic
directory of ember-api-docs. - The ember-api-docs will look at two files to see which versions of the ember documentation are available,
rev-index/ember-data.json
andrev-index/ember.json
. However you only generated one version's worth of jsonapi docs. So, to convince the app that your 2.11.18 version is the only one that exists, remove mentions of all other versions from those two files,ember-data.json
andember.json
. The result should look similar to this:
{
"data": {
"id": "ember",
"type": "project",
"attributes": {
"github-url": "https://github.com/emberjs/ember.js"
},
"relationships": {
"project-versions": {
"data": [{
"id": "ember-2.11.18",
"type": "project-version"
}]
}
}
},
"meta": {
"availableVersions": ["2.11.18"]
}
}
- Change the adapter to point to itself (your files in
public
) instead of an API endpoint by changing the host inapp/adapters/application.js
tohost: 'http://localhost:4200'
ember serve
and hopefully you'll see the app up and running- If links are broken for the modules, you can temporarily comment out these lines in
app/routes/project-version/modules/module.js
to get them working, unless your modules names include slashes. The app expects that all modules will be named likeember-something
so simple
module names likecomponent
will fail in the routing without this fix
getModel(typeName, params, transition) {
const projectID = transition.params['project-version'].project;
const compactVersion = transition.params['project-version'].project_version;
const projectVersion = this.get('metaStore').getFullVersion(projectID, compactVersion);
let klass = params[typeName];
// WHAT DOES THIS DO????
// if (!~klass.indexOf(projectID)) {
// klass = `${projectID}-${klass}`;
// }
return this.find(typeName, `${projectID}-${projectVersion}-${klass}`);
},
Debug log, because why not. Don't follow these:
This story has a happy ending. Here are my debugging steps, which may aid in coming up with a streamlined set of instructions:
Open ember.js and run yuidoc .
which generates a folder called docs
which contains a bunch of html files, but the important one is data.json
In ember-jsonapi-docs, comment out a few things:
// downloadExistingDocsToLocal()
// .then(() => fetchYuiDocs(projects, specificDocsVersion))
// .then(() =>
readDocs(projects, specificDocsVersion)
...
// .then(uploadToS3)
Based on errors thrown, it seems to expect a filetree that looks like this, so I made that structure:
tmp/json-docs/ember/projects
tmp/json-docs/ember-data/projects
tmp/rev-index/
I dropped data.json into tmp, and renamed the version stated in data.json to 2.16.0 because I am suspicious of the alpha.
Errors indicate a missing file at tmp/json-docs/ember/projects/ember.json
so I rename data.json to ember.json and put it at that location. Subsequent error of "cannot read relationships of undefined" indicates that a different file is needed. Sub in a file I got from Todd, trimmed down to only have 2.16.0. Name it ember.json. My YUIdoc output gets renamed back to data.json
Contents of ember.json
{
"data": {
"id": "ember",
"type": "project",
"attributes": {
"github-url": "https://github.com/emberjs/ember.js"
},
"relationships": {
"project-versions": {
"data": [{
"id": "ember-2.16.0",
"type": "project-version"
}]
}
}
},
"meta": {
"availableVersions": ["2.16.0"]
}
}
After yarn start
, nothing happens except:
Downloading docs for ember & ember-data
Reading project files
Done in 1.21s.
It seems that download & fetch are not skippable. The app needs all those byproducts in order to function.
So I added some AWS keys via export in the command line and put the download & fetch back, to see if I can just download something else and run that. (Run into AWS access problems, try a different key, etc)
Run yarn start -- --project ember --version 2.11.0
Lots of files get created. Based on the filetree, see that I should have put my docs into a folder called tmp/s3-docs/v2.16.0/ember-docs.json
, maybe??? Decide to just roll with 2.11.0 to see if it works.
Based on past work, I know that CORS is an issue, so I run this from just inside tmp/
http-server -p 8000 --cors
Open up ember-api-docs. Change the adapter host in app/adapters/application.js
to host: 'http://localhost:8000'
Start the ember-api-docs app with ember s
and open it in the browser. Error displayed in the DOM:
FetchError: invalid json response body at http://localhost:8000/rev-index/ember.json reason: Unexpected end of JSON input
Back end server confirms a failed GET for /rev-index/ember.json
but also indicates maybe some sort of passed preflight check??? The file is there in the tree...
[Sat Sep 02 2017 19:49:01 GMT-0400 (EDT)] "GET /rev-index/ember.json" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
[Sat Sep 02 2017 19:49:01 GMT-0400 (EDT)] "GET /rev-index/ember.json" Error (404): "Not found"
Check the network requests.
FetchError: invalid json response body at http://localhost:8000/rev-index/ember.json
Become suspicious of json. Remove trailing whitespace. No change in errors.
Become suspicious of the http server. Dump the files into public. Set the adapter host to host: '/'
. Get this error:
TypeError: Only absolute URLs are supported
Delete host altogether. Same error. Try setting host to http://locahost:4200
. See a new error:
FetchError: invalid json response body at http://localhost:4200/rev-index/ember-2.15.0.json reason: Unexpected token < in JSON at position 0
Follow that path in the app and see that it doesn't exist, because I ran the doc generator for 2.11.0. Try hard coding that into the adapter:
currentProject: '2.11.0',
currentProjectVersion: '2.11.0',
Same error. Search the codebase for 2.15. Remove everything but 2.11 from the availableVersions array in rev-index/ember.json
. Get a new error:
FetchError: invalid json response body at http://localhost:4200/rev-index/ember-undefined.json
This file doesn't exist. Go digging through Todd's files. It's not there either. Realize that there probably shouldn't be a file named "ember-undefined" and that's probably a bad filename parsing error. Go back to ember.json and ember-data.json and remove mentions of anything but 2.11. Regret not running the build of everything. Restart server just in case. Consider throwing my laptop out the window while I wait for it to build.
FINALLY it works.
Generate YUIdocs with modules changes, drop it into the 2.11.0 files. Get this error:
ember-2.11.0 has already been indexed in json-docs
Give it a bogus version name 2.11.18 as the folder name and version stated in the json. Change the ember.json and ember-data.json in ember-api-docs to reflect this. Drop the file into the place that 2.11 was. Restart ember server. Hard code 2.11.18 into the adapter. Hit a Fastboot error....
TypeError: Cannot read property 'split' of undefined
More than 2.11 json was needed. Go back to YUIdocs. Run generate with the bogus version name, and all of index.js intact in jsonapi-docs (except S3 upload).
yarn start -- --project ember --version 2.11.18
It will look like it is downloading things. Copy ALL the files over into public. Don't forget to limit the scope of ember.json and ember-data.json.
Great instructions. Followed and things working nicely. I think we can put this in a single, local script.
- copy over docs build output (using the generated version in the doc file as the tmp/s3-docs version directory)
- run the ember-jsonapi-docs build on the specified version
- copy output into ember-api-docs repo (assuming or supplying the repo location)
Additionally, we should just add an env variable, like USE_LOCAL_DATA that will set API_HOST to http://localhost:4200
would be cool to have a developer just do something like (assuming all repos cloned into the same parent directory):
cd ember.js
npm run docs
cd ../ember-jsonapi-docs
npm run generate-local-docs
cd ../ember-api-docs/
USE_LOCAL_DATA=true ember s
I have a script now that does some of this, but needs to be cleaned up. stay tuned...