Separate data fetching from console output

[greg-js]

I was at a Slack event the other day and was thinking that the general idea behind this package could be used for a great Slack bot. But to actually use it as such, there needs to be a way to require and execute the script so that it doesn't log to the console, but instead returns the information.

That got me thinking, maybe it makes sense to separate the data fetch entirely from the console output.

ie: we scrape the page for all the relevant, unformatted information and build an object, and then we return that (promise of an) object directly for people to do as they please, or we pass it to a function that then constructs the output for the console. The default behavior when installed globally would remain logging to the console, but as a locally installed module it could be useful in other projects, such as a Slack bot.

It would of course require a good deal of refactoring, but do you think this is worth doing? Either way, I'm free this evening so unless I get tempted by bourbon, I'll see if I can make a quick prototype ;-)

[rafaelrinaldi]

@greg-js Alright Greg, that makes sense. You mean, to separate the API from the CLI right? I agree that might be useful. I have to think on how to approach that though, I think in order to not break anything and to keep it simple we can just create mdn-api and then just change mdn to consume data from there instead. What do you think?

[greg-js]

Yep, that's exactly what I meant. I made a first pass at it earlier, but splitting it out into separate projects is a better idea.

FWIW, this is the minimum object necessary to format a result for the console in exactly the same way it's being done now - example for array.map:

{ url: 'https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/array/map',
  title: 'Array.prototype.map()',
  description: 'The map() method creates a new array with the results of calling a provided function on every element in this array.',
  method: 'map()',
  usage: 'arr.map(callback[, thisArg])',
  api: 
   [ { term: 'callback',
       definition: 'Function that produces an element of the new Array, taking three arguments:',
       level: 0 },
     { term: 'currentValue',
       definition: 'The current element being processed in the array.',
       level: 1 },
     { term: 'index',
       definition: 'The index of the current element being processed in the array.',
       level: 1 },
     { term: 'array',
       definition: 'The array map was called upon.',
       level: 1 },
     { term: 'thisArg',
       definition: 'Optional. Value to use as this when executing callback. Default value is the Window object',
       level: 0 } 
  ] 
}

(the levels are only for indenting callback arguments as per the other PR)

Of course, maybe some additional properties would be nice, like examples and the polyfill if there is one.

[greg-js]

I just took another look at that first pass I made at this back in April, finished it and decided to push it to a new branch in case you want to take a look at it - and since I realized it now satisfies my original intention, which was to make a slack bot out of this.

It does start from the other pull request I made to fix the API indentation, but apart from that, all I did here was separate the parsing of HTML from the MDN page (to ./parse.js) from the console formatting (./format.js), and added a console option which defaults to true.

The result is that if one were to install this package locally with npm and require it as in const mdn = require('mdn'), then mdn({keyword: 'array.map', language: 'js', shouldOpen: false, locale: 'en-us', toConsole: false}).then(data => console.log(JSON.stringify(data, null, 2))) will return this object:

{
  "url": "https://developer.mozilla.org/en-us/docs/Web/JavaScript/Reference/Global_Objects/array/map",
  "title": "Array.prototype.map()",
  "description": "The map() method creates a new array with the results of calling a provided function on every element in this array.",
  "method": "map()",
  "usage": "arr.map(callback[, thisArg])",
  "api": [
    {
      "term": "callback",
      "definition": "Function that produces an element of the new Array, taking three arguments:",
      "level": 0
    },
    {
      "term": "currentValue",
      "definition": "The current element being processed in the array.",
      "level": 1
    },
    {
      "term": "index",
      "definition": "The index of the current element being processed in the array.",
      "level": 1
    },
    {
      "term": "array",
      "definition": "The array map was called upon.",
      "level": 1
    },
    {
      "term": "thisArg",
      "definition": "Optional. Value to use as this when executing callback. Default value is the Window object",
      "level": 0
    }
  ]
}

When console is true, the above object is then used as a base for the console formatting in much the same way it was done before, and it seems to work out quite nicely.

It's backwards compatible (nothing changes if the package is installed globally or run from the command line) and creates potential for consuming the data in other ways. Wouldn't be hard to separate it into two separate projects either if you think that would be cleaner. Any thoughts?

[greg-js]

I'm closing this for now as it complicates things considerably and I don't need the Slack feature anymore, which was pretty much the only use case for it :-) I'll keep the branch around just in case one of us ever feels like taking it up again though.