[Request] Automatic package retrieval

[DarkGhostHunter]

I see that package updating is done through the update-database.ts.

I think that instead of adding data to the database almost manually, it should be the other way: use a simple file with the package data (type and where to find it), and let the script do the heavy lifting of gathering the rest of the information, like the description or keywords.

The main idea would be to have a separate JSON with the packages names and some overrides, and leave the database as a "cache". For example, the hypothetical base.json could be:

{
  "vendor/package-php": {
     // "type": "composer", // Always assume "composer"
  },
  "vendor/package-js": {
     "type": "npm",
     "description": "Something cool!"
  },
  // ...
}

From there, the script would be resumed as this:

1. For each package entry in base.json
   2.1. If a package is not in the database, retrieve the data from "source" (NPM/Packagist/GitHub).
   2.2. If a package is in the database:
   2.2.1. Check if has changed (If-Modified-Since: {last_execution}|time:$ < {last_execition})^1 and update. Or if there is a newer version too.
   2.3. Check if it contains data to override and apply it (like the description).
2. Save the time of execution.

[HassanZahirnia]

Hey @DarkGhostHunter ! Thanks for your feedback! 😊

My initial idea to handle the database was similar to what you suggested. But for for the sake of simplicity, easier maintenance and having typed data, I decided to use a simple variable export, at least for the start.

Things regarding the database structure, and the update script are not yet "finalized".

Since the project was launched just few days ago, I've been constantly tinkering with the shape of the data, and have been changing the update script and adding more features. So didn't want to make hard decisions about this kind of things right of the bat.

I will make a decision a little bit down the road, and in the meantime I will be open to any more suggestions 🌸