Chalarangelo/jsiqle

Scope sorting

Chalarangelo opened this issue · 2 comments

Chalarangelo commented

Sorting scopes based on a sorter function would be pretty nice. In fact, it could be a a major feature, especially if we can couple it with scope caching to optimize performance for data retrieval.

Chalarangelo commented

Update

Scope caching is in no way feasible at this time. However, scope sorting is pretty reasonable to implement and it will net a major performance optimization, as follows:

Current implementation

Supposing that one would query a scope and then sort it, the following steps must be taken:

  1. Call RecordSet.prototype.where().
  2. Call RecordSet.prototype.filter().
  3. Call RecordSet.prototype.entries().
  4. Spread the result to an array.
  5. Call Array.prototype.reduce().
  6. Call RecordSet() constructor to create a new RecordSet.
  7. Call callbackFn for each Record and RecordSet.prototype.set() if the result of the function is truthy.
  8. Call RecordSet.prototype.freeze().
  9. Call RecordSet.prototype.sort().
  10. Call RecordSet.prototype.entries().
  11. Spread the result to an array.
  12. Call comparatorFn for each pair of Record objects until the result is sorted.
  13. Call RecordSet() constructor to create a new RecordSet.
  14. Call RecordSet.prototype.freeze().

For obvious reason, this is not extremely performant.

Proposed implementation

Supposing that a scope can receive a third argument for the comparatorFn function either via an optional argument or as part of an object defining the scope, the code for calling the scope could be refactored to an internal $scopedWhere function as follows:

const $scopedWhere = Symbol.for('scopedWhere');

class RecordSet {
  [$scopedWhere](matcherFn, comparatorFn) {
    let matches = [];
    for (const entry of this.entries())
      if (matcherFn(value, key, this)) matches.push(entry);
    if (comparatorFn)
      matches.sort(([key1, value1], [key2, value2]) => comparatorFn(value1, value2, key1, key2));
    return new RecordSet({ iterable: matches, copyScopesFrom: this }).freeze();
  }
}

The steps are as follows:

  1. Call RecordSet.prototype[$scopedWhere]().
  2. Create an empty array.
  3. Call RecordSet.prototype.entries().
  4. Use a for...of loop over the iterator.
  5. Call matcherFn for each Record and Array.prototype.push()` if the result of the function is truthy.
  6. If comparatorFn is truthy, call Array.prototype.sort().
  7. Call comparatorFn for each pair of Record objects until the result is sorted.
  8. Call RecordSet() constructor to create a new RecordSet.
  9. Call RecordSet.prototype.freeze().

As shown above, this reduces the steps nearly in half and ensures that no unnecessary conversions are performed. It also optimizes memory usage by avoiding to spread the result of RecordSet.prototype.entries() to an array and instead uses a loop to iterate over the data, which can also boost performance.

Potential improvements

Generally speaking, Array.prototype.push() is considered pretty slow. Alternatively, spreading and filtering can be applied, if more performant.

Chalarangelo commented

Array.prototype.push() seems to perform better than Array.prototype.filter() (link):

image