facebook/Ax

Wishlist: Tracking Issue

lena-kashtelyan opened this issue · 6 comments

lena-kashtelyan commented

Feature requests marked as 'wishlist' will be gathered here going forward, in order to:

  • improve discoverability of issues that report bugs or ask questions,
  • help us easily assess these requests when roadmapping.

Please still feel free to open new issues for feature requests (or comment them here if they are short/clear), and we will take care of adding them to this post.

Status: will likely be addressed in the short-term

  • [DONE] Obtaining best parameters (parameters on the Pareto frontier) for multi-objective optimization in Service API (#656)
  • Support for Ax viz in Colab (#306)
  • Adding a conda distribution (#608, #614)
  • [DONE] Hierarchical search spaces (#140)
  • Support specifying fixed features in Service API (#746)
  • Support preference learning in Ax (#754)
  • Early-stopping tutorial in Ax (#851)

Status: will likely be addressed in the long-term

  • TensorBoard support (#248)
  • [IN PROGRESS] Optimization algorithm, suitable for combinatorial search spaces in Ax (#477)
  • Include performance benchmarks in Ax documentation beyond the BoTorch benchmarks published in https://arxiv.org/abs/1910.06403 (#496)
  • Exact equality parameter constraints (#510)
  • Automate selection of appropriate parameters for BoTorch components in Ax based on experiment and data size (#674)
  • "random_seed" setting in Loop and Service APIs guaranteeing full reproducibility (#605)
  • Constraints on ordered choice parameters (#710)
  • Make more common functions available with just import ax: #774
  • TRBO in Ax (#474)
  • Add getters/setters for Experiment._properties and Trial._properties (#566 (comment))
  • Support for MOO for hierarchical search spaces without flattening the search space (#1042)
  • Gaussian priors for BoTorch models in Ax (#1647)
  • Slice and contour plots for choice parameters (#1577, #1624)
  • Storage of Data for trials without Dataframes (associating potentially larger data, e.g. image data, with trials, and passing that data to the model directly; #880)
  • Official release of DeterministicModel and documentation for it (#1192, #935)
  • Multi-task GP tutorial (#2086)

Status: uncertain

  • Nonlinear parameter constraints (#153)
  • Adding "projects" to group Ax experiments (#189)
  • Integration with Nevergrad (#163)
  • Returning optimal ranges rather than single values for best parameters (#320)
  • Ax on ARM / Raspberry pie (#412, blocked on official PyTorch support for ARM)
  • ALEBO support for parameter constraints (#424)
  • Selective noise inference for some observations in a field experiment (#491)
  • Off-the-shelf models that do not use Gaussian noise (#491)
  • Support sampling individual parameters from specified distributions (#702)
  • Ability to specify input uncertainties (#751)
  • PAUSED trial status (#862)
  • Discrete fidelity parameter support (#979)
  • Accept numpy values as well as Ax primitives (#996)
  • Exposing more settings for ALEBO (#953)
  • Enable intersphinx mapping (#1227)
  • PyCharm support for viz (#1729)

Suggestions for setup changes:

  • Migration of website to Docusaurus 2 (#466, status: blocked)
rpanai commented

@lena-kashtelyan I already built a conda recipe see PR whenever we fix the windows compatibility I'll let you know.

  • ❤️1
rpanai commented

@lena-kashtelyan it is now available on conda-forge https://anaconda.org/conda-forge/ax-platform

  • ❤️1
lena-kashtelyan commented

Wow, thank you so much, @rpanai!