neulab/ExplainaBoard

Issues

• `TypeError: 'type' object is not subscriptable` when attempt to import or use CLI

  #604 opened 2 years ago by ttpro1995
  0

• `custom_features` and `custom_analyses` are not deserialized

  #596 opened 2 years ago by lyuyangh
  0

• Bump mypy version to 0.990

  #592 opened 2 years ago by tetsuok
  0

• Drop Python3.8 support

  #586 opened 2 years ago by odashi
  0

• add_tasks.md is out of date

  #583 opened 2 years ago by tetsuok
  0

• Consolidate quotes for strings

  #577 opened 2 years ago by odashi
  0

• Centralized type registry

  #446 opened 2 years ago by odashi
  7

• Add system metadata class

  #575 opened 2 years ago by odashi
  3

• Test of analysis on open-domain QA

  #495 opened 2 years ago by neubig
  1

• Potential issue with spearman R bootstrapping

  #537 opened 2 years ago by neubig
  10

• Change to `~=` requirements

  #472 opened 2 years ago by neubig
  1

• Is the current applicable condition of t-test correct?

  #509 opened 2 years ago by tetsuok
  22

• Use serialization subpackage for serialization and deserialization

  #426 opened 2 years ago by tetsuok
  1

• Add "create documentation" to add_tasks.md

  #556 opened 2 years ago by neubig
  1

• list[(name, Thing)] to dict[name, Thing], and remove MetricConfig and directly store configs in Metric

  #491 opened 2 years ago by odashi
  9

• Reconsider default number of buckets

  #557 opened 2 years ago by neubig
  0

• Avoid serializing functions

  #467 opened 2 years ago by odashi
  3

• test_generate_system_analysis in integration_tests.summarization_test.SummarizationTest is too slow

  #478 opened 2 years ago by tetsuok
  12

• Two places where tokenization should be standardized

  #473 opened 2 years ago by neubig
  4

• Lint Markdown files

  #536 opened 2 years ago by tetsuok
  5

• test_generate_system_analysis in integration_tests.summarization_test.SummarizationTest is flaky

  #547 opened 2 years ago by tetsuok
  0

• Metrics with different analysis levels have the same name

  #530 opened 2 years ago by qjiang002
  5

• AnalysisLevel does not own distinctive information

  #518 opened 2 years ago by odashi
  4

• Use setup.cfg to store every setup config

  #490 opened 2 years ago by odashi
  2

• t-test confidence level is wrong

  #510 opened 2 years ago by odashi
  1

• Decompose MetricResult

  #477 opened 2 years ago by odashi
  4

• Custom Analysis Todo List

  #500 opened 2 years ago by qjiang002
  1

• aggregate_stats and calc_metric_from_aggregate don't work in some cases.

  #497 opened 2 years ago by odashi
  0

• Separate batched/non-batched operations in Metric

  #498 opened 2 years ago by odashi
  0

• Increase the minimum sample size of bootstrapping

  #496 opened 2 years ago by odashi
  0

• Use SerializableDataclass to serialize and deserialize MetricConfig objects

  #427 opened 2 years ago by tetsuok
  0

• Add documentation for uploading custom features and custom analysis

  #458 opened 2 years ago by qjiang002
  1

• Publish package to PyPI on push tags

  #485 opened 2 years ago by tetsuok
  2

• `value` should be set to `None` under empty buckets

  #482 opened 2 years ago by OscarWang114
  2

• DataLab logging is too verbose in integration tests

  #471 opened 2 years ago by neubig
  3

• We need documents to introduce how to fix different types of CI Errors

  #480 opened 2 years ago by pfliu-nlp
  0

• We need a doc on how to perform unit/integration tests

  #479 opened 2 years ago by pfliu-nlp
  0

• How to name metrics when registering them

  #455 opened 2 years ago by pfliu-nlp
  8

• Concrete data type

  #466 opened 2 years ago by odashi
  2

• Document how custom dataset should be formatted

  #465 opened 2 years ago by pfliu-nlp
  0

• Document how to add customized features through the JSON config file

  #464 opened 2 years ago by pfliu-nlp
  0

• Data hosting for APE task is unstable

  #463 opened 2 years ago by neubig
  3

• Saving `is_training_set_available` in `sys_info` during `get_overall_statistics()`

  #453 opened 2 years ago by OscarWang114
  2

• Errors around Datalabs

  #449 opened 2 years ago by odashi
  4

• Bug: the calculation of metric stat for generation tasks

  #441 opened 2 years ago by pfliu-nlp
  2

• Bug: serialization failed for `EaaSMetricConfig`

  #444 opened 2 years ago by pfliu-nlp
  5

• Support Text-to-SQL task

  #447 opened 2 years ago by pfliu-nlp
  0

• ModuleNotFoundError: No module named 'explainaboard.metrics.auxiliary'

  #439 opened 2 years ago by noelchen90
  3

• `bucket_interval` is type-ambiguous

  #431 opened 2 years ago by neubig
  2

• Loader could have streaming option

  #438 opened 2 years ago by neubig
  0