How to use interactive tables (qgrid, itables, beakerx, ...)

[jhettler]

Hi everyone,
I would like to ask if someone uses any of interactive (filtering, sorting, paging, etc.) table visualization of pandas dataframe in knowledge repo. We use amazing qgrid extension https://github.com/quantopian/qgrid in jupyterlab, but I would like to get it or something similar working in knowledge repo. Is there a way how to do it or someone who could help with development? Ideas, hints, etc. welcome!

Thanks and regards,
Jakub

[bulam]

Hi Jakub,
Sorry about the delayed response. The Knowledge Repo supports HTML widgets. For example rendering this Jupyter notebook with knowledge repo add will preserve the interactive plotly chart in the final post. Internally, we also have an RStudio addin that converts interactive tables into HTML widgets for the knowledge repo, so it should definitely be possible to create something similar for Jupyter.

[jpzhangvincent]

Hi Jakub,
Sorry about the delayed response. The Knowledge Repo supports HTML widgets. For example rendering this Jupyter notebook with knowledge repo add will preserve the interactive plotly chart in the final post. Internally, we also have an RStudio addin that converts interactive tables into HTML widgets for the knowledge repo, so it should definitely be possible to create something similar for Jupyter.

Can you share the tips how to embed the HTML widgets in a post(generated from a Rmarkdown) for the knowledge repo ? Thanks!

[bulam]

@jpzhangvincent At a high level, you can put the HTML widgets in an iframe and put the iframes in your final rendered HTML post. Internally, we have used widgetframe to do this. Eventually the hope is to open source the RStudio addin as well to accompany the Knowledge Repo.

Here's some example code for doing this in R.

  | kp_render_internal <- function(input_file, df_print = "paged", force_s3_upload = FALSE) {
  | update_yaml_front_matter(input_file, "always_allow_html", "yes")
  | md_output_format <- rmarkdown::md_document(
  | preserve_yaml = TRUE,
  | df_print = df_print
  | )
  | # preserve html output instead of screen shot
  | md_output_format$knitr$opts_chunk$screenshot.force <- FALSE # nolint
  | # print SQL table properly
  | md_output_format$knitr$opts_knit$sql.print <- . %>% knitr::knit_print() %>% as.character() # nolint
  | # frame htmlwidgets in iframes
  | md_output_format$knitr$knit_hooks$evaluate <- function(code, envir, output_handler, ...) {
  | existing_value_renderer <- output_handler$value
  | output_handler$value <- function(x, ...) {
  | # embed html widget in iFrame
  | if (inherits(x, "htmlwidget")) {
  | x <- widgetframe::frameWidget(x)
  | }
  | existing_value_renderer(x, ...)
  | }
  | evaluate::evaluate(code, envir, output_handler = output_handler, ...)
  | }
  | # always append new line after plot because knowledge_repo can't grep properly
  | md_output_format$knitr$knit_hooks$plot <- function(x, ...) {
  | paste0(knitr::hook_plot_md(x, ...), "\n")
  | }
  |  
  | # set up S3 paths for dependencies upload
  | front_matter <- get_yaml_front_matter(readLines(input_file))
  | widgets_dir <- file.path(
  | "widgets/",
  | front_matter$path,
  | as.integer(Sys.time())
  | )
  | s3_path <- file.path("directory", widgets_dir)
  | s3_url <- glue("directory/{widgets_dir}")
  |  
  | # run this in a separate R process so we can kill it if we want
  | md_output <- callr::r(
  | function(input_file, output_format, knit_print) {
  | # load widgetframe for it to register its knit_print function
  | requireNamespace("widgetframe", quietly = TRUE)
  | # sneakyly overwrite knit_print.widgetframe with ours
  | registerS3method("knit_print", "widgetframe", knit_print, asNamespace("knitr"))
  |  
  | # redirect logging to stderr, which unfortunately doesn't show in callr
  | options("knitr.in.progress" = TRUE)
  | Rbnb::initialize_rbnb_loggers()
  |  
  | # print more columns
  | if (is.null(getOption("cols.print"))) {
  | options("cols.print" = 20)
  | }
  |  
  | # force kable to print html
  | options(knitr.table.format = "html")
  |  
  | # stop rmarkdown and pandoc from ruining our masterpiece
  | rmarkdown::render(
  | input_file,
  | output_format,
  | clean = FALSE,
  | run_pandoc = FALSE
  | )
  | },
  | list(
  | input_file = input_file,
  | output_format = md_output_format,
  | knit_print = knit_print_widgetframe(s3_path, s3_url, force_s3_upload)
  | ),
  | show = TRUE
  | )
  |  
  | input_dir <- dirname(tools::file_path_as_absolute(input_file))
  |  
  | # drop intermediate files other than our markdown
  | attr(md_output, "intermediates") %>%
  | purrr::discard(~ .x == md_output) %>%
  | purrr::map(~ file.path(input_dir, .x)) %>%
  | purrr::map(file.remove)
  |  
  | # inject html dependencies
  | knit_meta <- attr(md_output, "knit_meta")
  | md_output <- file.path(input_dir, md_output)
  | if (!is.null(knit_meta)) {
  | log_info("Processing HTML dependencies...")
  | meta <- knit_meta %>%
  | purrr::keep(~ class(.x) == "html_dependency") %>%
  | # attached paged table HTML dependencies
  | c(list(rmarkdown::html_dependency_pagedtable())) %>%
  | # keep unique dependencies only (first come first saved)
  | unique() %>%
  | # upload to s3 if not already
  | kp_migrate_dependencies(force_s3_upload) %>%
  | # render using hrefs which point to S3
  | htmltools::renderDependencies(srcType = "href") %>%
  | # unload require js which cause conflicts with htmlwidget
  | {htmltools::tags$script("define = null;") %++% .} %>% # nolint
  | # special preserve tag to avoid further preprocessing (although knowledge repo doesn't seem to honor this for now)
  | htmltools::htmlPreserve()
  |  
  | # write dependencies just after the front matter
  | output_lines <- readLines(md_output, warn = FALSE)
  | front_matter_end <- get_yaml_front_matter_line_number(output_lines)
  | append(output_lines, meta, front_matter_end) %>%
  | writeLines(md_output, useBytes = TRUE)
  | }
  |  
  | md_output
  | }