/ENCORE

ENCORE: Enhancing Computational Reproducibility

Primary LanguagePythonOtherNOASSERTION

ENCORE. Enhancing Computational Reproducibility

[TOC]

Note

  • This is the README.md file of the ENCORE template and is provided to give some background information about the project organization and to provide pointers to further information.

    • Version: V.4.x (main branch). Use this development template for new projects)

    • Previous ENCORE versions are found as separate branches starting from V4.1.1

General

ENCORE

Further information about ENCORE is also found in the ENCORE wiki. To facilitate further discussion about the develop of ENCORE we have set up Discussions.

The standardized File System Structure (sFSS) / Project Compendium

To organize a computational project into a project compendium, ENCORE has defined a sFSS which is a predefined but flexible directory structure to organize conceptual information, data, code, documentation, and (intermediate) results (e.g., tables, figures, text files). Each directory of the sFSS contains one or more predefined files to provide explanation and guidance for documentation. The sFSS is also referred to as a Project Compendium.

Markdown

The Markdown files (*.md) in the sFSS can be edited with any text editor but to show the markup one should use a Markdown viewer such as Typora (www.typora.io; Windows, Mac) or Notepad++ (Windows; install the MarkdownViewer plugin).

Project Team and Compendium Recipient

The (leader of the) Project Team is responsible for the organization and documentation of the project. The Step-by-Step ENCORE Guide and the explanation/instructions in the various README and other files are intended for the Project Team.

The Compendium Recipient is, for example, a direct colleague, a peer, or a reviewer who wants to inspect and/or reproduce the results of a project. Or it can be an archive such as Zenodo.

INFORMATION FOR THE COMPENDIUM RECIPIENT

Start to explore the project by opening Navigate.html in your browser.

Content of sFSS root directory

  • README.md. This file.
  • 2_CITATION.md. How to cite ENCORE and the FSS Navigator.
  • 3_LICENSE.md. License associated with the ENCORE template

Directories

  • .navigate. System directory used by the Compendium Navigator. Do not remove.
  • Data. Data used in the project. Note that data can also be located in the Processing (sub)directories depending on your needs.
  • Manuscript. (Final) manuscript of project
  • Processing. Software, documentation, and results. Also contains some general information about software environments.
  • ProjectDocumentation. General information about project.
  • Sharing. Restructured and/or selected FSS files shared with third-parties.

INFORMATION FOR THE PROJECT TEAM

ENCORE AUTOMATION

The GitHub repository ENCORE_AUTOMATION provides scripts to automate specific ENCORE tasks. Currently, the automatic creation of a local ENCORE project and and project repository on GitHub.com

ENCORE SUPPLEMENT

The GitHub repository ENCORE_SUPPLEMENT provides supplementary information to ENCORE such as documentation, manuscripts, and figures.

Sharing with a Compendium Recipient

  • Several files in the sFSS are provided in different formats (e.g., text, Markdown, LaTex, Microsoft Word, HTML). The Project Team has to remove the formats that will not be used. This prevents that these redundant files cause confusion when the project compendium is archived or shared with a Compendium Recipient.
  • The Step-by-Step-ENCORE-Guide can be removed since this document is only relevant for a Project Team setting up and using ENCORE.
  • Test_Navigate_Module.py can be removed.

Notes

  • Compendium Recipients can start to inspect a project: start by opening Navigate.html in a web browser.

Content of sFSS root directory

The following files need to be updated by the project owner

  • 0_PROJECT.md. Short description of project, contact person, and project team

  • 0_GETTINGSTARTED.txt. Template document (plain text format). Copy this file to your favourite editor to add content.

  • Examples:

    • 0_GETTINGSTARTED.docx. Template document (Microsoft Word format to show how to include links). The DOCX file can be saved as HTML (make sure you use utf-8 encoding).

    • 0_GETTINGSTARTED.tex. Template document (LaTex format to show how to include links). The LaTex file can be converted to HTML with Pandoc.

    • 0_GETTINGSTARTED.html. Example of exported HTML file used by the FSS navigator.

User documentation

  • 1_Step-by-Step-ENCORE-Guide.{pdf, docx}. User guide to use ENCORE (the File System Structure (FSS) and setting up a corresponding GitHub repository). Additional documentation is provided by the GitHub repository ENCORE_SUPPLEMENT

General

  • README.md. This file.

  • 2_CITATION.md. How to cite ENCORE and the FSS Navigator.

  • 3_LICENSE.md. License associated with the ENCORE template

  • .FSSignore. Currently not used.

Compendium (sFSS) Navigator

The Compendium Navigator is a Python program that generates Navigate.html when executed. This html file can be opened in a web-browser and guides the user to the most important parts of a projects.

  • The standalone program and source code are available. The Python code is part of this template. Compiled binaries are found on the ENCORE GitHub repository
  • Updates (e.g., bug fixes) of the Python code and executables are available from the Navigator repository Releases . Compatibility, with specific ENCORE templates is documented with each Release.
  • Navigator versions and Compatibility with ENCORE template versions is also available from the ENCORE wiki
  • If the sFSS is changed then Navigate.html can be updated by executing the Navigate.py script (if Python is installed on your computer), or by downloading and executing one of the executables. Open Navigate.html after running the program. The Navigator can be configured from the Navigation.conf text file. Execution takes a couple of seconds.
  • In principle, the Python script should run on any hardware (Windows, Mac, Unix). The compiled programs were compiled for specific operating systems (see below). The Navigator has been tested with Windows 10 and Windows 11 and with macOS 13.3.1 (Ventura), Apple M1 and macOS 10.13.6 (High Sierra), Intel Core i5.
  • For instructions to install Python and an environment to run Navigate.py see Compendium-Navigator repository.
  • There are no software dependencies. The Python code only uses standard Python libraries (see content of Navigate.py).
  • No specific installation is required.

Compendium Navigator files:

  • Navigate.html. Open in your browser to navigate the standardized file system.

  • Navigate.py. Standalone Python 3 script to generate Navigate.html to navigate the sFSS. Can be run from the command line (Navigate.py -h).

  • Navigate_U.sh. Shell script to run Navigate on Unix/Linux systems. Change the first line (#!/usr/bin/Python) if necessary. Make executable using chmod +x

  • There are also executables available for Windows and Mac OS. These are available from the GitHub release and also from Zenodo (DOI: https://doi.org/10.5281/zenodo.7985655):

    • Navigate_W.exe. Windows executable if you don't have Python installed (Navigate.exe -h).
    • Navigate_M. MacOS executable (macOS 13.3.1 (Ventura), Apple M1)
    • Navigate_MacIntel. MacOS executable (macOS 10.13.6 (High Sierra), Intel Core i5 )

  • Test_Navigate_Module.py. Python script to show how to use Navigate.py as module in other Python scripts. This may help to keep Navigate.html up-to-date without manually executing Navigate.py.

  • Navigation.conf. Configuration file for the FSS Navigator.

Directories

  • .navigate. System directory used by Navigate. Do not remove.
  • Data. Data used in the project. Note that data can also be located in the Processing (sub)directories depending on your needs.
  • Manuscript. (Final) manuscript of project
  • Processing. Software, documentation, and results. Also contains some general information about software environments.
  • ProjectDocumentation. General information about project.
  • Sharing. Restructured and/or selected FSS files shared with third-parties.