/Calibre-Web-Automated

Calibre-Web but automated and with Calibre features! Fully automate and simplify your eBook set up!

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

Calibre-Web Automated (formerly Calibre-Web Automator)

Calibre-Web Automated

Making Calibre-Web your dream, all-in-one self-hosted digital library solution.

Docker Pulls

Quick Access

Why does it exist? ๐Ÿ”“

Calibre, while a fantastic tool for its age, has several problems when containerised, including its reliance on a KasmVNC server instance for the UI, which is near impossible to use on mobile and is relatively resource-heavy if you're running a small, lower power server like I am.

For many, Calibre-Web has really swooped in to save the day, offering an alternative to a containerised Calibre instance that's quick, easy to set up, resource-light and with a much more modern UI to boot.

However, when compared to full-fat Calibre, it unfortunately lacks a few core features leading many to run both services in parallel, each serving to fill in where the other lacks, resulting in an often clunky, imperfect solution.

๐Ÿšจ HUGE NEW UPDATE ๐Ÿšจ - Version 2.0.0 - 28.08.2024

GitHub Release GitHub commits since latest release

๐Ÿšจ Please Update to the latest DockerHub image ASAP to avoid stability & import/ ingest issues that have been corrected from previous versions

View The changelog for more info. New features have been released

What Does it do? ๐ŸŽฏ

After discovering that using the DOCKER_MODS universal-calibre environment variable, you could gain access to Calibre's fantastic eBook conversion tools, both in the Web UI and in the container's CLI, I set about designing a similar solution that could really make the most of all of the tools available to try and fill in the gaps in functionality I was facing with Calibre-Web so that I could finally get rid of my bulky Calibre instance for good.

Features:

  • Automatic Imports - of .epub files into your Calibre-Web library โœจ

  • Automatic Conversion ๐Ÿ”ƒ - of newly downloaded books in 28 other formats into .epub for optimal compatibility with the widest number of eReaders, library homogeneity, and seamless functionality with Calibre-Web's excellent Send-to-Kindle Function.

  • A Weighted Conversion Algorithm: โš–๏ธ

    • Using the information provided in the Calibre eBook-converter documentation on which formats convert best into epubs, CWA is able to determine from downloads containing multiple eBook formats, which format will convert most optimally, ignoring the other formats to ensure the best possible quality and no duplicate imports
  • 28 Supported file types for conversion: ๐Ÿคฏ

    • .azw, .azw3, .azw4, .mobi, .cbz, .cbr, .cb7, .cbc, .chm, .djvu, .docx, .epub, .fb2, .fbz, .html, .htmlz, .lit, .lrf, .odt, .pdf, .prc, .pdb, .pml, .rb, .rtf, .snb, .tcr, .txtz

Cover Enforcement CWA

  • Automatic Enforcement of Changes made to Covers & Metadata through the Calibre-Web UI! ๐Ÿ‘€๐Ÿ“”

    • In stock Calibre-Web, any changes made to a book's Cover and/or Metadata are only applied to how the book appears in the Calibre-Web UI, changing nothing in the ebook's files like you would expect
    • This results in a frustrating situation for many CW users who utilise CW's Send-To-Kindle function, and are disappointed to find that the High-Quality Covers they picked out and carefully chosen Metadata they sourced are completely absent on all their other devices! UGH!
    • CWA's Automatic Cover & Metadata Enforcement Feature makes it so that WHATEVER you changes you make to YOUR books, are made to the books themselves, as well as in the Web UI, making what you see, what you get.
  • One Step Full Library Conversion ๐Ÿ”‚ - Any format -> .epub

    • Calibre-Web Automated has always been designed with .epub libraries in mind due to many factors, chief among which being the fact they are Compatible with the Widest Range of Devices, Ubiquitous as well as being Easy to Manage and Work with
    • Previously this meant that anyone with non-epub ebooks in their existing Calibre Libraries was unable to take advantage of all of Calibre-Web Automated's features reliably
    • So new to Version 1.2.0 is the ability for those users to quickly and easily convert their existing eBook Libraries, no matter the size, to .epub Version 3 format using a one-step CLI Command from within the CWA Container
    • This utility gives the user the option to either keep a copy of the original of all converted files in /config/processed_books or to trust the process and have CWA simply convert and replace those files (not recommended)
    • Full usage details can be found here
  • Simple CLI Tools for manual fixes, conversions, enforcements, history viewing ect. ๐Ÿ‘จโ€๐Ÿ’ป

    • Built-in command-line tools now also exist for:
      • Viewing the Edit History of your Library files (detailed above)
      • Listing all of the books currently in your Library with their current Book IDs
      • Manually enforcing the covers & metadata for ALL BOOKS in your library using the cover-enforcer -all command from within the container (RECOMMENDED WITH FIRST TIME USE)
      • Manually Enforcing the Covers & Metadata for any individual books by using the following command
      • cover-enforcer --dir <path-to-folder-containing-the-books-epub-here>
    • Full usage and documentation for all new CLI Commands can be found here CWA Database
  • Change Tracking Database ๐Ÿ“Š - In combination with the New Cover & Metadata Enforcement Features, a database now exists to keep track of any and all enforcements, both for peace of mind and to make the checking of any bugs or weird behaviour easier, but also to make the data available for statistical analysis or whatever else someone might want to use the data for

    • Full documentation can be found below here
  • NEW FEATURE - Library Auto-Detect ๐Ÿ“š๐Ÿ•ต๏ธ

    • Made to MASSIVELY simplify the setup process for both new and existing users alike
    • New Users without existing Libraries: ๐Ÿ†•
      • New users without existing Calibre Libraries no longer need to copy and paste metadata.db files and point to their location in the Web UI, CWA will now automatically detect the lack of Library in your given bind and automatically create a new one for you! It will even automatically register it with the Web UI so you can really hit the ground running
    • New or Existing Users with Existing Libraries:
      • Simply bind a directory containing your Calibre Library (search is done recursively so it doesn't matter how deep in the directory it is) and CWA will now automatically find it and mount it to the Web UI
      • Should you bind a directory with more than 1 Calibre Library in it, CWA will intelligently compare the disk sizes of all discovered libraries and mount the largest one
        • CWA supports only one library per instance though support for multiple libraries is being investigated for future releases
        • In the meantime, users with multiple libraries who don't want to consolidate them are advised to run multiple, parallel instances
  • NEW FEATURE - Easy Dark/ Light Mode Switching โ˜€๏ธ๐ŸŒ™

    • Switch between Light & Dark Modes in just one click from anywhere in the Web UI!
    • Simply click/tap the ๐Ÿ•ถ๏ธ icon on the Web UI's navbar and switch between themes at your leisure
  • NEW FEATURE - Internal Update Notification System ๐Ÿ›Ž๏ธ

    • Users will now be automatically notified of the availability of new updates from within the Web UI
      • Automatically triggered by a difference between the version number of the most recent GitHub release and the version installed
      • Set to only show once per calendar day until updated as to not be annoying
        • Visible to Admin users only
  • NEW FEATURE - Manual Library Refresh โ™ป๏ธ

    • Ever had books get stuck in the ingest folder after an unexpected power-cut ect.? Well say goodbye to having to manually copy the books to be ingested back in and out of the ingest folder, simply press the Refresh Library button on the navbar of the Web UI and anything still sitting in the ingest folder will be automatically ingested!
  • NEW FEATURE - Batch Editing & Deletion! ๐Ÿ—‚๏ธ๐Ÿ—„๏ธ

    • Say goodbye to clicking that edit button again, and again, and again just to remove or edit a single series!
    • To use, simply navigate to the Books Listpage on the left hand side of the Web UI, select the books you wish to edit/ delete and use the buttons either above the table or within the headers to do whatever you need!
    • Courtesy of @jmarmstrong1207

Calibre-Web Automated

UNDER ACTIVE DEVELOPMENT โš ๏ธ

  • Please be aware that while CWA currently works for most people, it is still under active development and that bugs and unexpected behaviours can occur while we work and the code base matures
  • I want to say a big thanks ๐Ÿ™ to the members of this community that have taken the time to participate in the testing and development of this project, especially to @jmarmstrong1207 who has been working tirelessly on improving the project since the release of Version 1.2.0
    • In recognition of this, @jmarmstrong1207 has now been promoted to a co-contributor here on the project, so feel free to also contact him with any issues, suggestions, ideas ect.
    • For any others that wish to contribute to this project in some way, please reach out on our Discord Server and see how you can best get involved:

Coming in Version 2.0.1 - Currently in Testing Phase ๐Ÿงช

  • Integrating some of the new Command-Line Features into the Web UI
    • Including Library Conversion, the Display of Conversion History and checking of the CWA Monitoring Services

Coming Soon - Currently Under Active Development ๐Ÿ—๏ธ

  • Robotic Reading ๐Ÿค–๐Ÿ“–
    • Bionic reading was initially released in 2022 as a method of formatting text to make it easier for people with ADHD and other concentration & reading issues to read faster, easier and to retain more of what they've read.
    • We were inspired by the concept and have created a ePub conversion algorithm inspired by the original Bionic Reading for those who like it or who may want to give it a go

Additional Features on our Roadmap ๐Ÿ›ฃ๏ธ๐ŸŒฑ

  • Release CWA also as a Docker Mod
  • Support for arm64 architectures
  • Adding tracking of ebook imports & deletions to the cwa.db
  • Improved metadata handling and conversion for comics & manga
  • Please suggest any ideas or wishes you might have! we're open to anything!

IMPORTANT NOTE: โšก Current users of Calibre-Web Automated version 1.2.2 or below should update using the latest DockerHub image to ensure stability and compatibility with future updates

How To Install ๐Ÿ“–

Quick Install ๐Ÿš€

  1. Download the Docker Compose template file using the command below:
curl -OL https://raw.githubusercontent.com/crocodilestick/calibre-web-automated/main/docker-compose.yml
  1. Move the compose file to an empty folder (e.g. ~/docker/calibre-web-automated/docker-compose.yml). This will be used to store the server data and library

  2. Navigate to where you downloaded the Compose file using cd and run:

docker compose up -d

And that's you off to the races! ๐Ÿฅณ HOWEVER to avoid potential problems and ensure maximum functionality, we recommend carrying out these Post-Install Tasks Here.


Using Docker Compose ๐Ÿ‹โญ(Recommended)

1. Setup the container using the Docker Compose template below: ๐Ÿ‹๐Ÿ“œ

---
services:
  calibre-web-automated:
    image: crocodilestick/calibre-web-automated:latest
    container_name: calibre-web-automated
    environment:
      - PUID=1000
      - PGID=100
      - TZ=UTC
      - DOCKER_MODS=lscr.io/linuxserver/mods:universal-calibre-v7.16.0
    volumes:
      - ./config:/config
      - ./cwa-book-ingest:/cwa-book-ingest
      - ./calibre-library:/calibre-library
      #- ./books:/books #Optional
      #- gmail.json:/app/calibre-web/gmail.json #Optional
    ports:
      - 8084:8083 # Change the first number to change the port you want to access the Web UI, not the second
    restart: unless-stopped
    
  • Explanation of the Container Bindings:
    • Make sure all 3 of the main bindings are separate directories, errors can occur when binds are made within other binds
    • /config - Can be any empty folder, used to store logs and other miscellaneous files that keep CWA running
    • /cwa-book-ingest - ATTENTION โš ๏ธ - All files within this folder will be DELETED after being processed. This folder should only be used to dump new books into for import and automatic conversion
    • /calibre-library - This should be bound to your Calibre library folder where the metadata.db file resides within and will be mounted automatically. If there are multiple libraries, it will find and mount the largest one.
      • If you don't have an existing Calibre Database, you can use the Docker Template folder for a quick install mentioned above.
    • /books (Optional) - This is purely optional, I personally bind /books to where I store my downloaded books so that they accessible from within the container but CWA doesn't require this
    • /app/calibre-web/gmail.json (Optional) - This is used to setup Calibre-Web and/or CWA with your gmail account for sending books via email. Follow the guide here if this is something you're interested in but be warned it can be a very fiddly process, I would personally recommend a simple SMTP Server

And just like that, Calibre-Web Automated should be up and running!

2. Recommended Post-Install Tasks:

Calibre-Web Quick Start Guide

  1. Open your browser and navigate to http://localhost:8084 or http://localhost:8084/opds for the OPDS catalog
  2. Log in with the default admin credentials (below)
  1. Configure your Calibre-Web instance via the admin page, referring to the Basic Configuration and UI Configuration guides
  2. Add books by having them placed in the folder you bound to cwa-book-ingest in your Docker Compose

โš ๏ธ ATTENTION โš ๏ธ

  • Downloading files directly into /cwa-book-ingest is not supported. It can cause duplicate imports and potentially a corrupt database. It is recommended to first download the books completely, then transfer them to /cwa-book-ingest to avoid any issues

Default Admin Login:

Username: admin
Password: admin123

Configuring CWA โš™๏ธ

  • If your Calibre Library contains any ebooks not in the .epub format, from within the container run the convert-library command.
    • Calibre-Web Automated's extra features only work with epubs and so failure to complete this step could result in unforeseen errors and general unreliability
    • Full usage can be found below in the Usage Section however the following command will automatically convert any non-epubs to epubs and store the original files in /config/processed_books:
convert-library --keep
  • Drop a book into your ingest folder and check everything is working correctly!

Usage ๐Ÿ”ง

Adding Books to Your Library

  • Simply move your newly downloaded or existing eBook files to the ingest folder which is /cwa-book-ingest by default or whatever you designated during setup if using the Script Install method. Anything you place in that folder will be automatically analysed, converted if necessary and then imported into your Calibre-Web library.
    • โš ๏ธ ATTENTION โš ๏ธ
      • Downloading files directly into /cwa-book-ingest is not supported. It can cause duplicate imports and potentially a corrupt database. It is recommended to first download the books completely, then transfer them to /cwa-book-ingest to avoid any issues

The Cover-Enforcer CLI Tool

usage: cover-enforcer [-h] [--log LOG] [--dir DIR] [-all] [-list] [-history] [-paths] [-v]

Upon receiving a log, valid directory or an "-all" flag, this script will enforce the covers and metadata of the corresponding books, making sure that each are correctly stored in
both the epubs themselves and the user's Calibre Library. Additionally, if an epub happens to be in EPUB 2 format, it will also be automatically upgraded to EPUB 3.

options:
  -h, --help     show this help message and exit
  --log LOG      Will enforce the covers and metadata of the books in the given log file.
  --dir DIR      Will enforce the covers and metadata of the books in the given directory.
  -all           Will enforce covers & metadata for ALL books currently in your calibre-library-dir
  -list, -l      List all books in your calibre-library-dir
  -history       Display a history of all enforcements ever carried out on your machine (not yet implemented)
  -paths, -p     Use with '-history' flag to display stored paths of all epubs in enforcement database
  -v, --verbose  Use with history to display entire enforcement history instead of only the most recent 10 entries

CWA Cover-Enforcer History Usage

The Convert-Library Tool

usage: convert-library [-h] [--replace] [--keep] [-setup]

Made for the purpose of converting ebooks in a calibre library not in epub format, to epub format

options:
  -h, --help     show this help message and exit
  --replace, -r  Replaces the old library with the new one
  --keep, -k     Creates a new epub library with the old one but stores the old files in /config/processed_books
  -setup         Indicates to the function whether or not it's being ran from the setup script or manually (DO NOT USE MANUALLY)

Checking the Monitoring Services are working correctly

  • Simply run the following command from within the container: cwa-check
  • If all 3 services come back as green and running they are working properly, otherwise there may be problems with your configuration/install

Further Development ๐Ÿ—๏ธ

  • Calibre-Web Automated (formerly Calibre-Web Automator) was made to be everything I need for my reading workflow, I personally love the new features and hope you do to!
  • While it already does everything I need, building it has been really fun and it's great to see so many other people get so much use out of it!
  • And so on top of continuing to perform maintenance and bugfixes, we are always open to suggestions and ideas from the community so if there's something you'd like to see, don't be afraid to ask!
  • To do so you can either open an Issue here on the project page or get in touch with us on our Discord Server here!
  • We're always looking for new people to help out too so if that sounds like you, let us know!