m-haisham/novelsave

add the option to retrieve metadata from www.wlnupdates.com or www.novelupdates.com

Closed this issue · 6 comments

Can you add the option to retrieve metadata from www.wlnupdates.com or www.novelupdates.com ?
So that the saved epubs can have all the metadata fields compiled.

This would require creating meta sources much akin to the current sources.
Plus creating a new action so that it can be used without affecting the current usage. Something like

actions:
  ...
  --meta META   url to retrieve additional meta information

Below is a list of meta fields

General

  • Description/Synopsis
    synopsis can be taken directly from the website from almost all sites

  • Genre/Tags
    tracking sites such as www.novelupdates.com does have a better collection of tags

  • Original Language
    for translated work it certainly is a worthy addition

  • Year (of release)
    for the spirit of completion

Variable

  • Ratings
    this is a very variable value so I don't think its worth

Unique

  • fanfiction
    the name of the original work is certainly something that could be included

What are your thoughts?

We just need to get all the epub metadata. Here is a real world example:

<dc:identifier id="book-id" opf:scheme="ISBN">1234567890X</dc:identifier> 
    <dc:title id="english">Battle Through the Heavens</dc:title>
    <meta refines="#english" property="title-type">english title</meta>
    <dc:title id="original">斗破苍穹</dc:title>
    <meta refines="#original" property="title-type">original title</meta>
    <dc:title id="alternative">Fights Break Sphere</dc:title>
    <meta refines="#alternative" property="title-type">alternative title</meta>
    <dc:language id="text-language">en</dc:language>
    <meta refines="#text-language" property="identifier-type" scheme="onix:codelist22">01</meta>
    <dc:language id="original-language">cn</dc:language>
    <meta refines="#original-language" property="identifier-type" scheme="onix:codelist22">02</meta>
    <dc:creator opf:role="aut" >Heavenly Silkworm Potato</dc:creator>
    <dc:creator opf:role="aut" >Tian Can Tu Dou</dc:creator>
    <dc:creator opf:role="aut" >天蚕土豆</dc:creator>
    <dc:creator opf:role="trl" >GravityTales</dc:creator>
    <dc:contributor opf:role="ill" >Hongbin Zhou</dc:contributor>
    <dc:publisher>Qidian</dc:publisher>
    <dc:subject>Action</dc:subject>
    <dc:subject>Adventure</dc:subject>
    <dc:subject>Fantasy</dc:subject>
    <dc:subject>Harem</dc:subject>
    <dc:subject>Martial Arts</dc:subject>
    <dc:subject>Xuanhuan</dc:subject>
    <dc:date opf:event="publication">2018-01-01T00:00:00Z</dc:date>
    <dc:source>urn:isbn:1234567890X</dc:source>
    <dc:description>"In a land where no magic is present. A land where the strong make the rules and the weak have to obey. A land filled with alluring treasures and beauty, yet also filled with unforeseen danger. Three years ago, Xiao Yan, who had shown talents none had seen in decades, suddenly lost everything. His powers, his reputation, and his promise to his mother. What sorcery has caused him to lose all of his powers? And why has his fiancee suddenly shown up? </dc:description>
 <link href="https://www.novelupdates.com/series/battle-through-the-heavens/" />
  </metadata> 

All the metadata in the example above is taken from Novelupdates.
An extensive API is provided on both websites. You can check wlnupdates.com API here:

https://github.com/fake-name/wlnupdates/blob/master/app/templates/api-docs.md

That makes things a whole lot easier.

It seems novelupdates does not have an api

Source: https://www.webnovel.com/book/lord-of-the-mysteries_11022733006234505
Metadata Source: https://www.wlnupdates.com/series-id/64956/lord-of-the-mysteries

Commandline

novelsave https://www.webnovel.com/book/lord-of-the-mysteries_11022733006234505 --meta https://www.wlnupdates.com/series-id/64956/lord-of-the-mysteries

Xml

<metadata xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:opf="http://www.idpf.org/2007/opf">
    <meta property="dcterms:modified">2020-12-21T13:25:05Z</meta>
    <meta name="generator" content="Ebook-lib 0.17.1"/>
    <dc:identifier id="id">aa939fac18b98616792efd551f6a6e57</dc:identifier>
    <dc:title>Lord of the Mysteries</dc:title>
    <dc:title type="alternate">诡秘之主 (new)</dc:title>
    <dc:title type="alternate">序列的戰爭</dc:title>
    <dc:title type="alternate">序列 的 戰爭 the most high</dc:title>
    <dc:title type="alternate">Lord of Enigma</dc:title>
    <dc:title type="alternate">序列的戰爭 (old)</dc:title>
    <dc:title type="alternate">Quỷ bí chi chủ</dc:title>
    <dc:title type="alternate">诡秘之主</dc:title>
    <dc:title type="alternate">Lord of Mysteries</dc:title>
    <dc:title type="alternate">Guǐmì Zhī Zhǔ</dc:title>
    <dc:title type="alternate">LoM</dc:title>
    <dc:title type="alternate">秘之主</dc:title>
    <dc:title type="alternate">LOTM</dc:title>
    <dc:language>en</dc:language>
    <dc:creator id="creator">Cuttlefish That Loves Diving</dc:creator>
    <dc:synopsis>With the rising tide of steam power and machinery, who can come close to being a Beyonder? Shrouded in the fog of history and darkness, who or what is the lurking evil that murmurs into our ears?&#13; Waking up to be faced with a string of mysteries, Zhou Mingrui finds himself reincarnated as Klein Moretti in an alternate Victorian era world where he sees a world filled with machinery, cannons, dreadnoughts, airships, difference machines, as well as Potions, Divination, Hexes, Tarot Cards, Sealed Artifacts… &#13; The Light continues to shine but mystery has never gone far. Follow Klein as he finds himself entangled with the Churches of the world—both orthodox and unorthodox—while he slowly develops newfound powers thanks to the Beyonder potions.&#13; Like the corresponding tarot card, The Fool, which is numbered 0—a number of unlimited potential—this is the legend of "The Fool."</dc:synopsis>
    <dc:contributor role="ill">A Chan</dc:contributor>
    <dc:contributor role="ill">Hantai Comics</dc:contributor>
    <dc:publisher>Qidian</dc:publisher>
    <dc:publisher>Webnovel</dc:publisher>
    <dc:subject>action</dc:subject>
    <dc:subject>adventure</dc:subject>
    <dc:subject>fantasy</dc:subject>
    <dc:subject>historical</dc:subject>
    <dc:subject>horror</dc:subject>
    <dc:subject>mature</dc:subject>
    <dc:subject>mystery</dc:subject>
    <dc:subject>shounen</dc:subject>
    <dc:subject>supernatural</dc:subject>
    <dc:subject>tragedy</dc:subject>
    <dc:subject>xuanhuan</dc:subject>
    <dc:date role="publication">Mon, 01 Jan 2018 00:00:00 GMT</dc:date>
    <source>https://www.webnovel.com/book/lord-of-the-mysteries_11022733006234505</source>
    <meta name="cover" content="cover-img"></meta>
  </metadata>

What do you think?