2.1.3 #12

Merged
olofvndrhr merged 8 commits from dev into master 2022-05-29 14:09:38 +02:00
12 changed files with 103 additions and 50 deletions

1
.gitignore vendored
View file

@ -9,6 +9,7 @@ downloads/
__pycache__/ __pycache__/
.pytest_cache/ .pytest_cache/
chaps.txt chaps.txt
mangas.txt
.idea/ .idea/
venv venv

View file

@ -9,43 +9,65 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Add support for more sites - Add support for more sites
## [2.1.3] - 2022-05-29
### Fixed
- Error-chapters and skipped-chapters list are now shown again
- The Interactive input version now matches `--version`
### Added
- Ability to list chapters with interactive input
### Changed
- Replace `exit()` with `sys.exit()`
- Renamed class methods to not look like dunder methods
- Script execution moved from `os.system()` to `subprocess.call()`
## [2.1.2] - 2022-05-20 ## [2.1.2] - 2022-05-20
### Fixed ### Fixed
- List chapters when none were specified - List chapters when none were specified
- Typos - Typos
### Added ### Added
- Ability to download whole volumes - Ability to download whole volumes
### Changed ### Changed
- Moved processing of list with links to input.py - Moved processing of list with links to input.py
- Updated README for volume and chapter selection - Updated README for volume and chapter selection
## [2.1.1] - 2022-05-18 ## [2.1.1] - 2022-05-18
### Fixed ### Fixed
- Progress bar on verbose output - Progress bar on verbose output
- Sonarqube link for CI - Sonarqube link for CI
- A few typos - A few typos
- Removed unnecessary escapes from file rename regex - Removed unnecessary escapes from file rename regex
### Added ### Added
- API template - API template
### Changed ### Changed
- Updated docker baseimage - Updated docker baseimage
- Rewrote app.py to a class - Rewrote app.py to a class
## [2.1.0] - 2022-05-16 ## [2.1.0] - 2022-05-16
### Fixed ### Fixed
- Detection of files. Now it will skip them again - Detection of files. Now it will skip them again
### Added ### Added
- Ability to save the chapters as pdf (only on amd64/x86) - Ability to save the chapters as pdf (only on amd64/x86)
- New output formats: rar, zip - New output formats: rar, zip
- Progress bar to show image download - Progress bar to show image download
@ -55,14 +77,15 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Removed duplicate code - Removed duplicate code
### Changed ### Changed
- How the variables are used inside the script - How the variables are used inside the script
- Variables have now the same name as in other scripts (mostly) - Variables have now the same name as in other scripts (mostly)
- Better retrying when a task fails - Better retrying when a task fails
## [2.0.8] - 2022-05-13 ## [2.0.8] - 2022-05-13
### Changed ### Changed
- Rewrote parts of script to be easier to maintain - Rewrote parts of script to be easier to maintain
- Moved the input script to the base folder - Moved the input script to the base folder
- Moved all arguments to a class - Moved all arguments to a class
@ -71,24 +94,26 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [2.0.7] - 2022-05-13 ## [2.0.7] - 2022-05-13
### Changed ### Changed
- Changed CI/CD Platform from Drone-CI to Woodpecker-CI - Changed CI/CD Platform from Drone-CI to Woodpecker-CI
- Release title is now only the version - Release title is now only the version
## [2.0.6] - 2022-05-11 ## [2.0.6] - 2022-05-11
### Fixed ### Fixed
- Filenames on windows (ntfs). Removed double quote from file and folder names
- Filenames on windows (ntfs). Removed double quote from file and folder names
## [2.0.5] - 2022-05-11 ## [2.0.5] - 2022-05-11
### Fixed ### Fixed
- Better error handling on "KeyboardInterrupt" - Better error handling on "KeyboardInterrupt"
- Release notes now fixed - Release notes now fixed
### Added ### Added
- New test cases
- New test cases
## [2.0.4] - 2022-05-10 ## [2.0.4] - 2022-05-10

View file

@ -60,7 +60,8 @@ See the docker [README](./docker/README.md)
## Options ## Options
> "--format" currently only works with "", "pdf", "zip", "rar" and "cbz". As it just renames the zip file with the new suffix (except pdf). For pdf creation you have to install img2pdf. > "--format" currently only works with "", "pdf", "zip", "rar" and "cbz". As it just renames the zip file with the new
> suffix (except pdf). For pdf creation you have to install img2pdf.
```txt ```txt
usage: manga-dlp.py [-h] (-u URL_UUID | --read READ | -v) [-c CHAPTERS] [-p PATH] [-l LANG] [--list] [--format FORMAT] [--forcevol] [--wait WAIT] [--verbose] usage: manga-dlp.py [-h] (-u URL_UUID | --read READ | -v) [-c CHAPTERS] [-p PATH] [-l LANG] [--list] [--format FORMAT] [--forcevol] [--wait WAIT] [--verbose]

View file

@ -7,7 +7,7 @@ LABEL build_version="Version:- ${VERSION} Build-date:- ${BUILD_DATE}"
LABEL maintainer="Ivan Schaller" LABEL maintainer="Ivan Schaller"
# manga-dlp version # manga-dlp version
ENV MDLP_VERSION=2.1.2 ENV MDLP_VERSION=2.1.3
# install packages # install packages
RUN \ RUN \

View file

@ -7,7 +7,7 @@ LABEL build_version="Version:- ${VERSION} Build-date:- ${BUILD_DATE}"
LABEL maintainer="Ivan Schaller" LABEL maintainer="Ivan Schaller"
# manga-dlp version # manga-dlp version
ENV MDLP_VERSION=2.1.2 ENV MDLP_VERSION=2.1.3
# install packages # install packages
RUN \ RUN \

View file

@ -1,33 +1,48 @@
from mangadlp.input import get_args import subprocess
import os
import sys import sys
mangadlp_version = "2.1.2" from mangadlp.input import get_args
mangadlp_version = "2.1.3"
def get_input(): def get_input():
print(f"Manga-DLP Version {mangadlp_version}") print(f"manga-dlp version: {mangadlp_version}")
print("Enter details of the manga you want to download:") print("Enter details of the manga you want to download:")
while True: while True:
try: try:
url_uuid = str(input("Url or UUID: ")) url_uuid = str(input("Url or UUID: "))
readlist = str(input("List with links (optional): ")) readlist = str(input("List with links (optional): "))
language = str(input("Language: ")) language = str(input("Language: ")) or "en"
chapters = str(input("Chapters: ")) list_chapters = str(input("List chapters? y/N: "))
if list_chapters.lower() != "y" or list_chapters.lower() != "yes":
chapters = str(input("Chapters: "))
except KeyboardInterrupt: except KeyboardInterrupt:
exit(1) sys.exit(1)
except: except:
continue continue
else: else:
break break
args = [f"-l {language}", f"-c {chapters}"]
args = [
"python3",
"manga-dlp.py",
"-l",
language,
"-c",
chapters,
]
if url_uuid: if url_uuid:
args.append(f"-u {url_uuid}") args.append("-u")
args.append(url_uuid)
if readlist: if readlist:
args.append(f"--read {readlist}") args.append("--read")
args.append(readlist)
if list_chapters.lower() == "y" or list_chapters.lower() == "yes":
args.append("--list")
# start script again with the arguments # start script again with the arguments
os.system(f"python3 manga-dlp.py {' '.join(args)}") subprocess.call(args)
if __name__ == "__main__": if __name__ == "__main__":

View file

@ -1,6 +1,9 @@
import re import re
import sys
from time import sleep from time import sleep
import requests import requests
import mangadlp.utils as utils import mangadlp.utils as utils
@ -43,7 +46,7 @@ class Mangadex:
except: except:
if counter >= 3: if counter >= 3:
print("ERR: Maybe the MangaDex API is down?") print("ERR: Maybe the MangaDex API is down?")
exit(1) sys.exit(1)
else: else:
print("ERR: Mangadex API not reachable. Retrying") print("ERR: Mangadex API not reachable. Retrying")
sleep(2) sleep(2)
@ -53,7 +56,7 @@ class Mangadex:
# check if manga exists # check if manga exists
if manga_data.json()["result"] != "ok": if manga_data.json()["result"] != "ok":
print("ERR: Manga not found") print("ERR: Manga not found")
exit(1) sys.exit(1)
return manga_data return manga_data
@ -66,7 +69,7 @@ class Mangadex:
# check for new mangadex id # check for new mangadex id
if not uuid_regex.search(self.url_uuid): if not uuid_regex.search(self.url_uuid):
print("ERR: No valid UUID found") print("ERR: No valid UUID found")
exit(1) sys.exit(1)
manga_uuid = uuid_regex.search(self.url_uuid)[0] manga_uuid = uuid_regex.search(self.url_uuid)[0]
return manga_uuid return manga_uuid
@ -86,7 +89,7 @@ class Mangadex:
title = alt_titles[self.language] title = alt_titles[self.language]
except: # no title on requested language found except: # no title on requested language found
print("ERR: Chapter in requested language not found.") print("ERR: Chapter in requested language not found.")
exit(1) sys.exit(1)
return utils.fix_name(title) return utils.fix_name(title)
# check if chapters are available in requested language # check if chapters are available in requested language
@ -120,7 +123,7 @@ class Mangadex:
# check for chapters in specified lang # check for chapters in specified lang
total_chapters = self.check_chapter_lang() total_chapters = self.check_chapter_lang()
if total_chapters == 0: if total_chapters == 0:
exit(1) sys.exit(1)
chapter_data = {} chapter_data = {}
last_chapter = ["", ""] last_chapter = ["", ""]

View file

@ -1,5 +1,6 @@
import re import re
import shutil import shutil
import sys
from pathlib import Path from pathlib import Path
import mangadlp.downloader as downloader import mangadlp.downloader as downloader
@ -11,7 +12,7 @@ from mangadlp.api.mangadex import Mangadex
class MangaDLP: class MangaDLP:
"""Download Mangas from supported sites. """Download Mangas from supported sites.
After initialization, start the script with the function __main__(). After initialization, start the script with the function get_manga().
:param url_uuid: URL or UUID of the manga :param url_uuid: URL or UUID of the manga
:param language: Manga language with country codes. "en" --> english :param language: Manga language with country codes. "en" --> english
@ -49,9 +50,9 @@ class MangaDLP:
self.download_wait = download_wait self.download_wait = download_wait
self.verbose = verbose self.verbose = verbose
# prepare everything # prepare everything
self.__prepare__() self._prepare()
def __prepare__(self) -> None: def _prepare(self) -> None:
# additional stuff # additional stuff
# set manga format suffix # set manga format suffix
if self.file_format and "." not in self.file_format: if self.file_format and "." not in self.file_format:
@ -70,10 +71,6 @@ class MangaDLP:
self.manga_chapter_list = self.api.chapter_list self.manga_chapter_list = self.api.chapter_list
self.manga_path = Path(f"{self.download_path}/{self.manga_title}") self.manga_path = Path(f"{self.download_path}/{self.manga_title}")
def __main__(self):
# start flow
self.get_manga()
def pre_checks(self) -> None: def pre_checks(self) -> None:
# prechecks userinput/options # prechecks userinput/options
# no url and no readin list given # no url and no readin list given
@ -81,7 +78,7 @@ class MangaDLP:
print( print(
f'ERR: You need to specify a manga url/uuid with "-u" or a list with "--read"' f'ERR: You need to specify a manga url/uuid with "-u" or a list with "--read"'
) )
exit(1) sys.exit(1)
# checks if --list is not used # checks if --list is not used
if not self.list_chapters: if not self.list_chapters:
if self.chapters is None: if self.chapters is None:
@ -89,15 +86,15 @@ class MangaDLP:
print( print(
f'ERR: You need to specify one or more chapters to download. To see all chapters use "--list"' f'ERR: You need to specify one or more chapters to download. To see all chapters use "--list"'
) )
exit(1) sys.exit(1)
# if forcevol is used, but didn't specify a volume in the chapters selected # if forcevol is used, but didn't specify a volume in the chapters selected
if self.forcevol and ":" not in self.chapters: if self.forcevol and ":" not in self.chapters:
print(f"ERR: You need to specify the volume if you use --forcevol") print(f"ERR: You need to specify the volume if you use --forcevol")
exit(1) sys.exit(1)
# if forcevol is not used, but a volume is specified # if forcevol is not used, but a volume is specified
if not self.forcevol and ":" in self.chapters: if not self.forcevol and ":" in self.chapters:
print(f"ERR: Don't specify the volume without --forcevol") print(f"ERR: Don't specify the volume without --forcevol")
exit(1) sys.exit(1)
# check the api which needs to be used # check the api which needs to be used
def check_api(self, url_uuid: str) -> type: def check_api(self, url_uuid: str) -> type:
@ -115,7 +112,7 @@ class MangaDLP:
# this is only for testing multiple apis # this is only for testing multiple apis
if api_test.search(url_uuid): if api_test.search(url_uuid):
print("Not supported yet") print("Not supported yet")
exit(1) sys.exit(1)
# no supported api found # no supported api found
print(f"ERR: No supported api in link/uuid found: {url_uuid}") print(f"ERR: No supported api in link/uuid found: {url_uuid}")
@ -164,10 +161,13 @@ class MangaDLP:
return_infos = self.archive_chapter(return_infos["chapter_path"]) return_infos = self.archive_chapter(return_infos["chapter_path"])
error_chapters.append(return_infos.get("error")) error_chapters.append(return_infos.get("error"))
skipped_chapters.append(return_infos.get("skipped")) skipped_chapters.append(return_infos.get("skipped"))
# check if chapter was skipped
# done with chapter try:
print("INFO: Done with chapter") return_infos["skipped"]
print("-----------------------------------------\n") # chapter was not skipped
except KeyError:
# done with chapter
print("INFO: Done with chapter\n")
# done with manga # done with manga
print(f"{print_divider}") print(f"{print_divider}")
@ -176,6 +176,10 @@ class MangaDLP:
skipped_chapters = list(filter(None, skipped_chapters)) skipped_chapters = list(filter(None, skipped_chapters))
if len(skipped_chapters) >= 1: if len(skipped_chapters) >= 1:
print(f"INFO: Skipped chapters:\n{', '.join(skipped_chapters)}") print(f"INFO: Skipped chapters:\n{', '.join(skipped_chapters)}")
# filter error list
error_chapters = list(filter(None, error_chapters))
if len(error_chapters) >= 1:
print(f"INFO: Chapters with errors:\n{', '.join(error_chapters)}")
print(f"{print_divider}\n") print(f"{print_divider}\n")
# once called per chapter # once called per chapter
@ -190,7 +194,7 @@ class MangaDLP:
) )
except KeyboardInterrupt: except KeyboardInterrupt:
print("ERR: Stopping") print("ERR: Stopping")
exit(1) sys.exit(1)
# check if the image urls are empty. if yes skip this chapter (for mass downloads) # check if the image urls are empty. if yes skip this chapter (for mass downloads)
if not chapter_image_urls: if not chapter_image_urls:
@ -251,7 +255,7 @@ class MangaDLP:
) )
except KeyboardInterrupt: except KeyboardInterrupt:
print("ERR: Stopping") print("ERR: Stopping")
exit(1) sys.exit(1)
except: except:
print(f"ERR: Cant download: '{chapter_filename}'. Skipping") print(f"ERR: Cant download: '{chapter_filename}'. Skipping")
# add to skipped chapters list # add to skipped chapters list

View file

@ -1,6 +1,8 @@
import shutil
import sys
from pathlib import Path from pathlib import Path
from time import sleep from time import sleep
import shutil
import requests import requests
import mangadlp.utils as utils import mangadlp.utils as utils
@ -29,7 +31,7 @@ def download_chapter(
raise ConnectionError raise ConnectionError
except KeyboardInterrupt: except KeyboardInterrupt:
print("ERR: Stopping") print("ERR: Stopping")
exit(1) sys.exit(1)
except: except:
if counter >= 3: if counter >= 3:
print("ERR: Maybe the MangaDex Servers are down?") print("ERR: Maybe the MangaDex Servers are down?")

View file

@ -1,15 +1,17 @@
import argparse import argparse
import mangadlp.app as app import sys
from pathlib import Path from pathlib import Path
mangadlp_version = "2.1.2" import mangadlp.app as app
mangadlp_version = "2.1.3"
def check_args(args): def check_args(args):
# check if --version was used # check if --version was used
if args.version: if args.version:
print(f"manga-dlp version: {mangadlp_version}") print(f"manga-dlp version: {mangadlp_version}")
exit(0) sys.exit(0)
# check if a readin list was provided # check if a readin list was provided
if not args.read: if not args.read:
# single manga, no readin list # single manga, no readin list
@ -47,7 +49,7 @@ def call_app(args):
args.wait, args.wait,
args.verbose, args.verbose,
) )
mdlp.__main__() mdlp.get_manga()
def get_args(): def get_args():

View file

@ -7,7 +7,7 @@ long_description = readme.read_text()
setuptools.setup( setuptools.setup(
name="manga-dlp", name="manga-dlp",
version="2.1.2", version="2.1.3",
author="Ivan Schaller", author="Ivan Schaller",
author_email="ivan@schaller.sh", author_email="ivan@schaller.sh",
description="A cli manga downloader", description="A cli manga downloader",

View file

@ -19,7 +19,7 @@ def test_full_api_mangadex():
download_wait=0.5, download_wait=0.5,
verbose=True, verbose=True,
) )
mdlp.__main__() mdlp.get_manga()
assert manga_path.exists() and manga_path.is_dir() assert manga_path.exists() and manga_path.is_dir()
assert chapter_path.exists() and chapter_path.is_file() assert chapter_path.exists() and chapter_path.is_file()