Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added urllib-downloader koubae #7

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Koubae
Copy link

@Koubae Koubae commented Dec 11, 2020

Added

download_images function

* extra_keywords default to None, so that pic can be search if only a keyword
* Change the default dir.
* Choose total pic to choosen value, if 'all' (capital doesn't matter) is passed, then download all found ulrs
* Parameter download, if False, then won't download, if True it will
* General refactoring

download_manager function

Same parameters as download_images but added

* multiprocess if set to True then it run in multiprocess mode
* debug => if set to True if will run log_to_stderr(logging.DEBUG)

logger

* Prints and add to log the messages from the functions

Fixed

* Readability 
* More user control

Removed

* os.getpid() for getting the os id I switch to current_process().name instead
we can also add Process().pid

* The download_page to downloader is delegaed to download both whole page or single url. 

Let me know what you think, I'm planning to add more functionallyty with argparse.

Signed-off-by: Koubae [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant