Skip to content

Commit

Permalink
v2.0.0 Release (#6)
Browse files Browse the repository at this point in the history
* Initial work on v2 candidate

* Additional changes for v2

* Reworked env and travis

* Added Changelog
  • Loading branch information
Justintime50 authored May 19, 2020
1 parent 75cebfa commit 9952389
Show file tree
Hide file tree
Showing 10 changed files with 379 additions and 19 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
.DS_Store
.idea
logs
.env
13 changes: 11 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,13 @@
language: bash
language: python

python:
- 3.6
- 3.7
- 3.8

install:
- pip install -r requirements.txt

script:
- shellcheck github-archive.sh
- shellcheck src/legacy/github-archive.sh
- pylint src/github_archive.py
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# CHANGELOG

## 2.0.0 (2020-05-18)

* Rewrote the script in Python.
* Added concurrency to clone/pull changes incredibly fast for large amounts of repos and gists.
* Added even more custimization and control
* Uniform log naming, better logging details
* Added changelog

## 1.1.0 (2020)

* Added more customization options
* Allowed organization and gists to be cloned/pulled
* Allowed private repos to be cloned/pulled
* Bug fixes

## 1.0.0 (2019)

* Wrote the script in Bash with Python as dependency
* Added some customization options
35 changes: 21 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# GitHub Archive

Powerful script to clone your entire GitHub instance or save it as an archive.
A powerful script to concurrently clone your entire GitHub instance or save it as an archive.

[![Build Status](https://travis-ci.com/Justintime50/github-archive.svg?branch=master)](https://travis-ci.com/Justintime50/github-archive)
[![MIT Licence](https://badges.frapsoft.com/os/mit/mit.svg?v=103)](https://opensource.org/licenses/mit-license.php)
Expand All @@ -16,30 +16,31 @@ Powerful script to clone your entire GitHub instance or save it as an archive.
- Clone/pull personal repos (public and private)
- Clone/pull organization repos (public and private)
- Clone/pull personal gists (public and private)
- Iterate over 100's of repos and gists
- Can be configured to run on a schedule to automate pulling changes
- Iterate over infinite number of repos and gists concurrently
- Great use case: Run on a schedule to automate pulling changes or keep a local backup of all your repos

### Configurable Settings

The power of GitHub Archive comes in its configuration. Maybe you only want to clone/pull your personal public repos or maybe you want to go all out and include private repos from you and all organizations you belong to including your gists. Customize the location repos are saved to, how long logs are kept for and the naming format they'll use. Iterate over 100's of repos and sit back while GitHub Archive does all the work.
The power of GitHub Archive comes in its configuration. Maybe you only want to clone/pull your personal public repos or maybe you want to go all out and include private repos from you and all organizations you belong to including your gists. Customize the location repos are saved to, how long logs are kept for. Iterate over 100's of repos concurrently and sit back while GitHub Archive does all the work.

- Personal repos (on/off)
- Organization repos (on/off)
- Personal Gists (on/off)
- Cloning (on/off)
- Pulling (on/off)
- Specify the number of repos to include (1 - ∞)
- Log retention life & filename scheme
- What organizations you'd like included
- Log retention life
- GitHub Archive location
- Which branch to pull from

## Install

**NOTE:** This project requires that you have Python installed. Python usually comes built-in on macOS and Linux.

```bash
# Copy the configuration file and edit for your needs.
cp .config.example .config
# Install dependencies
pip3 install -r requirements.txt

# Copy the environment file and edit for your needs.
cp .env.example .env
```

**For Private Repos:** You must have an SSH key generated on your local machine and added to your GitHub account.
Expand All @@ -62,18 +63,18 @@ GitHub Archive will clone any repo and gist that doesn't exist locally and pull
### Run Script

```bash
./github-archive.sh
python3 github_archive.py
```

### Shell Alias

```bash
# If using Bash insted of ZSH, use ~/.bash_profile
echo alias github-archive="/path/to/github-archive.sh" >> ~/.zshrc
echo alias github_archive="/path/to/github_archive.py" >> ~/.zshrc
source ~/.zshrc

# Usage of alias
github-archive
github_archive
```

### Launch Agent (Recommended on macOS)
Expand All @@ -96,5 +97,11 @@ launchctl start local.githubArchive.plist
```bash
crontab -e

0 1 * * * /path/to/github-archive.sh
0 1 * * * /path/to/github_archive.py
```

## Legacy Script

GitHub Archive was initially built with Bash partly because I was into shell scripting at the time and partly because I wanted to keep the script as dependency free as possible. Even still, the Bash script depended on Python. As Python will soon be removed from macOS by default, I decided to rewrite the script in pure Python for two reasons - 1) Once Python is no longer built-in, users would need to install Python anyway making the pure Bash script broken out of the box and 2) Rewriting allowed me to take full advantage of the power of Python. One large benefit of doing this was adding concurrency which is great for users with dozens or hundreds of repos across various organizations. Additional logging and stability improvements were also made.

If you'd like to use or view the legacy script, check out the `src/legacy` folder.
3 changes: 3 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
python-dotenv
PyGithub
pylint
44 changes: 44 additions & 0 deletions src/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
## GITHUB ARCHIVE CONFIGURATION ##
# Only edit the values of each variable, leaving variable names alone.

######################################################################
## USER CONFIGURATION ##

# Your GitHub token
TOKEN=""

# Enter "enable" to clone personal repos. Default: "enable"
USER_CLONE_ON="enable"

# Enter "enable" to pull personal repos. Default: "enable"
USER_PULL_ON="enable"

# Enter "enable" to clone gists. Default "disable"
GISTS_CLONE_ON="disable"

# Enter "enable" to pull gists. Default "disable"
GISTS_PULL_ON="disable"

######################################################################
## ORGANIZATIONAL CONFIGURATION ##

# Add each org name separated by comma. eg: "org1, org2, org3"
ORGS=""

# Enter "enable" to clone organization repos. Organization names must be entered in the "ORGS" variable below. Default: "disable"
ORGS_CLONE_ON="disable"

# Enter "enable" to pull organization repos. Organization names must be entered in the "ORGS" variable below. Default: "disable"
ORGS_PULL_ON="disable"

######################################################################
## ARCHIVE CONFIGURATION ##

# Number of days logs will be retained for. Default: "30"
LOG_LIFE="30"

# Where your archive will be housed. Default: "~/github-archive"
LOCATION="~/github-archive"

# Which branch each repo will pull from. Default: "master"
BRANCH="master"
Loading

0 comments on commit 9952389

Please sign in to comment.