Skip to content

Commit

Permalink
v2.1.0 - closes #9, closes #11
Browse files Browse the repository at this point in the history
  • Loading branch information
Justintime50 committed Jul 2, 2020
1 parent a328d09 commit 2c57d36
Show file tree
Hide file tree
Showing 11 changed files with 218 additions and 239 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,7 @@
.idea
logs
.env
*.egg-info
build
dist
__pycache__
6 changes: 3 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ python:
- 3.8

install:
- pip install -r requirements.txt
- pip3 install -e ."[dev]"

script:
- shellcheck src/legacy/github-archive.sh
- pylint src/github_archive.py
- shellcheck legacy/github-archive.sh
- pylint githubarchive/*.py
15 changes: 13 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,24 @@
# CHANGELOG

## 2.1.0 (2020-07-01)

* Replaced `requirements.txt` with `setup.py`
* Removed Launch Agent
* Set the default git pull behavior as `--ff-only` to avoid git message (closes #)
* Increased default timeout for git operations from `120` seconds to `180` seconds
* Replaced the majority of environment variables with command line args making customization easier as "feature flags"
* Various bug fixes and minor improvements
* Removed log deletion functionality
* Published to Pypi

## 2.0.1 (2020-05-30)

* Fixed the Python program line in the launch agent plist

## 2.0.0 (2020-05-18)

* Rewrote the script in Python.
* Added concurrency to clone/pull changes incredibly fast for large amounts of repos and gists.
* Rewrote the script in Python
* Added concurrency to clone/pull changes incredibly fast for large amounts of repos and gists
* Added even more custimization and control
* Uniform log naming, better logging details
* Added changelog
Expand Down
71 changes: 32 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,29 +21,24 @@ A powerful script to concurrently clone your entire GitHub instance or save it a

### Configurable Settings

The power of GitHub Archive comes in its configuration. Maybe you only want to clone/pull your personal public repos or maybe you want to go all out and include private repos from you and all organizations you belong to including your gists. Customize the location repos are saved to, how long logs are kept for. Iterate over 100's of repos concurrently and sit back while GitHub Archive does all the work.
The power of GitHub Archive comes in its configuration. Maybe you only want to clone/pull your personal public repos or maybe you want to go all out and include private repos from you and all organizations you belong to including your gists. Iterate over all your repos concurrently and sit back while GitHub Archive does the work.

- Personal repos (on/off)
- Organization repos (on/off)
- Personal Gists (on/off)
- Cloning (on/off)
- Pulling (on/off)
- What organizations you'd like included
- Log retention life
- GitHub Archive location
- Which branch to pull from
- Which branch to clone/pull from

## Install

```bash
# Install dependencies
pip3 install -r requirements.txt

# Copy the environment file and edit for your needs.
cp .env.example .env
pip3 install github-archive
```

**For Private Repos:** You must have an SSH key generated on your local machine and added to your GitHub account.
**For Private Repos:** You must have an SSH key generated on your local machine and added to your GitHub account as this tool uses the `ssh_url` to clone/pull.

### Automating SSH Passphrase Prompt (Recommended)

Expand All @@ -56,48 +51,46 @@ ssh-add

## Usage

GitHub Archive will clone any repo and gist that doesn't exist locally and pull those that do from the master branch of each repo and latest revision of each gist that you have access to - including organizations (if configured). You can run the script once, add an alias, or have it setup with a cron or Launch Agent and run occasionally to clone/pull any changes since it was last run.

**Merge Conflicts:** *Be aware that using GitHub Archive could lead to merge conflicts if you continually pull the same repos you work on without stashing or committing your changes. It is recommended to be used once for example on a new machine or setup as a separate archive from your development repositories. If you use GitHub Archive to pull in nighly changes from various repos, you should be religious about stashing or committing your changes or you will receive merge conflicts and the script may not complete running.*
GitHub Archive will clone any repo and gist that doesn't exist locally and pull those that do from the master branch of each repo and latest revision of each gist that you have access to - including organizations (if configured). **Merge Conflicts:** *Be aware that using GitHub Archive could lead to merge conflicts if you do not commit or stash your changes if using these repos as active development repos instead of simply an archive or one-time clone.*

### Run Script
By default, you only need to specify your GitHub token and GitHub Archive will clone/pull your personal repos for you. See below for customization options.

```bash
python3 github_archive.py
```

### Shell Alias

```bash
# If using Bash insted of ZSH, use ~/.bash_profile
echo alias github_archive="/path/to/github_archive.py" >> ~/.zshrc
source ~/.zshrc

# Usage of alias
github_archive
Basic Usage:
GITHUB_ARCHIVE_TOKEN=123... github_archive --user-clone --user-pull
Advanced Usage:
GITHUB_ARCHIVE_TOKEN=123... GITHUB_ARCHIVE_ORGS="org1, org2" GITHUB_ARCHIVE_LOCATION="~/custom_location" \
github_archive -uc -up -gc -gp -oc -op -b develop
Options:
-uc, --user-clone Clone personal repos (default: on)
-up, --user-pull Pull personal repos (default: on)
-gc, --gists-clone Clone personal gists (default: off)
-gp, --gists-pull Pull personal gists (default: off)
-oc, --orgs-clone Clone organization repos (default: off)
-op, --orgs-pull Pull organization repos (defaulf: off)
-b, --branch The branch to clone/pull from (default "master")
-h, --help Show usage info on for this tool
Environment variables
GITHUB_ARCHIVE_TOKEN - expects a string
GITHUB_ARCHIVE_ORGS - expects a string of comma separated orgs. eg: "org1, org2"
GITHUB_ARCHIVE_LOCATION - expects a string of an explicit location on your machine. eg: "~/custom_location"
```

### Launch Agent (Recommended on macOS)
## Development

Edit the path in the `plist` file to your script and logs as well as the time to execute, then setup the Launch Agent:
Install project with dev depencencies:

```bash
# Copy the plist to the Launch Agent directory
cp local.githubArchive.plist ~/Library/LaunchAgents

# Use `load/unload` to add/remove the script as a Launch Agent
launchctl load ~/Library/LaunchAgents/local.githubArchive.plist

# To `start/stop` the script from running, use the following
launchctl start local.githubArchive.plist
pip3 install -e ."[dev]"
```

### Cron
Lint the project:

```bash
crontab -e

0 1 * * * /path/to/github_archive.py
pylint githubarchive/*.py
```

## Legacy Script
Expand Down
Loading

0 comments on commit 2c57d36

Please sign in to comment.