From b6719da85c7bf4247d50b1cfea79ffa4b99d80a2 Mon Sep 17 00:00:00 2001 From: "Ryan D. Najac" Date: Wed, 11 Sep 2024 12:41:08 -0400 Subject: [PATCH] reorder docs --- docs/AWS/EC2.md | 70 ++++++++++++ docs/AWS/index.md | 105 +++++------------- docs/{Software => }/Packages.md | 0 docs/{Software => Protocols}/Jupyter.md | 0 docs/{Scripting => Software}/Bash.md | 0 docs/Software/Docker.md | 43 ++++++- docs/{Scripting => Software}/perl.md | 0 docs/{Tools => Software}/samtools.md | 0 docs/about.md | 1 - docs/{Tools/Illumina.md => illumina.md} | 2 +- docs/index.md | 2 +- .../{Tools/Linux-Command-Line.md => linux.md} | 0 12 files changed, 138 insertions(+), 85 deletions(-) rename docs/{Software => }/Packages.md (100%) rename docs/{Software => Protocols}/Jupyter.md (100%) rename docs/{Scripting => Software}/Bash.md (100%) rename docs/{Scripting => Software}/perl.md (100%) rename docs/{Tools => Software}/samtools.md (100%) rename docs/{Tools/Illumina.md => illumina.md} (99%) rename docs/{Tools/Linux-Command-Line.md => linux.md} (100%) diff --git a/docs/AWS/EC2.md b/docs/AWS/EC2.md index 2d58b51..8eba6d0 100644 --- a/docs/AWS/EC2.md +++ b/docs/AWS/EC2.md @@ -26,3 +26,73 @@ Click on the `Connect`, then `EC2 serial console` tab, and `Connect` again. Once connected, you can interact with the instance as if you were using a terminal. If a login prompt does not appear, try pressing `Enter`. + +## CLI Tools for interacting with EC2 instances + +### `ssh` + +edit `~/.ssh/config` + +- add the location of the private key file so you don't have to specify it with `-i` each time you connect +- add the `User` and `Hostname` fields so you don't have to specify them each time + +```sh +# example ~/.ssh/config +Host aws + Hostname ec2-1-234-5-6.compute-1.amazonaws.com + User ubuntu + IdentityFile ~/.ssh/aws.pem +``` + +### `scp` + +- "secure copy" files between computers using `ssh` +- this means the config from `~/.ssh/config` is used + +```sh +# copy file to home dir of remote aws instance +scp localfile.txt aws:~ + +# copy remote file to pwd +scp aws:~/remotefile.txt . +``` + +### `rsync` + +- useful for keeping all files in a directory up-to-date + +```sh +# copy all files in a directory to a remote server +rsync -avz --progress /path/to/local/dir/ aws:/path/to/remote/dir/ +``` + +### `tmux` + +Weird stuff can happen with "nested" sessions over `ssh`. +If you want to attach to a tmux session on a remote server, +you need to use the `-t` flag since `tmux` is not a login shell. + +```sh +ssh aws # works +ssh aws tmux a # huh? +ssh aws -t tmux a # ok +``` + +### `vim` + +Once you have ssh configured, you can use vim to edit files remotely thanks to +the `netrw` plugin that comes shipped with `vim`. + +```sh +vim scp://aws/remote/path/to/file +``` + +Copy current vim buffer to remote server + +```vim +:!scp % aws:~/path/to/remote/file +``` + +Inside vim, you can use the `:Explore` command to browse the remote server. +Read more about it with `:h netrw` from inside vim. + diff --git a/docs/AWS/index.md b/docs/AWS/index.md index de218a7..0a31266 100644 --- a/docs/AWS/index.md +++ b/docs/AWS/index.md @@ -1,105 +1,50 @@ # Amazon Web Services -Command Line Interface References: +This page covers setting up the command-line interface for AWS services. +For more information on AWS services, see one of these pages: -- [AWS S3 CLI](https://docs.aws.amazon.com/cli/latest/reference/s3/index.html) -- [AWS EC2 CLI](https://docs.aws.amazon.com/cli/latest/reference/ec2/index.html) -- [AWS IAM CLI](https://docs.aws.amazon.com/cli/latest/reference/iam/index.html) -- [AWS Glacier CLI](https://docs.aws.amazon.com/cli/latest/reference/glacier/index.html) +- [S3](s3.md) +- [EC2](ec2.md) +- [IAM](iam.md) ## Installation Fresh Ubuntu instances require [installation](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html): ```sh -#!/bin/bash -# -## Install AWS CLI v2 curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" -unzip awscliv2.zip && sudo ./aws/install - -# cleanup -rm -rf awscliv2.zip ./aws - -# add completion to your shell -complete -C '/usr/local/bin/aws_completer' aws -# on macOS, add the following to your .zshrc: -# autoload bashcompinit && bashcompinit -# autoload -Uz compinit && compinit - -# configure credentials; use the ones from the IAM user you created -# AWS Access Key ID [None]: -# AWS Secret Access Key [None]: -# Default region name [None]: us-east-1 -# Default output format [None]: [json|text|table] -aws configure -``` - -## Other CLI Tools - -### `ssh` - -edit `~/.ssh/config` +unzip awscliv2.zip && sudo ./aws/install && rm -rf ./awscliv2.zip ./aws +``` -- add the location of the private key file so you don't have to specify it with `-i` each time you connect -- add the `User` and `Hostname` fields so you don't have to specify them each time +Next, add tab completion to your shell: ```sh -# example ~/.ssh/config -Host aws - Hostname ec2-1-234-5-6.compute-1.amazonaws.com - User ubuntu - IdentityFile ~/.ssh/aws.pem -``` - -### `scp` - -- "secure copy" files between computers using `ssh` -- this means the config from `~/.ssh/config` is used - -```sh -# copy file to home dir of remote aws instance -scp localfile.txt aws:~ - -# copy remote file to pwd -scp aws:~/remotefile.txt . +complete -C '/usr/local/bin/aws_completer' aws ``` -### `rsync` - -- useful for keeping all files in a directory up-to-date +If you're using `zsh` (the shell on macOS), add the following to your `.zshrc` +so that the completion script is loaded when you start a new shell: ```sh -# copy all files in a directory to a remote server -rsync -avz --progress /path/to/local/dir/ aws:/path/to/remote/dir/ +autoload bashcompinit && bashcompinit +autoload -Uz compinit && compinit ``` -### `tmux` - -Weird stuff can happen with "nested" sessions over `ssh`. -If you want to attach to a tmux session on a remote server, -you need to use the `-t` flag since `tmux` is not a login shell. +Finally, configure your credentials: ```sh -ssh aws # works -ssh aws tmux a # huh? -ssh aws -t tmux a # ok -``` - -### `vim` - -Once you have ssh configured, you can use vim to edit files remotely thanks to -the `netrw` plugin that comes shipped with `vim`. - -```sh -vim scp://aws/remote/path/to/file +aws configure ``` -Copy current vim buffer to remote server +- AWS Access Key ID [None]: +- AWS Secret Access Key [None]: +- Default region name [None]: us-east-1 +- Default output format [None]: [json|text|table] -```vim -:!scp % aws:~/path/to/remote/file -``` +## Further Reading +Command Line Interface References: -Inside vim, you can use the `:Explore` command to browse the remote server. -Read more about it with `:h netrw` from inside vim. +- [AWS S3 CLI](https://docs.aws.amazon.com/cli/latest/reference/s3/index.html) +- [AWS EC2 CLI](https://docs.aws.amazon.com/cli/latest/reference/ec2/index.html) +- [AWS IAM CLI](https://docs.aws.amazon.com/cli/latest/reference/iam/index.html) +- [AWS Glacier CLI](https://docs.aws.amazon.com/cli/latest/reference/glacier/index.html) diff --git a/docs/Software/Packages.md b/docs/Packages.md similarity index 100% rename from docs/Software/Packages.md rename to docs/Packages.md diff --git a/docs/Software/Jupyter.md b/docs/Protocols/Jupyter.md similarity index 100% rename from docs/Software/Jupyter.md rename to docs/Protocols/Jupyter.md diff --git a/docs/Scripting/Bash.md b/docs/Software/Bash.md similarity index 100% rename from docs/Scripting/Bash.md rename to docs/Software/Bash.md diff --git a/docs/Software/Docker.md b/docs/Software/Docker.md index 54d493e..e6ed2e3 100644 --- a/docs/Software/Docker.md +++ b/docs/Software/Docker.md @@ -1,13 +1,52 @@ # Docker -## Build on local machine (amd64) +Docker is a platform for developing, shipping, and running applications in +containers. It enables developers to package an application with all of its +dependencies into a standardized unit for software development. Docker +containers are lightweight and contain everything needed to run the application, +including code, runtime, system tools, system libraries, and settings. + +There are two installations: + +- Docker Desktop on Linux +- Docker Engine + +Docker Desktop is the easiest way to get started with Docker on Linux. +[Source](https://docs.docker.com/desktop/install/linux/). + +## Installation + +> [!CAUTION] +> Always check the [official Docker documentation](https://docs.docker.com/engine/install/ubuntu/) for the most +> up-to-date instructions. The following steps are for Ubuntu 22.04 + + +First, remove any old or unofficial versions of Docker. + +```sh +for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done +``` + +### Ubuntu installation + +First, [set up Docker's `apt` repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository). + +[Then...](https://docs.docker.com/desktop/install/linux/ubuntu/#install-docker-desktop) + +> [!NOTE] +> I had to enter the BIOS and enable virtualization to get Docker Desktop to work on my machine. + + +## Docker Engine Examples + +### Build on local machine (amd64) ```sh docker buildx create --use docker buildx build --platform linux/amd64 -t rdn2108/cbmf:v1.0 . ``` -## Push to Docker Hub +### Push to Docker Hub ```sh docker buildx build --platform linux/amd64 -t rdn2108/cbmf:v1.0 . --push diff --git a/docs/Scripting/perl.md b/docs/Software/perl.md similarity index 100% rename from docs/Scripting/perl.md rename to docs/Software/perl.md diff --git a/docs/Tools/samtools.md b/docs/Software/samtools.md similarity index 100% rename from docs/Tools/samtools.md rename to docs/Software/samtools.md diff --git a/docs/about.md b/docs/about.md index dd41c68..4601e62 100644 --- a/docs/about.md +++ b/docs/about.md @@ -13,4 +13,3 @@ is a collaborative effort and what good does documentation do if it's not These files are written using [GitHub Flavored Markdown \(GFM\)](https://github.github.com/gfm/), a superset of the [original](https://daringfireball.net/projects/markdown/syntax) lightweight markup language with plain text formatting syntax. -Try editing this page by clicking on the link in the top right corner! diff --git a/docs/Tools/Illumina.md b/docs/illumina.md similarity index 99% rename from docs/Tools/Illumina.md rename to docs/illumina.md index a5a6a65..176de33 100644 --- a/docs/Tools/Illumina.md +++ b/docs/illumina.md @@ -1,4 +1,4 @@ -# Illumina +# Illumina Sequencing If you used a core facility, Azenta, or another commercial service for sequencing, they will send link to directly download the de-multiplexed diff --git a/docs/index.md b/docs/index.md index aa1d617..75a12e8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -26,7 +26,7 @@ echo "Hello, world!" > [!TIP] > If you get `zsh: command not found: #` when you try to run a copied command, -> run `setopt interactivecomments` first. Read about why +> run `setopt interactivecomments` first. Read about why this happens > [here](https://stackoverflow.com/a/11873793/26469286). ## Contributing diff --git a/docs/Tools/Linux-Command-Line.md b/docs/linux.md similarity index 100% rename from docs/Tools/Linux-Command-Line.md rename to docs/linux.md