# gec **Repository Path**: impredicative/gec ## Basic Information - **Project Name**: gec - **Description**: Mirror of github.com/impredicative/gec, a Bash utility for git+gocryptfs - **Primary Language**: Shell - **License**: AGPL-3.0 - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2020-11-08 - **Last Updated**: 2022-08-15 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # gec **`gec`** is a command-line utility written in Bash with convenience commands for using [gocryptfs](https://github.com/rfjakob/gocryptfs) with git. It refrains from doing anything clever, making it possible to naively fallback to the underlying gocryptfs or git commands if a need should arise. It transparently uses data encryption, both at rest and on the remotes. It uses version control and leverages redundant remote storage. The implemented remote commands require GitHub and Bitbucket. Git users will be at home with it. > **:warning: Before continuing, save the link to the official [Gitee mirror](https://gitee.com/impredicative/gec) of this repo.** ## Contents * [Screenshot](#screenshot) * [Links](#links) * [Limitations](#limitations) * [Requirements](#requirements) * [Installation](#installation) * [Development](#development) * [Setup](#setup) * [Directories](#directories) * [Workflow](#workflow) * [Commands](#commands) * [Migration](#migration) ## Screenshot ![](sample.png) ## Links | Caption | Link | |-----------|-----------------------------------------------| | Repo | https://github.com/impredicative/gec | | Changelog | https://github.com/impredicative/gec/releases | | Mirror | https://gitee.com/impredicative/gec | ## Limitations The known applicable size limits for [GitHub](https://stackoverflow.com/a/59479166/) and [Bitbucket](https://stackoverflow.com/a/73244577/) are tabulated below. If a hard limit is violated during a `commit`, `gec` will attempt to check it and error early, otherwise a `push` will fail. | Size of | SI Value | Type | Enforcer | Action by `gec` | |---------|----------|------|-----------|-----------------| | File | 100M | Hard | GitHub | Error | | Push | 2G | Hard | GitHub | Error | | Repo | 5G | Soft | GitHub | (none) | | File | 1G | Hard | Bitbucket | (none) | | Push | 3.5G | Hard | Bitbucket | (none) | | Repo | 4G | Hard | Bitbucket | Error | Due to the use of the gocryptfs `-sharedstorage` option, a hardlink cannot be created in a decrypted repo. ## Requirements Linux is required along with a few tools which are covered in the [**Installation**](#installation) section. A dedicated [GitHub](https://github.com/) account and [Bitbucket](https://bitbucket.org/) account is required with an identical username on both sites! If using Firefox, the [Multi-Account Containers](https://addons.mozilla.org/en-US/firefox/addon/multi-account-containers/) add-on can be very useful to separate them from your primary accounts on these sites. ## Installation These steps were tested on Ubuntu. On other distros, ensure that the executables are available in the PATH. Ensure `curl` ≥7.74.0, `git` ≥2.25.1, [`jq`](https://github.com/stedolan/jq/releases) ≥1.5: ```shell sudo apt install curl git jq ``` Install [`gocryptfs`](https://github.com/rfjakob/gocryptfs/releases) ≥2.2.1: ```shell RELEASE=$(curl https://api.github.com/repos/rfjakob/gocryptfs/releases | jq -r .[0].tag_name) wget -qO- https://github.com/rfjakob/gocryptfs/releases/download/${RELEASE}/gocryptfs_${RELEASE}_linux-static_amd64.tar.gz | sudo tar -xz -f - -C /usr/local/sbin/ gocryptfs gocryptfs-xray ``` Install [`git-sizer`](https://github.com/github/git-sizer/releases) ≥1.3.0: ```shell VERSION=$(curl https://api.github.com/repos/github/git-sizer/releases | jq -r .[0].tag_name | tr -d v) wget -qO- https://github.com/github/git-sizer/releases/download/v${VERSION}/git-sizer-${VERSION}-linux-amd64.zip | sudo busybox unzip - git-sizer -d /usr/local/sbin/ sudo chmod +x /usr/local/sbin/git-sizer ``` Install `gec`: ```shell # Install program RELEASE=$(curl https://api.github.com/repos/impredicative/gec/releases | jq -r .[0].tag_name) sudo wget https://raw.githubusercontent.com/impredicative/gec/${RELEASE}/gec.sh -O /usr/local/sbin/gec sudo chmod +x /usr/local/sbin/gec # Install completion script if using Bash: mkdir -p ~/.local/share/bash-completion/completions wget https://raw.githubusercontent.com/impredicative/gec/${RELEASE}/completion.bash -O ~/.local/share/bash-completion/completions/gec source ~/.local/share/bash-completion/completions/gec # This is automatic if bash-completion is installed, but can do manually for current terminal only. # Install completion script if using Fish: mkdir -p ~/.config/fish/completions wget https://raw.githubusercontent.com/impredicative/gec/${RELEASE}/completion.fish -O ~/.config/fish/completions/gec.fish source ~/.config/fish/completions/gec.fish # This is automatic, but can do manually for current terminal only. ``` For future updates to `gec`, running `gec install` will install its latest release and also its shell completion scripts. ## Development ```shell # Link program git clone git@github.com:impredicative/gec.git cd ./gec sudo ln -s "${PWD}/gec.sh" /usr/local/sbin/gec # Link completion script if using Bash: mkdir -p ~/.local/share/bash-completion/completions ln -s "${PWD}/completion.bash" ~/.local/share/bash-completion/completions/gec source ~/.local/share/bash-completion/completions/gec # This is automatic if bash-completion is installed, but can do manually for current terminal only. # Link completion script if using Fish: mkdir -p ~/.config/fish/completions ln -s "${PWD}/completion.fish" ~/.config/fish/completions/gec.fish source ~/.config/fish/completions/gec.fish # This is automatic, but can do manually for current terminal only. ``` ## Setup In the steps below, `` refers to an identical username in both GitHub and Bitbucket. On each device: 1. Run `gec config core.owner ` once for all future repos. 2. Run `ssh-keygen -f ~/.ssh/id_gec` once to create a new SSH key. Use and securely save a passphrase for this key to minimize the risk of any unauthorized push. 3. Add the `~/.ssh/id_gec.pub` file for the key created above into the `` account in both GitHub and Bitbucket. 4. Create or prepend (not append) to `~/.ssh/config` the specific contents: ```shell Match host github.com,bitbucket.org exec "[[ $(git config user.name) = gec ]]" IdentityFile ~/.ssh/id_gec ``` 5. Run `chmod go-rw ~/.ssh/config` to tighten permissions of the file as is advised in `man ssh_config`. 6. Run `gec test.ssh` to test GitHub and Bitbucket access via SSH, ensuring that the `` name is printed for both. 7. Run `gec test.token` to test GitHub and Bitbucket access via a personal access token for each. The GitHub token must have access to the `repo` and `delete_repo` scopes. The Bitbucket token must have access to the `repository:read`, `repository:admin` and `repository:delete` scopes. ## Directories Storage repos are created in `~/gec/`. This location is created automatically. Both encrypted and decrypted repos and their files are organized in this location. Although this location is not currently configurable, a softlink or hardlink can be used to redirect it elsewhere if needed. For each repo, these directories are created and used: | Location | Description | |-------------------------------|-----------------------------------------------| | `~/gec/encrypted/` | git repo contents | | `~/gec/encrypted//.git` | .git directory of git repo | | `~/gec/encrypted//fs` | encrypted filesystem contents within git repo | | `~/gec/decrypted/` | decrypted filesystem mountpoint | ## Workflow To create and provision a new repo: * `gec init ` * `gec use ` * `touch .Trash-${UID}` (_Avoids deleting files to Trash on Ubuntu_) To provision an existing repo: * `gec clone ` To use a provisioned repo, these are some of the many commands: * `gec pull []` (_If and when changed on remote_) * `gec use []` (_Remember to `exit` the shell after using_) * `gec status []` * `gec done "a non-secret commit message"` (_If files changed_) * `gec umount ` Refer to the [repo-specific commands](#repo-specific) section for details on using the commands in the workflows above. ## Commands ### Repo-agnostic * **`config []`**: Get or set a value of key from configuration file `~/.gec`. To list all values, specify `-l`. * **`install []`**: Update to the named or latest release of `gec`. * **`list`**: Alias of `ls`. * **`ls [pattern]`**: List the output of the `state` command for matching repos in `~/gec/encrypted`. If specifying a pattern, it may need to be quoted. * **`lock`**: Unmount all mounted repos. * **`test.ssh`**: Test access to GitHub and Bitbucket via SSH. * **`test.token`**: Test access to GitHub and Bitbucket via a personal access token for each. ### Repo-specific In the commands below, `` refers to an identical repository name, e.g. "travel-ak", in both GitHub and Bitbucket. It can be auto-determined if a command is run from its encrypted or decrypted directory. When it can be auto-determined, to disambiguate a command's arguments that follow, it can alternatively be specified as a period. The minimally relevant repo-specific commands are listed in the [**Workflow**](#workflow) section. #### Informational * **`check.dec []`**: Check decrypted file sizes. Error if a size limit is exceeded. The repo must be in a mounted state. It is run automatically by `commit` when needed if mounted. * **`check.git []`**: Check encrypted file sizes, and use `git-sizer` to check the size of the git repo. Error if a size limit is exceeded. It is run automatically by `commit` when needed. * **`du []`**: Print the human-friendly disk usage of the git repo directory for a depth of one. * **`du.dec []`**: Print the human-friendly disk usage of the decrypted directory for a depth of one. The repo must be in a mounted state. * **`du.enc []`**: Print the human-friendly disk usage of the encrypted filesystem directory for a depth of one. * **`info []`**: Alias of `status`. * **`log [] [options]`**: Print the git log for the last ten commits. Options, if any, are forwarded to `git log`. If specifying any options, to auto-determine ``, specify a period in its place. * **`logs []`**: Alias of `log`. * **`state []`**: Print the repo mount state, .git directory disk usage, encrypted filesystem directory disk usage, total disk usage, and repo name. * **`status []`**: Print the repo name, mount state, and short git status. If mounted, also print the change status of decrypted paths plus the mount information. #### Remote oriented A [GitHub token](https://github.com/settings/tokens/new) and a [Bitbucket token](https://bitbucket.org/-/profile/personal_access_tokens) are required for these commands. For your security, the tokens are not saved by `gec`. * **`create `**: Idempotently create the repo in GitHub and Bitbucket. The required GitHub token must have access to the `repo` scope. The required Bitbucket token must have access to the `repository:read` and `repository:admin` scopes. * **`del []`**: Delete an existing repo in GitHub and Bitbucket. The required GitHub and Bitbucket tokens must have access to their `delete_repo` and `repository:delete` scopes respectively. Also see the `rm` and `destroy` commands. #### git oriented * **`amend [] [""]`**: Add and amend all changes to most recent commit. If `` is not specified, it is kept unchanged. `` is not encrypted. To auto-determine ``, specify a period in its place. * **`clone `**: Clone and configure a preexisting repo from GitHub into its git repo directory, and add its Bitbucket URL. * **`commit ""`**: Add and commit all changes. `` is not encrypted. To auto-determine ``, specify a period in its place. * **`gc [] [options]`**: Run git garbage collection on the repo. Options, if any, are forwarded to `git gc`. If specifying any options, to auto-determine ``, specify a period in its place. * **`pull []`**: Pull commits from remote. For safety, only a fast-forward pull is made, and a prerequisite is that the repo must be in a dismounted state. * **`push []`**: Push commits to remote. * **`reset.url []`**: Update the locally configured remote URLs for the repo to the expected ones for GitHub and Bitbucket, removing all other URLs. * **`send ""`**: (`commit`+`push`) Add, commit, and push all changes. `` is not encrypted. To auto-determine ``, specify a period in its place. Also see the `done` command. #### gocryptfs oriented * **`dismount`**: Alias of `umount`. * **`init.fs []`**: Initialize the encrypted filesystem for an empty repo. No commit or push is made. A new password is requested. The password and a printed master key must be securely saved. * **`mount []`**: Mount a repo in read-write mode into its decrypted mountpoint if not already mounted as such. Also see the `use` command. * **`mount.ro []`**: Mount a repo in read-only mode into its decrypted mountpoint if not already mounted as such. Also see the `use.ro` command. * **`mount.rw`**: Alias of `mount`. Also see the `use.rw` command. * **`umount [] [-f]`**: Unmount a repo if it is mounted. To force an unmount, specify `-f`. * **`unmount`**: Alias of `umount`. #### System * **`rm `**: Interactively remove all local directories of the repo. Also see the `del` and `destroy` commands. * **`shell []`**: Provide a shell into the git repo directory. * **`shell.dec []`**: Provide a shell into the decrypted mountpoint of a mounted repo. * **`shell.enc []`**: Provide a shell into the encrypted filesystem directory. #### Compound * **`init `**: (`create`+`clone`+`init.fs`+`send`) Create new repo remotely, clone it locally, initialize encrypted filesystem, commit, and push. A new password is requested. The password and a printed master key must be securely saved. The required GitHub token must have access to the `repo` scope. The required Bitbucket token must have access to the `repository:read` and `repository:admin` scopes. * **`destroy `**: (`rm`+`del`) Interactively remove all local repo directories, and delete repo from GitHub and Bitbucket. * **`done ""`**: (`check.dec`+`umount`+`send`) Add, commit, and push all changes. Unmount repo if mounted. `` is not encrypted. To auto-determine ``, specify a period in its place. * **`rename `**: Rename repo remotely and locally, and update its locally configured remotes. The repo must be in a dismounted state. The new name must not already be in use. The required GitHub and Bitbucket tokens must have access to their `repo` and `repository:admin` scopes respectively. After renaming, remember to store the password for the repo under its new name. * **`use []`**: (`mount`+`shell.dec`) Mount read-write if not already mounted as such, and provide a shell into the decrypted mountpoint. * **`use.ro []`**: (`mount.ro`+`shell.dec`) Mount read-only if not already mounted as such, and provide a shell into the decrypted mountpoint. * **`use.rw`**: Alias of `use`. ## Migration If updating from v0.4, first verify the [Setup](#setup) steps. Next, for each repo: 1. If it exists remotely but not locally, obtain it using `gec clone `. 2. Check its repo size as printed by `gec check.git `. 3. If the repo size is ≤4.0G, run this sequence once: ```shell gec shell # Enters repo shell gec create # Creates repo in Bitbucket. gec reset.url # Adds Bitbucket remote and removes obsolete remote. gec push # Pushes to Bitbucket. exit # Exits repo shell. ``` 4. If `gec push` fails with the push size exceeding 2G, run an alternative command instead such as the one below to push commits individually. At this time, ignore errors about failing to push commits to GitHub. ```shell git log --format=%H --reverse | xargs -i git push origin {}:refs/heads/$(git branch --show-current) ``` 5. If the repo size is >4.0G, it cannot be migrated. One or more new repos can however be created to replace it. In each new repo, limit each push to a max of 2G.