Setup Modern Command Line Tools

here’s a quick guide for setting up modern command line environments on a fresh macOS Monterey 12+ as of late 2021.

Enjoy.

stop saving screenshots directly to destop

We no longer need to defaults write com.apple.screencapture to stop all screenshots going to ~/Desktop/ by default. Instead, just do a regular CMD+SHIFT+5 then click the lower screen toolbar Options menu, and select a better directory for all future screen captures!

install homebrew:

/bin/bash -c \
    "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# then run the commands output at the end to insert `brew` into your shell path

install modern tools:

# Fix macOS awful built in tools they refuse to update because they are a
# multi-trillion dollar company refusing to confront modern licenses or
# just outright pay for public use of software written by individuals.
# mac /bin/ls is awful, so coreutils ls is better (installed as 'gls').
# mac rsync stopped updating in 2006(!) so install the latest version manually.
# install modern langs/dependencies/utils/extensions for multiple needs too.
brew install zsh-autosuggestions zsh-completions bash bash-completion@2 tmux coreutils fswatch
brew install ripgrep neovim bat fzf fd jq git awk yt-dlp diff-so-fancy exiftool autossh rename
brew install openblas cmake libxml2 apache-arrow openssl ffmpeg pandoc kerl aspell
brew install awscli nmap rsync sassc watch
brew install dnsmasq htop wget
brew install ansible cowsay pv

# instead of homebrew pyenv, you may want to clone `pyenv` from the main repo
# because `pyenv` often adds new release spec files to their repo weeks before cutting
# new official releases for homebrew. They should eventually refactor releases
# to be independent from the code releases themselves, right?
# You can also hack around homebrew delayed updates by finding the version file from
# the main `pyenv` repo and copying the spec into homebrew dir:
# /opt/homebrew/opt/pyenv/plugins/python-build/share/python-build
brew install zig rust lua luarocks pyenv npm rebar3 perl zlib ctags

# fix autocomplete permission to not warn on startup
chmod -R go-w '/opt/homebrew/share/zsh'
chmod -R 755 /opt/homebrew/share

# install pyenv shell environment helpers
echo 'eval "$(pyenv init --path)"' >> ~/.zprofile
echo 'eval "$(pyenv init -)"' >> ~/.zshrc

# reload settings into current console
source ~/.zshrc ~/.zprofile

configure things good

# install good helpers for things
npm install -g uglify-js 

# install python versions you like to run:
pyenv install 3.12.0
pyenv global 3.12.0

# configure git properly along with good helpers
git config --global user.name "Matt Stancliff"
git config --global user.email "matt@genges.com"
git config --global user.useConfigOnly true

# tell git to use symlinks if repo has symlinks (otherwise it DUPLICATES THE FILE ugh)
git config --global core.symlinks true

# put all your dirty files in here (.DS_Store, tags, __pycache__, etc)
git config --global core.excludesfile ~/.gitignore_global

# recent git releases enabled auto-pager for "git branch" which is hella'nnoying
git config --global pager.branch false
git config --global init.defaultBranch main

# have rebase auto-rebase any other branches with the same root
git config --global rebase.updateRefs true

# stop git from dropping commit messages because the editor return code isn't stable
git config --global core.editor nvim -f

git config --global alias.llog "log --pretty=fuller"
git config --global alias.logs "log --graph --pretty=format:'%C(magenta)%h%Creset -%C(red)%d%Creset %s %C(dim green)(%cr) %C(cyan)<%an>%Creset' --abbrev-commit"
git config --global alias.rl "reflog --format='%C(auto)%h %<|(20)%gd %C(blue)%cr%C(reset) %gs (%s)'"

git config --global alias.tagdate 'log --date-order --graph --tags --simplify-by-decoration --pretty=format:"%ai %h %d"'
git config --global alias.branchbydate "for-each-ref --count=30 --sort=-committerdate refs/heads/ --format='%(refname:short)'"
git config --global alias.branchcolor '!for ref in $(git for-each-ref --sort=-committerdate --format="%(refname)" refs/heads/ refs/remotes ); do git log -n1 $ref --pretty=format:"%Cgreen%cr%Creset %C(yellow)%d%Creset %C(bold blue)<%an>%Creset%n" | cat ; done | awk '"'! a["'$0'"]++'"

# short for "github-add". usage: git gha [github user] [reponame]
git config --global alias.gha '!f() { git remote add $1 https://github.com/$1/$2.git; }; f'

# short for "git pull request". usage: git gpr [github issue-id] [local name]
git config --global alias.gpr '!f() { git fetch origin pull/$1/head:$2; }; f'

git config --global core.pager "diff-so-fancy | bat -p"
git config --global interactive.diffFilter "diff-so-fancy --patch"

# automatically push tags on regular push
git config --global push.followTags true

# Note: if you don't like the light color shading from these defaults,
#       edit ~/.gitconfig and remove section: [color "diff-highlight"]
diff-so-fancy --set-defaults

setup global python dev/experiment environment

# Install some global libraries for general use outside of full venvs
# (will have to re-do this when you change your `pyenv global` versions; or you
#  could keep it all in a venv too, but it's just for quick console testing of random things)
pip install pip ipython pandas numpy orjson poetry pendulum visidata litecli jupyterlab black ruff ufmt -U

# Give your default ipython environment good imports automatically:
mkdir -p ~/.ipython/profile_default/startup && nvim ~/.ipython/profile_default/startup/00-startup.py

import importlib

import numpy as np
import pandas as pd

import io
import os
import sys
import time
import orjson
import asyncio
import datetime
import pendulum as p

from pathlib import Path
from dataclasses import dataclass, field
from collections import defaultdict, Counter

configure optimal DNS performance, in /opt/homebrew/etc/dnsmasq.conf add:

server=8.8.8.8
server=4.2.2.1
server=4.2.2.2
server=4.2.2.3
server=4.2.2.4
server=4.2.2.5
server=4.2.2.6
server=8.8.4.4

# Can also block specific hostnames and subdomains from resolving:
# '#' appended means forward to default servers (i.e. skip blocking).
# no characters after '/' means return NXDOMAIN for the entire *.x.tld (unless manually excepted)
# After changing your config: sudo brew services restart dnsmasq
server=/api.twitter.com/#
server=/tpop-api.twitter.com/#
server=/twitter.com/
server=/ycombinator.com/
server=/old.reddit.com/#
server=/reddit.com/
server=/redd.it/
server=/facebook.com/
server=/fb.com/
server=/meta.com/
server=/openai.com/

brew edit dnsmasq and change service launch to (may need to monitor updates and re-change if versions flip underneath you):

service do
  run [opt_sbin/"dnsmasq", "--all-servers", "--keep-in-foreground", "-C", etc/"dnsmasq.conf", "-7", etc/"dnsmasq.d,*.conf"]
  keep_alive true
end

start local dns resolver with performance and content fixes

sudo brew services start dnsmasq -v

then go to network settings and fix your DNS server to 127.0.0.1

flush all DNS caches:

sudo killall -HUP mDNSResponder
sudo killall mDNSResponderHelper
sudo dscacheutil -flushcache

# for your browsers, make sure to edit the configs
# and disable dns-over-https (if you don't care about it)
# then flush the browser caches too.
# firefox DNS cache is hiding under:
# about:networking#dns

# also disable dns-over-https using checkbox inside the
# "Network Settings" panel at the bottom of:
# about:preferences#general

# also in about:config search for extensions.pocket.enabled and disable it.

set up neovim for modern editing:

git clone --depth 1 \
    https://github.com/wbthomason/packer.nvim \
    ~/.local/share/nvim/site/pack/packer/start/packer.nvim

configure neovim with modern plugins

mkdir -p ~/.config/nvim/lua/ && nvim ~/.config/nvim/lua/plugins.lua

return require('packer').startup(function()
  -- Packer can manage itself
  use 'wbthomason/packer.nvim'

  use {'ms-jpq/coq_nvim', branch = 'coq'}
  use {'ms-jpq/coq.artifacts', branch = 'artifacts'}
  use {'ms-jpq/coq.thirdparty', branch = '3p'}
end)

tell neovim to load plugins on startup in ~/.config/nvim/init.vim

lua require('plugins')
let g:coq_settings = { 'auto_start': 'shut-up', 'display.icons.mode': 'none' }

set nocompatible            " disable compatibility to old-time vi
set showmatch               " show matching 
set ignorecase              " case insensitive 
set hlsearch                " highlight search 
set incsearch               " incremental search
set tabstop=4               " number of columns occupied by a tab 
set softtabstop=4           " see multiple spaces as tabstops so <BS> does the right thing
set expandtab               " converts tabs to white space
set shiftwidth=4            " width for autoindents
set autoindent              " indent a new line the same amount as the line just typed
set wildmode=longest,list   " get bash-like tab completions
filetype plugin indent on   " allow auto-indenting depending on file type
syntax on                   " syntax highlighting
filetype plugin on
set ttyfast                 " Speed up scrolling in Vim

run in nvim after the files are in place (may require run/exit/run again):

:COQdeps
:PackerUpdate

Install good shell defaults and auto-helper scripts

sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

~/.zshrc updates:

# many themes, but my requirements are:
# prompt must include hostname
# prompt must include full path (not just current directory)
# any metadata like current branches/repos in prompt must not make the prompt line "too long"
# no date/time in prompt
ZSH_THEME="kphoen"
EDITOR=nvim
VISUAL=nvim
PAGER=bat

# zsh has an awful tab completion bug unless this is defined
LC_ALL=en_US.UTF-8
LANG=en_US.UTF-8

# tab complete the entire first suggestion by default instead
# of stalling out and requiring a second tab for "menu select"
setopt menu_complete

alias ls="gls --color=auto"
alias moretty="sudo sysctl -w kern.tty.ptmx_max=768"
alias nolo='exiftool -GPSLongitude= -GPSLatitude='
alias po=poetry
alias goodnight='pmset displaysleepnow'
alias pi='echo "scale=60; 4*a(1)" | bc -lq'
alias vim=nvim

# stop warnings from autocomplete files having weird permissions
ZSH_DISABLE_COMPFIX=true

# completions from brew zsh-completions
if type brew &>/dev/null; then
  FPATH=$(brew --prefix)/share/zsh-completions:$FPATH

  autoload -Uz compinit
  compinit
fi



# extract frames from a video to individual image files.
# usage: toImgs [movie path] [prefix for output files]
function toImgs() {
    # the "-r 7" says dump 7 images per second of video.
    # You can increase or decrease 'r N' as necessary.
    ffmpeg -i "$1" -r 7 -qscale:v 2 "$2"-%0000d.jpg
}

# add a console "note" command to open a markdown file in the
# current directory named with the current date and any suffix
# provided as arguments to the command.
# usage: note shopping list
function note() {
    note_name=""
    if [[ ! -z "$*" ]]; then
        # Use all arguments as trailing name for note
        # (spaces replaced by dashes)
        note_name="-${*// /-}"
    fi
    nvim $(date "+%Y-%m-%d")$note_name.md
}

# create directory and jump into it with one command
function ccd() {
    mkdir -p "$1"
    cd "$1"
}

# get the binary log of a number
# if you like using 'l' as the default 'ls' shorthand, don't do this i guess.
unalias l
function l()  {
     echo "scale=10; l($1)/l(2)" | bc -lq
}

# do a 2^x
function 2x()  {
    echo "2^$1" | bc -lq                 
}

# do a e^x
function e()  {
    echo "e($1)" | bc -lq                 
}

edit kphoen theme ~/.oh-my-zsh/themes/kphoen.zsh-theme for more compactness:

PROMPT='%{$fg[red]%}%n%{$reset_color%}@%{$fg[magenta]%}%m%{$reset_color%}:%{$fg[blue]%}%~%{$reset_color%}$ '
RPROMPT='${return_code}$(git_prompt_info)$(git_prompt_status)%{$reset_color%}'

try to pre-trigger all the macOS directory protection warnings up front

In your home directory, run “find .” or “fd > /dev/null” and accept all OS warnings about allowing Terminal to access your directories so future usage doesn’t unexpectedely encounter errors. You’ll also get an OS warning if you access any remote disks and also local disks in /Volumes/ from a terminal session too.

Install better terminal fonts:

# clone
git clone https://github.com/powerline/fonts.git --depth=1
# install
cd fonts
./install.sh
# clean-up a bit
cd ..
rm -rf fonts

# Good fonts are "Liberation Mono Powerline" and "Meslo LG S for Powerline"
# other fonts are bad
# do not use other fonts
# you have been warned

Other recent setup problems:

many python packages still don’t have arm64/M1 binaries yet, so pip/poetry will try to build from source and we need extra arguments for the bulids to find our homebrew locations:

compiling some python packages (scipy specifically) requires:

LDFLAGS="-L/opt/homebrew/opt/openblas/lib"
CPPFLAGS="-I/opt/homebrew/opt/openblas/include"
PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig"
OPENBLAS=/opt/homebrew/opt/openblas/lib/

You can also currently install a scipy 1.8.0 prerelease arm binary package, but arm binary packages don’t exist for current fully released versions (but manually installing custom binaries from other sources breaks poetry deps in a dozen different ways too; best to wait for a new official release to get projects working again).

python package numba is limited to llvm10 which homebrew won’t provide for us, so we need to build it manually: download llvm10, build, then provide extra build environment options where needed:

mkdir -p ~/build
cd ~/build
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.1/llvm-10.0.1.src.tar.xz
tar xfvp llvm*
cd llvm*.src
mkdir build
cd build
cmake ..
make -j12

oh, and of course there’s also a bug with LLVM 10 they never fixed where it uses a bad libxml2 link, so you need to make sure you have libxml2 installed from brew then:

cd /opt/homebrew/opt/libxml2/lib
ln -sf libxml2.2.dylib liblibxml2.tbd.dylib

Now you have llvm-config at ~/build/llvm-10.0.1.src/bin/llvm-config and you can re-try your failed numba dependency building/installing with:

OPENSSL_ROOT_DIR=/opt/homebrew/opt/openssl LLVM_CONFIG=~/build/llvm-10.0.1.src/build/bin/llvm-config pip install numba

erlang versions

# Note: due to the arm switch, many old erlang versions can't
# be compiled natively on all systems anymore (modern ones work everywhere).
#
# If you need an old erlang version, you can copy a working kerl-installed
# directory from an intel system to your arm mac, then Rosetta2 will do
# the right things (but some shared libs like openssl will still fail, so hope you
# weren't doing erl crypto stuck on old versions).

# Attempting to hack around the "old versions don't compile on arm" problem by
# cross-compiling doesn't seem to work because the erlang build system is
# fairly jank and is still fairly poorly structured autoconf, so the basic
# attempt of this doesn't work:
# # CC="clang --target=x86_64-apple-darwin" kerl build 17.5

kerl build 24.1.6
kerl install ~/kerl/24.1.6
kerl status
kerl cleanup all
source ~/kerl/24.1.6/activate