A Review of ZSH Completion
Prologue
After many years of research, I’ve finally come to an important conclusion:
Writing completion scripts sucks.
In consideration of this, I have organized these sections in order of greatest importance to the topic (excepting this first section), so the devoted reader can stop reading this article at whatever point you lose interest.
Like me, you probably began using ZSH as a result of increasingly rabid advocacy stirring your soul into dissatisfaction with the inflexible completion system of other shells (c.f bash).
At that time, ZSH’s comprehensive documentation was the best source of information on the topic. ZSH’s man pages are written in encyclopedic reference style while also containing many digressions, anecdotes and ’end notes’ (some of which have footnotes themselves) in a medium generally lacking both. This provides the manpages a unique distinction for a technical document - that of supporting range of interpretations and the creation of a robust semi-scholarly enterprise of interpreting the “true” meaning of the documentation.
Since then, the ZSH guide (a separate document) has been updated with details of the new completion system. Although it contains some of the structural details required for quickly writing correct completion scripts, completion details common to many utilities aren’t included.
In response, I’ve tried to compile some principles, mailing-list wisdom and other disambiguation for this hairy topic.
The structure of standard argument completion.
ZSH’s completion system is contextual, sensitive to the point at which it is invoked, consequently a common pattern is to progressively backtrack to current point, determining what options and what subcommands have been set thus far, pruning the selection of completion options.
If you’ve never written completion scripts before, you might not yet have been exposed to the ultimate structure of option parsing that completion scripts (generally) successfully distill. It will be a profitable use of your time to get a high-level view of completion script’s structure unintentionally defy it and spend time working against, rather than with the grain of the system.
- An entry point (e.g
git
) will processes the command line string. - A function for each subcommand such as
diff
, typically wrapping_arguments
- One function for each type of object, referred to as a tag in ZSH nomenclature.
- In some cases, functions to handle special values, files, sockets, fifos.
The actual mechanics of achieving this typically mean popping off the head/first word of the line, feeding that word into a case statement, calling the appropriately specialized function for whatever completion option has been passed in.
Another important feature of ZSH’s completion system is that there are many special utility functions, variables and options that make many common tasks easier, a great deal of writing completion scripts is to use these appropriately.
Git is a good command to analyse, not just because it is comprehensive, but also because it reflects many common option handling patterns; Git is run by a single command, git, followed by either flags or an argument giving the particular git command, again followed by options to that command. A high level view of what-must-be-completed-when in git demonstrates it’s recursive nature immediately.
The structure of _arguments
The function _arguments has been described as having `the syntax from hell’, but with the arguments already laid out in front of you it doesn’t look so bad. The are three types of argument: options to _arguments itself, arguments saying how to handle options to the command (i.e. `p4 diff’), and arguments saying how to handle normal arguments to the command.
The following is an excerpt from the p4 diff completion function (in turn excerpted from the manual) and covers some important fundamentals of argument parsing.
(( $+functions[_git-diff] )) || _git-diff() { _arguments -s : \ '-f[diff every file]' \ '-t[include non-text files]' \ '(-sd -se -sr)-sa[opened files, different or missing]' \ '(-sa -se -sr)-sd[unopened files, missing]' \ '(-sa -sd -sr)-se[unopened files, different]' \ '(-sa -sd -se)-sr[opened files, same as depot]' \ '-d-[select diff option]:diff option:' \ '((b\:ignore\ blanks c\:context n\:RCS s\:summary' \ 'u\:unified w\:ignore\ all\ whitespace))' \ "*::file:_perforce_files" }
Flags
The excerpt above only has one flag, and it’s the most meaningful flag as well, but I won’t waste an opportunity to briefly cover some other very handy flags:
-s
conveys that single-letter options are allowed, i.e. they can be combined
as in -ft
, although this doesn’t prevent you from acceping multiple word
options either. -w
is related; in combination with -s
it means that the
options can stack even if one of them itself takes an argument.
For example, tar -cf $FILE
could be processed in this way, as the option after
-f
indicates the file we’d be processing (further options AFTER -f
would be
valid as well)
-S
is completely unrelated, it indicates that the completion function
shouldn’t complete options after --
, which is a common UNIX ’pattern’ to
indicate options have ended.
The optspec
The long strings of option specification that follows the flags to _arguments
and a colon are known known as ~optspec~s or option specification.
Option Naming
_arguments
broadly supports 7 different option specification varieties, all of
which can be directly followed by a bracketed explanation string.
specification | description |
---|---|
*optspec | Here, optspec is one of the remaining forms below |
-/+optname | Plus or Minus the option |
-optname- | The first argument must be supplied here |
-optname+ | The first argument must be supplied with a + |
Utility Functions
Creating a dummy first argument
The following is an extract of the iproute2 argument handling in _ip
local args args=( # Command word /$'[^\0]#\0'/ 'l*ink:configure network device:$link_cmds' \ 'addrlabel:manage addrlabel:$addrlabel_cmds' \ 'a*ddr:manage protocol address:$addr_cmds' \ ) _regex_arguments _command
_pick_variant
to add options depending upon the version of a program.
local arguments # We supply a regex to _pick_variant, in this case checking gor the string `gnu` if ! _pick_variant gnu=gnu unix --help; then arguments=('-g[This flag only works on gnu distributions of this binary]') else arguments=('-a[Otherwise this flag is available]') fi
Match an ambiguous clause with _guard
The _guard
can break between two tags, dependent upon the regex; if this
doesn’t seem extraordinarily useful to you, you’re not alone – In the body of
existing ZSH completion scripts, _guard
is typically used an the action for
the specification pased into _arguments
and similar functions.
The zshcompsys
manpage itself describes behavior reminiscent of the completion
behavior of fc(1)
_guard
As an example, consider a command taking the options -n and -none, where -n must be followed by a numeric value in the same word.
zshcompsys(4)
The _fc
completion demonstrates this here:
if [[ -n $state ]]; then zstyle -s ":completion:${curcontext}:" list-separator sep || sep=-- if [[ -z ${line:#*=*} ]] && compset -P '*='; then _message -e replacements 'replacement' elif [[ -prefix [0-9] ]]; then events=( ${(0)"$(printf "%-${#HISTNO}.${#HISTNO}s $sep %s\0" "${(kv)history[@]}")"} ) _wanted -2V events expl "$state_descr" compadd -M "B:0=" -ld events - \ "${events[@]%% *}" elif [[ -prefix - ]]; then for num cmd in "${(kv@)history}"; do (( num=num - HISTNO )) events+=( "${(r.1+$#HISTNO.)num} $sep $cmd" ) done _wanted -2V events expl "$state_descr" compadd -ld events - \ "${events[@]%% *}" else _wanted events expl "$state_descr" compadd -S '' - \ ${${history%%[=[:IFS:]]*}:#[0-9-]*} || _guard "[0-9]#" event fi fi && ret=0
Examples
A statement about these examples should be made here
Delimited values with final option
A common scenario that occurs in commands such as libcap
’s capability
manipulation toolchain, bintools
and coreutils
is the requirement to
complete a list of arbitrary keywords, each with a unix-style (equal sign)
option after each one.
An example of such a command is exemplified by setcap
You might initially look at the chmod
completion, and this would get you far,
however the completion script itself is quite long. The core of the unix options
completion lies in the following.
list_terminator='*[=]' # Corresponds to `=` delimiter=',' # The character that delimits the list options=("e:effective", "i:inheritable", "p:permitted") # Valid options case $state in # compset -P checks if we've reached a user entering a $list_terminator if compset -P $list_terminator; then _describe -t options "options" options else # Otherwise complete from these list of items. _values -s $delimiter items 'foo[Description of foo]' \ 'bar[Description of bar]' fi ;; esac
Operating system specific flags with $OSTYPE
local arguments arguments=('-b[Base argument]') # We might add additional arguments based on the operating system if [[ "$OSTYPE" = (freebsd*|darwin*) ]]; then arguments+=('-m[OSX or FreeBSD Specific Flag]') fi if [[ $OSTYPE = solaris* ]]; then arguments+=('-s[Solaris specific flag]') fi if [[ $OSTYPE = linux* ]]; then arguments+=('-l[Linux specific flag]') fi
Completion from a dynamic list
There are two ways to go about this. Both require that you create a function
that calls compadd
with the list of words you want completed.
typedef -a _tmux_words _tmux_list() { compadd -a _tmux_words }
Up to you to figure out how to populate the _tmux_words
array. The function
that eventually calls compadd
can do as much other work as you like to decide
whether to call compadd
at all; see for example the _expand_alias
function
in the zsh distribution. 1
With that in place, you can do either:
Create a key binding that invokes it, leaving normal completion alone.
compdef -k _tmux_list complete-word ^XT
Add a function to your “completer” style.
zstyle ':completion:*' completer _complete _tmux_list _correct
Don’t use the above zstyle literally; find the one you are presently
using and insert _tmux_list
at the point where you want those words
tried as possible completions.
Caching variables during completion
Depending on whether you mean all completions for the current command
line or just all repetitions of completion for the same word (e.g.,
cycling through a menu) there may be different approaches to this.
Within completion on a single word, you can look at the _oldlist
completer for an example.
Based on your additional explanation, though, I suspect that’s not what you’re after, but the basic idea is still the same: Create a function which you reference at the beginning of the completer zstyle. That function tests (somehow) whether the cached state needs to be refreshed.
Bart Schaefer describes a crude procedure to cache the value value of $HISTNO
and then reload the cache if it has changed.
_xrcache() { if (( $_xr_HISTNO != $HISTNO )) then _xr_HISTNO=$HISTNO _xr_output=$(xrandr -q) fi return 1 # always "fail" so other completers are tried } zstyle ':completion:*' completer _xrcache _oldlist _expand _complete # etc.
Manual ordering of completion alternatives
You can prevent alphabetical sorting by passing -V
and the matchname: compadd -V unsorted - $revarray
Bart Schaefer also discusses compadd -V unsorted -a revarray
for large arrays:
Notable zstyle
options
Hidden completion list
This sort of question occassionally appears on newsgroups from time to time:
I want to have the alternatives offered by consecutive presses of alt-e, and I don’t want the alternatives to be listed below the command line. To achieve this, I have had to set the option
BASH_AUTO_LIST
. If this option is not set, a list of alternatives is displayed as soon as I hit alt-e (and at the same time the first alternative is put on the command line, which is good). But I don’t want this option to be set globally. I have not been able to figure out how to make this menu NOT appear for this particular completion, but without setting the global option. Is there a way to achieve this?
The answer is to set the hidden
zstyle
, which can be done like this:
zstyle ':completion:*list-comp:*' hidden all
But hidden is looked up from _description
which you don’t call.
You could add _wanted
around the compadd but all the hidden style
actually does is cause the -n
option to be passed to compadd which you
could do directly.
Style and Convention
ZSH completion scripts are (fortunately) never given the opportunity to evolve into the complex balls of mud that a ’real’ programming environment affords; consequently there is much less attention given to the stylistic debates that are tied to other languages.
This said, there are a few, largely unwritten, rules and conventions that are
Descriptions
Always use description. This is important. Really. Always use descriptions. If
you have just written down a compadd
without a $expl[@]
(or equivalent), you
have just made an error. Even in helper functions where you use a $@
: if you
can’t be sure that there is a description in the arguments, add one. You can
(and, in most cases, should) then give the description you generated after the
$@
. This makes sure that the, probably more specific, description given by the
calling function takes precedence over the generic one you have just generated.
And it really isn’t that complicated, is it? Think about a string people might want to see above the matches (in singular – that’s used throughout the completion system) and do:
local expl _description tag expl <descr> compadd "$expl@]" - <matches ...>
Note that this function also accepts -V
and -J
, optionally (in the same
word) preceded by 1
or 2
to describe the type of group you want to use. For
example:
_description tag expl '...' compadd "$expl[@]" -1V foo - ... # THIS IS WRONG!!!
is not the right way to use a unsorted group. Instead do:
_description -1V tag expl '...' compadd "$expl[@]" - ...
and everything will work fine.
Also, if you are about to add multiple different types of matches, use multiple
calls to _description
and add them with multiple calls to compadd
. But in
almost all cases you should then add them using different tags anyway, so, see
above.
And since a tag directly corresponds to a group of matches, you’ll often be using the tags function that allows you to give the explanation to the same function that is used to test if the tags are requested (again: see above). Just as a reminder:
_wanted [ -[1,2]V | -[1,2]J ] <tag> expl <descr> <cmd> ...
and
_requested [ -[1,2]V | -[1,2]J ] <tag> expl <descr> [ <cmd> ... ]
is all you need to make your function work correctly with both tags and description at the same time.
Terminology
spec
: Argument Specificationtag
: The ’varieties’ of types of objects that are valid completions, e.x a command that takes a set of permissions OR a file as it’s next argument.
Variables
$state
: The canonical variable for processing which tag you are in.$expl
: An idiom for options normally given to compadd at some point, typically an array$descr
: Argument description variables
External Resources
Footnotes:
I picked _expand_alias
because it’s explicitly
designed to be usable as either a key binding or a completer entry. Note
#compdef at the top of the source file.
Comments
You can add a comment to this page by writing one on it's Github Issue Page