Rabbit Hole: Fisher

Tagged: rabbithole

Welcome! My Rabbit Hole posts are a series of blog posts where I explore a code base written by someone else and try and understand how a certain part of it works. Today's post is about the fish shell plugin manager Fisher.

Setting up a new computer

I received a new computer for work this week and that meant setting up all my dotfiles and so on. I use fish for my shell - it's great! I also use a plugin called z which is also great! z allows you to jump to common directories. To install z, according to its readme, I was to use Fisher. And now... I want to know how Fisher works!

But first some background.

About Fisher

I don't know much about Fisher, but I was surprised to find that it was created by Jorge Bucaran who also created the lovely HyperApp library - an excellent tiny dom that is rather Elm-ish, if I recall correctly.

From the readme we learn the following about Fisher:

  • 100% pure-Fish—easy to contribute to or modify.

  • Blazing fast concurrent plugin downloads.

  • Zero configuration out of the box.

  • Oh My Fish! plugin support.

Mostly, I'm curious about:

  1. Where do plugins get installed?

  2. How do plugins get fetched?

  3. How are plugins "registered" so that they can be invoked from my fish-shell?

Exploring the repo

I clone the repo to my otherppl/ folder and start poking around. Running tree reveals there aren't too many files in this project (finally, I've chosen something a bit smaller to practice reading and learning from other codebases!)

❯ tree
.
├── LICENSE.md
├── README.md
├── completions
│   └── fisher.fish
├── functions
│   └── fisher.fish
└── tests
    ├── fisher.fish
    └── ponyo
        ├── conf.d
        │   └── ponyo.fish
        └── functions
            └── ponyo.fish

6 directories, 7 files

I'm not exactly interested in completions, but the functions folder should be a good start.

Setting up fisher

So, having set up z the other day require that I install fisher, which I did by following its readme which instructed me to run this curl command:

curl -sL https://git.io/fisher | source && fisher install jorgebucaran/fisher

What's interesting here is that the above command doesn't actually move any scripts into your assumed path - from my still limited terminal knowledge, it pipes the results of the curl request into the source utility. Let's learn about that quickly:

man source

DESCRIPTION
       source  evaluates  the  commands  of the specified file in the current shell as a new block of code. This is different from starting a new process to perform the
       commands (i.e. fish < FILENAME) since the commands will be evaluated by the current shell, which means that changes in shell variables will  affect  the  current
       shell.  If additional arguments are specified after the file name, they will be inserted into the $argv variable. The $argv variable will not include the name of
       the sourced file.

Ok, so the actual curl'd url is fisher itself! And then, we're using the fisher command to install fisher somewhere - we don't know where yet, but that's coming up next.

Where do plugins get installed?

I try running which fisher to see if it can location some kind of script or binary. It doesn't return anything. I guess this is because it's just a function and so the which command can't locate it. I guess that means it's time to start reading some code!

I've never looked at fish syntax, but it's different enough from bash (which I also don't know super well). Alright, enough excuses.

The file only has two functions, and the first one is the one we are interested in.

Immediately I see that the function sets a local variable called fisher_path to $__fish_config_dir, which then gets used in the setting of a var fish_plugins that creates a concatenated string:

    set --query fisher_path || set --local fisher_path $__fish_config_dir
    set --local fisher_version 4.3.1
    set --local fish_plugins $__fish_config_dir/fish_plugins

It was probably easier to just paste that code sample above. I don't see $__fish_config_dir being set anywhere in this file, which leads me to believe that there it is an inherent variable belong to fish. Let's see if it shows up in our terminal

❯ ls $__fish_config_dir
completions    conf.d         config.fish    fish_plugins   fish_variables functions

Great, that answers the first question - I see the fish_plugins folder.

How do plugins get fetched?

The fisher function switches over several commands, let's look for install.

        case install update remove
            isatty || read --local --null --array stdin && set --append argv $stdin

            set --local install_plugins
            set --local update_plugins
            set --local remove_plugins
            set --local arg_plugins $argv[2..-1]
            set --local old_plugins $_fisher_plugins
            set --local new_plugins

            # .....

Now our work is set out for us. This case statement branch is the bulk of the file, and it appears that we are handling installation, updating and removal of plugins in this block.

I have zero idea what isatty is doing and I had to look up what set does when no value is given to set:

man set

#...
If only a variable name has been given, set sets the variable to the empty list.

#...
# while we're at it, let's see what *local* does.
-l  or --local forces the specified shell variable to be given a scope that is local to the current block, even if a variable with the given name exists and is non-local

Back to isatty:

man isatty

NAME isatty - test if a file descriptor is a terminal

SYNOPSIS isatty [FILE DESCRIPTOR]

DESCRIPTION isatty tests if a file descriptor is a terminal (as opposed to a file). The name is derived from the system call of the same name, which for historical reasons refers to a teletypewriter (TTY).

FILE DESCRIPTOR may be either the number of a file descriptor, or one of the strings stdin, stdout, or stderr. If not specified, zero is assumed.

If the specified file descriptor is a terminal device, the exit status of the command is zero. Otherwise, the exit status is non-zero. No messages are printed to standard error.

Looks like we are getting a bit of a history lesson here. Or at least a historical-reference. I don't know much about teletypewriter's - I'll save that for another post - so I'm going to have to make some educated guesses about terminology. I'll think of isatty as something that tests a value and passes when the file is not a "file". I'm not sure if something constitutes a file beyond my os-level understanding of it, but I can always come back to this area if I get lost later.

It seems that if isatty fails, then we use read - looks like I need to do another man page lookup:

man read

NAME read - read line of input into variables

#...

-l or --local makes the variables local. -z or --null marks the end of the line with the NUL character, instead of newline. This also disables interactive mode. -a or --list stores the result as a list in a single variable. This option is also available as --array for backwards compatibility.

I'm starting to hit a wall here, and in true rabbit fashion I can feel every part of me wanting to scream and run away! Ugh, and what is happening at the end of this line:

isatty || read --local --null --array stdin && set --append argv $stdin

set, --append, argv, and $stdin are all question marks for me, and I can feel my will power depleting. Let's take a detour to try and find where the actual plugins get fetched.

This block seems promising:

if set --query argv[2]
    for plugin in $new_plugins
        if contains -- "$plugin" $old_plugins
            test "$cmd" = remove &&
                set --append remove_plugins $plugin ||
                set --append update_plugins $plugin
        else if test "$cmd" = install
            set --append install_plugins $plugin
        else
            echo "fisher: Plugin not installed: \"$plugin\"" >&2 && return 1
        end
    end
else #...

I don't really know what set --query argv[2] is doing yet, but I see that we are --append'ing to the "install_plugins" variable ... a "$plugin"! By that reasoning I should be able to look at where $install_plugins is iterated over and find how they are fetched.

Speaking of fetched - I just found a variable called $fetch_plugins - it seems that fisher pushes all plugins that a user wants to update and/or install into a single list that they fetch.

for plugin in $fetch_plugins
    set --local source (command mktemp -d)
    set --append source_plugins $source

    command mkdir -p $source/{completions,conf.d,functions}

    fish --command "
        if test -e $plugin
            command cp -Rf $plugin/* $source
        else
            set temp (command mktemp -d)
            set name (string split \@ $plugin) || set name[2] HEAD
            set url https://codeload.github.com/\$name[1]/tar.gz/\$name[2]

            echo Fetching (set_color --underline)\$url(set_color normal)

            if curl --silent \$url | tar -xzC \$temp -f - 2>/dev/null
                command cp -Rf \$temp/*/* $source
            else
                echo fisher: Invalid plugin name or host unavailable: \\\"$plugin\\\" >&2
                command rm -rf $source
            end
            command rm -rf \$temp
        end

        set files $source/* && string match --quiet --regex -- .+\.fish\\\$ \$files
    " &

    set --append pid_list (jobs --last --pid)
end

Above, we see iteration over the $fetch_plugins array, and for each one a temporary directory is created via mktemp -d and set to the source variable - which I'm guessing represents the source code of the plugin to update/install.

After that there is a long string command that is passed to fish (seems sort of like eval in lisp or js). Here's where the magic happens! This command breaks apart the arguments to fisher install and uses them to construct the url to find the package at. It seems that codeload.github.com is the place to do that - I have never visited that sub-domain, and when I try to reach it I am just redirected to my home page on Github.

Anyway, curl is run and the results get copied to the local $source variable.

How are plugins "registered" so that they can be invoked from my fish-shell?

The last question I have is hopefully a simple answer - after having run fisher I see that what I thought was a directory (~/.config/fish/fish_plugins) was actually just a file that lists the plugins the user has installed. Instead, the packages are installed under ~/.config/fish/functions - I see z and fisher there now!

I'm going to make one last educated guess and say that anything in that functions/ directory is automatically callable by fish.

Wrap up

Woah, I did it! My last two attempts at writing a Rabbit-Hole post were much more... beleaguering. This ended being not too complicated. I had to read a fair few manpages, but definitely learned a few things.

That's it for now. As always, thanks for reading.