Satisfying Haskell Dependencies

Heath Raftery -
You almost certainly do not want to use stack install.

Okay, fine.

I've been wilfully ignoring this advice since I first stumbled on it, because I want a global package manager goddammit. Look, I know pipenv and its container ilk are the doctor's recommendation, and that all dependencies should be specified by version and installed only within a project scope. But that's for day jobs with the luxury of a "project" to maintain. I'm an amateur (as often as I can be), experimenting with a new tool for love, and creating a dozen "projects" a day (something, to be honest, I expect day jobbers might be doing too). I want to install some interesting packages, and have them available in whatever REPL, IDE or command line interface I fire up next. I don't want to create a dependencies list for each script I write. I just want to import it, and if it's not found, solve it forever more (on my computer at least) with one fell install thePackage.

But Alexis King's An Opinionated Guide to Haskell is the last shot against my defences. It's time to stop glossing over package management.

This article is the summary of many hours navigating the obsolete and incomplete available guidance, culminating in Just Bloody Doing It. The results are really clear, and I hope this serves as an accurate and informative guide for at least a few hours more!

The Results

Let me get right to it and start with the end. Here are the four practical ways to satisfy package dependencies in Haskell in (early February) 2022:

1. Global installs

You can still do global installs.

Here's how:

  • Forget stack and just cabal install --lib the-package every time. Make you run this from outside any existing Cabal package to avoid adding to that specific package.
  • Now libraries from the installed package can be imported automagically in GHCi, and you can compile programs that rely on them simply by running ghc *.hs
  • Note the installed packages may or may not appear in ghc-pkg list, so that command becomes pretty useless.

2. Use Stack as a run command

You can use stack directly. Here's how:

stack exec --package containers --package optparse-generic [...and so on] -- runghc hello.hs

While this is good to know, it seems pretty useless for anything other than just running the script.

3. Create a Stack project

The real deal. Here's how:

  • Run stack new my-project hraftery/minimal
  • Put your code in Main.hs and your dependencies in my-project.cabal (and maybe stack.yaml - see Stack Deep Dive).
  • Use stack build to automagically pull and build all dependencies.

My hraftery/minimal project template is up to date, and is very light and very easy to use. You can build up from there to suit your needs. Or you can start with the official default template by not specifying one (ie. stack new my-project). Jump to the Stack Project Templating section to see how and why I created a leaner template.

4. Use a Stack script

Your Haskell file itself can become an executable Stack script. Here's how:

  • Add -- stack --resolver lts-6.25 script --package the-package to the top of your .hs file.
  • This is really designed for script only usage, not compilation.

Evaluating Findings

To understand which of the 4 methods was going to solve my package management quandary, I put each of them to task in my typical use case. Ultimately I want to do three things with my Haskell code: Edit, Test; and Run. I suspect I'm not the only one.

For example, at the moment my Edit; Test; Run environments are mostly VS Code; GHCi; and GHC respectively. So that's what I used for the evaluation.

Note I consider an operational Haskell Language Server to be critical functionality for Edit - insights from code inspection (eg. type inference and HLint advice) are just too damn glorious in Haskell to do without.

Here's what I discovered:

Method Edit Test Run
Global installs Fine, once you use the Magic Trick! See Stack Deep Dive. Fine. Either load the script file (eg. :l main) or use import directly. You may be prompted to run :set package first, which generally works fine. Fine. Simply ghc *.hs to get the executable.
Stack directly Bad. The editor doesn't know your command. Bad. The intepreter doesn't know your command. Good, if you ever get here!
Stack project Yes... but you have to open the project folder, not a parent folder! Good. Just use stack ghci instead of ghci. Builds all dependencies on launch. The best. stack run will do it all.
Stack script Bad. According to this. Assumed bad. Bad. Interpreter only.

Conclusion

Look, I've gone full circle on this. If your "project" will live for less than a day (eg. it's a key experiment for a bigger project or you're just using Haskell as a tool to get a job done) then just use global installs. You might need to reset version compatibility for your next experiment, but the tools are much more stable now, and the time investment is relatively insignificant.

Now I have it working reliably, Global Installs will suit me just fine for the majority of work I do. I'm not interested in a hour long YaML goose chase because I want to try splitOn instead of split for a few minutes. I know, it's sacrilege, but the tooling has got a lot better since Moses wrote Thou Shall Not Globally Install, and in my case it's good enough. Which is not only enough, it's good.

Where Global Installs no longer make sense (eg. I do need to manage dependency versions or I want my work to be more portable) I'll be creating a Stack project using what I now know is required to be a "real" Stack project.

Exploration Phase

This section is a collection of the more illuminating findings from my research.

An Opinionated Guide

I love me an opinionated guide. Alexis King's is great. She says:

stack is not a package manager, it is a build tool. It does not manage a set of “installed” packages; it simply builds targets and their dependencies.
So a new Haskeller decides to install lens, types stack install lens, and then later tries stack uninstall lens, only to discover that no such command exists.
stack install is not like npm install. stack install is like make install. It is nothing more than an alias for stack build --copy-bins, and all it does is build the target and copy all of its executables into some relatively global location like ~/.local/bin. This is usually not what you want.

Ah ha! So stack install is a trap for the unwary. Right, I get it. Let's go ahead and try adding a project dependencies file for every silly little script I write. Alexis' guidelines are:

  • Install packages per-project using the ordinary stack build command.
  • Per-compiler with stack build --copy-compiler-tool

Alas, this simply doesn't work for a standalone script. Often in these instructions there's an implicit assumption that you have a project already established. But I don't, and I'm trying desperately to avoid as much baggage as possible for every little script I write.

Stack Deep Dive

Turns out I'm not gunna get out of this without learning Stack projects for real instead of copying & pasting my way to competence.

As is often the Haskell experience, the available guides are hard to take guidance from. A lot has changed in Haskell's lifetime, and the experts are quite removed from the experience of newbies. To get to the bottom of this, I was going to have to try it out for myself.

Here's the key truths I found:

  • Cabal is a build system, which is used by Stack.
  • A Cabal package has:
    • A name and version
    • 0 or 1 libraries
    • 0 or more executables
  • package.yaml is used by hpack to create my-package.cabal. So:
    • if you don't use hpack, don't use package.yaml.
    • if you do use hpack, don't touch my-package.cabal.
    • According to this thread hpack doesn't add a lot these days that can't be managed just as easily in .cabal directly, so I'm going to opt to not use it for simplicity.
  • A Stack project has:
    • A resolver (eg. which Stackage Snapshot to use for packages)
    • Extra dependencies to override the resolver
    • 0 or more Cabal packages
    • GHC options
  • So a build-depends/dependencies entry in my-package.cabal/pacakage.yaml is a build requirement, while a extra-deps entry in stack.yaml specifies which version to use to satisfy requirements.
    • Stack requires the dependency to be listed in both places, but only if it's not in the snapshot.
    • With the purpose made clear, all you need to customise your my-package.cabal and stack.yaml files is nicely (and extensively) documented here and here, respectively.
  • The base dependency contains the Prelude and support libraries. See https://wiki.haskell.org/Base_package
  • Magic Trick: Whenever stuff is not working, run the magic command first:
    • rm ~/.ghc/x86_64-darwin-8.10.7/environments/default
    • Obviously, your paths may vary based on your OS, architecture and GHC version.
    • I don't know what this file does, but this is the solution to about 300 mysterious issues.
  • Lots of things work only by coincidence, which is maddening when you try to make it work consistently. Every approach works just fine, if you happen to have the right cache or local installs. So careful experimentation is necessary to figure out reliability.

Stack Project Templating

None of the standard Stack project templates are simple. While simple is less complicated than the default, it is far from simple and requires a lot of metadata. Starting there makes it very hard to figure out how to effectively and reliably modify the project files to suit your project. scotty-hello-world is really short, but seems to be missing some standard files, so who knows what "minimal" is? The Internet doesn't seem to know.

So I sought to find out.

And voilà! It turns out there is so much out-of-date and unnecessary cruft in the standard Stack templates, which makes them very hard to learn from. From this minimal starting point, you can build up to suit your needs, purposefully and consciously.

{-# START_FILE {{name}}.cabal #-}
cabal-version:       2.4
name:                {{name}}
version:             0.1.0.0

executable {{name}}
  main-is:             Main.hs
  default-language:    Haskell2010
  build-depends:
    base >= 4.14,

{-# START_FILE Main.hs #-}
module Main where

main :: IO ()
main = do
  putStrLn "hello world"

{-# START_FILE stack.yaml #-}
snapshot: lts-18.24

I've stuck a copy of this at https://github.com/hraftery/stack-templates/blob/main/minimal.hsfiles so anyone can use it by simply calling stack new my-project hraftery/minimal.

Note if you exclude the stack.yaml file, you're quietly back to Global Installs, which is really handy for debugging your installs.

To make sure my minimal template was sufficient, I used a Haskell program which imported a few libraries that I was sure I hadn't already stashed in some cache on my local machine somewhere. Specifically, I tested with this in my .cabal file (see how I took advantage of modern Cabal and politely left a trailing comma for you in the template?):

  build-depends:
    base >= 4.14,
    split,
    search-algorithms,
    cereal,
    safecopy

and this after the module line at the top of my Main.hs file.

import Data.List.Split (splitOn)
import Algorithm.Search (dijkstra, pruning)
import Data.SafeCopy
import Data.Serialize.Get
import Data.Serialize.Put

and no changes to stack.yaml! It turns out split, search-algorithms, cereal and even safecopy are all part of the Stackage Snapshot, so no stack additions are required. If you do venture outside Stackage, you'll need to also add an extra-deps to stack.yaml. See the "both places" comment in Stack Deep Dive.

Key References