- Walking the path of opam, treading on solid ground
- First step: installing opam
- Second step: initialisation
- Acclimating to the environment
- Switches, tailoring your workspace to your vision
- The official opam-repository, the safe for all your packages
- Installing packages in your current switch
- Conclusion
Opam 101: The First Steps
Welcome, dear reader, to a new series of blog posts!
This series will be about everything opam
. Each article will cover a specific
aspect of the package manager, and make sure to dissipate any confusion or
misunderstandings on this keystone of the OCaml distribution!
Each technical article will be tailored for specific levels of engineering --
everyone, be they beginners, intermediate or advanced in the OCaml Arts will
find answers to some questions about opam
right here.
Checkout each article's
tags
to get an idea of the entry level required for the smoothest read possible!
- Walking the path of opam, treading on solid ground
- First step: installing opam
- Second step: initialisation
- Acclimating to the environment
- Switches, tailoring your workspace to your vision
- The official opam-repository, the safe for all your packages
- Installing packages in your current switch
- Conclusion
New to the expansive OCaml sphere? As said on the official opam website, opam has been a game changer for the OCaml distribution, since it first saw the light of day here, almost a decade ago.
Walking the path of opam, treading on solid ground
We are aware that it can be quite a daunting task to get on-board with the OCaml distribution. Be it because of its decentralised characteristics: plethora of different tools, a variety of sometimes clashing modi operandi and practices, usually poorly documented edge use-cases, the variety of ways to go about having a working environment or many a different reason...
We have been thinking about making it easier for everyone, even the more
confirmed Cameleers, by releasing a set of blogposts progressively detailing
the depths at which opam
can go.
Be sure to read these articles from the start if you are new to the beautiful world of OCaml and, if you are already familiar, use it as a trust-worthy documentation on speed-dial... You never know when you will have to setup an opam installation while off-the-grid, do you ?
Are you ready to dive in ?
First step: installing opam
First, let's talk about installing opam.
DISCLAIMER: In this tutorial, we will only be addressing a fresh install of
opam
on Linux and Mac. For more information about a Windows installation, stay tuned with this blog!
One would expect to have to interact with the package manager of one's
favourite distribution in order to install opam
, and, to some extent, one
would be correct. However, we cannot guarantee that the version of opam you
have at your disposal through these means is indeed the one expected by this
tutorial, and every subsequent one for that matter.
You can check that here, make
sure the version available to you is 2.1.5
or above.
Thus, in order for us to guarantee that we are on the same version, we will use the installation method found here and add an option to specify the version of opam we will be working with from now on.
Note that if you don't add the --version 2.1.5
option to the following
command line, the script will download and install the latest opam release.
The cli of opam
is made to remain consistent between versions so, unless you
have a very old version, or if you read this article in the very distant
future, you should not have problems by not using the exact same version as
we do. For the sake of consistency though, I will use this specific version.
$ bash -c "sh <(curl -fsSL https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh) --version 2.1.5"
This script will download the necessary binaries for a proper installation of
opam
. Once done, you can move on to the nitty gritty of having a working
opam
environment with opam init
.
Second step: initialisation
The first command to launch, after the initial opam
binaries have been
downloaded and opam
has been installed on your system, is opam init
.
This is when you step into the OCaml distribution for the first time.
opam init
does several crucial things for you when you launch it, and the
rest of this article will detail what exactly these crucial things are and what
they mean:
- it checks some required and recommended tools;
- it syncs with the official OCaml opam-repository, which you can find here;
- it sets up the opam environment in your
*rc
files; - it creates a switch and installs an ocaml-compiler for you;
Lets take a step-by-step look at the output of that command:
$ opam init
No configuration file found, using built-in defaults.
Checking for available remotes: rsync and local, git, mercurial, darcs.
Perfect!
<><> Fetching repository information ><><><><><><><><><><><><><><><><>
[default] Initialised
<><> Required setup - please read <><><><><><><><><><><><><><><><><><>
In normal operation, opam only alters files within ~/.opam.
However, to best integrate with your system, some environment
variables should be set. If you allow it to, this initialisation
step will update your bash configuration by adding the following
line to ~/.profile:
test -r ~/.opam/opam-init/init.sh && . ~/.opam/opam-init/init.sh > /dev/null 2> /dev/null || true
Otherwise, every time you want to access your opam installation,
you will need to run:
eval $(opam env)
You can always re-run this setup with 'opam init' later.
Do you want opam to modify ~/.profile? [N/y/f]
(default is 'no', use 'f' to choose a different file) y
User configuration:
Updating ~/.profile.
[NOTE] Make sure that ~/.profile is well sourced in your ~/.bashrc.
<><> Creating initial switch 'default' (invariant ["ocaml" {>= "4.05.0"}] - initially with ocaml-base-compiler)
<><> Installing new switch packages <><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml" {>= "4.05.0"}]
<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><>
∗ installed base-bigarray.base
∗ installed base-threads.base
∗ installed base-unix.base
∗ installed ocaml-options-vanilla.1
⬇ retrieved ocaml-base-compiler.5.1.0 (https://opam.ocaml.org/cache)
∗ installed ocaml-base-compiler.5.1.0
∗ installed ocaml-config.3
∗ installed ocaml.5.1.0
∗ installed base-domains.base
∗ installed base-nnp.base
Done.
The main result for an opam init
call is to setup what is called your opam root
. It does so by creating a ~/.opam
directory to operate inside of.
opam
modifies and writes in this location only as a default.
First, opam
checks that there is at least one required tool for syncing to
the opam-repository
. Then it checks what backends are available in your
system. Here all are available: rsync, git, mercurial, and darcs
. They will
be used to sync repositories or packages.
$ opam init
No configuration file found, using built-in defaults.
Checking for available remotes: rsync and local, git, mercurial, darcs.
Perfect!
Then, opam
fetches the default opam repository: opam.ocaml.org
.
<><> Fetching repository information ><><><><><><><><><><><><><><><><>
[default] Initialised
Secondly, opam
requires your input in order to configure your shell for the
smoothest possible experience. For more details about the opam environment,
refer to the next section.
Something interesting to remember for later is, in the excerpt below, we grant opam with the permission to edit the
~/.profile
file. This part of the Quality of Life features for an everyday use anopam
environment and we will detail how so below.
<><> Required setup - please read <><><><><><><><><><><><><><><><><><>
In normal operation, opam only alters files within ~/.opam.
However, to best integrate with your system, some environment
variables should be set. If you allow it to, this initialisation
step will update your bash configuration by adding the following
line to ~/.profile:
test -r ~/.opam/opam-init/init.sh && . ~/.opam/opam-init/init.sh > /dev/null 2> /dev/null || true
Otherwise, every time you want to access your opam installation,
you will need to run:
eval $(opam env)
You can always re-run this setup with 'opam init' later.
Do you want opam to modify ~/.profile? [N/y/f]
(default is 'no', use 'f' to choose a different file) y
User configuration:
Updating ~/.profile.
[NOTE] Make sure that ~/.profile is well sourced in your ~/.bashrc.
The next action is the installation of your very first switch
alongside a
version of the OCaml compiler, by default a compiler >= 4.05.0
to be exact.
For more information about what is a switch
be sure to read the rest of the
article.
<><> Creating initial switch 'default' (invariant ["ocaml" {>= "4.05.0"}] - initially with ocaml-base-compiler)
<><> Installing new switch packages <><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml" {>= "4.05.0"}]
<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><>
∗ installed base-bigarray.base
∗ installed base-threads.base
∗ installed base-unix.base
∗ installed ocaml-options-vanilla.1
⬇ retrieved ocaml-base-compiler.5.1.0 (https://opam.ocaml.org/cache)
∗ installed ocaml-base-compiler.5.1.0
∗ installed ocaml-config.3
∗ installed ocaml.5.1.0
∗ installed base-domains.base
∗ installed base-nnp.base
Done.
Great! So let's focus on the actions performed by the opam init
call!
Acclimating to the environment
Well, as said previously, the first action was to setup an opam root
in your
$HOME
directory, (i.e: ~/.opam
). This is where opam
will operate. opam
will never modify other locations in your filesystem without notifying you
first.
An opam
root is made to resemble a linux-like architecture. You will find
inside it directories such as /usr
, /etc
, /bin
and so on. This is by
default where opam
will store everything relative to your system-wide
installation. Config files, packages and their configurations, and also
binaries.
This leads us to the need for an eval $(opam env)
call.
Indeed, in order to make your binaries and such accessible as system-wide
tools, you need to update all the relevant environment variables (PATH
,
MANPATH
, etc.) with all the locations for all of your everyday OCaml tools.
To see what variables are exported when evaluating the opam env
command, you
can check the following codeblock:
$ opam env
OPAM_SWITCH_PREFIX='~/.opam/default'; export OPAM_SWITCH_PREFIX;
CAML_LD_LIBRARY_PATH='~/.opam/default/lib/stublibs:~/.opam/default/lib/ocaml/stublibs:~/.opam/default/lib/ocaml'; export CAML_LD_LIBRARY_PATH;
OCAML_TOPLEVEL_PATH='~/.opam/default/lib/toplevel'; export OCAML_TOPLEVEL_PATH;
MANPATH=':~/.opam/default/man'; export MANPATH;
PATH='~/.opam/default/bin:$PATH'; export PATH;
Remember when we granted opam init
with the permission to edit the ~/.profile
file, earlier in this tutorial ? That comes in handy now: it keeps us from
having to use the eval $(opam env)
more than necessary.
Indeed, you would otherwise have to call it every time you launch a new shell
among other things. What it does instead is adding hook at prompt level that
keeps opam
environment synced, updating it every time the user presses
Enter
. Very handy indeed.
Switches, tailoring your workspace to your vision
The second task accomplished by opam init
was installing the first switch
inside your fresh installation.
A switch
is one of opam's core operational concepts, it's definition can vary
depending on your exact use-case but in the case of OCaml, a switch
is a
named pair:
- an arbitrary version of the OCaml compiler
- a list of packages available for that specific version of the compiler.
In our example, we see that the only packages installed in the process were the
dependencies for the OCaml compiler version 5.1.0
inside the switch
named
default
.
<><> Creating initial switch 'default' (invariant ["ocaml" {>= "4.05.0"}] - initially with ocaml-base-compiler)
<><> Installing new switch packages <><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml" {>= "4.05.0"}]
<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><>
∗ installed base-bigarray.base
∗ installed base-threads.base
∗ installed base-unix.base
∗ installed ocaml-options-vanilla.1
⬇ retrieved ocaml-base-compiler.5.1.0 (https://opam.ocaml.org/cache)
∗ installed ocaml-base-compiler.5.1.0
∗ installed ocaml-config.3
∗ installed ocaml.5.1.0
∗ installed base-domains.base
∗ installed base-nnp.base
Done.
You can create an arbitrary amount of parallel switches
in opam. This allows
users to manage parallel, independent OCaml environments for their
developments.
There are two types of switches
:
global switches
have their packages, binaries and tools available anywhere on your computer. They are useful when you consider a givenswitch
to be your default and most adequate environment for your everyday use ofopam
and OCaml.local switches
on the other hand are only available in a given directory. Their packages and binaries are local to that specific directory. This allows users to make specific projects have their own self-contained working environments. The local switch is automatically selected byopam
as the current one when you are located inside the appropriate directory. More details on local switches below.
The default behaviour for opam
when creating a switch
at init-time is to
make it global and name it default
.
$ opam switch show
default
$ opam switch
# switch compiler description
→ default ocaml.5.1.0 default
Now that you have a general understanding of what exactly is a switch
and how
it is used, let's get into how you can go about manually creating your first
switch
.
Creating a global switch
NB: Remember that
opam
's command-line interface is beginner friendly. You can, at any point of your exploration, use the--help
option to have every command and subcommand explained. You may also checkout the opam cheat-sheet that was released a while ago and might still hold some precious insights on opam's cli.
So how does one create a switch
? The short answer is bafflingly
straightforward:
# Installs a switch named "my-switch" based OCaml compiler version > 4.05.0
# Here 4.05 is the default lower compiler version opam selects when unspecified
$ opam switch create my-switch
Easy, right? Now let's imagine that you would like to specify a later version
of the OCaml compiler. The first thing you would want to know is which version
are available for you to specify, and you can use opam list
for that.
Other commands can be used to the same effect but we prefer introducing you to
this specific one as it may also be used for any other package available via
opam
.
So, as for any other package than ocaml
itself, opam list
will give you all
available versions of that package for your currently active switch
. Since we
don't yet have an OCaml compiler installed, it will list all of them so that we
may pick and choose our favourite to use for the switch
we are making.
$ opam list ocaml
# Packages matching: name-match(ocaml) & (installed | available)
# Package # Installed # Synopsis
ocaml.3.07 -- The OCaml compiler (virtual package)
ocaml.3.07+1 -- The OCaml compiler (virtual package)
ocaml.3.07+2 -- The OCaml compiler (virtual package)
ocaml.3.08.0 -- The OCaml compiler (virtual package)
(...)
ocaml.4.13.1 -- The OCaml compiler (virtual package)
ocaml.4.13.2 -- The OCaml compiler (virtual package)
(...)
ocaml.5.2.0 -- The OCaml compiler (virtual package)
Let's use it for a switch:
# Installs a switch named "my-switch" based OCaml compiler version = 4.13.1
$ opam switch create my-switch ocaml.4.13.1
That's it, for the first time, you have manually created your own global switch
tailored to your specific needs, congratulations!
NB: Creating a switch can be a fairly time-consuming task depending on whether or not the compiler version you have queried from
opam
is already installed on your machine, typically in a previously createdswitch
. Every time you askopam
to install a version of the compiler, it will first scour your installation for a locally available version of that compiler to save you the time necessary for downloading, compiling and installing a brand new one.
Now, onto local switches
.
Creating a local switch
As said previously, the use of a local switch
is to constrain a specific
OCaml environment to a specific location on your workstation.
Let's imagine you are about to start a new development called my-project
.
While preparing all necessary pre-requisites for it, you notice something
problematic: your global default
switch is drastically incompatible with the
dependencies of your project.
In this imaginary situation, you have a default
global switch that is useful
for most of your other tasks but now have only one project that differs from
your usual usage of OCaml.
To remedy this situation, you could go about creating another global switch
for your upcoming dev requirements on my-project
and proceed to install all
relevant packages and remake a full switch
from scratch for that specific
project. However this would require you to always keep track of which one is
your currently active switch
, while possibly having to regularly oscillate
between your global default
switch and your alternative global my-project
switch which you could understandably find to be suboptimal and tedious to
incorporate to your workflow on the long run.
That's when local switches
come in handy because they allow you to leave the
rest of your OCaml dev environment unaffected by whatever out-of-bounds or
specific workload you're undertaking. Additionally, the fact that opam
automatically selects your local switch
as your current active one as soon as
you step inside the relevant directory makes the developers's context switch
seemless.
Let's examine how you can create such a switch
:
# Hop inside the directory of your project
$ cd my-project
# We consider your project already has an opam file describing only
# its main dependency: ocaml.4.14.1
$ opam switch create .
<><> Installing new switch packages <><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml" {>= "4.05.0"}]
<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><>
∗ installed base-bigarray.base
∗ installed base-threads.base
∗ installed base-unix.base
∗ installed ocaml-system.4.14.1
∗ installed ocaml-config.2
∗ installed ocaml.4.14.1
Done.
$ opam switch
# switch compiler description
→ /home/ocp/my-project ocaml.4.14.1 /home/ocp/my-project
default ocaml.5.1.0 default
my-switch ocaml.4.13.1 my-switch
[NOTE] Current switch has been selected based on the current directory.
The current global system switch is default.
Here it is, you can now hop into your local switch /home/ocp/my-project
whenever you have time to deviate from your global environment.
The official opam-repository, the safe for all your packages
Among all the things that opam init
did when it was executed, there is still
one detail we have yet to explain and that's the first action of the process:
retrieving packages specification from the official OCaml opam-repository
.
Explaining what exactly an opam-repository
is requires the recipient to have
a slightly deeper understanding of how opam
works than the average reader
this article was written for might have; so you will have to wait for us to go
deeper into that subject in another blogpost when the time is ripe.
What we will do now though is explain what the official OCaml
opam-repository
is and how it relates to our use of opam
in this blog post.
The Official OCaml opam-repository
is an open-source project where all released software of the OCaml
distributions are referenced. It holds different compilers, basic tools,
thousands of libraries, approximatively 4500 packages in total as of today and
is configured to be the default repository for opam
to sync to. You may add
your own repositories for your own use of opam
, but again, that's a subject
for another time.
In case the repository itself is not what you are looking for, know that all packages available throughout the entire OCaml distribution may be browsed directly on ocaml.org.
It is essentially a collection of opam packages
described in opam file
format. Checkout the manual for
more information about the opam file
format.
A short explanation for it is that an opam package
file holds every
information necessary for opam
to operate and provide. The file lists all of
the packages direct dependencies, where to find its source code, the names and
emails of maintainers and authors, different checksums for each archive release and the
list goes on.
Here's a quick example for you to have an idea of what it looks like:
opam-version: "2.0"
synopsis: "OCaml bindings to Zulip API"
maintainer: ["Dario Pinto <dario.pinto@ocamlpro.com>"]
authors: ["Mohamed Hernouf <mohamed.hernouf@ocamlpro.com>"]
license: "LGPL-2.1-only WITH OCaml-LGPL-linking-exception"
homepage: "https://github.com/OCamlPro/ozulip"
doc: "https://ocamlpro.github.io/ozulip"
bug-reports: "https://github.com/OCamlPro/ozulip/issues"
dev-repo: "git+https://github.com/OCamlPro/ozulip.git"
tags: ["zulip" "bindings" "api"]
depends: [
"ocaml" {>= "4.10"}
"dune" {>= "2.0"}
"ez_api" {>= "2.0.0"}
"re"
"base64"
"json-data-encoding" {>= "1.0.0"}
"logs"
"lwt" {>= "5.4.0"}
"ez_file" {>= "0.3.0"}
"cohttp-lwt-unix"
"yojson"
"logs"
]
build: [ "dune" "build" "-p" name "-j" jobs "@install" ]
url {
src: "https://github.com/OCamlPro/ozulip/archive/refs/tags/0.1.tar.gz"
checksum: [
"md5=4173fefee440773dd0f8d7db5a2e01e5"
"sha512=cb53870eb8d41f53cf6de636d060fe1eee6c39f7c812eacb803b33f9998242bfb12798d4922e7633aa3035cf2ab98018987b380fb3f380f80d7270e56359c5d8"
]
}
Okay so now, how do we go about populating a switch
with packages and really get started?
Installing packages in your current switch
It's elementary. This simple command will do the trick of trying to install a
package, and its dependencies, in your currently active switch
.
$ opam install my-package
I say trying because opam
will notify you if the current package version
and its dependencies you are querying are or not compatible with the current
state of your switch
. It will also offer you solutions for the compatibility
constraints between packages to be satisfiable: it may suggest to upgrade some
of your packages, or even to remove them entirely.
The key thing about this process is that opam
is designed to solve
compatibility constraints in the global graph of dependencies that the OCaml
packages form. This design is what makes opam
the average Cameleer's best
friend. It will highlight inconsistencies within dependencies, it will figure
out a way for your specific query to be satisfiable somehow and save you a
lot of headscratching, that is, if you are willing to accommodate a bit of
getting-used to.
The next command allows you to uninstall a package from your currently active
switch
as well as the packages that depend on it:
$ opam remove my-package
And the two following will update
the state of the repositories opam
is
synchronised with and upgrade
the packages installed while always keeping
package compatibility in mind.
$ opam update
$ opam upgrade
Conclusion
Here it is, you should now be knowledgeable enough about opam
to jump right in
the OCaml discovery!
Today we learned everything elementary about opam
.
From installation, to initialisation and explanations about the core concepts
of the opam
environment, switches
, packages and the Official OCaml
opam-repository
.
Be sure to stay tuned with our blog, the journey into the rabbit hole has only
started and opam
is a deep one indeed!
Thank you for reading,
From 2011, with love,
The OCamlPro Team
About OCamlPro:
OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from experts with a state-of-the-art knowledge of programming languages theory and practice.
- We provide audit, support, custom developer tools and training for both the most modern languages, such as Rust, Wasm and OCaml, and for legacy languages, such as COBOL or even home-made domain-specific languages;
- We design, create and implement software with great added-value for our clients. High complexity is not a problem for our PhD-level experts. For example, we helped the French Income Tax Administration re-adapt and improve their internally kept M language, we designed a DSL to model and express revenue streams in the Cinema Industry, codename Niagara, and we also developed the prototype of the Tezos proof-of-stake blockchain from 2014 to 2018.
- We have a long history of creating open-source projects, such as the Opam package manager, the LearnOCaml web platform, and contributing to other ones, such as the Flambda optimizing compiler, or the GnuCOBOL compiler.
- We are also experts of Formal Methods, developing tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users' Club) and using them to prove safety or security properties of programs.
Please reach out, we'll be delighted to discuss your challenges: contact@ocamlpro.com or book a quick discussion.
Most Recent Articles
2024
- Archéologie de la Généalogie: prise en main et optimisation d’un logiciel vieux de 30 ans
- Alt-Ergo 2.6 is Out!
- Flambda2 Ep. 3: Speculative Inlining
- opam 2.2.0 release!
- Flambda2 Ep. 2: Loopifying Tail-Recursive Functions
- Fixing and Optimizing the GnuCOBOL Preprocessor
- OCaml Backtraces on Uncaught Exceptions
- Opam 102: Pinning Packages
- Flambda2 Ep. 1: Foundational Design Decisions
- Behind the Scenes of the OCaml Optimising Compiler Flambda2: Introduction and Roadmap
- Lean 4: When Sound Programs become a Choice
- Opam 101: The First Steps
2023
- Maturing Learn-OCaml to version 1.0: Gateway to the OCaml World
- The latest release of Alt-Ergo version 2.5.1 is out, with improved SMT-LIB and bitvector support!
- 2022 at OCamlPro
- Autofonce, GNU Autotests Revisited
- Sub-single-instruction Peano to machine integer conversion
- Statically guaranteeing security properties on Java bytecode: Paper presentation at VMCAI 23
- Release of ocplib-simplex, version 0.5
- The Growth of the OCaml Distribution