new opam features: "opam build"
UPDATE: after discussions following this post, this feature was abandoned with the interface presented below. See this post for the details and the new interface!
The new opam 2.0 release, currently in beta, introduces several new features.
This post gets into some detail on the new
opam build command, its purpose,
its use, and some implementation aspects.
opam build is run from the source tree of a project, and does not rely on a
pre-existing opam installation. As such, it adds a new option besides the
existing workflows based on managing shared OCaml installations in the form of
What does it do ?
Typically, this is used in a fresh git clone of some OCaml project. Like when
pinning the package, opam will find and leverage package definitions found in
the source, in the form of
- if opam hasn't been initialised (no
~/.opam), this is taken care of.
- if no switch is otherwise explicitely selected, a local switch is used, and
created if necessary (i.e. in
- the metadata for the current project is registered, and the package installed after its dependencies, as opam usually does
This is particularly useful for distributing projects to people not used to
opam and the OCaml ecosystem: the setup steps are automatically taken care of,
and a single
opam build invocation can take care of resolving the dependency
chains for your package.
If building the project directly is preferred, adding
--deps-only is a good
way to get the dependencies ready for the project:
opam build --deps-only eval $(opam config env) ./configure; make; etc.
Note that if you just want to handle project-local opam files,
opam build can
also be used in your existing switches: just specify
or make sure the
OPAMSWITCH variable is set. E.g.
opam build --no-autoinit --deps-only is a convenient way to get the dependencies for the local project
ready in your current switch.
The installation of the packages happens as usual to the prefix corresponding to
the switch used (
<project-root>/_opam/ for a local switch). But it is
--install-prefix, to further install the package to the system:
opam build --install-prefix ~/local
will install the results of the package found in the current directory below ~/local.
The dependencies of the package won't be installed, so this is intended for programs, assuming they are relocatable, and not for libraries.
Choosing custom repositories
The user can pre-select the repositories to use on the creation of the local switch with:
opam build --repositories <repos>
<repos> is a comma-separated list of repositories, specified either as
name if already configured on the system.
Multiple packages are commonly found to share a single repository. In this case,
opam build registers and builds all of them, respecting cross-dependencies.
The opam files to use can also be explicitely selected on the command-line.
In this case, specific opam files must be named
The choice of the compiler, on automatic initialisation, is either explicit,
--compiler option, or automatic. In the latter case, the default
selection is used (see
opam init --help, section "CONFIGURATION FILE" for
details), but a compiler compatible with the local packages found is searched
from that. This allows, for example, to choose a system compiler when available
and compatible, avoiding a recompilation of OCaml.
--install-prefix, the normal installation is done, then the
tracking of package-installed files, introduced in opam 2.0, is used to extract
the installed files from the switch and copy them to the prefix.
The packages installed through
opam build are not registered in any
repository, and this is not an implicit use of
opam pin: the rationale is that
packages installed this way will also be updated by repeating
opam build. This
means that when using other commands, e.g.
opam upgrade, opam won't try to
keep the packages to their local, source version, and will either revert them to
their repository definition, or remove them, if they need recompilation.
This is still in beta: there are still rough edges, please experiment and give feedback! It is still possible that the command syntax and semantics change significantly before release.
Another use-case that we are striving to improve is sharing of development
setups (share sets of pinned packages, depend on specific remotes or git hashes,
etc.). We have many
improve on this, but
is not, as of today, a direct solution to this. In particular, installing this
way still relies on the default opam repository; a way to define specific
options for the switch that is implicitely created on
opam build is in the
Louis Gesbert (16 March 2017 at 14 h 31 min):
Some discussion on a better naming and making some parts of this more widely available in the opam CLI is ongoing at https://github.com/ocaml/opam/issues/2882
Hez Carty (16 March 2017 at 17 h 23 min):
Is it possible/planned to support sharing of compilers across local (or global) switches? It would be very useful to have a global 4.04.0+flambda switch including only the compiler itself or the compiler + basic tools like ocp-indent and merlin. Then a number of projects could share this base installation but have their own locally installed dependencies without duplicating the entire build time per-project.
Louis Gesbert (17 March 2017 at 10 h 10 min):
Sharing compilers, or other packages across switches is not supported at the moment. However:
You can still use the global
system compileron any switch, local or not, to avoid its recompilation What is planned, as a first step, for after the 2.0 release, is to add a cache of compiled packages. Hooks are already in place to allow this, and opam is able to track the files installed by each package already, so the most difficult part is probably going to be the relocation issues with OCaml itself.
A cache is an easier solution to warrant consistency: with shared switches, the problem of reinstallations and keeping everything consistent gets much more complex — what happens when you change the compiler of your “master” switch ?
Hez Carty (20 March 2017 at 16 h 46 min):
That sounds great, thank you. Should make this kind of local switch more useful when working with large numbers of projects.
OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from state-of-the art programming languages like OCaml and Rust.
We design, create and implement custom ad-hoc software for our clients. We also have a long experience in developing and maintaining open-source tooling for OCaml, such as Opam, TryOCaml, ocp-indent, ocp-index and ocp-browser, and we contribute to the core-development of OCaml, notably with our work on the Flambda optimizer branch.
Another area of expertise is that of Formal Methods, with tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users'). We also provide vocational trainings in OCaml and Rust, and we can build courses on formal methods on-demand. Please reach out, we'll be delighted to discuss your challenges: email@example.com or book a quick discussion.