Packing and Functors
We have recently worked on modifying the OCaml system to be able to pack a set of modules within a functor, parameterized on some signatures. This page presents this work, funded by Jane Street.
All the patches on this page are provided for OCaml version 3.12.1.
Packing Functors
Installation of the modified OCaml system
The patch for OCaml 3.12.1 is available here:
ocaml+libfunctor-3.12.1.patch.gz (26 kB)
To use it, you can use the following recipe, that will compile and
install the patched version in ~/ocaml+libfunctor-3.12.1/bin/
.
~% wget http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz
~% tar zxf ~/ocaml-3.12.1.tar.gz
~% cd ocaml-3.12.1
~/ocaml-3.12.1% wget ocamlpro.com/code/ocaml+libfunctor-3.12.1.patch.gz
~/ocaml-3.12.1% gzip -d ocaml+libfunctor-3.12.1.patch.gz
~/ocaml-3.12.1% patch -p1 < ocaml+libfunctor-3.12.1.patch
~/ocaml-3.12.1% ./configure –prefix ~/ocaml+libfunctor-3.12.1
~/ocaml-3.12.1% make coldstart
~/ocaml-3.12.1% make ocamlc ocamllex ocamltools
~/ocaml-3.12.1% make library-cross
~/ocaml-3.12.1% make bootstrap
~/ocaml-3.12.1% make all opt opt.opt
~/ocaml-3.12.1% make install
~/ocaml-3.12.1% cd ~
~% export PATH=$HOME/ocaml+libfunctor-3.12.1/bin:$PATH
Note that it needs to bootstrap the compiler, as the format of object files is not compatible with the one of ocaml-3.12.1.
Usage of the lib-functor patch.
Now that you are equiped with the new system, you can start using it. The lib-functor patch adds two new options to the compilers ocamlc and ocamlopt:
-
-functor <interface_file>
: this option is used to specify that the current module is compiled with the interface files specifying the argument of the functor. This option should be used together with -for-pack, where is the name of the module in which the current module will be embedded. -
-pack-functor <module>
: this option is used to pack the modules. It should be used with the option -o <object_file> to specify in which module it should be embedded. Thespecified with -pack-functor specifies the name of functor that will be created in the target object file.
If the interface x.mli contains :
type t
val compare : t -> t -> int
and the files xset.ml
and xmap.ml
contain respectively :
module T = Set.Make(X)
module T = Map.Make(X)
Then :
~/test% ocamlopt -c -for-pack Xx -functor x.cmi xset.ml
~/test% ocamlopt -c -for-pack Xx -functor x.cmi xmap.ml
~/test% ocamlopt -pack-functor MakeSetAndMap -o xx.cmx xset.cmx xmap.cmx
will construct a compiled unit whose signature is (that you can get
with ocamlopt -i xx.cmi
, see below) :
module MakeSetAndMap :
functor (X : sig type t val compare : t -> t -> int end) -> sig
module Xset : sig
module T : sig
type elt = X.t
type t = Set.Make(X).t
val empty : t
val is_empty : t -> bool
…
end
end
module Xmap : sig
module T : sig
type key = X.t
type ‘a t = ‘a Map.Make(X).t
val empty : ‘a t
val is_empty : ‘a t -> bool
…
end
end
end
Other extension: printing interfaces
OCaml only allows you to print the interface of a module or interface by compiling its source with the -i option. However, you don’t always have the source of an object interface (in particular, if it was generated by packing), and you might still want to do it.
In such a case, the lib-functor patch allows you to do that, by using the -i option on an interface object file:
~/test% cat > a.mli
val x : int
~/test% ocamlc -c -i a.mli
val x : int
~/test% ocamlc -c -i a.cmi
val x : int
Other extension: packing interfaces
OCaml only allows you to pack object files inside another object file (.cmo or .cmx). When doing so, you can either provide an source interface (.mli) that you need to compile to provide the corresponding object interface (.cmi), or the object interface will be automatically generated by exporting all the sub-modules within the packed module.
However, sometimes, you would want to be able to specify the interfaces of each module separately, so that:
-
you can reuse most of the interfaces you already specified
-
you can use a different interface for a module, that the one used to compile the other modules. This happens when you want to export more values to the other internal sub-modules than you want to export to the user.
In such a case, the lib-functor patch allows you to do that, by using the -pack option on interface object files:
test% cat > a.mli
val x : int
test% cat > b.mli
val y : string
test% ocamlc -c a.mli b.mli
test% ocamlc -pack -o c.cmi a.cmi b.cmi
test% ocamlc -i c.cmi
module A : sig val x : int end
module B : sig val y : string end
Using ocp-pack
to pack source files
Installation of ocp-pack
Download the source file from:
ocp-pack-1.0.1.tar.gz
(20 kB, GPL Licence, Copyright OCamlPro SAS)
Then, you just need to compile it with:
~% tar zxf ocp-pack-1.0.1.tar.gz
~% cd ocp-pack-1.0.1
~/ocp-pack-1.0.1% make
~/ocp-pack-1.0.1% make install
Usage of ocp-pack
ocp-pack
can be used to pack source files of modules within just one
source file. It allows you to avoid the use of the -pack
option, that
is not always supported by all ocaml tools (for example,
ocamldoc
). Moreover, ocp-pack
tries to provide the correct locations
to the compiler, so errors are not reported within the generated
source file, but within the original source files.
It supports the following options:
% ocp-pack -help
Usage:
ocp-pack -o target.ml [options] files.ml*
Options:
-o <filename.ml> generate filename filename.ml
-rec use recursive modules
all .ml files must have a corresponding .mli file
-pack-functor <modname> create functor with name <modname>
-functor <filename.mli> use filename as an argument for functor
-mli output the .mli file too
.ml files without .mli file will not export any value
-no-ml do not output the .ml file
-with-ns use directory structure to create a hierarchy of modules
-v increment verbosity
–version display version information
ocp-pack
automatically detects interface sources and implementation
sources. When only the interface source is available, it is assumed
that it is a type-only module, i.e. no val items are present inside.
Here is an example of using ocp-pack
to build the ocamlgraph package:
test% ocp-pack -o graph.ml
lib/bitv.ml lib/heap.ml lib/unionfind.ml
src/sig.mli src/dot_ast.mli src/sig_pack.mli
src/version.ml src/util.ml src/blocks.ml
src/persistent.ml src/imperative.ml src/delaunay.ml
src/builder.ml src/classic.ml src/rand.ml src/oper.ml
src/path.ml src/traverse.ml src/coloring.ml src/topological.ml
src/components.ml src/kruskal.ml src/flow.ml src/graphviz.ml
src/gml.ml src/dot_parser.ml src/dot_lexer.ml src/dot.ml
src/pack.ml src/gmap.ml src/minsep.ml src/cliquetree.ml
src/mcs_m.ml src/md.ml src/strat.ml
test% ocamlc -c graph.ml
test% ocamlopt -c graph.ml
The -with-ns option can be used to automatically build a hierarchy of modules. With that option, sub-directories are seen as sub-modules. For example, packing a/x.ml, a/y.ml and b/z.ml will give a result like:
[code language=”fsharp”] module A = struct module X = struct … end module Y = struct … end end module B = struct module Z = struct … end end [/code]
Packing modules as functors
The -pack-functor
and -functor
options provide the same behavior
as the same options with the lib-functor patch. The only difference is
that -functor
takes the interface source as argument, not the
interface object.
Packing recursive modules
When trying to pack modules with ocp-pack
, you might discover that
your toplevel modules have recursive dependencies. This is usually
achieved by types declared abstract in the interfaces, but depending
on each other in the implementations. Such modules cannot simply
packed by ocp-pack
.
To handle them, ocp-pack
provides a -rec
option. With that option,
modules are put within a module rec construct, and are all required to
be accompagnied by an interface source file.
Moreover, in many cases, OCaml is not able to compile such recursive modules:
-
For typing reasons: recursive modules are typed in an environment containing only an approximation of other recursive modules signatures
-
For code generation reasons: recursive modules can be reordered depending on their shape, and this reordering can generate an order that is actually not safe, leading to an exception at runtime
To solve these two issues in most cases, you can use the following patch (you can apply it using the same recipe as for lib-functor, and even apply both patches on the same sources):
ocaml+rec-3.12.1.patch.gz
With this patch, recursive modules are typed in an environment that is enriched progressively with the final types of the modules as soon as they become available. Also, during code generation, a topological order is computed on the recursive modules, and the subset of modules that can be initialized using in that topological order are immediatly generated, leaving only the other modules to be reordered.
About OCamlPro:
OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from experts with a state-of-the-art knowledge of programming languages theory and practice.
- We provide audit, support, custom developer tools and training for both the most modern languages, such as Rust, Wasm and OCaml, and for legacy languages, such as COBOL or even home-made domain-specific languages;
- We design, create and implement software with great added-value for our clients. High complexity is not a problem for our PhD-level experts. For example, we helped the French Income Tax Administration re-adapt and improve their internally kept M language, we designed a DSL to model and express revenue streams in the Cinema Industry, codename Niagara, and we also developed the prototype of the Tezos proof-of-stake blockchain from 2014 to 2018.
- We have a long history of creating open-source projects, such as the Opam package manager, the LearnOCaml web platform, and contributing to other ones, such as the Flambda optimizing compiler, or the GnuCOBOL compiler.
- We are also experts of Formal Methods, developing tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users' Club) and using them to prove safety or security properties of programs.
Please reach out, we'll be delighted to discuss your challenges: contact@ocamlpro.com or book a quick discussion.
Most Recent Articles
2024
- Alt-Ergo 2.6 is Out!
- Flambda2 Ep. 3: Speculative Inlining
- opam 2.2.0 release!
- Flambda2 Ep. 2: Loopifying Tail-Recursive Functions
- Fixing and Optimizing the GnuCOBOL Preprocessor
- OCaml Backtraces on Uncaught Exceptions
- Opam 102: Pinning Packages
- Flambda2 Ep. 1: Foundational Design Decisions
- Behind the Scenes of the OCaml Optimising Compiler Flambda2: Introduction and Roadmap
- Lean 4: When Sound Programs become a Choice
- Opam 101: The First Steps
2023
- Maturing Learn-OCaml to version 1.0: Gateway to the OCaml World
- The latest release of Alt-Ergo version 2.5.1 is out, with improved SMT-LIB and bitvector support!
- 2022 at OCamlPro
- Autofonce, GNU Autotests Revisited
- Sub-single-instruction Peano to machine integer conversion
- Statically guaranteeing security properties on Java bytecode: Paper presentation at VMCAI 23
- Release of ocplib-simplex, version 0.5
- The Growth of the OCaml Distribution