Module and module unit#

A module consists of one or more module units. A module unit is a special kind of translation unit. A module unit should almost always start with a module declaration. The syntax of the module declaration is:

[export] module module_name[:partition_name];

Terms enclosed in [] are optional. module_name and partition_name follow the rules for a C++ identifier, except that they may contain one or more period (.) characters. Note that a . in the name has no semantic meaning and does not imply any hierarchy.

In this document, module units are classified as:

• Primary module interface unit

• Module implementation unit

• Module partition interface unit

• Internal module partition unit

A primary module interface unit is a module unit whose module declaration is export module module_name; where module_name denotes the name of the module. A module should have one and only one primary module interface unit.

A module implementation unit is a module unit whose module declaration is module module_name;. Multiple module implementation units can be declared in the same module.

A module partition interface unit is a module unit whose module declaration is export module module_name:partition_name;. The partition_name should be unique within any given module.

An internal module partition unit is a module unit whose module declaration is module module_name:partition_name;. The partition_name should be unique within any given module.

In this document, we use the following terms:

• A module interface unit refers to either a primary module interface unit or a module partition interface unit.

• An importable module unit refers to either a module interface unit or an internal module partition unit.

• A module partition unit refers to either a module partition interface unit or an internal module partition unit.

Built Module Interface#

A Built Module Interface (or BMI) is the precompiled result of an importable module unit.

Global module fragment#

The global module fragment (or GMF) is the code between the module; and the module declaration within a module unit.

Quick Start#

Let’s see a “hello world” example that uses modules.

// Hello.cppm
module;
#include <iostream>
export module Hello;
export void hello() {
  std::cout << "Hello World!\n";
}

// use.cpp
import Hello;
int main() {
  hello();
  return 0;
}

Then, on the command line, invoke Clang like:

$ clang++ -std=c++20 Hello.cppm --precompile -o Hello.pcm
$ clang++ -std=c++20 use.cpp -fmodule-file=Hello=Hello.pcm Hello.pcm -o Hello.out
$ ./Hello.out
Hello World!

In this example, we make and use a simple module Hello which contains only a primary module interface unit named Hello.cppm.

A more complex “hello world” example which uses the 4 kinds of module units is:

// M.cppm
export module M;
export import :interface_part;
import :impl_part;
export void Hello();

// interface_part.cppm
export module M:interface_part;
export void World();

// impl_part.cppm
module;
#include <iostream>
#include <string>
module M:impl_part;
import :interface_part;

std::string W = "World.";
void World() {
  std::cout << W << std::endl;
}

// Impl.cpp
module;
#include <iostream>
module M;
void Hello() {
  std::cout << "Hello ";
}

// User.cpp
import M;
int main() {
  Hello();
  World();
  return 0;
}

Then, back on the command line, invoke Clang with:

# Precompiling the module
$ clang++ -std=c++20 interface_part.cppm --precompile -o M-interface_part.pcm
$ clang++ -std=c++20 impl_part.cppm --precompile -fprebuilt-module-path=. -o M-impl_part.pcm
$ clang++ -std=c++20 M.cppm --precompile -fprebuilt-module-path=. -o M.pcm
$ clang++ -std=c++20 Impl.cpp -fprebuilt-module-path=. -c -o Impl.o

# Compiling the user
$ clang++ -std=c++20 User.cpp -fprebuilt-module-path=. -c -o User.o

# Compiling the module and linking it together
$ clang++ -std=c++20 M-interface_part.pcm -fprebuilt-module-path=. -c -o M-interface_part.o
$ clang++ -std=c++20 M-impl_part.pcm -fprebuilt-module-path=. -c -o M-impl_part.o
$ clang++ -std=c++20 M.pcm -fprebuilt-module-path=. -c -o M.o
$ clang++ User.o M-interface_part.o  M-impl_part.o M.o Impl.o -o a.out

We explain the options in the following sections.

How to enable standard C++ modules#

Standard C++ modules are enabled automatically when the language standard mode is -std=c++20 or newer.

How to produce a BMI#

To generate a BMI for an importable module unit, use either the --precompile, --precompile-reduced-bmi, or -fmodule-output command line options.

The --precompile option generates the BMI as the output of the compilation with the output path specified using the -o option.

The --precompile-reduced-bmi option generates a Reduced BMI (See the following section for the definition of Reduced BMI) as the output of the compilation with the output path specified using the -o option.

The -fmodule-output option generates the BMI as a by-product of the compilation. If -fmodule-output= is specified, the BMI will be emitted to the specified location. If -fmodule-output and -c are specified, the BMI will be emitted in the directory of the output file with the name of the input file with the extension .pcm. Otherwise, the BMI will be emitted in the working directory with the name of the input file with the extension .pcm.

Generating BMIs with --precompile is referred to as two-phase compilation because it takes two steps to compile a source file to an object file. Generating BMIs with -fmodule-output is called one-phase compilation. The one-phase compilation model is simpler for build systems to implement while the two-phase compilation has the potential to compile faster due to higher parallelism. As an example, if there are two module units A and B, and B depends on A, the one-phase compilation model needs to compile them serially, whereas the two-phase compilation model can be compiled as soon as A.pcm is available, and thus can be compiled simultaneously with the A.pcm to A.o compilation step.

File name requirements#

By convention, importable module unit files should use .cppm (or .ccm, .cxxm, or .c++m) as a file extension. Module implementation unit files should use .cpp (or .cc, .cxx, or .c++) as a file extension.

A BMI should use .pcm as a file extension. The file name of the BMI for a primary module interface unit should be module_name.pcm. The file name of a BMI for a module partition unit should be module_name-partition_name.pcm.

Clang may fail to build the module if different extensions are used. For example, if the filename of an importable module unit ends with .cpp instead of .cppm, then Clang cannot generate a BMI for the importable module unit with the --precompile option because the --precompile option would only run the preprocessor (-E). If using a different extension than the conventional one for an importable module unit you can specify -x c++-module before the file. For example,

// Hello.cpp
module;
#include <iostream>
export module Hello;
export void hello() {
  std::cout << "Hello World!\n";
}

// use.cpp
import Hello;
int main() {
  hello();
  return 0;
}

In this example, the extension used by the module interface is .cpp instead of .cppm, so it cannot be compiled like the previous example, but it can be compiled with:

$ clang++ -std=c++20 -x c++-module Hello.cpp --precompile -o Hello.pcm
$ clang++ -std=c++20 use.cpp -fprebuilt-module-path=. Hello.pcm -o Hello.out
$ ./Hello.out
Hello World!

Module name requirements#

[module.unit]p1:

All module-names either beginning with an identifier consisting of std followed by zero or more digits or containing a reserved identifier ([lex.name]) are reserved and shall not be specified in a module-declaration; no diagnostic is required. If any identifier in a reserved module-name is a reserved identifier, the module name is reserved for use by C++ implementations; otherwise it is reserved for future standardization.

Therefore, none of the following names are valid by default:

std
std1
std.foo
__test
// and so on ...

Using a reserved module name is strongly discouraged, but -Wno-reserved-module-identifier can be used to suppress the warning.

Specifying BMI dependencies#

There are 3 ways to specify a BMI dependency:

1. -fprebuilt-module-path=<path/to/directory>.

2. -fmodule-file=<path/to/BMI> (Deprecated).

3. -fmodule-file=<module-name>=<path/to/BMI>.

The -fprebuilt-module-path option specifies the path to search for BMI dependencies. Multiple paths may be specified, similar to using -I to specify a search path for header files. When importing a module M, the compiler looks for M.pcm in the directories specified by -fprebuilt-module-path. Similarly, when importing a partition module unit M:P, the compiler looks for M-P.pcm in the directories specified by -fprebuilt-module-path.

The -fmodule-file=<path/to/BMI> option causes the compiler to load the specified BMI eagerly and directly. The -fmodule-file=<module-name>=<path/to/BMI> option causes the compiler to load the specified BMI for the module specified by <module-name> when necessary. The main difference is that -fmodule-file=<path/to/BMI> will load the BMI eagerly, whereas -fmodule-file=<module-name>=<path/to/BMI> will only load the BMI lazily, as will -fprebuilt-module-path. The -fmodule-file=<path/to/BMI> option for named modules is not suggested unless you know what you’re doing as it may change the order the compiler handles the code.

When these options are specified in the same invocation of the compiler, the -fmodule-file=<path/to/BMI> option takes precedence over -fmodule-file=<module-name>=<path/to/BMI>, which takes precedence over -fprebuilt-module-path=<path/to/directory>.

Note: all BMI dependencies must be specified explicitly, either directly or indirectly. See https://github.com/llvm/llvm-project/issues/62707 for details.

When compiling a module implementation unit, the BMI of the corresponding primary module interface unit must be specified because a module implementation unit implicitly imports the primary module interface unit.

[module.unit]p8

A module-declaration that contains neither an export-keyword nor a module-partition implicitly imports the primary module interface unit of the module as if by a module-import-declaration.

The -fprebuilt-module-path=<path/to/directory>, -fmodule-file=<path/to/BMI>, and -fmodule-file=<module-name>=<path/to/BMI> options may be specified multiple times. For example, the command line to compile M.cppm in the previous example could be rewritten as:

$ clang++ -std=c++20 M.cppm --precompile -fmodule-file=M:interface_part=M-interface_part.pcm -fmodule-file=M:impl_part=M-impl_part.pcm -o M.pcm

When there are multiple -fmodule-file=<module-name>= options for the same <module-name>, the last -fmodule-file=<module-name>= overrides the previous -fmodule-file=<module-name>= option.

Remember that module units still have an object counterpart to the BMI#

While module interfaces resemble traditional header files, they still require compilation. Module units are translation units, and need to be compiled to object files, which then need to be linked together as the following examples show.

For example, the traditional compilation processes for headers are like:

src1.cpp -+> clang++ src1.cpp --> src1.o ---,
hdr1.h  --'                                 +-> clang++ src1.o src2.o ->  executable
hdr2.h  --,                                 |
src2.cpp -+> clang++ src2.cpp --> src2.o ---'

And the compilation processes for module units are like:

              src1.cpp ----------------------------------------+> clang++ src1.cpp -------> src1.o -,
(header unit) hdr1.h    -> clang++ hdr1.h ...    -> hdr1.pcm --'                                    +-> clang++ src1.o mod1.o src2.o ->  executable
              mod1.cppm -> clang++ mod1.cppm ... -> mod1.pcm --,--> clang++ mod1.pcm ... -> mod1.o -+
              src2.cpp ----------------------------------------+> clang++ src2.cpp -------> src2.o -'

As the diagrams show, we need to compile the BMI from module units to object files and then link the object files. (However, this cannot be done for the BMI from header units. See the section on header units for more details.)

BMIs cannot be shipped in an archive to create a module library. Instead, the BMIs(*.pcm) are compiled into object files(*.o) and those object files are added to the archive instead.

clang-cl#

clang-cl supports the same options as clang++ for modules as detailed above; there is no need to prefix these options with /clang:. Note that cl.exe options to emit/consume IFC files are not supported. The resulting precompiled modules are also not compatible for use with cl.exe.

We recommend that build system authors use the above-mentioned clang++ options with clang-cl to build modules.

Consistency Requirements#

Modules can be viewed as a kind of cache to speed up compilation. Thus, like other caching techniques, it is important to maintain cache consistency, which is why Clang does very strict checking for consistency.

// M.cppm
export module M;

// Use.cpp
import M;

$ clang++ -std=c++20 M.cppm --precompile -o M.pcm
$ clang++ -std=c++23 Use.cpp -fprebuilt-module-path=.

$ clang++ -std=c++20 M.cppm --precompile -o M.pcm
# Inconsistent optimization level.
$ clang++ -std=c++20 -O3 Use.cpp -fprebuilt-module-path=.
# Inconsistent debugging level.
$ clang++ -std=c++20 -g Use.cpp -fprebuilt-module-path=.

$ clang++ -std=c++20 M.cppm --precompile -o M.pcm
# Inconsistent optimization level.
$ clang++ -std=c++20 -O3 -DNDEBUG Use.cpp -fprebuilt-module-path=.

Name Mangling#

The declarations in a module unit which are not in the global module fragment have new linkage names.

For example,

export module M;
namespace NS {
  export int foo();
}

The linkage name of NS::foo() is _ZN2NSW1M3fooEv. This couldn’t be demangled by previous versions of the debugger or demangler. As of LLVM 15.x, llvm-cxxfilt can be used to demangle this:

$ llvm-cxxfilt _ZN2NSW1M3fooEv
  NS::foo@M()

The result should be read as NS::foo() in module M.

The ABI implies that something cannot be declared in a module unit and defined in a non-module unit (or vice-versa), as this would result in linking errors.

Despite this, it is possible to implement declarations with a compatible ABI in a module unit by using a language linkage specifier because the declarations in the language linkage specifier are attached to the global module fragment. For example:

export module M;
namespace NS {
  export extern "C++" int foo();
}

Now the linkage name of NS::foo() will be _ZN2NS3fooEv.

Module Initializers#

All importable module units are required to emit an initializer function to handle the dynamic initialization of non-inline variables in the module unit. The importable module unit has to emit the initializer even if there is no dynamic initialization; otherwise, the importer may call a nonexistent function. The initializer function emits calls to imported modules first followed by calls to all of the dynamic initializers in the current module unit.

Translation units that explicitly or implicitly import a named module must call the initializer functions of the imported named module within the sequence of the dynamic initializers in the translation unit. Initializations of entities at namespace scope are appearance-ordered. This (recursively) extends to imported modules at the point of appearance of the import declaration.

If the imported module is known to be empty, the call to its initializer may be omitted. Additionally, if the imported module is known to have already been imported, the call to its initializer may be omitted.

Interactions with Reduced BMI#

With reduced BMI, non-cascading changes can be more powerful. For example,

// A.cppm
export module A;
export int a() { return 44; }

// B.cppm
export module B;
import A;
export int b() { return a(); }

$ clang++ -std=c++20 A.cppm -c -fmodule-output=A.pcm  -fmodules-reduced-bmi -o A.o
$ clang++ -std=c++20 B.cppm -c -fmodule-output=B.pcm  -fmodules-reduced-bmi -o B.o -fmodule-file=A=A.pcm
$ md5sum B.pcm
6c2bd452ca32ab418bf35cd141b060b9  B.pcm

And let’s change the implementation for A.cppm to:

export module A;
int a_impl() { return 99; }
export int a() { return a_impl(); }

and recompile the example:

$ clang++ -std=c++20 A.cppm -c -fmodule-output=A.pcm  -fmodules-reduced-bmi -o A.o
$ clang++ -std=c++20 B.cppm -c -fmodule-output=B.pcm  -fmodules-reduced-bmi -o B.o -fmodule-file=A=A.pcm
$ md5sum B.pcm
6c2bd452ca32ab418bf35cd141b060b9  B.pcm

We should find the contents of B.pcm remain the same. In this case, the build system is allowed to skip recompilations of TUs which solely and directly depend on module B.

This only happens with a reduced BMI. With reduced BMIs, we won’t record the function body of int b() in the BMI for B so that the module A doesn’t contribute to the BMI of B and we have less dependencies.

Reduce duplications#

While it is valid to have duplicated declarations in the global module fragments of different module units, it is not free for Clang to deal with the duplicated declarations. A translation unit will compile more slowly if there are a lot of duplicated declarations between the translation unit and modules it imports. For example:

// M-partA.cppm
module;
#include "big.header.h"
export module M:partA;
...

// M-partB.cppm
module;
#include "big.header.h"
export module M:partB;
...

// other partitions
...

// M-partZ.cppm
module;
#include "big.header.h"
export module M:partZ;
...

// M.cppm
export module M;
export import :partA;
export import :partB;
...
export import :partZ;

// use.cpp
import M;
... // use declarations from module M.

When big.header.h is big enough and there are a lot of partitions, the compilation of use.cpp may be significantly slower than the following approach:

module;
#include "big.header.h"
export module m:big.header.wrapper;
export ... // export the needed declarations

// M-partA.cppm
export module M:partA;
import :big.header.wrapper;
...

// M-partB.cppm
export module M:partB;
import :big.header.wrapper;
...

// other partitions
...

// M-partZ.cppm
export module M:partZ;
import :big.header.wrapper;
...

// M.cppm
export module M;
export import :partA;
export import :partB;
...
export import :partZ;

// use.cpp
import M;
... // use declarations from module M.

Reducing the duplication from textual includes is what improves compile-time performance.

To help users to identify such issues, we add a warning -Wdecls-in-multiple-modules. This warning is disabled by default and it needs to be explicitly enabled or by -Weverything.

ABI non-breaking styles#

module;
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
export module your_library;
export namespace your_namespace {
  using decl_1;
  using decl_2;
  ...
  using decl_n;
}

module;
#include "third_party/A/headers.h"
#include "third_party/B/headers.h"
...
#include "third_party/Z/headers.h"
export module your_library;
#define IN_MODULE_INTERFACE
extern "C++" {
  #include "header_1.h"
  #include "header_2.h"
  ...
  #include "header_n.h"
}

#ifdef IN_MODULE_INTERFACE
#define EXPORT export
#else
#define EXPORT
#endif

#ifndef IN_MODULE_INTERFACE
#include "third_party/A/headers.h"
#endif

#include "header_x.h"

...

ABI breaking style#

The term ABI breaking may sound like a bad approach. However, this style forces consumers of the library use it in a consistent way. e.g., either always include headers for the library or always import modules. The style prevents the ability to mix includes and imports for the library.

The pattern for ABI breaking style is similar to the export extern-C++ style.

module;
#include "third_party/A/headers.h"
#include "third_party/B/headers.h"
...
#include "third_party/Z/headers.h"
export module your_library;
#define IN_MODULE_INTERFACE
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"

#if the number of .cpp files in your project are small
module :private;
#include "source_1.cpp"
#include "source_2.cpp"
...
#include "source_n.cpp"
#else // the number of .cpp files in your project are a lot
// Using all the declarations from third-party libraries which are
// used in the .cpp files.
namespace third_party_namespace {
  using third_party_decl_used_in_cpp_1;
  using third_party_decl_used_in_cpp_2;
  ...
  using third_party_decl_used_in_cpp_n;
}
#endif

(And add EXPORT and conditional include to the headers as suggested in the export extern-C++ style section.)

The ABI with modules is different and thus we need to compile the source files into the new ABI. This is done by an additional part of the interface unit:

#if the number of .cpp files in your project are small
module :private;
#include "source_1.cpp"
#include "source_2.cpp"
...
#include "source_n.cpp"
#else // the number of .cpp files in your project are a lot
// Using all the declarations from third-party libraries which are
// used in the .cpp files.
namespace third_party_namespace {
  using third_party_decl_used_in_cpp_1;
  using third_party_decl_used_in_cpp_2;
  ...
  using third_party_decl_used_in_cpp_n;
}
#endif

If the number of source files is small, everything can be put in the private module fragment directly (it is recommended to add conditional includes to the source files as well). However, compile time performance will be bad if there are a lot of source files to compile.

Note that the private module fragment can only be in the primary module interface unit and the primary module interface unit containing the private module fragment should be the only module unit of the corresponding module.

In this case, source files (.cpp files) must be converted to module implementation units:

#ifndef IN_MODULE_INTERFACE
// List all the includes here.
#include "third_party/A/headers.h"
...
#include "header.h"
#endif

module your_library;

// Following off should be unchanged.
...

The module implementation unit will import the primary module implicitly. Do not include any headers in the module implementation units as it avoids duplicated declarations between translation units. This is why non-exported using declarations should be added from third-party libraries in the primary module interface unit.

If the library is provided as libyour_library.so, a modular library (e.g., libyour_library_modules.so) may also need to be provided for ABI compatibility.

// header.h
#pragma once

#include <cstdint>

namespace example {
class C {
public:
    std::size_t inline_get() { return 42; }
    std::size_t get();
};
}

// src.cpp
#include "header.h"

std::size_t example::C::get() {
    return 43 + inline_get();
}

$nm -ACD libexample.so
libexample.so:0000000000001130 W example::C::inline_get()
libexample.so:0000000000001110 T example::C::get()

// example.cppm
export module example;
import std;
// and other third-party modules, if any
#define IN_MODULE_WRAPPER
#include "header.h" // omit changing of header.h for brevity

// src.module.cpp
module example;
#define IN_MODULE_IMPL
#include "src.cpp" // omit changing of src.cpp for brevity

$llvm-nm -ACD libexample.so
libexample.so: 0000000000001060 T initializer for module example
libexample.so: 0000000000001150 W example::C::inline_get()
libexample.so: 0000000000001130 T example::C::get()
libexample.so: 0000000000001180 T example::C@example::inline_get()
libexample.so: 0000000000001160 T example::C@example::get()

#include "header.h"

namespace user {
    void user_def(example::C& c) {

    }
}

$ llvm-nm -ACD libuser.so
libuser.so: 0000000000001100 T user::user_def(example::C&)

import example;

namespace user {
    void user_def(example::C& c) {

    }
}

$ llvm-nm -ACD libuser.so
libuser.so: 0000000000001100 T user::user_def(example::C@example&)

// your_header.h
#include <cstdint>

inline void your_interface() {}

// your_header.h
#ifndef IN_MODULE_WRAPPER
#include <cstdint>
#endif

#ifdef IN_MODULE_WRAPPER
#define MY_EXPORT export
#else
#define MY_EXPORT
#endif

#ifdef IN_MODULE_WRAPPER
#define MY_INLINE
#else
#define MY_INLINE inline
#endif

MY_EXPORT MY_INLINE void your_interface() {}

// your_module_interface.cppm
export module your_library;
import std;
#define IN_MODULE_WRAPPER
#include "your_header.h"

Providing a header to skip parsing redundant headers#

Many redeclarations shared between translation units cause Clang to have slower compile-time performance. Further, there are known issues with include after import. Even when that issue is resolved, users may still get slower compilation speed and larger BMIs. For these reasons, it is recommended to not include headers after importing the corresponding module. However, it is not always easy if the library is included by other dependencies, as in:

#include "third_party/A.h" // #include "your_library/a_header.h"
import your_library;

or

import your_library;
#include "third_party/A.h" // #include "your_library/a_header.h"

For such cases, it is best if the library providing both module and header interfaces also provides a header which skips parsing so that the library can be imported with the following approach that skips redundant redeclarations:

import your_library;
#include "your_library_imported.h"
#include "third_party/A.h" // #include "your_library/a_header.h" but got skipped

The implementation of your_library_imported.h can be a set of controlling macros or an overall controlling macro if using #pragma once. Then headers can be refactored to:

#pragma once
#ifndef YOUR_LIBRARY_IMPORTED
...
#endif

If the modules imported by the library provide such headers, remember to add them to your_library_imported.h too.

Importing modules#

When there are library dependencies providing modules, the module dependencies should be imported in your module as well. Many existing libraries will fall into this category once the std module is more widely available.

module;
import modules_from_third_party;
#define IN_MODULE_INTERFACE
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
export module your_library;
export namespace your_namespace {
  using decl_1;
  using decl_2;
  ...
  using decl_n;
}

export module your_library;
import modules_from_third_party;
#define IN_MODULE_INTERFACE
extern "C++" {
  #include "header_1.h"
  #include "header_2.h"
  ...
  #include "header_n.h"
}

export module your_library;
import modules_from_third_party;
#define IN_MODULE_INTERFACE
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"

#if the number of .cpp files in your project are small
module :private;
#include "source_1.cpp"
#include "source_2.cpp"
...
#include "source_n.cpp"
#endif

Problems Background#

As discussed before, the redeclaration in different TU is one of the major problems of using modules from the perspective of the compiler. The redeclaration pattern is a major trigger of compiler bugs. And even if the compiler accepts the redeclaration pattern as expected, the compilation performance will be affected too.

e.g,

// a.h
#pragma once
class A { ... };

// a.cppm
module;
#include "a.h"
export module a;
export using ::A;

// a.cc
import a;
#include "a.h"
A a;

Here in a.cc, we have redeclaration for A, one from a.cppm and one from a.cc itself.

To avoid the redeclaration pattern, in previous section, we suggested users to comment out thirdparty headers manually.

And here we will introduce another approach to avoid such redeclaration pattern by using clang module map.

Clang Module Map Background#

Clang Module Map is a feature of Clang Header Modules. See Clang Module for full introduction of Clang Header Modules. Here we would only introduce Clang Header Modules to make this document self contained.

// a.h
#pragma once
class A { ... };

// a.cppm
module;
#include "a.h"
export module a;
export using ::A;

// a.cc
import a;
#include "a.h"
A a;

Examples#

To use Clang Module Map for C++20 Named Modules, end users have to wait for the support from build systems. Here we ignore the build systems to help users to understand the mechanism.

Here is an example of using clang module map to replace a header to an import of a module.

// a.h
#pragma once
static_assert(false, "don't include a.h");

// main.cpp
#include "a.h"
int main() {
    return 0;
}

// a.cppm
module;
#include <iostream>
export module a;
struct Init {
    Init() {
        std::cout << "Module 'a' got imported" << std::endl;
    }
};
Init a;

// a.cppm.modulemap
module a {
  header "a.h"
}

Then invoke Clang with:

$ clang++ -std=c++20 a.cppm -c -fmodule-output=a.pcm -o a.o
$ clang++ -std=c++20 main.cpp -fmodule-map-file=a.cppm.modulemap -fmodule-file=a=a.pcm a.o -o main
$ ./main
Module 'a' got imported

We can find that the header file a.h is not included actually (otherwise the compilation should fail due to the static assert). And it imports the module a and then the varaible in module a got initialized.

The secret comes from the flag -fmodule-map-file=a.cppm.modulemap, the content of a.cppm.modulemap says: map the #include of a.h to the import to module a. Then when the compiler sees #include "a.h", the compiler won’t include a.h actually but tries to import the module a. And the from the command line -fmodule-file=a=a.pcm, the compiler get the module file of module a, then module file of module a get imported and the inclusion of a.h is skipped.

Then we can try to use the mechanism to avoid redeclaration pattern for header wrapping modules.

// a.h
#pragma once
class A { ... };

// a.cppm
module;
#include "a.h"
export module a;
export using ::A;

// a.cc
import a;
#include "a.h"
A a;

// a.cppm.modulemap
module a {
  header "a.h"
}

Similarly, when we compile a.cc, if we add the flag -fmodule-map-file=a.cppm.modulemap, the compiler will map the inclusion of a.h to the import of module a. And the module a is already imported. So we avoid the redeclaration of class A in a.cc.

An imaginable problem with this approach maybe the hidden inclusion. e.g,

// b.h
#pragma once
struct B {};

// a.h
#pragma once
#include "b.h"
struct A { B b; };

// b.cppm
export module b;
export extern "C++" struct B { };

// a.cppm
export module a;
import b;
export extern "C++" struct A { B b; };

// test.cc
import a;
#include "a.h"
A a;
B b;

// a.cppm.modulemap
module a {
  header "a.h"
}

// b.cppm.modulemap
module b {
  header "b.h"
}

The example is valid if we don’t use the module map:

$ clang++ -std=c++20 b.cppm -c -fmodule-output=b.pcm -o b.o
$ clang++ -std=c++20 a.cppm -c -fmodule-output=a.pcm -fmodule-file=b=b.pcm -o a.o
$ clang++ -std=c++20 test.cc -fmodule-file=a=a.pcm  -fmodule-file=b=b.pcm  -fsyntax-only

But if we enable the module map, the example is invalid:

$ clang++ -std=c++20 test.cc -fmodule-map-file=a.cppm.modulemap -fmodule-file=a=a.pcm -fmodule-map-file=b.cppm.modulemap -fmodule-file=b=b.pcm -fsyntax-only
test.cc:4:1: error: declaration of 'B' must be imported from module 'b' before it is required
    4 | B b;
      | ^
b.cppm:2:28: note: declaration here is not visible
    2 | export extern "C++" struct B { };
      |                            ^
1 error generated.

A suggested convention for end users and build systems#

As said, the build system is a vital role in this strategy. However, for build systems, it is not easy to support clang explicit header modules or support the module map with C++20 named modules generally. The complexity for build system won’t be less than supporting C++20 named modules.

So here we suggest a convention between end users and build systems to ease the implementation burden of build systems and help end users to avoid the redeclaration problem from mixing #include and import.

For end users who is the author of header based library offering named module wrappers, The header’s interface should be a subset of the module interface excluding user-facing macros.

• Extract all user facing headers into a single header file. Since C++20 named modules

• For each named module interface, provide a module map file to map the interface headers to the named module. The name of the module map should be the name of the module interface unit plus .modulemap.

The number of the module map may not be a lot sicne this is still a header based library.

For build systems,

• For each Translation Units, if the unit doesn’t import any named modules, stop. This is not what we want.

• If the TU imports named module, for all imported named module unit, look up for the module map file in the same path of the imported module unit with the name of the module unit plus .modulemap. e.g., if the name of the module unit is a.cppm, we should lookup for a.cppm.modulemap.

• For the found module map, pass -fmodule-map-file=<module_map_file_path> to the clang compiler.

The point of the approach is, the build system can reuse the result of C++20 named modules to manage depencies. So that the implementation burden of build systems is largely reduced.

Preload the std module#

We can assume almost all C++ TU depends on the STL. So we can assume it is safe to import std in almost all C++ TUs. And clang have a mechanism to load the corresponding module eagerly: -fmodule-file=<path-to-BMI>.

So if we built the std module’s BMI ahead of time, we can pass -fmodule-file=<path-to-BMI-of-std-module> so that the code:

// hello.cpp
#include <string>
#include <iostream>
int main() {
  std::string hello = "Hello World.\n";
  std::cout << hello << std::endl;
  return 0;
}

will be effectively equal to:

import std;
#include <string>
#include <iostream>
int main() {
  std::string hello = "Hello World.\n";
  std::cout << hello << std::endl;
  return 0;
}

Let’s take a quick measure (the numbers may vary in different environments):

$ time clang++ -std=c++23 hello.cpp -o hello

real    0m1.245s
user    0m1.154s
sys     0m0.085s

and let’s build the std module first and preload it without changing hello.cpp.

$ clang++ -std=c++23 <path-to-std-module>/std.cppm -c -o std.o -fmodule-output=std.pcm
$ time clang++ -std=c++23 hello.cpp -o hello -fmodule-file=std.pcm

real    0m0.795s
user    0m0.707s
sys     0m0.084s

we should be able to observe a clear compilation reduction.

Clang has some optimizations for the case of import-before-include but also the pattern import-before-include may trigger compiler internal issues. But bugs are bugs. They should be reported and fixed.

However, it is straight-forward that

import std;
#include <string>
#include <iostream>

will be slower to be compiled than

import std;

And due to compiler internal data structures, the mixed form will be much more slower than the pure import style due to the same declration apears in different TU issues (see above sections for more discussions).

Use Module Map to replace includes to imports#

So is there any tricks we can use to replace the includes with the imports without changing the source code? Yes, clang has a module map. Although clang’s module map was invented for clang header module maps, it can be used with named modules as well.

Let’s start with a simple example:

// std.modulemap
module std [system] {
  requires cplusplus

  header "algorithm"
  header "bitset"
  header "complex"
  header "deque"
  header "exception"
  header "fstream"
  header "functional"
  header "iomanip"
  header "ios"
  header "iosfwd"
  header "iostream"
  header "istream"
  header "iterator"
  header "limits"
  header "list"
  header "locale"
  header "map"
  header "memory"
  header "new"
  header "numeric"
  header "ostream"
  header "queue"
  header "set"
  header "sstream"
  header "stack"
  header "stdexcept"
  header "streambuf"
  header "string"
  header "typeinfo"
  header "utility"
  header "valarray"
  header "vector"
  header "array"
  header "atomic"
  header "chrono"
  header "codecvt"
  header "condition_variable"
  header "forward_list"
  header "future"
  header "initializer_list"
  header "mutex"
  header "ratio"
  header "regex"
  header "scoped_allocator"
  header "system_error"
  header "thread"
  header "tuple"
  header "typeindex"
  header "unordered_map"
  header "unordered_set"
  header "optional"
  header "any"
  header "variant"
}

the paths of the files referenced in module map is relative to the module map. So they can’t be associated with the system headers.

There are two solutions for this. First, we can create the softlinks for the headers:

mkdir -p std-module-map
cp std.modulemap std-module-map/.
cd std-module-map

libcxx_dir=...

awk -F '"' '/^[[:space:]]*header[[:space:]]+"/ {print $2}' std.modulemap |
while IFS= read -r header; do
  ln -s "$libcxx_dir/$header" "$header"
done

then

$ time bin/clang++ -std=c++23 hello.cpp std.o -o hello $LIBCXX_FLAGS -fmodule-map-file=std-module-map/std.modulemap -fmodule-file=std=std.pcm  -Rmodule-import

hello.cpp:1:2: remark: importing module 'std' from 'std.pcm' [-Rmodule-import]
    1 | #include <string>
      |  ^

real    0m0.171s
user    0m0.128s
sys     0m0.043s

We can see the compilation time is highly reduced!

The flag -Rmodule-import tells if the compiler replace headers includes to imports actually.

The other solution to associate the std headers with the module map is to use clang’s VFS system.

// vfs.yaml
{
  'version': 0,
  'overlay-relative': True,
  'roots': [
    {
      'name': '<path-to-std-headers>',
      'type': 'directory',
      'contents': [
        {
          'name': 'std.modulemap', 'type': 'file',
          'external-contents': 'std.modulemap'
        },
      ]
    }
  ]
}

then

$ time clang++ -std=c++23 hello.cpp std.o -o hello -fmodule-map-file=<path-to-std-headers>/std.modulemap -fmodule-file=std=std.pcm -ivfsoverlay vfs.yaml -Rmodule-import
hello.cpp:1:2: remark: importing module 'std' from 'std.pcm' [-Rmodule-import]
    1 | #include <string>
      |  ^

real    0m0.172s
user    0m0.122s
sys     0m0.049s

the vfs.yaml means to place the std.modulemap file to the directory of std headers virtually.

Why not use clang header modules?#

Clang header modules have two modes:

• Clang implicit header modules: compiler will manage the modules builds.

• Clang explicit header modules: build system will manage the modules builds.

Clang implicit header modules sounds nice but it is prety fragile. Users need to be careful to manage the headers and builds to make it effective.

Clang explicit header modules are much better. This is the way the inventors of clang header modules are using. But they need supports from build systems.

Extends to other modules#

Extends this to other modules has some problems as:

• Dependency management: we can assume almost all TU needs std module. But we can’t assume this for other modules.

• Preprocessor state management: The use of module map to replace headers includes to imports of named modules loses the information about macros. We can do this for std module because the ‘exported’ macro from STL is really limited and centralized. We can exclude the headers which define macros only from the module map. But this may not be true for other modules.

These problems are solvable if we introduce new features. This may be possible if the community has more interests.

What build systems need to do#

Except automating the steps I described above, an important step to manage the flags the consumer used. E.g., if they are two TUs, one is using C++20 and the other is using C++23, the build system needs to build two BMIs of the std module so that the consumers can consume successfully.

Summary#

This section introduces two tricks to use std modules without changing the source code.

These tricks needs support from build systems to make them smoothy in real cases.

The preload tricks will load the std module to all the TUs. It is relatively slower than a clean rewrite (replace #include to imports). And it may trigger more compiler internal bugs. But it should be faster than the pure includes.

The module map solution can make the compilation much more faster. But it needs more steps.

Including headers after import is not well-supported#

The following example is accepted:

#include <iostream>
import foo; // assume module 'foo' contain the declarations from `<iostream>`

int main(int argc, char *argv[])
{
    std::cout << "Test\n";
    return 0;
}

but if the order of #include <iostream> and import foo; is reversed, then the code is currently rejected:

import foo; // assume module 'foo' contain the declarations from `<iostream>`
#include <iostream>

int main(int argc, char *argv[])
{
    std::cout << "Test\n";
    return 0;
}

Both of the above examples should be accepted.

This is a limitation of the implementation. In the first example, the compiler will see and parse <iostream> first then it will see the import. In this case, ODR checking and declaration merging will happen in the deserializer. In the second example, the compiler will see the import first and the #include second which results in ODR checking and declarations merging happening in the semantic analyzer. This is due to a divergence in the implementation path. This is tracked by #61465.

Ignored preferred_name Attribute#

When Clang writes BMIs, it will ignore the preferred_name attribute on declarations which use it. Thus, the preferred name will not be displayed in the debugger as expected. This is tracked by #56490.

Inconsistent filename suffix requirement for importable module units#

Currently, Clang requires the file name of an importable module unit to have .cppm (or .ccm, .cxxm, .c++m) as the file extension. However, the behavior is inconsistent with other compilers. This is tracked by #57416.

Incorrect ODR violation diagnostics#

ODR violations are a common issue when using modules. Clang sometimes produces false-positive diagnostics or fails to produce true-positive diagnostics of the One Definition Rule.

Using TU-local entity in other units#

Module units are translation units, so the entities which should be local to the module unit itself should never be used by other units.

The C++ standard defines the concept of TU-local and exposure in basic.link/p14, basic.link/p15, basic.link/p16, basic.link/p17, and basic.link/p18.

However, Clang doesn’t formally support these two concepts. This results in unclear or confusing diagnostic messages. Further, Clang may import TU-local entities to other units without any diagnostics. This is tracked by #78173.

Quick Start#

The following example:

import <iostream>;
int main() {
  std::cout << "Hello World.\n";
}

could be compiled with:

$ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
$ clang++ -std=c++20 -fmodule-file=iostream.pcm main.cpp

How to produce BMIs#

Similar to named modules, --precompile can be used to produce a BMI. However, that requires specifying that the input file is a header by using -xc++-system-header or -xc++-user-header.

The -fmodule-header={user,system} option can also be used to produce a BMI for header units which have a file extension like .h or .hh. The argument to -fmodule-header specifies either the user search path or the system search path. The default value for -fmodule-header is user. For example:

// foo.h
#include <iostream>
void Hello() {
  std::cout << "Hello World.\n";
}

// use.cpp
import "foo.h";
int main() {
  Hello();
}

could be compiled with:

$ clang++ -std=c++20 -fmodule-header foo.h -o foo.pcm
$ clang++ -std=c++20 -fmodule-file=foo.pcm use.cpp

For headers which do not have a file extension, -xc++-header (or -xc++-system-header, -xc++-user-header) must be used to specify the file as a header. For example:

// use.cpp
import "foo.h";
int main() {
  Hello();
}

$ clang++ -std=c++20 -fmodule-header=system -xc++-header iostream -o iostream.pcm
$ clang++ -std=c++20 -fmodule-file=iostream.pcm use.cpp

How to specify BMI dependencies#

-fmodule-file can be used to specify a BMI dependency (or multiple times for more than one BMI dependency).

With the existing implementation, -fprebuilt-module-path cannot be used for header units (because they are nominally anonymous). For header units, use -fmodule-file to include the relevant PCM file for each header unit.

This is expected to be solved in a future version of Clang either by the compiler finding and specifying -fmodule-file automatically, or by the use of a module-mapper that understands how to map the header name to their PCMs.

Compiling a header unit to an object file#

A header unit cannot be compiled to an object file due to the semantics of header units. For example:

$ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
# This is not allowed!
$ clang++ iostream.pcm -c -o iostream.o

Include translation#

The C++ standard allows vendors to convert #include header-name to import header-name; when possible. Currently, Clang does this translation for the #include in the global module fragment. For example, the following example:

module;
import <iostream>;
export module M;
export void Hello() {
  std::cout << "Hello.\n";
}

is the same as this example:

module;
#include <iostream>
export module M;
export void Hello() {
    std::cout << "Hello.\n";
}

$ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
$ clang++ -std=c++20 -fmodule-file=iostream.pcm --precompile M.cppm -o M.cpp

In the latter example, Clang can find the BMI for <iostream> and so it tries to replace the #include <iostream> with import <iostream>; automatically.