Improve documentation on building with cmake inside of a conda environment
This issue originally came up while building a package from source with cmake inside of a docker build, but the underlying issue is broader than that (I've verified by replicating the issue in a local conda environment at CIT). In short, I do not believe that the current instructions here under the Cmake
tab are complete (and therefore are not at present correct).
Right now, the instructions say that to build a package from source when the build system is cmake, one should do:
cmake <directory> ${CMAKE_ARGS}
where the variable CMAKE_ARGS
is defined inside the conda environment when one has installed conda compiler packages into that environment.
However, with just that, I have encountered build failures. After correspondence with Duncan, he pointed out that what that variable is set to depends on whether or not the conda environment detects that it is in a 'conda build'. Hence he advised setting CONDA_BUILD=1
before activating (or reactivating) the environment. While that allowed the build to succeed, the install step still failed, as it tried to install into system locations rather than the conda environment. On looking at the lines that toggle the definition of CMAKE_ARGS
depending on the value of CONDA_BUILD
, one can see that the installation flags are determined via the variable PREFIX
, which does not appear to be set simply by activating the environment. If I manually set it to the same value as CONDA_PREFIX
when the environment is active, then the project I was working on successfully built and installed into the conda environment as desired. However I do note that looking at the same lines in the activation script, they also depend on BUILD_PREFIX
, so I am not sure whether that could cause failures as well for other projects.
All of which to say: as far as I can tell, a sufficient procedure for building a package that relies on cmake in such a way that it is installed into a conda environment seems nontrivial. I believe we certainly want to support this, and to document whatever way is best to advise our users, if only because many of our users will develop software whose production version will be deployed via conda, so it will be important for them to build it and test it in a way that is as "close as possible" to what will be deployed.
I should emphasize that all I'm commenting on at the moment is the tab on those instructions specific to Cmake
. I would be surprised if the Autotools
instructions were incomplete (though I haven't tested), and on other occasions the Python
instructions have worked for me. I have never used Meson
and have no idea about that.
The best solution I have come up with thus far (in particular, trying not to make assumptions about what shell is in use) is the following, assuming that I already have a conda environment named myenv
which has all of the dependencies of my package and the necessary build tools (cmake
, compilers, etc.):
conda env config vars set CONDA_BUILD=1 -n myenv
conda activate myenv
conda env config vars set PREFIX=$CONDA_PREFIX # You must do this after activating to get the value of $CONDA_PREFIX
conda activate myenv # Need to activate again to pick up the new value of CMAKE_ARGS
cmake $CMAKE_ARGS . # Arguably, some users may want to unset both CONDA_BUILD and PREFIX once they're done building
I relied on these instructions for setting environment variables in a shell-independent way (which also does not pollute the environment once the conda environment is deactivated).
I would be pleased to learn there is a simpler way, and was also not sure if this is getting complex enough that the simple tab for cmake
projects is insufficient.