Commit 2f79d7e9 authored by Duncan Macleod's avatar Duncan Macleod

Merge branch 'compiling' into 'master'

Added docs section on compiling

Closes #10

See merge request !31
parents 7c3a70cb 2de581d9
.. _lscsoft-conda-compiling:
*************************************
Compiling code in a Conda environment
*************************************
.. note::
The :ref:`lscsoft-conda-pre-built-environments` should contain all
of the packages needed to build most things, if packages are missing
please `open a Request ticket
<https://git.ligo.org/lscsoft/conda/issues/new?issuable_template=Request>`_
to request their addition.
To compile packages without using ``conda build`` you will need to build
a conda environment that contains the necessary packages.
The only tricky thing about this is installing the compilers,
`read this
<https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html>`__.
You will need to **activate** the environment properly so as to have the
compiler tools set the necessary environment variables (``CC`` etc).
.. note::
There is now a suite of conda-forge packages designed to provide a
compiler for your architecture.
See the `compilers <https://anaconda.org/conda-forge/compilers>`__
package, and its dependencies.
=====================
Installation prefices
=====================
Conda's environment configuration mean that dynamic linking is much
more tightly controlled than outside of an environment.
This means that, for example, building shared object libraries and
installing them into custom directories that you append to environment
variables such as ``LD_LIBRARY_PATH`` may not resolve links properly.
The way to avoid this is simply to always install code directly into
a conda environment.
For autotools builds this would just mean using ``--prefix=${CONDA_PREFIX}``
during ``./configure``.
==================
Compiling on macOS
==================
Compiling on macOS using conda compilers is not absolutely trivial, there is
a small set of steps that need to be followed to configure your environment.
.. _lscsoft-conda-compiling-macos-sdk:
~~~~~~~~~~~~~~~~~~~~~~~~
Installing the macOS SDK
~~~~~~~~~~~~~~~~~~~~~~~~
Conda compilers will try and compile software for the oldest compatible
version of macOS, which is 10.9 most of the time.
So, you will likely need to install the MacOSX10.9 SDK, which you can do
as follows:
.. code-block:: bash
mkdir -p /opt
cd /opt
wget https://github.com/phracker/MacOSX-SDKs/releases/download/10.13/MacOSX10.9.sdk.tar.xz
tar -xf MacOSX10.9.sdk.tar.xz
Then, set the ``CONDA_BUILD_SYSROOT`` environment variable to match the new
SDK:
.. code-block:: bash
export CONDA_BUILD_SYSROOT="/opt/MacOSX10.9.sdk"
~~~~~~~~~~~~~~~~~~~~~~
Configuring the linker
~~~~~~~~~~~~~~~~~~~~~~
In order to resolve links, especially during ``./configure`` steps for
autotools builds, you will need to 'activate' your build environment with
the following steps:
.. code-block:: bash
conda activate <env-name>
export PREFIX="${CONDA_PREFIX}"
export CONDA_BUILD="1"
conda activate <env-name>
......@@ -15,5 +15,6 @@
usage/index
environments/index
tips/index
compiling/index
packaging/index
tips/index
.. _lscsoft-conda-packaging:
*****************************
Packaging software with conda
......@@ -6,6 +7,8 @@ Packaging software with conda
This document contains a brief primer on packaging software for distribution
with `conda <https://conda.io>`__.
.. _lscsoft-conda-packaging-recipe:
=================
What is a recipe?
=================
......@@ -32,6 +35,9 @@ directory (``bld.bat`` for Windows).
The ``build.sh`` script is a shell script that can do whatever you need it to do
(`example <https://github.com/conda-forge/nds2-client-feedstock/blob/492436f4fa3c6b8a62dbc86dc68d79f818d251e3/recipe/build.sh>`__).
.. _lscsoft-conda-packaging-conda-forge:
=======================================
Packaging something new for conda-forge
=======================================
......@@ -52,6 +58,9 @@ examples from LSCSoft Conda (feel free to suggest your own):
- C/C++: `nds2-client <https://github.com/conda-forge/nds2-client-feedstock/tree/492436f4fa3c6b8a62dbc86dc68d79f818d251e3/recipe/>`__,
- Multiple outputs: `lal <https://github.com/conda-forge/lal-feedstock/tree/13e6fd9ba32f15b25aaf68bc6ffe106398d171b1/recipe>`__.
.. _lscsoft-conda-packaging-github:
==========================================
Maintaining a conda-forge recipe on GitHub
==========================================
......@@ -68,6 +77,9 @@ that the recipe is out of date.
This is mainly done by searching for tarballs in the same location as the
current release tarball that suggest a newer version has been uploaded.
.. _lscsoft-conda-packaging-mkl:
===================================
Building packages against Intel MKL
===================================
......@@ -92,6 +104,9 @@ build from the command line using something like this:
conda build recipe --variants "{blas_impl: \"mkl\", mkl: \"2018\"}"
.. _lscsoft-conda-packaging-testing:
======================
Testing package builds
======================
......@@ -116,16 +131,16 @@ e.g:
cd lal-feedstock
conda build recipe
-----
Hacks
-----
=============================
Common issues and workarounds
=============================
If your build has failed, try one of the following hacky workarounds.
If that doesn't work, try google.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
----------------------------------------------------
(linux) C compiler doesn't see the ``USER`` variable
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
----------------------------------------------------
Add ``USER`` to the ``build/script-env`` list of variables to be inherited
into ``build.sh``:
......@@ -136,18 +151,14 @@ into ``build.sh``:
script_env:
- USER
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(macOS) Build tries to use the wrong version of the macOS SDK
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can download the MacOSX10.9 SDK as follows:
.. _lscsoft-conda-packaging-hacks-macos-sdk:
.. code-block:: bash
-------------------------------------------------------------
(macOS) Build tries to use the wrong version of the macOS SDK
-------------------------------------------------------------
mkdir -p /opt
cd /opt
wget https://github.com/phracker/MacOSX-SDKs/releases/download/10.13/MacOSX10.9.sdk.tar.xz
tar -xf MacOSX10.9.sdk.tar.xz
Make sure and read :ref:`lscsoft-conda-compiling-macos-sdk`.
Then you can tell ``conda-build`` about this by adding the following
lines to ``~/conda_build_config.yaml`` (create it if you don't have it):
......@@ -157,9 +168,9 @@ lines to ``~/conda_build_config.yaml`` (create it if you don't have it):
CONDA_BUILD_SYSROOT: # [osx]
- /opt/MacOSX10.9.sdk # [osx]
-----------------
=================
Docker containers
-----------------
=================
If you prefer to work in a pristine environment, you can use the
`condaforge/linux-anvil
......@@ -171,31 +182,4 @@ when building packages for production.
Compiling packages in a Conda environment, not using conda-build
================================================================
.. note::
The :ref:`lscsoft-conda-pre-built-environments` should contain all
of the packages needed to build most things, if packages are missing
please `open a ticket <https://git.ligo.org/lscsoft/conda/issues/>`__
to request their addition.
To compile packages without using ``conda build`` you will need to build
a conda environment that contains the necessary packages.
The only tricky thing about this is installing the compilers,
`read this
<https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html>`__.
You will need to **activate** the environment properly so as to have the
compiler tools set the necessary environment variables (``CC`` etc).
.. note::
There is now a suite of conda-forge packages designed to provide a
compiler for your architecture.
See the `compilers <https://anaconda.org/conda-forge/compilers>`__
package, and its dependencies.
Lastly, on macOS you will most likely need to set the ``CONDA_BUILD_SYSROOT``
environment variable to match the custom MacOS SDK mentioned above:
.. code-block:: bash
export CONDA_BUILD_SYSROOT="/opt/MacOSX10.9.sdk"
This section has been moved to :ref:`lscsoft-conda-compiling`.
......@@ -46,7 +46,7 @@ details for some common operating systems:
The following examples should get you far enough to use TeX with
the Matplotlib Python library. If you find that other packages
are needed, please
`open a Bug ticket https://git.ligo.org/lscsoft/conda/issues/new?issuable_template=Bug>`_.
`open a Bug ticket <https://git.ligo.org/lscsoft/conda/issues/new?issuable_template=Bug>`_.
^^^^^^^^
Macports
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment