Conda: Environment configuration

Michael Mikhailidi -

My imaginary opponent shrugs his shoulders in bewilderment on this topic. Who needs to manage packages and environments in the age of ubiquitous containerization, clouds, and 5G? You should if you are a thousand miles south of the Beltway with motley mobile coverage. Internet in the rentals is good enough for me but scarcely covers the whole family's needs. Not to mention that even "cold" cloud infrastructure costs you money, and when it does not, compute instance is powerful just enough to run a small Flask/HTML5 application.

That intro supposedly leads you to the point: you own (with a probability of .95, according to Google Analytics) a powerful computing device that you can use as a standalone system with minimal connectivity requirements for developing, debugging, and testing applications. And it would be a good idea - to use virtual environments to limit cross-project dependencies and keep your OS package list under control.

I'll fast forward the obvious prises to virtual development environments, skipping the part where I moved from Python's pip, wheels, and virtualenv to Conda. The primary reason is that Conda helps me manage quite different environments and requirements, most of the time not related to Python at all. For example, I have a WSO2 API Manager installed into the Conda-controlled environment. However, I have missed configuring session variables within a virtual environment and restoring it when I deactivate it or switch to another project. I played with the idea to have separate shell scripts for project activation/deactivation, but then I found a more convenient solution. Yes, you still should create scripts to set up and clean up environment variables, but now it's a part of the environment, and you would not miss it when you backup it or commit it to the source code repository.  

Let's go through the practical example. for my WSO2 environment, I want to have:

  • Set up a JAVA_HOME variable to run the API Manager server
  • Update PATH variables to search API-M and API CLI commands
  • Change my current folder to the environment location
  • Restore my session when I deactivate the environment.  

I have Miniconda3 already installed and configured in my WSL2 Ubuntu, so just a few more steps:

  1. Create a new Conda environment
## Create a new environment with the name wso2am
(base) mmikhail@my-laptop:~$ conda create -n wso2am
(base) mmikhail@my-laptop:~$ conda info --envs

# conda environments:
#
base          *  /home/mmikhail/miniconda3
tftests          /home/mmikhail/miniconda3/envs/tftests
wso2am           /home/mmikhail/miniconda3/envs/wso2am
  1. Activate the new environment
(base) mmikhail@my-laptop:~$ conda activate wso2am
(wso2am) mmikhail@my-laptop:~$ cd ~/miniconda3/envs/wso2am/
(wso2am) mmikhail@my-laptop:~/miniconda3/envs/wso2am$
  1. Create activation and deactivation scripts
$ mkdir -p ./etc/conda/activate.d
$ mkdir -p ./etc/conda/deactivate.d
$ touch ./etc/conda/activate.d/env_vars.sh
$ touch ./etc/conda/deactivate.d/env_vars.sh

It'a a time to complete shell scripts with commands.

Activation steps in the ./etc/conda/activate.d/env_vars.sh

#!/bin/sh
# Set JAVA_HOME to run WSO2 API-M
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
# SAVE the Original PATH variable
export SAVE_APIM_PATH=$PATH
# Current WSO2 API-M installation 
export WSO2AM=wso2am-4.1.0
# Adjust PATH variables
export PATH=$CONDA_PREFIX/$WSO2AM/bin:\
$CONDA_PREFIX/apictl:$PATH
# Go to the environment home
cd $CONDA_PREFIX

Deactivation commands forthe ./etc/conda/deactivate.d/env_vars.sh

#!/bin/sh
# Clean up variables 
unset JAVA_HOME
unset WSO2AM
#Restore the original PATH values
export PATH=$SAVE_APIM_PATH
# Go to the my home folder
cd $HOME

This setup does exactly what I wanted with someextra perks:

  • I use standard conda (or mamba if you want to) commands with no additional wrappers
  • Environment configuration files are part of the environment and could be transferred or containerized if I need to.
  • Deactivation script gives you an opportunity to restore session environment to the original state.