1.6.1. Overview¶
This page is neither a recommendation to use Conda, nor is it a fulltutorial on using it. It is just meant to be a helpful starting pointto Conda, if it sounds appealing to you. Examples here are just formanaging a few Python needs for AFNI.
Python is increasingly used in the imaging world. Everyone and theirsheepadoodle seems to be writing in it and coming across it in somepart of their daily work. Part of its appeal is that there exist manyuseful modules for carrying out tasks (plotting, machine learning,statistical modeling, writing in XKCD font, etc.).
Along with the flexibility and broad appeal of Python comes a need formanaging and maintaining dependencies: making sure the necessarydependencies are installed, adding/updating modules, and using therequisite version of Python. For example, there were major changesbetween Python 2 and 3 (just try evaluating 1/2 in each case), andoccasionally across sub-versions code can break; we can only wait andsee what will happen with Python 4.
Anyway, managing potentially multiple versions of Python, as well asdiferent sets of dependencies can be tricky. There are different waysof doing this, but through personal experience I have found using“Conda” to be (by far) the easiest and most straightforward. We canbasically set up one or more environments (or envs) withspecific Python versions, modules and dependencies, and we can swapback and forth between them fairly easily—they are each separate anddistinct, and don’t get in each other’s way. We can set one to be“on” by default, not really think about it often, and then just changeto another environment if we need to. Nice.
We will use the specific installation form of Conda called“Miniconda”, because it starts of being light-weight, and you canthen add whatever you need. Some official reference pages for thisare:
These notes apply to either Linux or Mac OS. Miniconda can also beinstalled on Windows directly, but since AFNI doesn’t run there, whocares?
These notes are only for setting up Conda for a single user.Instructions for installing Conda for multiple users is described onthe conda website here.
Very important note: you should not need your sudopassword to install Conda on your system, nor to run later “conda”commands. Using sudo with conda will just cause headacheslater, so don’t do it.
1.6.2. Set up Conda (verbose)¶
Download+install miniconda¶
To download+install, copy+paste:
... for Linux:
curl -OL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.shbash Miniconda3-latest-Linux-x86_64.sh -b
... for Mac (intel chip):
curl -OL https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.shbash Miniconda3-latest-MacOSX-x86_64.sh -b
... for Mac (M1 chip):
curl -OL https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-arm64.shbash Miniconda3-latest-MacOSX-arm64.sh -b
The
curlprogram should be installed on your computer afterfollowing AFNI setup instructions. You can also copy+paste thelink into a browser and download it that way.NB: The
-boption runs the installer in “batch mode”, sayingyes to all prompts: install into home directory, accept licenses.To go through those options manually, don’t include that option.To initialize conda in shells, copy+paste:
~/miniconda3/bin/conda init bash tcsh zsh
This puts conda setup text into each of the above shell’s
~/.*rcfiles (the relative path tocondamust be given whenusing the batch mode installation in the previous step).NB: Conda/miniconda gets setup with path names hardwired into itsfiles, so you cannot just move your “miniconda3” directory andupdate these path locations later and still have it work.
Make updates known to terminal
Do one of the following:
Open a new terminal. You should now see a text string like“(base)” to the left of your terminal prompt. (Below, we show youcan optionally disable that text.)
Source your shell’s
~/.*rcfile (to know your current shell,typeecho $0):... for bash:
source ~/.bashrc... for tcsh or csh:
source ~/.cshrc... for zsh:
source ~/.zshrc
You should see a string “(base)” string stuck before your terminalprompt now. (Below, you can optionally disable that text.)
Type
conda -Vto see the version number. NB: It should be atleast 4.6.
Update conda version (opt)¶
When you run some Conda commands, you might receive promptings toupdate your Conda version. Sometimes explicit commands forcopy+pasting are even provided. Go ahead and do so. The followingcommand can also be used to update your Conda (or to check that youare update, at least):
conda update -n base -c defaults conda
To check your current Conda version, type:
conda --version
Load/exit basic conda environments¶
To see the list of currently available conda environments, type:
conda env list
The name of each available environment appears in the first column(its file location appears in the second). The currently activeenvironment has an asterisk * after its name (one might not beloaded).
To load or “activate” an environment in that list called ENV_NAME,type:
conda activate ENV_NAME
To exit or “deactivate” the current environment, type:
conda deactivate
Make new envs from cmd line: AFNI minimal Python¶
There are many aspects to creating a new environment. We only providethe most basic here. For example, conda can manage much morecomplicated environments, beyond loading just Python+modules.
Here is an example of creating a new environment from a command line,one that packages the most recent Python version (which will likelyhave major version number “3”) and a few useful modules (whoseunspecified version numbers will be whatever conda decides, withMatplotlib being at least 2.2.3):
conda create -y \ -n py3_afni_tiny \ python \ "matplotlib>=2.2.3" numpy scipy \ "flask>=2.1.2" \ "flask-cors>=3.0.10"
This new environment’s name is “py3_afni_tiny”; I called it thisbecause that is basically the minimal set of modules used within AFNI(at present). As of July 2, 2023, in now includes the Python-flaskmodule dependencies used for running a local server, so the buttons inthe APQC HTML save information automatically; this functionality onlyappears possible in Python v3.
To make a similar setup for Python 2.7 (no earlier versions of Pythonshould be used), one could run:
conda create -y \ -n py2_afni_tiny \ python=2.7 \ "matplotlib>=2.2.3" numpy scipy \ pillow
Now, if I type conda list env, I will see a list of all myavailable environments (where ${USER} would actually be replacedby my username):
# conda environments:#base * /home/${USER}/miniconda3py2_afni_tiny /home/${USER}/miniconda3/envs/py2_afni_tinypy3_afni_tiny /home/${USER}/miniconda3/envs/py3_afni_tiny
As noted above, to switch to py3_afni_tiny, I would type:
conda activate py3_afni_tiny
To see what modules are installed in your active environment (andtheir version numbers) you can run:
conda list
... which, in the current “py3_afni_tiny” would be as follows (andyou might have slightly different things):
- show list output y/n -So, in this environment, I could run a program that importsmatplotlib, whereas in the “base” environment, I couldn’t.
Make new envs from text file: AFNI minimal Python¶
This is the command line style to create a new environment (perhapspreferable to command line style, for easier saving and sharing),including both specific and minimal package dependency versions:
Make a new text file called environment_ex1.yml:
name: py3_afni_tiny # TO MAKE: conda env create -f env*_ex1.ymldependencies: - python - "matplotlib>=2.2.3" # minimal ver for 'pythonic' APQC HTML - numpy - scipy - "flask>=2.1.2" # for local server for APQC HTML - "flask-cors>=3.0.10" # for local server for APQC HTML
... and then run:
conda env create -f environment_ex1.yml
Make a new text file called environment_ex2.yml:
name: py2_afni_tiny # TO MAKE: conda env create -f env*_ex2.ymldependencies: - python=2.7 - "matplotlib>=2.2.3" # minimal ver for 'pythonic' APQC HTML - numpy - scipy - pillow - ipython - flask # for local server for APQC HTML - flask-cors # for local server for APQC HTML
... and then run:
conda env create -f environment_ex2.yml
(Bonus, because I like IPython, and jupyter-notebooks arecommon) Make a new text file calledenvironment_ex3.yml:
name: py3_afni_mid # TO MAKE: conda env create -f env*_ex3.ymldependencies: - python - "matplotlib>=2.2.3" # minimal ver for 'pythonic' APQC HTML - pillow - numpy - scipy - "flask>=2.1.2" # for local server for APQC HTML - "flask-cors>=3.0.10" # for local server for APQC HTML - "ipython>=8.0" # minimal ver for tab-autocomplete to work - jupyter - prompt_toolkit=3.0.* # >=2 for jupyter to autocomplete; >=3 for ipython
... and then run:
conda env create -f environment_ex3.yml
Add to+update conda environments¶
To add a new package or module NEW_PACK to an existing environmentENV_NAME, one can use the following syntax:
conda install -n ENV_NAME NEW_PACK
... so, for example example, you could add the scipy module to oneof the above environments with:
conda install -n py2_afni_tiny ipython
To update a module or package CURR_PACK in a currently activeenvironment, you can use:
conda update CURR_PACK
... for example,:
conda update matplotlib
So, let’s say you want one primary environment on your OS to have allyour packages of interest loaded, so you don’t have to hop betweenenvironments when using different programs. You could make one thathas everything you know you need loaded now, and then in the futureyou could simply keep adding to it. This might be useful with AFNI,in particular, because there are so few requirements here (modernPython with a very small number of modules).
Specify default environment for the terminal¶
By default, conda will load the “base” environment in any newterminal. To instead have a different environment ENV_NAME loadedin each new terminal/shell, we can add the line conda activateENV_NAME in the shell’s ~/.*rc file somewhere after the #>>> conda initialize >>> lines.
Since I am running “bash” shell, I have added the following line in my~/.bashrc file (by opening that file with a text editor):
conda activate py3_afni_tiny
After sourcing that file or opening a new terminal, conda env listshould show that environment loaded, in this and in any new terminals.If that did not work, please check that that the conda version is atleast 4.6 (via conda -V).
If you do choose to automatically activate your own env like this,then you might also want to run this in a terminal:
conda config --set auto_activate_base false
so that conda doesn’t pre-load the “base” environment unnecessarily(taking a bit of time).
Disable conda prompt string (opt)¶
Personally I don’t like having the name of the conda environmentalways appearing before my prompt, like “(base)” or whatever. To notdisplay that text, you can run:
conda config --set changeps1 False
To make your existing terminal recognize this change, source yourshell’s ~/.*rc file, e.g. source ~/.bashrc or source~/.cshrc. Or open a new terminal.
If in the future you want to re-enable this behavior, then you canalways run:
conda config --set changeps1 True
These commands edit a text file called ~/.condarc. You can openit and see what defaults/settings you have made, if you wish.
1.6.3. Set up Conda (quick)¶
Download and install
... for Linux:
curl -OL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.shbash Miniconda3-latest-Linux-x86_64.sh -b
... for Mac (intel chip):
curl -OL https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.shbash Miniconda3-latest-MacOSX-x86_64.sh -b
... for Mac (M1 chip):
curl -OL https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-arm64.shbash Miniconda3-latest-MacOSX-arm64.sh -b
Initialize conda in shells
~/miniconda3/bin/conda init bash tcsh zsh
Make updates known to terminal
Open a new terminal, or source your shell’s
~/.*rcfile.Remove annoying prompt string (opt)
conda config --set changeps1 False
Update conda version (opt)
conda update -n base -c defaults conda
... and to display conda version:
conda --version
Make new envs from cmd line: AFNI minimal Python
This is the command line style to create a new environment,including both specific and minimal package dependency versions:
conda create -y \ -n py3_afni_tiny \ python \ "matplotlib>=2.2.3" numpy scipy \ "flask>=2.1.2" \ "flask-cors>=3.0.10"conda create -y \ -n py2_afni_tiny \ python=2.7 \ "matplotlib>=2.2.3" numpy scipy \ pillow
See the next section for a slightly better way.
Make new envs from text file: AFNI minimal Python
This is the command line style to create a new environment (perhapspreferable to command line style, for easier saving and sharing),including both specific and minimal package dependency versions:
Make a new text file called environment_ex1.yml:
name: py3_afni_tiny # TO MAKE: conda env create -f env*_ex1.ymldependencies: - python - "matplotlib>=2.2.3" # minimal ver for 'pythonic' APQC HTML - numpy - scipy - "flask>=2.1.2" # for local server for APQC HTML - "flask-cors>=3.0.10" # for local server for APQC HTML
... and then run:
conda env create -f environment_ex1.yml
Make a new text file called environment_ex2.yml:
name: py2_afni_tiny # TO MAKE: conda env create -f env*_ex2.ymldependencies: - python=2.7 - "matplotlib>=2.2.3" # minimal ver for 'pythonic' APQC HTML - numpy - scipy - pillow - ipython - flask # for local server for APQC HTML - flask-cors # for local server for APQC HTML
... and then run:
conda env create -f environment_ex2.yml
(Bonus, because I like IPython, and jupyter-notebooks arecommon) Make a new text file calledenvironment_ex3.yml:
name: py3_afni_mid # TO MAKE: conda env create -f env*_ex3.ymldependencies: - python - "matplotlib>=2.2.3" # minimal ver for 'pythonic' APQC HTML - pillow - numpy - scipy - "flask>=2.1.2" # for local server for APQC HTML - "flask-cors>=3.0.10" # for local server for APQC HTML - "ipython>=8.0" # minimal ver for tab-autocomplete to work - jupyter - prompt_toolkit=3.0.* # >=2 for jupyter to autocomplete; >=3 for ipython
... and then run:
conda env create -f environment_ex3.yml
Load an existing environment
Copy+paste:
conda activate ENV_NAME
For example, from above to setup for AFNI:
conda activate py3_afni_tiny
Activate an env by default
To activate some env
ENV_NAMEby default, putcondaactivate ENV_NAMEin your shell’s~/.*rcfile, after the# <<< conda initialize <<<line.For example, to set up for AFNI,put
conda activate py3_afni_tinythere.NB1: This assumes your conda version (
conda -V) is atleast 4.6.NB2: If you do automatically activate your own env, then alsocopy+paste the following to not pre-load the “base” env (addingunnecessary time):
conda config --set auto_activate_base false
NB3: In general, you don’t want to keep appending different
conda activate ...commands in a~/.*rcfile, as each onetakes a bit of time.Add to an existing environment
Once you have built an environment, if you decide you anotherpackage that you might have forgotten, you can do so with:
conda install -n ENV_NAME PACK_NAME
For example,
conda install -n py2_afni_tiny pandas
1.6.4. Short list of conda commands¶
List available modules (starred/asterisked one is active):
conda env list
Deactivate current module:
conda deactivate
Activate/switch to a specific environment/module:
conda activate ENV_NAME
See module+version list in current env:
conda list
Update a package in the current environment:
conda update PACKAGE
Add a package to some environment:
conda install -n ENV_NAME PACK_NAME
Add a channel to an active environment (with top priority amongchannels):
conda config --add channels NEW_CHANNEL
Add a channel to an active environment (with bottom priorityamong channels):
conda config --append channels NEW_CHANNEL
Remove an existing environment (ENV_NAME cannot be active whenthis command is run):
conda remove --name ENV_NAME --all
Update conda program version:
conda update -n base -c defaults conda
1.6.5. A note on making envs, re. AFNI and more¶
It is entirely up to you, Dear User, what modules you install and howyou organize your environments (and if you even choose to useConda). At the moment AFNI has very minimal Python requirements. Infact, the AFNI set of recommended modules might simply fit insidethose requirements that you have for other software/uses, and youmight not need to do anything new.
We certainly don’t anticipate or desire a person to set up onespecific environment for running AFNI, then another for running someother software, and then another for another project... While thatis possible, it seems annoying and inefficient, and often unnecessary.So, hopefully, you can set up one environment (or a small number ofthem) and not have to switch too much.
1.6.6. Fancier things with Conda¶
There are a lot of fancy things that can be done with Conda that wewill not describe here. A good starting point is the ManagingEnvironments documentation.
Cloning¶
One concept with Conda is cloning environments: if I can setup aConda environment on my laptop with a certain set of modules, eachwith a certain version number, then I can “clone” it and use thatexact recipe to setup a duplicate environment on a different computer.This is a nice concept for reproducibility (as sometimes usingdifferent version numbers of modules can affect outputs/results).
More on cloning and building identical conda envs can be read.
Note that in practice, truly duplicating environments exactly isactually pretty tough. Getting very close might be good enough formost purposes, though, in practice.
View Conda Cheatsheet¶
It’s here: the conda cheatsheet.
Make conda environments, more generally¶
The environment builder works a bit like a package manager, where itcan get a lot of common modules from a default, central repository,but if you want more specialized ones, you might have to add fromanother place. To add new repositories to pull from, you add achannel to your Conda setup.
Let’s say you want to add the Sphinx module with cloud-theme support(I doubt you will, but just as an example). If you try:
conda create -y \ -n py37_afni_with_sph \ python=3.7 \ matplotlib numpy scipy \ "flask>=2.1.2" \ "flask-cors>=3.0.10" \ sphinx cloud_sptheme
You will likely get the following message:
- show text output y/n -Collecting package metadata (current_repodata.json): doneSolving environment: failed with repodata from current_repodata.json, will retry with next repodata source.Collecting package metadata (repodata.json): doneSolving environment: failedPackagesNotFoundError: The following packages are not available from current channels: - cloud_spthemeCurrent channels: - https://repo.anaconda.com/pkgs/main/linux-64 - https://repo.anaconda.com/pkgs/main/noarch - https://repo.anaconda.com/pkgs/r/linux-64 - https://repo.anaconda.com/pkgs/r/noarchTo search for alternate channels that may provide the conda package you'relooking for, navigate to https://anaconda.organd use the search bar at the top of the page.
This message: 1) tells us our current channels don’t contain thismodule; 2) shows us our current channels; and 3) helpfully directs usto a webpage to search for a new channel that might have it.
So, searching for “cloud_sptheme” at https://anaconda.org/, one of thetop package owners appears to be “conda-forge” (and this is a fairlylarge platform). So, to add it to my repository list for gettingmodules, I would run:
conda config --add channels conda-forge
Then, I can retry my conda create .. command above, which shouldresult in success this time.
Thus, if you try to build an environment and get told that somedesired module can’t be found, you can search for it amongst availablechannels, add that channel to your Conda setup, and try again.
1.6.7. Comments on AFNI-specific conda environments¶
Conda can be used to create environments with a very wide range ofdependencies. For the purposes of interacting with AFNI, we tend tojust focus on Python modules, since all other dependencies shouldessentially be sorted out by following the main installationinstructions for a given OS.
AFNI has very few Python module dependencies. Most AFNI Pythonprograms in fact have no non-base module dependencies. However, ifyou are going to use conda/miniconda to set up a Python environmentanyways, you might as well install any possible modules, too.Moreover, some programs you might want to use in conjunction with AFNImight have additional dependencies, which you could add to yourAFNI-running environment, too.
Every Python program in the AFNI distribution runs in either Python2.7 or Python 3.* except for the distributed version of PrantikKundu’s multi-echo FMRI-processing program meica.py, which must berun in Python 2.7 (because of when it was written and its developerhas left the field).
Some comments then about possible AFNI environments to make, ordependencies to combine with your other
for most projects with AFNI, you could use either of the
py2_afni_tinyorpy3_afni_tinyenvironments, describedabove, on their own or with their dependencies added to apre-existing environment. NB: there is no strict Python versionrequirement for AFNI. Since Python 2.7 is technically deprecatednow, you might opt for a Python 3.* recipe, all other things beingequal.If you will be using Prantik’s older
meica.pyprogram, thenyou would want a Python 2.7-based environment available, such aspy2_afni_tiny. It could be used as your main environment, orit could just be one you switch in-and-out of for running thatspecific program. (Being able to easily switch environments isbasically the raison-d’etre of conda.)If you want to incorporate the modern TEDANA (Dupre et al., 2021) into your AFNI processing ofME-FMRI data, then here is an example of an environmentincorporating both sets of dependencies. You could make a new textfile called environment_afni_ted.yml:
name: py3_afni_ted # conda env create -f env*afni_ted.ymlchannels: - conda-forge - defaultsdependencies: - python - "matplotlib>=2.2.3" # minimal ver for 'pythonic' APQC HTML - numpy - scipy - "flask>=2.1.2" # for local server for APQC HTML - "flask-cors>=3.0.10" # for local server for APQC HTML - pip # used for TEDANA - pip: - tedana # getting TEDANA
... and then run:
conda env create -f environment_afni_ted.yml
NB: these reflect current TEDANA installation instructions, whichmight change over time. We will try to keep up with those.
