A Better Way to Wrangle Python Environments
virtualenv begrudgingly. I know it’s what you should do, but setting up and maintaining virtualenvs manually gets old quickly. For the past few months, I’ve been using autoenv to automate this process, and now I use virtual environments much more consistently.
Why You Should Care
Python’s import system takes a bit of time to get used to. For the first ~year of my Python journey, just seeing the
$PYTHONPATH environment variable threw up so many “Here Be Dragons” red flags that I’d break down and nuke my Python setup to avoid dealing with it.
PYTHONPATH, really, is just the same as the normal
PATH: it specifies to python where it should look for packages when you call
import. However, for most projects you’ll want to maintain a
requirements.txt to allow your users to install the
pip requirements for your project.
However, if your
PYTHONPATH is cluttered with packages that are irrelevant to the project you’re working on, getting a consistently correct, lean
requirements.txt is impossible.
This is why you really should use virtual environments for all python projects. Virtual environments allow you to create a separate set of python packages away from your system-level packages, thus keeping your
PYTHONPATH clean and your
pip freeze > requirements.txt runs accurate.
This keeps your system-level packages in-tact, your project dependencies clear, and you’ll probably never have to go down the rabbit hole of fixing a broken
$PYTHONPATH. If worse comes to worst, just nuke your virtual environment and reinstall painlessly from your
Setting up Autoenv
Installing Autoenv is pretty trivial. It’s as simple as:
pip install autoenv echo "source `which activate.sh`" >> ~/.bashrc
Autoenv acts like a
.bashrc for every folder on your machine. Just put a
.env file in your project with some commands to execute, and every time you
cd into that directory you’ll automatically get everything setup.
.env file should do two things:
- Clear your
PYTHONPATHvariable to prevent confusion with system-level installed packages.
- Activate your
Here’s my ideal
export PYTHONPATH=. source /path/to/project/directory/venv/bin/activate
Short. Simple. Extensible. Just add additional exports as necessary if you want to set other environment variables.
In fact, you can get this all setup with two commands:
virtualenv venv echo "export PYTHONPATH=.\nsource `pwd`/venv/bin/activate" > .env
cd into your project directory, and you’ll have your virtual environment setup and ready for package installations.
.env files in each of the parent directories of your current directory. So, if I
~/Documents/MyProject, Autoenv looks for:
… and executes them in that order.
This means, you can put a script in
~/.env and it’ll be executed prior to the project-specific
You can take advantage of this to automatically
deactivate virtual environments.
Simply make a
.env in your home directory with this little script:
if which deactivate >/dev/null; then deactivate fi
deactivate a virtual environment if you have one activated. So, if you
cd outside your project, the virtual environment of that project is automatically deactivated. Awesome!