There are (too) many guides out there about how to install Python on Mac OS X. So why another one? Cause i found most of the other ways to lead to long-term maintenance debt that is unnecessary and can easily be avoided. Also many of the other guides aren’t very easy to extend in “but i also need that library”-cases. So here I’ll briefly explain how to set-up a scientific python stack based on homebrew. The advantages are that this stack will never conflict with your system’s core and homebrew opens up a whole new world of easy to access unix tools via a simple brew install ...
.
This step-by-step installation guide to setup a scientific python environment has been tested on Mac OS X Mavericks 10.9 / Yosemite 10.10 / El Capitain 10.11 / Sierra 10.12. It will probably also work in the following versions (if not, let me know in the comments).
An older version of this setup guide can be found here: Scientific Python on Mac OS X 10.8 with homebrew, main changes: rename of a tap + changes wrt. openblas as Accelerate was fixed in OS X 10.9
Needless to say: Make a backup (Timemachine)
First install homebrew.
Follow their instructions, then come back here.
If you don’t have a clean install, some of the following steps might need minor additional attention (like changing permissions chmod
, chown
, chgrp
or overwriting existing files in the linking step with brew link --overwrite package_that_failed
. In this case i can only recommend a backup again).
In general: execute the following commands one line at a time and read the outputs! If you read some warnings about “keg-only” that’s fine, it just means that brew won’t “hide” your system’s stuff behind the stuff it installed itself so it doesn’t cause problems… brewed stuff will still use it.
# set up some taps and update brew
brew tap homebrew/science # a lot of cool formulae for scientific tools
brew tap homebrew/python # numpy, scipy, matplotlib, ...
brew update && brew upgrade
# install a brewed python
brew install python
# and/or if you want to give python3 a go (it's about time):
#brew install python3 # repeat/replace the following pip commands with pip3
A word about brewed python: this is what you want!
It’s more up to date than the system python, it will come with pip and correctly install in the brew directories, working together well with brewed python libs that aren’t installable with plain pip. This also means that pip by default will work without sudo as all of homebrew, so if you ever run or have to run sudo pip ...
because of missing permissions, then you’re doing it wrong! Also, don’t be afraid of multiple pythons on your system: you have them anyhow (python2 and python3) and it’s an advantage, as we’ll make sure that nothing poisons your system python and that you as a user & developer will use the brewed python:
hash -r python # invalidate bash's lookup cache for python
which python
# should say /usr/local/bin/python
echo $PATH
# /usr/local/bin should appear in front of /usr/bin
If this is not the case you’d probably end up not using brewed python. Please check your brew install with brew doctor
, it will probably tell you that you should consider updating your paths in ~/.bashrc
. You can either follow its directions or create a ~/.profile
file like this one: ~/.profile. If you performed these steps, please close your terminal windows and open a new one for the changes to take effect. Test the above again.
Even if the above check worked, run the following anyhow and read through its output (no output is good):
brew doctor
Pay special attention if this tells you to install XQuartz, and if it does, install it! You’ll need it anyhow…
Now after these preparations, let’s really start installing stuff… below you’ll mostly find one package / lib per line. For each of them and for their possible options: they’re a recommendation that might save you some trouble, so i’d recommend to install all of them as i write here, even if specifying some of the options will compile brew packages from source and take a bit longer…
# install PIL, imagemagick, graphviz and other
# image generating stuff
brew install libtiff libjpeg webp little-cms2
pip install Pillow
brew install imagemagick --with-fftw --with-librsvg --with-x11
brew install graphviz --with-librsvg --with-x11
brew install cairo
brew install py2cairo # this will ask you to download xquartz and install it
brew install qt pyqt5
# install virtualenv, nose (unittests & doctests on steroids)
pip install virtualenv
pip install nose
# install numpy and scipy
# nowadays there are two good ways to install numpy and scipy: via pip or via brew.
# PICK ONE:
# - i prefer pip for proper virtualenv support and more up-to-date versions.
# - brew builds are a bit older, but handy if you need a source build
pip install numpy
pip install scipy
# OR:
# (if you want to run numpy and scipy with openblas also remove comments below:)
#brew install openblas
#brew install numpy # --with-openblas
#brew install scipy # --with-openblas
# test the numpy & scipy install
python -c 'import numpy ; numpy.test();'
python -c 'import scipy ; scipy.test();'
# some cool python libs (if you don't know them, look them up)
# matplotlib: generate plots
# pandas: time series stuff
# nltk: natural language toolkit
# sympy: symbolic maths in python
# q: fancy debugging output
# snakeviz: cool visualization of profiling output (aka what's taking so long?)
#brew install Caskroom/cask/mactex # if you want to install matplotlib with tex support and don't have mactex installed already
brew install matplotlib --with-cairo --with-tex # cairo: png ps pdf svg filetypes, tex: tex fonts & formula in plots
pip install pandas
pip install nltk
pip install sympy
pip install q
pip install snakeviz
# ipython/jupyter with parallel and notebook support
brew install zmq
pip install ipython[all]
# html stuff (parsing)
pip install html5lib cssselect pyquery lxml BeautifulSoup
# webapps / apis (choose what you like)
pip install Flask Django tornado
# semantic web stuff: rdf & sparql
pip install rdflib SPARQLWrapper
# graphs (graph metrics, social network analysis, layouting)
pip install networkx
# maintenance: updating of pip libs
pip list --outdated # see Updating section below
Have fun 😉
Updating
OK, let’s say it’s been a while since you installed things with this guide and you now want to update all the installed libs. To do this you should first upgrade everything installed with brew like this:
brew update && brew outdated && brew upgrade
Afterwards for upgrading pip packages (globally or in a virtualenv) you can just run
pip list --outdated
to get a list of outdated packages and then manually update them with:
pip install -U package1 package2 ...
If you want a tiny bit more comfort you can use the pip-review package to do this interactively:
pip install pip-review
Once installed you should be able to run the following either in a virtualenv or globally for your whole system:
pip-review -i # for interactive mode, -a to upgrade all which is dangerous
It will check your installed packages for new versions and give you a list of outdated packages. I’d recommend to run it with the -i
option to interactively install the upgrades.
A word of warning about the brewed packages: If i recommended to install a package with brew above that’s usually for a good reason like the pip version not working properly. If you’re a bit more advanced, you can try to upgrade them with pip, but i’d recommend to properly unlink them with brew unlink <package>
before, as some pip packages might run into problems otherwise. If you find the pip package works like a charm then, please let me know in the comments below so i can update this guide. In general i prefer the pip packages as they’re more up to date, work in virtual environments and can then easily be updated with pip-review
.
As always: If you liked this, think something is missing or wrong leave a comment.
Updates to this guide:
2014-03-02: include checking of $PATH
for Mike
2015-03-17: enhanced many explanations, provided some useful options for packages, general workover
2015-04-15: included comment about installing mactex via cask if not there already, thanks to William
2015-06-05: Pillow via pip and Updating section
2015-11-01: pip-review
(was detached from pip-tools
) + alternative
2016-02-15: hash -r python
(invalidate bash python bin lookup cache)
2016-12-21: updates for sierra, brew upgrade
, python3 and some more comments
2017-03-30: updated pyqt’s package name to pyqt5
Pingback: Scientific Python on Mac OS X 10.8 with homebrew | Jörn's Blog
Pingback: 20140714 Monday Numpy, PyQt4, Pyaudio | YOUNG
Pingback: Python | Programmer Tea Time
Thanks, this was really helpful!
😉
Really thank you. You save me lots of time;-)
This is really good thank you! But I had problems with Scipy,
[cc_bash]
python -c import scipy; assert not scipy.test().failures
Python version 2.7.9 (default, Jan 7 2015, 11:50:42) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)]
nose version 1.3.3
Traceback (most recent call last):
File “”, line 1, in
AssertionError
Error: scipy: failed
Failed executing: python -c import\ scipy;\ assert\ not\ scipy.test().failures
/usr/local/Library/Homebrew/formula.rb:707:in `block in system’
/usr/local/Library/Homebrew/formula.rb:664:in `open’
/usr/local/Library/Homebrew/formula.rb:664:in `system’
/usr/local/Library/Taps/homebrew/homebrew-python/scipy.rb:88:in `block (2 levels) in ‘
/usr/local/Library/Homebrew/language/python.rb:25:in `call’
/usr/local/Library/Homebrew/language/python.rb:25:in `block in each_python’
/usr/local/Library/Homebrew/language/python.rb:17:in `each’
/usr/local/Library/Homebrew/language/python.rb:17:in `each_python’
/usr/local/Library/Taps/homebrew/homebrew-python/scipy.rb:87:in `block in ‘
/usr/local/Library/Homebrew/formula.rb:617:in `block in run_test’
/usr/local/Library/Homebrew/extend/fileutils.rb:21:in `mktemp’
/usr/local/Library/Homebrew/formula.rb:614:in `run_test’
/usr/local/Library/Homebrew/cmd/test.rb:38:in `block (2 levels) in test’
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/timeout.rb:66:in `timeout’
/usr/local/Library/Homebrew/cmd/test.rb:37:in `block in test’
/usr/local/Library/Homebrew/cmd/test.rb:15:in `each’
/usr/local/Library/Homebrew/cmd/test.rb:15:in `test’
/usr/local/Library/brew.rb:135:in `’
[/cc_bash]
Can you suggest anything to remedy this?
after updating to the recent version i had a similar problem identified by the
[cc_bash]ImportError: No module named _csr[/cc_bash]
Following the discussion in https://github.com/Homebrew/homebrew-python/issues/185#issuecomment-67534979 it seems this is caused by a leftover from previous installs and can be fixed with:
[cc_bash]find /usr/local/lib/python2.7/site-packages/scipy -name ‘*.pyc’ -delete[/cc_bash]
After this [cci_bash]ipython -c ‘import scipy; scipy.test()'[/cci_bash] seems to only report one failure, namely this one: https://github.com/python-pillow/Pillow/issues/1074 . Seems this is fixed in >0.15.0 and a false positive, so either wait for brew to pick up scipy 0.15 or just ignore 😉
Pingback: Coding theano under remote ubuntu server from local Mac (在本地mac机器上,写、跑、调试、看-远程ubuntu上的theano代码) | 挖沙啦挖沙啦
Pingback: Python Scientific Computing on Mac OS X Yosemite | Tung-Wei Lin
I’ve been struggling with this the whole day, until stumbling upon this. Seriously amazing – thank you!
I just ran into a problem with this. You need LaTeX installed before you can install `matplotlib`. That’s easy to fix with `brew install Caskroom/cask/mactex`. Although it appears to take forever!
I finally got mactex to install, but still can’t get matplotlib to install –with-tex. I just removed the –with-tex to install without latex and it worked fine. I hardly ever use tex anyway.
weird, worked for me… what errors did you get?
I had the same problem. Installed MacTex using:
brew install Caskroom/cask/mactex
Installed with no errors:
brew install Caskroom/cask/mactex
==> brew cask install Caskroom/cask/mactex
==> Downloading http://tug.org/cgi-bin/mactex-download/MacTeX.pkg
######################################################################## 100.0%
==> No checksum defined for Cask mactex, skipping verification
==> Running installer for mactex; your password may be necessary.
==> Package installers may write to any location; options such as --appdir are ignored.
Password:
==> installer: Package name is MacTeX-2015
==> installer: Installing at base path /
==> installer: The install was successful.
Then when installing matlib it said:
brew install matplotlib --with-cairo --with-tex
==> Installing matplotlib from homebrew/python
matplotlib: A LaTeX distribution is required for Homebrew to install this formula.
Make sure that "/usr/texbin", or the location you installed it to, is in
your PATH before proceeding.
You can install with Homebrew Cask:
brew install Caskroom/cask/mactex
You can download from:
https://www.tug.org/mactex/
matplotlib: `dvipng` not found. This is optional for Matplotlib.
You can install with Homebrew Cask:
brew install Caskroom/cask/matctex
Error: An unsatisfied requirement failed this build.
this looks like a path problem to me… can you run [cci]brew doctor[/cci] and [cci]echo $PATH[/cci]?
OK – I solved this problem. It’s necessary to open a new shell. The $PATH variable didn’t contain the Tex path in it. When you open a new shell, it should be there.
joern, I didn’t notice your reply before I posted mine. Yes – that seems to have been the issue. I opened a new shell window and all of a sudden there were a lot more entries in
echo $PATH
than there were before. And it included /Library/TeX/texbin.Do you know how to install numpy with openblas for python3?
yes, i think you can just run [cci_bash]brew install numpy –with-openblas –with-python3[/cci_bash].
You can see these flags with [cci_bash]brew info numpy[/cci_bash].
Thank you, it helps me a lot. Now I’m able to use theano with python. However, when I try to import theano with ipython I got the following error:
/Library/Python/2.7/site-packages/theano/gof/compiledir.py in ()
1 from __future__ import print_function
—-> 2 import six.moves.cPickle as pickle
3 import errno
4 import logging
5 import os
ImportError: No module named cPickle
I have tried to change line 2 in the cmodule.py from
import six.moves.cPickle as pickle to import six.moves.pickle as pickle
and also to
import six.moves.pickle
Any idea of what is wrong?
I’m not exactly sure, but it looks a lot like you didn’t install theano after this guide / didn’t properly follow this guide. If you had, the path would not be [cci]/Library/Python2.7/site-packages[/cci] (that is the system path), but more like [cci]/usr/local/lib/python2.7/site-packages/[/cci] (that is the brew python’s system wide path).
I’d uninstall all packages that you installed under [cci]/Library/Python2.7/site-packages[/cci] and then re-install them with brew python’s pip. Before you do so, make sure that you’re actually running brew python’s pip: [cci]which pip[/cci] on your commandline should return [cci]/usr/local/bin/pip[/cci]. Also when running [cci]which python[/cci] it should return [cci]/usr/local/bin/python[/cci].
Thanks for the information, good job.
I have a problem with the graph-tool installation. As you can see all is prepared and allready to install:
| => brew info graph-tool
homebrew/science/graph-tool: stable 2.7 (bottled), HEAD
http://graph-tool.skewed.de/
Not installed
From: https://github.com/Homebrew/homebrew-science/blob/master/graph-tool.rb
==> Dependencies
Build: pkg-config ✔
Required: boost ✔, boost-python ✔, cairomm ✔, cgal ✔, py2cairo ✔, gnome-icon-theme ✔, librsvg ✔, pygobject3 ✔
Recommended: google-sparsehash ✔, gtk+3 ✔, homebrew/python/numpy ✔, homebrew/python/scipy ✔, homebrew/python/matplotlib ✔
But the installation never finish:
| => brew install graph-tool
==> Installing graph-tool from homebrew/homebrew-science
==> Installing dependencies for homebrew/science/graph-tool: librsvg
==> Installing homebrew/science/graph-tool dependency: librsvg
==> Downloading https://download.gnome.org/sources/librsvg/2.40/librsvg-2.40.10.tar.xz
Already downloaded: /Library/Caches/Homebrew/librsvg-2.40.10.tar.xz
==> ./configure –prefix=/usr/local/Cellar/librsvg/2.40.10_1 –disable-Bsymbolic –enable-tools=yes –enable-pixbuf-loader=yes –enable-introspection=yes
==> make install gdk_pixbuf_binarydir=/usr/local/Cellar/librsvg/2.40.10_1/lib/gdk-pixbuf-2.0/2.10.0/loaders gdk_pixbuf_moduledir=/usr/local/Cellar/librsvg/2.40.10_1/lib/gdk-pixbuf-2.0/2.10.0/loaders
It never finish the “make install”. Do you know how to solve it?
Thanks
hmmm, what you posted doesn’t seem to include an error message, sorry… maybe open an issue in github?
This worked perfectly on El Capitan. Make sure you’ve updated Xcode (via the App Store) and your Xcode command line tools:
xcode-select –install
I picked the pip install path for scipy and numpy; you will need to run `brew link numpy`
It worked fine, and thanks for the useful advice.
But although I installed pip-tools, I don’t have access to any pip-review command…
But I do have a pip-compile and a pip-sync (as explained in https://github.com/nvie/pip-tools)
On that Web site there is no mention of pip-review whatsoever, doesn’t it exist?
Thanks again
yes, the pip-tools were updated since writing this guide and don’t provide the pip-review command anymore. I updated the guide.
Thanks! This save me a lot of time! 🙂
Thanks for the fantastic guide! However, pip’s numpy gave the following errors during the test, any ideas how to fix this:
ERROR: test_scripts.test_f2py
———————————————————————-
Traceback (most recent call last):
File “/usr/local/lib/python2.7/site-packages/nose/case.py”, line 197, in runTest
self.test(*self.arg)
File “/usr/local/lib/python2.7/site-packages/numpy/testing/decorators.py”, line 146, in skipper_func
return f(*args, **kwargs)
File “/usr/local/lib/python2.7/site-packages/numpy/tests/test_scripts.py”, line 68, in test_f2py
code, stdout, stderr = run_command([f2py_cmd, ‘-v’])
File “/usr/local/lib/python2.7/site-packages/numpy/tests/test_scripts.py”, line 48, in run_command
proc = Popen(cmd, stdout=PIPE, stderr=PIPE)
File “/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py”, line 710, in __init__
errread, errwrite)
File “/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py”, line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
———————————————————————-
Ran 5919 tests in 34.415s
FAILED (KNOWNFAIL=3, SKIP=8, errors=1)
Scipy test output gave:
Ran 18421 tests in 125.988s
OK (KNOWNFAIL=97, SKIP=1183)
Is this fine?
hmm, this seems like a minor bug in numpy to me… i made a bugreport nevertheless: https://github.com/numpy/numpy/issues/6718
Thank you SO MUCH for clearing this up. I like playing with all the new python packages as they come out and I had like 8 versions of python installed that weren’t all working with my pip installed libraries. This is the grunt work that isn’t so fun to blog about, but THANK YOU again.
😉 you’re welcome
As of now, on some newer Mac systems you can stumble upon breaked python install in brew. When runninng numpy/scipy test, you can get this error:
ImportError: dlopen(/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so, 2): Symbol not found: __PyCodecInfo_GetIncrementalDecoder
Referenced from: /usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_io.so
Expected in: flat namespace
The easy way to handle this is to downgrade to 2.7.10. For this you’ll need to do following:
$ brew unlink python
$ cd /usr/local/Library/
$ git checkout 10e9b5bf267010a34bad60a392206571909fedef /usr/local/Library/Formula/python.rb
$ brew install python
and voila!
$ python –version
Python 2.7.10
didn’t have that problem after upgrading to Python 2.7.11 myself, but thanks for the feedback. Note that the fix you mention is only a temporary solution and might cause conflicts when updating brew… if you really need an older python version for whatever reason, i’d recommend installing pyenv instead (brew install pyenv) and configure it as you wish.
I also ran into this problem. The root cause is explained here in superbatfish’s response to one of the comments: http://stackoverflow.com/questions/34386527/symbol-not-found-pycodecinfo-getincrementaldecoder-trying-to-register-a-packa
The relevant comment: “The fundamental problem here is that bash’s hash cache can become incorrect immediately after you install a new version of python. (Try type python to verify that’s the problem.) The person you quoted may have accidentally fixed this via all the uninstalling/reinstalling they described, but the straightforward fix is to type hash -r python into the terminal. – superbatfish”
rehashing python fixed this for me without having to downgrade to 2.10.
Hope this helps someone out there
thanks, updated the post
Thank you!
This helped me greatly fix my python environment after upgrading to El Capitan.
Thank you.
Thank you !!!!
hash -r python saved my day. In the middle of writing a research paper I though I screwed up everything.
Thanks again.
I think this installs python2.7, I want to install python3.
Would simply saying:
brew tap homebrew/python3
be sufficient? Can you tell me how to install this setup with python 3?
Thanks, great setup manual!
I m not able to install mathplotlib.I have installed mactex but still i am getting the error. Error is given below:
==> Installing matplotlib from homebrew/python
matplotlib: A LaTeX distribution is required for Homebrew to install this formula.
Make sure that “/usr/texbin”, or the location you installed it to, is in
your PATH before proceeding.
TeXRequirement unsatisfied!
You can install with Homebrew-Cask:
brew cask install mactex
You can download from:
https://www.tug.org/mactex/
matplotlib:
dvipng
not found. This is optional for Matplotlib.DvipngRequirement unsatisfied!
You can install with Homebrew-Cask:
brew cask install matctex
Error: An unsatisfied requirement failed this build.
My system had Mac OS Sierra. v 10.12
Can u help me with this please.
if you don’t have mactex i guess you don’t want latex support, just remove the –with-tex option then
Looks like you don’t need –all in the brew update:
myname-MBP:~ myname$ brew update && brew outdated && brew upgrade –all
Warning: We decided to not change the behaviour of
brew upgrade
sobrew upgrade --all
is equivalent tobrew upgrade
without any otherarguments (so the
--all
is a no-op and can be removed).Also, is there any particular reason you suggest brew for numpy and scipy? It looks like they install older versions. Also what is the advantage to openblas? I got an error when using “brew install numpy –with openblas” and just went with pip.
thanks for the feedback. Updated the
--all
and clarified brewing numpy and scipyThanks for the excellent step-by-step instructions and keeping this page up to date! And also for the previous post – John Cleese on Creativity – which made excellent viewing while the installs did their thing. I commented on John Cleese post with new URL for the video.
Only one minor change to your steps:
$ brew install qt pyqt
Error: No available formula with the name “pyqt”
==> Searching for similarly named formulae…
This similarly named formula was found:
pyqt5
To install it, run:
brew install pyqt5
I ran brew install qt pyqt5 and all seems well.
Thanks again!
yes, seems this is the updated module’s name. thanks for reporting 🙂