{"id":566,"date":"2014-02-25T03:38:55","date_gmt":"2014-02-25T02:38:55","guid":{"rendered":"http:\/\/joernhees.de\/blog\/?p=566"},"modified":"2017-03-30T13:44:17","modified_gmt":"2017-03-30T11:44:17","slug":"scientific-python-on-mac-os-x-10-9-with-homebrew","status":"publish","type":"post","link":"https:\/\/joernhees.de\/blog\/2014\/02\/25\/scientific-python-on-mac-os-x-10-9-with-homebrew\/","title":{"rendered":"Scientific Python on Mac OS X 10.9+ with homebrew"},"content":{"rendered":"

There are (too) many guides out there about how to install Python on Mac OS X. So why another one? Cause i found most of the other ways to lead to long-term maintenance debt that is unnecessary and can easily be avoided. Also many of the other guides aren’t very easy to extend in “but i also need that library”-cases. So here I’ll briefly explain how to set-up a scientific python stack based on homebrew. The advantages are that this stack will never conflict with your system’s core and homebrew opens up a whole new world of easy to access unix tools via a simple brew install ...<\/code>.<\/p>\n

This step-by-step installation guide to setup a scientific python environment has been tested on Mac OS X Mavericks 10.9 \/ Yosemite 10.10 \/ El Capitain 10.11 \/ Sierra 10.12. It will probably also work in the following versions (if not, let me know in the comments).
\nAn older version of this setup guide can be found here: Scientific Python on Mac OS X 10.8 with homebrew<\/a>, main changes: rename of a tap + changes wrt. openblas as Accelerate was fixed in OS X 10.9<\/a><\/span><\/p>\n

Needless to say: Make a backup (Timemachine)<\/p>\n

First install homebrew<\/a>.
\nFollow their instructions, then come back here.<\/p>\n

If you don’t have a clean install, some of the following steps might need minor additional attention (like changing permissions chmod<\/code>, chown<\/code>, chgrp<\/code> or overwriting existing files in the linking step with brew link --overwrite package_that_failed<\/code>. In this case i can only recommend a backup again).<\/p>\n

In general: execute the following commands one line at a time and read the outputs! If you read some warnings about “keg-only” that’s fine, it just means that brew won’t “hide” your system’s stuff behind the stuff it installed itself so it doesn’t cause problems… brewed stuff will still use it.<\/p>\n

# set up some taps and update brew\nbrew tap homebrew\/science # a lot of cool formulae for scientific tools\nbrew tap homebrew\/python # numpy, scipy, matplotlib, ...\nbrew update && brew upgrade\n\n# install a brewed python\nbrew install python\n\n# and\/or if you want to give python3 a go (it's about time):\n#brew install python3  # repeat\/replace the following pip commands with pip3\n<\/code><\/pre>\n
A word about brewed python: this is what you want!<\/p>\n
It’s more up to date than the system python, it will come with pip and correctly install in the brew directories, working together well with brewed python libs that aren’t installable with plain pip. This also means that pip by default will work without sudo as all of homebrew, so if you ever run or have to run sudo pip ...<\/code> because of missing permissions, then you’re doing it wrong! Also, don’t be afraid of multiple pythons on your system: you have them anyhow (python2 and python3) and it’s an advantage, as we’ll make sure that nothing poisons your system python and that you as a user & developer will use the brewed python:<\/p>\n
hash -r python  # invalidate bash's lookup cache for python\nwhich python\n# should say \/usr\/local\/bin\/python\n\necho $PATH\n# \/usr\/local\/bin should appear in front of \/usr\/bin\n<\/code><\/pre>\n
If this is not the case you’d probably end up not using brewed python. Please check your brew install with brew doctor<\/code>, it will probably tell you that you should consider updating your paths in ~\/.bashrc<\/code>. You can either follow its directions or create a ~\/.profile<\/code> file like this one: ~\/.profile<\/a>. If you performed these steps, please close your terminal windows and open a new one for the changes to take effect. Test the above again.<\/p>\n
Even if the above check worked, run the following anyhow and read through its output (no output is good):<\/p>\n
brew doctor\n<\/code><\/pre>\n
Pay special attention if this tells you to install XQuartz, and if it does, install it! You’ll need it anyhow…<\/p>\n
Now after these preparations, let’s really start installing stuff… below you’ll mostly find one package \/ lib per line. For each of them and for their possible options: they’re a recommendation that might save you some trouble, so i’d recommend to install all of them as i write here, even if specifying some of the options will compile brew packages from source and take a bit longer…<\/p>\n
# install PIL, imagemagick, graphviz and other\n# image generating stuff\nbrew install libtiff libjpeg webp little-cms2\npip install Pillow\nbrew install imagemagick --with-fftw --with-librsvg --with-x11\nbrew install graphviz --with-librsvg --with-x11\nbrew install cairo\nbrew install py2cairo # this will ask you to download xquartz and install it\nbrew install qt pyqt5\n\n# install virtualenv, nose (unittests & doctests on steroids)\npip install virtualenv\npip install nose\n\n# install numpy and scipy\n# nowadays there are two good ways to install numpy and scipy: via pip or via brew.\n# PICK ONE:\n# - i prefer pip for proper virtualenv support and more up-to-date versions.\n# - brew builds are a bit older, but handy if you need a source build\npip install numpy\npip install scipy\n# OR:\n# (if you want to run numpy and scipy with openblas also remove comments below:)\n#brew install openblas\n#brew install numpy # --with-openblas\n#brew install scipy # --with-openblas\n\n# test the numpy & scipy install\npython -c 'import numpy ; numpy.test();'\npython -c 'import scipy ; scipy.test();'\n\n# some cool python libs (if you don't know them, look them up)\n# matplotlib: generate plots\n# pandas: time series stuff\n# nltk: natural language toolkit\n# sympy: symbolic maths in python\n# q: fancy debugging output\n# snakeviz: cool visualization of profiling output (aka what's taking so long?)\n#brew install Caskroom\/cask\/mactex  # if you want to install matplotlib with tex support and don't have mactex installed already\nbrew install matplotlib --with-cairo --with-tex  # cairo: png ps pdf svg filetypes, tex: tex fonts & formula in plots\npip install pandas\npip install nltk\npip install sympy\npip install q\npip install snakeviz\n\n# ipython\/jupyter with parallel and notebook support\nbrew install zmq\npip install ipython[all]\n\n# html stuff (parsing)\npip install html5lib cssselect pyquery lxml BeautifulSoup\n\n# webapps \/ apis (choose what you like)\npip install Flask Django tornado\n\n# semantic web stuff: rdf & sparql\npip install rdflib SPARQLWrapper\n\n# graphs (graph metrics, social network analysis, layouting)\npip install networkx\n\n# maintenance: updating of pip libs\npip list --outdated  # see Updating section below\n<\/code><\/pre>\n
Have fun \ud83d\ude09<\/p>\n
Updating<\/h2>\n
OK, let’s say it’s been a while since you installed things with this guide and you now want to update all the installed libs. To do this you should first upgrade everything installed with brew like this:<\/p>\n
brew update && brew outdated && brew upgrade\n<\/code><\/pre>\n
Afterwards for upgrading pip packages (globally or in a virtualenv) you can just run<\/p>\n
pip list --outdated\n<\/code><\/pre>\n
to get a list of outdated packages and then manually update them with:<\/p>\n
pip install -U package1 package2 ...\n<\/code><\/pre>\n
If you want a tiny bit more comfort you can use the pip-review package to do this interactively:<\/p>\n
pip install pip-review\n<\/code><\/pre>\n
Once installed you should be able to run the following either in a virtualenv or globally for your whole system:<\/p>\n
pip-review -i # for interactive mode, -a to upgrade all which is dangerous\n<\/code><\/pre>\n
It will check your installed packages for new versions and give you a list of outdated packages. I’d recommend to run it with the -i<\/code> option to interactively install the upgrades.<\/p>\n
A word of warning about the brewed packages: If i recommended to install a package with brew above that’s usually for a good reason like the pip version not working properly. If you’re a bit more advanced, you can try to upgrade them with pip, but i’d recommend to properly unlink them with brew unlink <package><\/code> before, as some pip packages might run into problems otherwise. If you find the pip package works like a charm then, please let me know in the comments below so i can update this guide. In general i prefer the pip packages as they’re more up to date, work in virtual environments and can then easily be updated with pip-review<\/code>.<\/p>\n
As always: If you liked this, think something is missing or wrong leave a comment.<\/p>\n
Updates to this guide:<\/h3>\n
2014-03-02: include checking of $PATH<\/code> for Mike
\n2015-03-17: enhanced many explanations, provided some useful options for packages, general workover
\n2015-04-15: included comment about installing mactex via cask if not there already, thanks to William
\n2015-06-05: Pillow via pip and Updating section
\n2015-11-01: pip-review<\/code> (was detached from pip-tools<\/code>) + alternative
\n2016-02-15: hash -r python<\/code> (invalidate bash python bin lookup cache)
\n2016-12-21: updates for sierra, brew upgrade<\/code>, python3 and some more comments
\n2017-03-30: updated pyqt’s package name to pyqt5<\/p>\n","protected":false},"excerpt":{"rendered":"
Scientific python setup guide for Mac OS X 10.9 Mavericks with homebrew<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[2],"tags":[20,57,69,77,96,132,146,147,149,156,171],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pYA5n-98","jetpack-related-posts":[{"id":526,"url":"https:\/\/joernhees.de\/blog\/2013\/06\/08\/mac-os-x-10-8-scientific-python-with-homebrew\/","url_meta":{"origin":566,"position":0},"title":"Scientific Python on Mac OS X 10.8 with homebrew","date":"2013-06-08","format":false,"excerpt":"(newer version of this guide) A step-by-step installation guide to setup a scientific python environment based on Mac OS X and homebrew. Needless to say: Make a backup (Timemachine) First install homebrew. Follow their instructions, then come back here. If you don't have a clean install, some of the following\u2026","rel":"","context":"In "Coding"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":19,"url":"https:\/\/joernhees.de\/blog\/2010\/06\/28\/python-and-encoding\/","url_meta":{"origin":566,"position":1},"title":"Python and encoding","date":"2010-06-28","format":false,"excerpt":"Well, first real post, so let's start easy. I've been working a lot with python lately, and came across a nice short How to Use UTF-8 with Python which also makes the difference between unicode and utf8 very clear. The howto also links to another valuable source: Characters vs. Bytes,\u2026","rel":"","context":"In "Coding"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":314,"url":"https:\/\/joernhees.de\/blog\/2010\/12\/15\/python-unicode-doctest-howto-in-a-doctest\/","url_meta":{"origin":566,"position":2},"title":"Python unicode doctest howto in a doctest","date":"2010-12-15","format":false,"excerpt":"Another thing which has been on my stack for quite a while has been a unicode doctest howto, as I remember I was quite lost when I first tried to test encoding stuff in a doctest. So I thought the ultimate way to show how to do this would be\u2026","rel":"","context":"In "Coding"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":256,"url":"https:\/\/joernhees.de\/blog\/2010\/09\/21\/how-to-convert-hex-strings-to-binary-ascii-strings-in-python-incl-8bit-space\/","url_meta":{"origin":566,"position":3},"title":"How to convert hex strings to binary ascii strings in python (incl. 8bit space)","date":"2010-09-21","format":false,"excerpt":"As i come across this again and again: How do you turn a hex string like \"c3a4c3b6c3bc\" into a nice binary string like this: \"11000011 10100100 11000011 10110110 11000011 10111100\"? The solution is based on the Python 2.6 new string formatting: >>> \"{0:8b}\".format(int(\"c3\",16)) '11000011' Which can be decomposed into 4\u2026","rel":"","context":"In "Coding"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":91,"url":"https:\/\/joernhees.de\/blog\/2010\/07\/21\/sort-python-dict-by-values\/","url_meta":{"origin":566,"position":4},"title":"Sort python dictionaries by values","date":"2010-07-21","format":false,"excerpt":"Perhaps you already encountered a problem like the following one yourself: You have a large list of items (let's say URIs for this example) and want to sum up how often they were viewed (or edited or... whatever). A small one-shot solution in python looks like the following and uses\u2026","rel":"","context":"In "Coding"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":81,"url":"https:\/\/joernhees.de\/blog\/2010\/07\/19\/min-heap-in-python\/","url_meta":{"origin":566,"position":5},"title":"Min-Heap in Python","date":"2010-07-19","format":false,"excerpt":"I recently wanted to implement a small event system where events can have different priorities. So for example the event with highest priority (lowest value) should be handled first. Python comes with a heapq module which can transform a list into a heap in a way that it stays a\u2026","rel":"","context":"In "Coding"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/posts\/566"}],"collection":[{"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/comments?post=566"}],"version-history":[{"count":12,"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/posts\/566\/revisions"}],"predecessor-version":[{"id":833,"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/posts\/566\/revisions\/833"}],"wp:attachment":[{"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/media?parent=566"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/categories?post=566"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/joernhees.de\/blog\/wp-json\/wp\/v2\/tags?post=566"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}