-
Notifications
You must be signed in to change notification settings - Fork 8
/
Copy pathnasty-python-monstrosities.html
196 lines (184 loc) · 10.8 KB
/
nasty-python-monstrosities.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
<!DOCTYPE html>
<html>
<head>
<link rel="canonical" href="https://hardmath123.github.io/nasty-python-monstrosities.html"/>
<link rel="stylesheet" type="text/css" href="/static/base.css"/>
<title>Nasty Python Monstrosities - Comfortably Numbered</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
<link rel="alternate" type="application/rss+xml" title="Comfortably Numbered" href="/feed.xml" />
<!--
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script>
MathJax.Hub.Config({
tex2jax: {inlineMath: [['$','$']]}
});
</script>
-->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css" integrity="sha384-Um5gpz1odJg5Z4HAmzPtgZKdTBHZdw8S29IecapCSB31ligYPhHQZMIlWLYQGVoc" crossorigin="anonymous">
<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.js" integrity="sha384-YNHdsYkH6gMx9y3mRkmcJ2mFUjTd0qNQQvY9VYZgQd7DcN7env35GzlmFaZ23JGp" crossorigin="anonymous"></script>
<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/contrib/auto-render.min.js" integrity="sha384-vZTG03m+2yp6N6BNi5iM4rW4oIwk5DfcNdFfxkk9ZWpDriOkXX8voJBFrAO7MpVl" crossorigin="anonymous"></script>
<script>
document.addEventListener("DOMContentLoaded", function() {
renderMathInElement(document.body, {
// customised options
// • auto-render specific keys, e.g.:
delimiters: [
{left: '$$', right: '$$', display: true},
{left: '$', right: '$', display: false},
{left: '\\begin{align}', right: '\\end{align}', display: true},
{left: '\\(', right: '\\)', display: false},
{left: '\\[', right: '\\]', display: true}
],
// • rendering keys, e.g.:
throwOnError : false
});
});
</script>
</head>
<body>
<header id="header">
<script src="static/main.js"></script>
<div>
<a href="/"><span class="left-word">Comfortably</span> <span class="right-word">Numbered</span></a>
</div>
</header>
<article id="postcontent" class="centered">
<section>
<h1>Nasty Python Monstrosities</h1>
<center><em><p>Why I’m not going to distribute any Python ever again.</p>
</em></center>
<h4>Wednesday, February 4, 2015 · 4 min read</h4>
<p><strong>Scenario:</strong> I’ve written a simple Python command-line tool. It consists of
more than one file, because separation of code is important. It only needs to
run on UNIX systems and shouldn’t require sudo to install. How do I distribute
this code in a safe, user-friendly way? How do I teach my intro CS class how to
proudly share their creations with the world?</p>
<p><strong>Answer:</strong> You don’t.</p>
<p>Distributing Python code should be a solved problem. It’s not. Or, rather, it’s
a <a href="http://xkcd.com/927/">standards problem</a>. There are too many tools out
there, and there’s no clear roadmap that explains which subset does what. I’m
talking about Pip, Virtualenv, PyPi, wheels, eggs, distutils, distribute,
setuptools, easy_install, and twine. Some of these have been incorporated into
others, and are therefore obsolete. Some of these are endorsed by a PEP.
(What’s a PEP? Do I care?)</p>
<p>There is certainly not one obvious way to do it. Even if you’re Dutch.</p>
<p>Assuming you have picked out a subset of tools to use, though, there’s no
guarantee that they will cooperate. A Pip flag, for instance, might work. It
might not work. It might do stuff that you didn’t expect. It might create files
without telling you (correction: it’ll be noted cryptically in the Pip log). It
might pass it on to <code>setup.py</code>, or it might not, or it might reject a valid
<code>setup.py</code> argument because it doesn’t understand it, or it might pass it on to
Python erroneously.</p>
<p>My Pip log contains monstrosities such as:</p>
<p>(Enumeration of all the things here that make me cry is left as an exercise to
the reader.)</p>
<pre><code>Running command /usr/bin/python -c "import setuptools, tokenize;
__file__='/path/to/setup.py'; exec(compile(getattr(tokenize, 'open',
open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop
--no-deps --user
</code></pre><p>In short, it might eat your laundry depending on local atmospheric conditions,
and then it’ll flip a coin to decide whether or not to tell you.</p>
<hr>
<p>Let’s talk about publishing. PyPi is the “standard” repository. But it makes you
host your own tarballs, so essentially all it does is match package names to
URLs (all other information is duplicated in the tarball itself).</p>
<p>Uploading to PyPi happens <em>over an insecure connection</em>. <a href="https://packaging.python.org/en/latest/distributing.html#upload-your-distributions">They recognize this
in the
docs</a>,
and recommend installing a <em>separate</em> client that lets you publish over SSL.</p>
<p>They also keep telling users to use OpenID. The OpenID that <a href="https://developers.google.com/accounts/docs/OpenID#shutdown-timetable">Google is
deprecating</a>
in a couple of months. Of course, <a href="https://pypi.python.org/pypi?:action=login&provider=Google">trying to log in with
Google</a> throws an
internal server error.</p>
<p>Apparently, the whole system is so confusing that every time you upload, you
are encouraged to do a dry run on the test repository just to make sure you
understand it.</p>
<p>Now, my Python program was not a module, but a command-line tool. <a href="http://blog.habnab.it/blog/2013/07/21/python-packages-and-you/">The way you
specify a command-line
tool</a> in Python
looks like this:</p>
<pre><code class="lang-python"># ...
entry_points={
'console_scripts': ['name_to_link = packagename.submodule:function_name'],
},
# ...
</code></pre>
<p>Here’s what setuptools does. It parses that monstrous string and extracts
information. It then creates a <em>new</em> Python script in a <code>$PATH</code>‘d directory.
That script <em>imports</em> your script as a module, and then calls the function you
specify. It’s anyone’s guess whether important scriptey things such as
environment variables or command-line arguments are preserved.</p>
<p>This is all in complete ignorance of the whole <code>if __name__ == '__main__'</code>
infrastructure, for whatever reason. Of course, you can <em>also</em> make things
runnable with <code>python -m packagename.submodule</code> by including a <code>__main__</code> file.
Yum.</p>
<p>Even if it did the reasonable thing and symlinked your script directly, it
would actually <em>modify</em> your file. That’s right. It <a href="https://docs.python.org/2/distutils/setupscript.html#installing-scripts">replaces shebang
lines</a>
with the path to the currently-running Python interpreter. Because Explicit Is
Better Than Implicit (tm).</p>
<p>There is no reason to do this. UNIX provides a very helpful idiom: using
<code>#!/usr/bin/env python arguments...</code>. That’ll search the $PATH for the
right interpreter and use it. No black magic needed, and you can provide flags
to the python interpreter if you so desire.</p>
<p>Only God—make that Guido—knows what’ll happen if you’re trying to install a
Python 3 script using a Python 2.7. Or want to distribute something runnable
with <code>ipython</code>.</p>
<hr>
<p>By the way, if you think <em>installing</em> is bad: you would think uninstalling
should be a standardized, well-thought-out, documented process, right? You
wish. There setup.py-installed programs have no fixed uninstallation procedure.
If you Google around, you are led to <a href="http://stackoverflow.com/a/1550235/3053429">this highly-upvoted StackOverflow
answer</a>, which suggests using an
option that lists all generated files, and <em>piping the output</em> to <code>xargs rm
-rf</code>.</p>
<p>Let’s hope they don’t have a file with a space in it, or <em>bad</em> things are going
to happen. Keep in mind that Python is recommended for newbies, the kinds who
will gladly copy-and-paste shell commands from the Internet.</p>
<hr>
<p>Here’s the deal. Flat is <em>not</em> better than nested. Nested is better than flat.
There’s a reason we use parsers to convert strings to abstract syntax trees.
There’s a reason LISP is easier to write than x86 assembly. There’s a reason
for the existence of the term “dependency tree”. They are <em>literally</em> nested
structures. Importing from a sibling directory should not be the harrowing
experience it is in Python.</p>
<p>And that’s why I love npm. Not because it’s written in a beautiful language or
because it pretends to be easy-to-use, but because it’s simple and elegant and
it doesn’t do too much or too little. Modules are nested prettily in their
directories, and the <em>only</em> file npm touches is <code>package.json</code>.</p>
<p>Scripts are simply linked into npm’s bin folder. Your “script” could be a Bash
program, for all it cares.</p>
<p>Installing is one command, uninstalling is one command, publishing is one
command, and everything is one tool operating on one repository.</p>
<p>npm is transparent. Python’s plethora of packaging plakavacs are anything but.</p>
</section>
<div id="comment-breaker">◊ ◊ ◊</div>
</article>
<footer id="footer">
<div>
<ul>
<li><a href="https://github.com/kach">
Github</a></li>
<li><a href="feed.xml">
Subscribe (RSS feed)</a></li>
<li><a href="https://twitter.com/hardmath123">
Twitter</a></li>
<li><a href="https://creativecommons.org/licenses/by-nc/3.0/deed.en_US">
CC BY-NC 3.0</a></li>
</ul>
</div>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-46120535-1', 'hardmath123.github.io');
ga('require', 'displayfeatures');
ga('send', 'pageview');
</script>
</footer>
</body>
</html>