1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
<title>Python package maintenance — Gentoo Python Guide documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/sphinx_highlight.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Maintenance of Python implementations" href="interpreter-maintenance.html" />
<link rel="prev" title="QA checks and warnings" href="qawarn.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="python-package-maintenance">
<h1>Python package maintenance<a class="headerlink" href="#python-package-maintenance" title="Permalink to this heading">¶</a></h1>
<section id="support-for-python-2">
<h2>Support for Python 2<a class="headerlink" href="#support-for-python-2" title="Permalink to this heading">¶</a></h2>
<p>Since Python 2.7 reached EOL, Gentoo is currently phasing out support
for Python 2. Unless your package or its reverse dependencies really
need it, you should omit it from <code class="docutils literal notranslate"><span class="pre">PYTHON_COMPAT</span></code>. If you’re adding
a new package and it does not support Python 3, do not add it.</p>
<p>Many upstreams are removing Python 2 support from new releases of their
software. We remove it proactively whenever reverse dependencies permit
in order to anticipate this and avoid having to deal with lots
of reverse dependencies afterwards.</p>
<p>Packages that do not support Python 3 and are unlikely to start
supporting it soon are being slowly removed.</p>
</section>
<section id="which-implementations-to-test-new-packages-for">
<h2>Which implementations to test new packages for?<a class="headerlink" href="#which-implementations-to-test-new-packages-for" title="Permalink to this heading">¶</a></h2>
<p>The absolute minimum set of targets are the current default targets
found in <code class="docutils literal notranslate"><span class="pre">profiles/base/make.defaults</span></code>. However, developers
are strongly encouraged to test at least the next Python 3 version
in order to ease future transition, and preferably all future versions.</p>
<p>Marking for PyPy3 is optional. At this moment, we do not aim for wide
coverage of PyPy3 support.</p>
</section>
<section id="adding-new-python-implementations-to-existing-packages">
<h2>Adding new Python implementations to existing packages<a class="headerlink" href="#adding-new-python-implementations-to-existing-packages" title="Permalink to this heading">¶</a></h2>
<p>New Python implementations can generally be added to existing packages
without a revision bump. This is because the new dependencies are added
conditionally to new USE flags. Since the existing users can not have
the new flags enabled, the dependencies do not need to be proactively
added to existing installations.</p>
<p>This usually applies to stable packages as well as new Python targets
are generally <code class="docutils literal notranslate"><span class="pre">use.stable.mask</span></code>-ed. This means that stable users
will not be able to enable newly added flags and therefore the risk
of the change breaking stable systems is minimal.</p>
</section>
<section id="which-packages-can-be-co-maintained-by-the-python-project">
<h2>Which packages can be (co-)maintained by the Python project?<a class="headerlink" href="#which-packages-can-be-co-maintained-by-the-python-project" title="Permalink to this heading">¶</a></h2>
<p>A large part of the Python ecosystem is fairly consistent, making it
feasible for (co-)maintenance by the Gentoo Python team.</p>
<p>As a rule of thumb, Python team is ready to maintain packages specific
to the Python ecosystem and useful for the general population of Python
programmers. This includes Python interpreters and tooling, packages
purely providing Python modules and extensions and utilities specific
to the Python language.</p>
<p>However, the Python team has limited manpower, therefore it may reject
packages that have high maintenance requirements. As a rule, Python
team does not accept packages without working tests.</p>
<p>If your package matches the above profile, feel free to ask a member
of the Python project whether they would like to (co-)maintain
the package. However, if you are not a member of the project, please
do not add us without asking first.</p>
</section>
<section id="porting-packages-to-a-new-eapi">
<h2>Porting packages to a new EAPI<a class="headerlink" href="#porting-packages-to-a-new-eapi" title="Permalink to this heading">¶</a></h2>
<p>When porting packages to a new EAPI, please take care not to port
the dependencies of Portage prematurely. This generally includes
<code class="docutils literal notranslate"><span class="pre">app-portage/gemato</span></code>, <code class="docutils literal notranslate"><span class="pre">dev-python/setuptools</span></code> and their recursive
dependencies.</p>
<p>Ideally, these ebuilds carry an appropriate note above their EAPI line,
e.g.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># please keep this ebuild at EAPI 7 -- sys-apps/portage dep</span>
<span class="n">EAPI</span><span class="o">=</span><span class="mi">7</span>
</pre></div>
</div>
<p>This does not apply to test dependencies — they are not strictly
necessary to install a new Portage version.</p>
</section>
<section id="monitoring-new-package-versions">
<h2>Monitoring new package versions<a class="headerlink" href="#monitoring-new-package-versions" title="Permalink to this heading">¶</a></h2>
<section id="pypi-release-feeds">
<h3>PyPI release feeds<a class="headerlink" href="#pypi-release-feeds" title="Permalink to this heading">¶</a></h3>
<p>The most efficient way to follow new Python package releases are
the feeds found on <a class="reference external" href="https://pypi.org/">PyPI</a>. These can be found in the package’s
“Release history” tab, as “RSS feed”.</p>
<p>The Gentoo Python project maintains a comprehensive <a class="reference external" href="https://projects.gentoo.org/python/release-feeds.opml">list of PyPI feeds
for packages</a> in <code class="docutils literal notranslate"><span class="pre">dev-python/</span></code> category (as well as other important
packages maintained by the Python team) in OPML format.</p>
</section>
<section id="checking-via-pip">
<h3>Checking via pip<a class="headerlink" href="#checking-via-pip" title="Permalink to this heading">¶</a></h3>
<p>The <a class="reference internal" href="#pip-list-outdated">pip list --outdated</a> command described in a followup section
can also be used to verify installed packages against their latest PyPI
releases. However, this is naturally limited to packages installed
on the particular system, and does not account for newer versions being
already available in the Gentoo repository.</p>
</section>
<section id="repology">
<h3>Repology<a class="headerlink" href="#repology" title="Permalink to this heading">¶</a></h3>
<p><a class="reference external" href="https://repology.org/">Repology</a> provides a comprehensive service for tracking distribution
package versions and upstream releases. The easiest ways to find Python
packages present in the Gentoo repository is to search by their
maintainer’s e-mail or category (e.g. <code class="docutils literal notranslate"><span class="pre">dev-python</span></code>). When searching
by name, the majority of Python-specific package use <code class="docutils literal notranslate"><span class="pre">python:</span></code> prefix
in their Repology names.</p>
<p>Unfortunately, Repology is very susceptible to false positives.
Examples of false positives include other distributions using custom
version numbers, replacing packages with forks or simply Repology
confusing different packages with the same name. If you find false
positives, please use the ‘Report’ option to request a correction.</p>
<p>Please also note that Repology is unable to handle the less common
version numbers that do not have a clear mapping to Gentoo version
syntax (e.g. <code class="docutils literal notranslate"><span class="pre">.post</span></code> releases).</p>
</section>
</section>
<section id="routine-checks-on-installed-python-packages">
<h2>Routine checks on installed Python packages<a class="headerlink" href="#routine-checks-on-installed-python-packages" title="Permalink to this heading">¶</a></h2>
<p>The following actions are recommended to be run periodically on systems
used to test Python packages. They could be run e.g. via post-sync
actions.</p>
<section id="pip-check">
<h3>pip check<a class="headerlink" href="#pip-check" title="Permalink to this heading">¶</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">check</span></code> (provided by <code class="docutils literal notranslate"><span class="pre">dev-python/pip</span></code>) can be used to check
installed packages for missing dependencies and version conflicts:</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ python3.10 -m pip check
meson-python 0.6.0 requires ninja, which is not installed.
cx-freeze 6.11.1 requires patchelf, which is not installed.
openapi-spec-validator 0.4.0 has requirement openapi-schema-validator<0.3.0,>=0.2.0, but you have openapi-schema-validator 0.3.0.
cx-freeze 6.11.1 has requirement setuptools<=60.10.0,>=59.0.1, but you have setuptools 62.6.0.
</pre></div>
</div>
<p>This tool checks the installed packages for a single Python
implementation only, so you need to run it for every installed
interpreter separately.</p>
<p>In some cases the issues are caused by unnecessary version pins
or upstream packages listing optional dependencies as obligatory.
The preferred fix is to fix the package metadata rather than modifying
the dependencies in ebuild.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>pip does not support the <code class="docutils literal notranslate"><span class="pre">Provides</span></code> metadata, so it can
produce false positives about <code class="docutils literal notranslate"><span class="pre">certifi</span></code> dependency. Please ignore
these:</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>httpcore 0.15.0 requires certifi, which is not installed.
httpx 0.23.0 requires certifi, which is not installed.
sphobjinv 2.2.2 requires certifi, which is not installed.
requests 2.28.0 requires certifi, which is not installed.
</pre></div>
</div>
</div>
</section>
<section id="pip-list-outdated">
<h3>pip list --outdated<a class="headerlink" href="#pip-list-outdated" title="Permalink to this heading">¶</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">list</span> <span class="pre">--outdated</span></code> (provided by <code class="docutils literal notranslate"><span class="pre">dev-python/pip</span></code>) can be used
to check whether installed packages are up-to-date. This can help
checking for pending version bumps, as well as to detect wrong versions
in installed metadata:</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ pip3.11 list --outdated
Package Version Latest Type
------------------------ ----------------- ------- -----
dirty-equals 0 0.4 wheel
filetype 1.0.10 1.0.13 wheel
mercurial 6.1.3 6.1.4 sdist
node-semver 0.8.0 0.8.1 wheel
PyQt-builder 1.12.2 1.13.0 wheel
PyQt5 5.15.6 5.15.7 wheel
PyQt5-sip 12.10.1 12.11.0 sdist
PyQtWebEngine 5.15.5 5.15.6 wheel
Routes 2.5.1.dev20220522 2.5.1 wheel
selenium 3.141.0 4.3.0 wheel
sip 6.6.1 6.6.2 wheel
sphinxcontrib-websupport 1.2.4.dev20220515 1.2.4 wheel
uri-template 0.0.0 1.2.0 wheel
watchfiles 0.0.0 0.15.0 wheel
watchgod 0.0.dev0 0.8.2 wheel
</pre></div>
</div>
<p>Again, the action applies to a single Python implementation only
and needs to be repeated for all of them.</p>
<p>Particularly note the packages with versions containing only zeroes
in the above list — this is usually a sign that the build system
does not recognize the version correctly. In some cases, the only
working solution would be to sed the correct version in.</p>
<p>The additional <code class="docutils literal notranslate"><span class="pre">dev</span></code> suffix is usually appended via <code class="docutils literal notranslate"><span class="pre">tag_build</span></code>
option in <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>. This causes the version to be considered
older than the actual release, and therefore the respective options need
to be stripped.</p>
</section>
<section id="gpy-verify-deps">
<h3>gpy-verify-deps<a class="headerlink" href="#gpy-verify-deps" title="Permalink to this heading">¶</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">gpy-verify-deps</span></code> (provided by <code class="docutils literal notranslate"><span class="pre">app-portage/gpyutils</span></code>) compares
the ebuild dependencies of all installed Python packages against their
metadata. It reports the dependencies that are potentially missing
in ebuilds, as well as dependencies potentially missing
<code class="docutils literal notranslate"><span class="pre">[${PYTHON_USEDEP}]</span></code>. For the latter, it assumes that all
dependencies listed in package metadata are used as Python modules.</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ gpy-verify-deps
[...]
=dev-python/tempest-31.0.0: missing dependency: dev-python/oslo-serialization [*]
=dev-python/tempest-31.0.0: missing dependency: dev-python/cryptography [*]
=dev-python/tempest-31.0.0: missing dependency: dev-python/stestr [*]
=dev-python/versioningit-2.0.0: missing dependency: dev-python/tomli [*]
=dev-python/versioningit-2.0.0: missing dependency: dev-python/importlib_metadata [python3.8 python3.9]
=dev-python/wstools-0.4.10-r1: missing dependency: dev-python/setuptools [*]
</pre></div>
</div>
<p>The check is done for all installed interpreters. The report indicates
whether the dependency upstream is unconditional (<code class="docutils literal notranslate"><span class="pre">[*]</span></code>) or specific
to a subset of Python implementations.</p>
<p>Similarly to <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">check</span></code> results, every dependency needs to be
verified. In many cases, upstream metadata lists optional or build-time
dependencies as runtime dependencies, and it is preferable to strip them
than to copy the mistakes into the ebuild.</p>
</section>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">Gentoo Python Guide</a></h1>
<h3>Navigation</h3>
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="preface.html">Preface</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpreter.html">Python interpreters</a></li>
<li class="toctree-l1"><a class="reference internal" href="eclass.html">Choosing between Python eclasses</a></li>
<li class="toctree-l1"><a class="reference internal" href="basic.html">Common basics</a></li>
<li class="toctree-l1"><a class="reference internal" href="any.html">python-any-r1 — build-time dependency</a></li>
<li class="toctree-l1"><a class="reference internal" href="single.html">python-single-r1 — single-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="multi.html">python-r1 — multi-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils.html">distutils-r1 — standard Python build systems</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils-legacy.html">distutils-r1 legacy concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="helper.html">Common helper functions</a></li>
<li class="toctree-l1"><a class="reference internal" href="depend.html">Advanced dependencies</a></li>
<li class="toctree-l1"><a class="reference internal" href="test.html">Resolving test suite problems</a></li>
<li class="toctree-l1"><a class="reference internal" href="pytest.html">pytest recipes</a></li>
<li class="toctree-l1"><a class="reference internal" href="concept.html">Advanced concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="expert-multi.html">Expert python-r1 usage</a></li>
<li class="toctree-l1"><a class="reference internal" href="buildsys.html">Integration with build systems written in Python</a></li>
<li class="toctree-l1"><a class="reference internal" href="porting.html">Porting tips</a></li>
<li class="toctree-l1"><a class="reference internal" href="migration.html">Migration guides</a></li>
<li class="toctree-l1"><a class="reference internal" href="qawarn.html">QA checks and warnings</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Python package maintenance</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#support-for-python-2">Support for Python 2</a></li>
<li class="toctree-l2"><a class="reference internal" href="#which-implementations-to-test-new-packages-for">Which implementations to test new packages for?</a></li>
<li class="toctree-l2"><a class="reference internal" href="#adding-new-python-implementations-to-existing-packages">Adding new Python implementations to existing packages</a></li>
<li class="toctree-l2"><a class="reference internal" href="#which-packages-can-be-co-maintained-by-the-python-project">Which packages can be (co-)maintained by the Python project?</a></li>
<li class="toctree-l2"><a class="reference internal" href="#porting-packages-to-a-new-eapi">Porting packages to a new EAPI</a></li>
<li class="toctree-l2"><a class="reference internal" href="#monitoring-new-package-versions">Monitoring new package versions</a></li>
<li class="toctree-l2"><a class="reference internal" href="#routine-checks-on-installed-python-packages">Routine checks on installed Python packages</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="interpreter-maintenance.html">Maintenance of Python implementations</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="qawarn.html" title="previous chapter">QA checks and warnings</a></li>
<li>Next: <a href="interpreter-maintenance.html" title="next chapter">Maintenance of Python implementations</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>document.getElementById('searchbox').style.display = "block"</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2020, Michał Górny, license: CC BY 4.0.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 5.2.3</a>
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
<a href="_sources/package-maintenance.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
|
GitWeb