Python Development Workflow¶
- 2017-current: Git
- PEP 512: Migrating from hg.python.org to GitHub
- We are now live on GitHub! (Feb 2017)
- 4 weeks with the new workflow: what needs changing?
- Pull request #1 (Feb 2017)
- 2011-2017: Mercurial
- PEP 385: Migrating from Subversion to Mercurial. “2011-03-05: final conversion (tentative)”
- PEP 374: Choosing a distributed VCS for the Python project
- Mapping of Subversion revision to Mercurial commit:
Misc/svnmap.txtfile in the Python code base.
- Example with Subversion revision 68121: https://hg.python.org/lookup/r68121 redirects to http://svn.python.org/view?view=revision&revision=68121 but sadly this service is down (tested in September 2020)
- 2005-2011: Subversion (SVN)
- 1990-2005: CVS
- Python migrated to Subversion to 2005, but I don’t when CVS started to be used. 1989 is the year when Guido created Python.
- Initially hosted on
cvs.python.org, it migrated to Sourceforge
- The first CVS commit was made at Aug 9, 1990. (converted as a Git commit)
Python Forge and Bug Tracker¶
- 2022-current: GitHub
- 2007-2022: Roundup with Rietveld
- CPython instance: https://github.com/psf/bpo-tracker-cpython
- Modified instance of Roundup
- Report Tracker Problem: https://github.com/python/psf-infra-meta/issues
- Migration from SourceForge to Roundup in 2007
- Status of new issue tracker (29 Oct 2006) by Brett Cannon
- Re: [Python-Dev] PEP 581: Using GitHub Issues for CPython (8 Mar 2019) by Barry Warsaw
- tracker-discuss mailing list
- Weekly “Summary of Tracker Issues” sent to python-dev since May 13, 2007. First email: Summary of Tracker Issues: “ACTIVITY SUMMARY (05/06/07 - 05/13/07)” (May 2007).
- Special links:
- Weekly summary by email: https://github.com/psf/bpo-tracker-cpython/blob/master/scripts/roundup-summary
- OLD: https://hg.python.org/tracker/roundup/
- OLD: https://hg.python.org/tracker/python-dev/
- 2000-2007: SourceForge
- May 2000: “We move all development (CVS at the time, and bug tracking) to SourceForge. This roughly coincided with PythonLabs leaving CNRI, so clearly we couldn’t continue running infra off of their systems.” (email)
- The first commit mentioning SF patch was made at June 20, 2000
- Redirection URLs like:
Rietveld was used for code reviews:
- Based on patch files
- bugs.python.org had a tool to create a review from a Mercurial repository
- Support multiple “versions” of a patch
- People who comment cannot unsubscribe: new comments are sent by emails
- User database disconnected from bugs.python.org database
- The server was not reliable and failed often
- In 2008/2009, Rietveld was an external service where to upload patches
- Example: https://codereview.appspot.com/14105
- Python-ideas: Yield-from patches on Rietveld (March 2009): “I have uploaded the patches for my yield-from implementation to Rietveld if anyone wants to take a look at them: http://codereview.appspot.com/20101/show”
- Tracker-discuss: Adding a “Rietveld this” button? (march 2009) about roundup/rietveld integration
- python-dev: Define a place for code review in Python workflow (Jul 2010, same idea proposed again),
- Python-Dev: View tracker patches with ViewVC? (Jul 2010, early discussions about the actual implementation),
- python-checkins, tracker: Import rietveld (30 Sep 2010): Rietveld integration added, with a few more commits in October.
- Python-Dev: Rietveld integration into Roundup (October 2010) by Martin von Loewis
Workflow in 2010-2018 (before GitHub, buildbots testing code once merged)¶
Buildbot was only running after changes were pushed upstream. It was common that a change broke the Windows support, and so core devs pushed an “attempt to fix Windows” commit, and then a second one, etc.
To propose a change, a contributor had to open an issue in the bug tracker (Roundup at bugs.python.org), and attach a patch file. A core developer had to
- Download the patch locally
- Apply the patch file
- Fix conflicts: when the day was older than 1 day, conflicts were very likely
Misc/NEWS was basically always in conflict, especially on merges.
Changes were first fixed in the oldest supported branch, and then forward-ported to newer branches. For example, fixed in 2.6, and ported to 2.7, 3.0 and 3.1.
When Subversion was used, a Subversion “property” (in practice, a text file tracked by Subversion) listed the revision number of all “merged” changes. For example, when a change made in the 2.6 branch was merged into the 2.7 branch, it was added to this list. It was likely that this property file was in conflict. Sadly, it was a text file made of a single line with thousands of revision numbers. Text editors are not convenient to edit such file. It was barely possible to fix a conflict in this property.
A new tool to review (comment) patches was linked to Roundup: Rietveld. It was possible to generate a patch from a fork the Mercurial repository, and then get a review page. Rietveld supported multiple revisions of the same change. Drawback: the tool was not well integrated with Roundup. For example, there was no way to unsubscribe from a review.
GitHub Python organization¶
GitHub CPython project¶
- GitHub cpython administrators:
- Brett Cannon
- Release managers (ex: Ned Deily)
- GitHub uses mention-bot: https://github.com/facebook/mention-bot
- IRC notifications: http://n.tkte.ch/ – see: https://discuss.python.org/t/replacement-for-irc-github-service/805
cherry-pick for backports¶
- Check Python CLA (service run by Mariatta Wyjaya)
- Bug reports: https://github.com/python/miss-islington/issues
- Code: https://github.com/python/miss-islington
Mariatta is the primary maintainer. The bot runs in Heroku.
The bot runs cherry-picker to backport changes in CPython.