Cogito, Ergo Sumana for 2021 July

# 01 Jul 2021, 12:58PM: Software Bill of Materials & the US Federal Government: In February, the United States's President Biden signed an executive order on the US's supply chains; he followed this up with an EO in May specifically concentrating on improving cybersecurity. To quote Tidelift's summary, "in essence, this order is a striking attempt to create a new global standard for cybersecurity that all organizations around the world will need to ensure their software supply chain meets or exceeds in the near future."

Because of the EO, the US's National Telecommunications and Information Administration requested comment "on the minimum elements for a Software Bill of Materials (SBOM), and what other factors should be considered in the request, production, distribution, and consumption of SBOMs". I heard about this thanks to Jacob Kaplan-Moss, who suggested the Python Software Foundation could share its open source perspective. So I led an effort to submit a comment on behalf of the Packaging Working Group of the PSF - thanks to Morgan Mayo (PSF Director of Resource Development) for some of the prose!

The Packaging WG comment, along with comments from 80+ other individuals and groups, are now up at the NTIA website. Check out Section II ("Background context about the Python packaging toolchain and ecosystem") for a simplified yet still confusing diagram. Section IV of our comment ("Infrastructure funding") is pretty short; for a longer treatment on a related topic, see Tidelift's comment "Software bills of materials are important—but they won't work at scale if we don't pay the maintainers".

Filed under: Open Source and Free Culture Programming:Python Transparency in Government Code

# (1) 02 Jul 2021, 12:56PM: A Comedy Memory: I'm having trouble getting started on things I ought to do today, so here's a story I think I've never told here before.

In early 2011, I travelled to San Francisco for work, and one evening I went to an open-mic comedy night at BrainWash. I signed up for a slot -- maybe I got slot #4 or 5. And the first or second person to go up was just bad. He was not just misogynist or unfunny, but both -- it was as though he had forgotten that you will actually need punchlines to make people laugh and that demeaning women and fantasizing about drinking alcohol is insufficient. For instance: "I went to Wondercon and I saw a woman who said that on Saturday she dressed up as Batwoman and on Sunday she dressed up as Dark Phoenix, and I said, 'all anyone wants is [raunchy desire elided]'."

He finished his gross set and slouched out the door, and then several minutes later I went up, and said:

I'm in San Francisco because I just got a job with the nonprofit behind Wikipedia. So I've decided that this is the open mic that anyone can edit, and I'm going to edit a routine you heard earlier tonight.

And I then repeated the earlier guy's setups, but then added actual punchlines. For example, my punchline about the cosplayer was feigned shock at mixing DC and Marvel, and I turned a setup about soy milk that had ended with "you could make a White Russian out of that" into a joke about white vegans.

My audience loved it. I loved it. I told folks about it the next day and one of my colleagues said that I should just make that my M.O., going to open mics and improving upon an act that had come before. I said that I did not want to get beat up. But also I think it's a bit rare that someone else's comedic incompetence and my talent line up such that I can so immediately dunk on someone by spontaneously riffing on their work, like Mozart does to Salieri in Amadeus.

Filed under: Comedy Memoir

# 03 Jul 2021, 04:45PM: Researching The Leadership Gap for Legacy Projects: I've given a lot of conference talks recently. As part of the PyCon US Maintainers' Summit in May, I delivered an eight-minute talk, "Researching the leadership gap for legacy projects". The video is now available, and here's the written version (I did not use slides).

Contents:

Introducing the question/problem
A little more about the problem I see
Tooling hypothesis
Corporate involvement hypothesis (hypotheses)
New problems hypothesis
Values and culture hypothesis
How would we check?

Hi! I'm Sumana Harihareswara of Changeset Consulting and I'm not using any slides today, so you're free to take a few minutes to stretch, fold some laundry, what have you.

Introducing the question/problem

We have a pipeline for getting folks with coding skills to start or contribute to open source projects. That's great and I'm glad.

But I'm pretty sure we don't have a pipeline to recruit or grow contributors with leadership and management skills. A maintainer of a widely-used project *is* a manager, but since skilled managers are scarce in open source, we're seeing important projects stumble, or even wither, for lack of managerial work. (At least, this is what I've seen, and if you are seeing something to the contrary, I want to hear about it.) And I think this is a factor in the open source sustainability problem.

Knowing why this is happening can help us fix it. So in this talk I'll share a few hypotheses: one about the tooling we build and use, one about the effects of corporate involvement, one about changes in the problems we're trying to address, and one about values and culture. And I'll talk about how we would check whether any of these are true. I hope that considering this question will aid your efforts in focusing time and energy on things that will make a difference to project sustainability.

A little more about the problem I see

Founders start projects but are unprepared for the managerial demands of maintaining things that other people depend on. And, when legacy projects stagnate, contributors don't know how to take them over and stabilize them by solving common strategic, team, communication, workflow, and financial problems. Since they don't know how to rehab existing projects, individuals and orgs fork or start new ones, exacerbating fragmentation headaches.

So what hypotheses do we have?

Tooling hypothesis

Here's one. It used to be that, if you were going to run an open source project, you had to make the tooling platform yourself. You had to set up and administer, and maybe even build, your own bug tracker, source code repository, wiki, documentation site, and tarball release repository. Now these are hosted services -- GitHub, Read the Docs, PyPI. That means that it's easier to start a project, but that also means it's easier to start a project without learning a lot about the value and capabilities of these platforms along the way. And then project founders are less equipped to use those things well.

Corporate involvement hypothesis (hypotheses)

Here's another: it changes things when companies get more and more involved in open source, or hire people from open source. Maybe big companies hire the people who have managerial skills, and sometimes take them out of open source work entirely, and leave behind the ones who don't. Or maybe open source would be failing worse without corporate involvement, and corporate engagement is sort of subsidizing the poor management that we would otherwise see.

New problems hypothesis

Here's another: what got us here won't get us there. We're trying to address new or unsolved or undersolved problems and areas in open source -- distributed and cloud computing, getting telemetry while protecting user privacy, user experience design that can stand up to the best that industry can offer, diversity, equity, and inclusion, and so on. So, in general, open source contributors have grown a certain median level of leadership skill, but we're seeing cracks in the infrastructure because of the new loads they are trying to handle.

Values and culture hypothesis

Here's another: Let's talk about our culture and values, and how that affects contributor retention and promotion. We in open source recognize and promote people for the code they write, but we're bad at recognizing and valuing people for the managerial contributions they make -- we treat glue work https://noidea.dog/glue as invisible. So we don't attract or retain people with managerial skills or with the interest in growing in that direction.

And I'd like to thank Amye Scavarda for some thoughts about some of these hypotheses.

How would we check?

So. How would we check whether any of these are right? A few thoughts. We could talk with the scholars who were funded by the Ford and Sloan Foundation grants on critical digital infrastructure to get their thoughts. We could work with the CHAOSS people, the open source metrics working group, to construct a means to quantitatively measure maintainership actions happening within a project, and find out what other attributes it correlates with -- like whether or not those particular projects, or the domain areas they're in, were particularly influenced by companies. We could look for conversations from 15 years ago to check what our forebearers were saying, whether they thought this was likely to be a problem. We could try to offer explicit skills learning opportunities to contributors, to see whether people are interested, and whether we can find ways to retain the people who take those offers as open source maintainers.

My gut says that the source of the problem is a mix of the corporate involvement and values and culture hypotheses. So that's the basis I'm working from. I am working on a book outlining and teaching the skills open source software maintainers need, and teaching these skills to contributors who have never managed public-facing projects before. It'll be a textbook or a self-help guide for new and current maintainers of existing projects (what I call "brownfield projects", as opposed to "greenfield") and will focus on teaching specific project management skills (such as initial project audit, grantwriting, bug triage, and meeting facilitation) in the context of open source. If you go to changeset.nyc, under "resources" you'll find a free sample.

But let's talk during the Q&A session on Thursday. Which, if any, of these hypotheses ring true to you, and how could someone check whether you're right?

Thanks.

Filed under: Conferences and Performances Deliverables Management and Leadership Open Source and Free Culture Open Source and Free Culture:Advice:Maintainership book Work:Management and Leadership

# 03 Jul 2021, 05:32PM: Some Novel Python Packaging/Distribution/Inspection/Installation Projects:

People who program in Python have an easier time hearing about package-related tools that have been around for a while and that are under the banner of the Python Packaging Authority, or that are commercially supported (this simplified diagram showcases a lot of them). And if you're looking for canonical guidance on what tools to use, check out packaging.python.org and tell your colleagues.

But -- since open source and open standards make things interoperable -- people also develop new tools for their specific needs in packaging, distribution, inspection, and installation, and sometimes I come across them when people announce them. I haven't tried any of these yet but here's a list of some stuff I noticed from the last few years.

Pypitoken, "A library for generating and manipulating PyPI tokens"

Thoth, "an enhanced server-side resolution offered to the Python community" (related: thoth-solver: "A tool for aggregating Python package metadata" and Dependency Monkey which "can compute all the possible combinations of packages that can occur in a resolved software stack and verify the given stack works well")

installer, "a low-level library for installing wheel distributions"

Dotlock "is a package management tool similar to pipenv, but with a different philosophy: instead of acting as a wrapper around pip, dotlock handles package resolution natively."

simpleindex provides "a lightweight PEP-503 private index/proxy" that declares routing rules to serve files from local directories. Also see pywharf.

Mach-nix "allows one to package Python projects and environments with Nix, requiring minimal knowledge of Nix.... Why would you want to use this tool? Reproducible builds with all build and run-time dependencies provided by the same package manager, regardless of whether they're Python dependencies or not."

ipwhl: a downstream repository in which "Each repo release will ensure a single version for a project for each platform, and one can use it to replace PyPI for both build and runtime dependencies for reproducibility." Per the repo for "interplanetary wheels (or floating cheeses)": "platform-unique, singly-versioned Python binary distributions backed by IPFS for security and reproducibility."

Python devirtualizer: "a preliminary implementation which manages shared packages so that only one copy of each package version is required."

pip-deepfreeze: "a small tool that aims at managing the dependencies of a Python application in a virtual environment."

And one more thing that is a PyPA project: the Python Advisory DB. After public discussion, there's a new community-owned repository of security advisories for packages published on https://pypi.org.

Filed under: Programming:Python