Recently Martin updated the Package Index to allow users to leave comments, and it appears that this innovation has been contentious. As a result of the rumblings Martin created a poll to determine whether the feature should continue in its present form or be modified in various ways. Here are the results as at the time of this writing:
| Allow ratings and comments on all packages (status quo) | 223 | ||||
| Allow package owners to disallow comments (ratings unmodified) | 137 | ||||
| Allow comments, but only send them to package owners (ratings unmodified) | 33 | ||||
| Disallow comments (ratings unmodified) | 24 | ||||
| Disallow ratings and comments (status three months ago) | 88 |
This is all very well, but unfortunately it appears that PyPi is now boxed into a corner. Even if the most popular option is implemented (retaining the current situation, where the newly-added rating and comment feature is allowed on all packages) this guarantees that a majority of those voting will have their favored option rejected. I suppose this demonstrates that you can give people too many choices.
My own discomfort with PyPi goes rather deeper. While I think that it's great that we have a central repository to support setuptools (even though release 0.6 is now three years into its release cycle and onto its eleventh release candidate) and now distribute, I would like to see it become much more usable than it currently is. It would be easy to see this as an attack on the implementers and maintainers (which it is not intended to be: the maintainers of all the software I have mentioned have done valuable work that I could not). Honestly, it isn't.
In reality I think it would be good if they had more help. Particularly the kind of help that let them package the facilities this excellent tool PyPi provides, in a much more obvious way. Even if this means complaining about the way things currently are.
Almost as a side note, I ended up following a twisty little maze of (web) passages all alike which finally led me to the Python issue tracker. Since it showed an apparently remembered login name and password I assumed all I had to do was click the Login button and all would be well, but apparently not. So I did what any user would do, and followed the Lost your login? link.
Alas, neither my email address nor my user name was recognized on that page, so I decided the only thing I could do was to register an account (even though I know I have submitted bugs in the past). So I went through the registration process only to be presented with the following unhelpful message:
OK, so what the heck is all that about? How am I now supposed to proceed if by chance I have burning information about a bug in the Python system? I have said before, and I will say again, that as an interface between Python's users and its developers the issue tracker sucks. I am sure it is very useful for the developers, but as an input collection mechanism it seems to be only slightly less valuable than a customer service desk staffed only by a notice reading "go away" (I am exaggerating for dramatic effect here).
[Edit: this was apparently a bug, which has now been fixed at least for the Python bug tracker.]
I suspect that what is needed here is the e-mail equivalent of a help desk, where people with no knowledge of the infrastructure can exchange messages with a team of real human beings who know what the score is and can make any necessary inputs to the issue tracking system on their behalf. Call them user proxies, if you like. I am aware that in the high-tech world of open source this may be seen as a heresy, but people still have their uses, dammit, and they clamor to be useful even as the capitalist world declares them redundant by the hundreds of thousands.
Now I know before I post it that some people are not going to like this article: they will either say that I shouldn't be complaining if I'm not prepared to fix what I'm complaining about, or that I should not be making a noise about something that third parties will use as evidence that the Python world is in some sort of disarray. Frankly I don't buy either of those arguments.
The Python world has recently gone through a long-drawn-out and extremely energetic discussion about increasing the diversity of the community. As an existing community we are fighting an uphill battle, because it's even more difficult to change the constituency of existing communities than it is to recruit a diverse mix to new ones. Just the same this has had some very positive results, not least the publication of a diversity statement that I think the Python Software Foundation has every right to be proud of -- it might seem like a simple piece of text, but it was a hard-won development that even cost us the resignation of a member.
Yet despite all that work, we apparently haven't yet got to the stage where the Python community includes people who can look at a less-than-optimal interface and say out loud "we need to do something about this". I don't know if this is because we are too close to PyPi to be able to acknowledge its faults, or because people fear hostile responses if they make negative comments about the infrastructure, or (perhaps most likely) because they don't want to offend those who have invested their time and effort into producing something, at least, which is more than most of us do. I do know that it frustrates the hell out of me.
So let me put a public stake in the ground here. I have visited a number of local Python user groups in the past year (the first PSF chairman to do so, as far as I can tell). Almost everyone I have spoken to along the above lines has been enthusiastic about making things better, and willing to volunteer to help make the necessary improvements. So now I need to hear from the broader Python community about what's "wrong" (less pejoratively: what we should change to make things better). This means you.
The Python Software Foundation is currently looking quite critically at next year's activities and the budget to support those activities. If we are going to make a real difference to the perception of Python and to its adoption as a serious IT solution to a broad range of problems then we need broad involvement from the whole community, not just the PSF membership. I am investigating a number of ways in which the PSF could encourage a broader involvement, and it would be helpful at this point if there were general evidence of a desire by non-members to get more involved in Python: not just its development, but its community.
If this piece isn't enough to get a decent discussion going then I suppose I should just resign as PSF chairman and look elsewhere for a community that gives a damn. I honestly don't think that will be what I need to do. I'm really hoping you guys don't let me down here.
25 comments:
"If this piece isn't enough to get a decent discussion going then I suppose I should just resign as PSF chairman and look elsewhere for a community that gives a damn."
Whooooaaa! Steve. Please don't.
FWIW, I did vote and while doing so discovered that a few of my favorite packages (Polygon among them) had been ported to Python 3.x.
I'm not high powered enough a programmer to have any kind of opinion on this. My only thought on voting yes for comments was that it might (unintentional irony) generate some worthwhile discussion on the respective packages along with the inevitable spam.
Sorry it's been such a headache. 2009 has been a bit of a bear for the weary active Python community member.
Hang in there.
For the record, the URL you gave (http://bugs.python.org/setuptools/) seems to be the setuptools issue tracker. The Python issue tracker itself is at http://bugs.python.org/. There is some customization so I don't know if your experience would have been the same.
As for the discussion, while I agree it would be a good thing to have one I don't have a lot to say myself. PyPI is a critical piece of infrastructure but I don't know enough about how people use it to give any useful advice. Perhaps other people are in the same situation as me, which would explain why there hasn't been much of a discussion.
I agree with the general idea of bringing more people to contribute to Python (either through code or other forms of contribution). Contributing to a programming language (especially a major, elegant one) should be quite attractive for many people and I sometimes wonder why we don't have more volunteers. Ideally there should be so many candidates that we'd have to refuse most of them :-)
I don't think change will happen within the current code base, or structure. I think alternatives must be made, competition must be had and only then will we end up with a great, flexible and user friendly system.
Some people have already started working on something "else". There's plenty of PyPi clones out there to back that "else".
I agree with you vehemently on all accounts.
then I suppose I should just resign as PSF chairman
(panicked) WE'LL SAY ANYTHING YOU WANT JUST DON'T TALK THAT WAY.
As for assitance with the bug process, I wonder if help@python.org would be willing to add that to its mission? Of course, help@python.org has an ongoing problem that the people who would most benefit from it don't know it exists. But, for bug report purposes, a sizable "For help:" message at the head of the issue tracker would help.
As for help@python.org's general publicity problem, maybe a mention of help@python.org should be worked into the Python interactive help... or the Python interpreter should include a "hopeless flailing detection" routine that brings up help@python.org...
Hey, wait, there's something I don't understand...
The standard open source ethos is, "if you don't like something, fix it" -- not "if you don't like something, complain about it." For example, I refuse to bother Martin with my wacky ideas about (for example) including an advogato-based trust metric in PyPI because I simply don't have time to do it myself. (Note, a trust metric could be a solution to the commenting problem, IMO.)
So I think you're swimming against the stream here. Everybody can (and did, AFAICR) have an opinion on the diversity statement, and there were relatively few ways to discern what were "good" opinions and what weren't. But the community generally doesn't take so kindly to people expressing opinions without effort on software issues - and rightly so. Talk is cheap; work (support, programming, etc.) is hard.
So I don't think we need more people saying out loud that we need to do something -- I think we need more people *doing something*, and perhaps a better path to get people involved.
@Jesse, more alternatives to PyPI are going to fragment the community just like easy_install. Not necessarily pro or con, but something to think about.
Python actually has way better infrastructure than most open source projects... for instance git doesn't even *have* a bug tracker.
That said, you are right to criticize. Many people in the open source community seem to be allergic to criticism of anything related to open source, but this is not rational. For problems to be solved they must be recognized...
Developers always make the ultimate call about what needs to be fixed, but if they are receptive to complaints and engaged with the non-developer community, they will be able to make that call better and in a way that will bring in more users.
Yes, please, don't resign!!
@Catherine, I like the idea:
"I see this is the third time you've typed F*** YOU at the command prompt. Would you like to be connected to the Python help line?"
@titus I don't buy the fragmentation argument right now. The reason I don't is simple - we have a monoculture right now. We have zero choices - I'll happily point you to the setuptools debacle as evidence as to why this is a bad thing. Without competition, or a reason to improve, or even community driven alternatives, there's no reason to buck the status quo.
About polls and surveys: there has definitely been a tradition of "framing" polls, especially around Python packaging, so that the results are shaped to favour particular outcomes. This may not be deliberate, but the original "what do you use for packaging?" poll mostly neglected the far-from-marginal use-case that system packaging happens to address. I need distutils to apply its (or Python's) knowledge of extension modules and package layouts, not to replicate all sorts of other things in a suboptimal fashion. Sadly, my impression is that opinions such as my own are now being filtered out - it's like .deb and .rpm files don't exist and we're not supposed to talk about them.
(Another case of "framing" a debate happens around the Python language and implementations. People would rather talk about language evolution in terms of Ruby, C#, Haskell, Erlang and Google's new language than look at Python's closest relatives. Again, those Python-like languages just don't exist.)
I'm not saying that the recent Package Index poll was inappropriately "framed", but sometimes it might be better to just ask people and get qualitative responses, rather than adding up some preferences and trying to guess why people are upset. I only update my software's entries on the Package Index out of habit, just to let people know that my code sees attention periodically, and I'm unlikely to use it as a serious distribution or feedback mechanism in the foreseeable future, but it's useful as a catalogue (or index!) of works. Where's my voting option in that poll?
About volunteering: I know how Martin probably feels about writing code and being told that it isn't a good thing because I've seen this behaviour before myself. The Python community seems to have a higher density of people who will gladly criticise volunteer efforts but won't do anything themselves apart from push some agenda or other. My best community-related decision of 2009 was to volunteer for a bunch of people who seem grateful for the stuff I've done, whereas in the Python scene I can only imagine the same stuff being met with "Why don't you junk all that rubbish and just use DjylonGears like us?"
My opinion here may seem paranoid or maybe petty, but doesn't it explain why it's so hard to get things done under the python.org banner? That people decide that everything at python.org has to be gold-plated and mandatory round-the-table discussions are required to get consensus and to dampen down the chorus of ridicule about what someone may have suggested?
Steve, don't take the status quo personally: at least you're not pretending that everything is flawless. Even the cloud of my own cynicism doesn't obscure some potential solutions to some of these crises. At least you're not telling us that "Steve knows best" and leaving it at that.
@Carl: You don't need to be a high-powered programmer to have an opinion. I suppose my major point was that we should be listening more carefully to the broader user community.
@Antoine: It looks as though the issue is that Firefox assumes that the password is the same for both trackers. I recently saw a similar problem report about inability to create an account on the Python tracker, though. And why don't the sites make it clear in their headings what they are tracking issues for? That's poor UI design.
I believe we have to ask ourselves why we aren't snowed under with volunteer developers. It's a serious concern for the future.
@Jesse: This isn't just about PyPi, but I do regard the PyPi situation as symptomatic of a broader malaise. It would be a shame to leave a trail of half-finished solutions along the wayside on our path to some nebulous ideal.
@Catherine: If the help@python.org team would be prepared to take it on that would be great, but the real problem is the one you identify - the channels are hidden from the majority of users. It could certainly use some more publicity.
@Titus: this isn't a programming problem, it's a cultural problem, so I am trying to fix it. It's a bad mistake to assume that the solution to every problem is more technology - I could spend weeks rewriting PyPi and come up with something that's worse than the current version.
I agree whole-heartedly about the better path to get involved - and I share your concern about fragmenting the development efforts. With a larger developer pool this might be less of a concern, though.
@Brendan: Thanks for the support. I too think that criticism and recognition of the need to improve is a positive thing. I also agree that the Python team has indeed put a terrific infrastructure together, but this isn't just about technology.
@Jesse: The problems seem to stem more from dependence on a single developer than on a single code base. Competition is good when there's no shortage of resources, but where developers are concerned that isn't the position right now.
@Paul: Your opinion seems neither paranoid or petty. I'm not sure who "the people" making these decisions about python.org are, but I do know that more people should be involved. There seems to be very little enthusiasm for innovation among the present maintainers - this could be because they are at full stretch already, though I don't see the results in the web site if that is the case.
You may be a cynic, but you have great Python credentials. I don't claim to know best, and need help from a broad cross-section of the Python community to try and improve things.
@steve - frankly, it's both. You and I both know it's difficult to get changes into the main site, and sub-sites such as PyPi are no exception. It's part social, and part technical.
When I say "build a competitor" - I mean both socially, and technically. I would rather have 10 failed starts spawn 1 good one, then to sit around trying to get consensus on what changes are the best, trying not to offend anyone, etc.
Having 30 volunteers offer to fix something, or change something fundamental doesn't solve the fact that the people on the other side may not be listening, or can simply say "no".
@Jesse: Well we certainly agree on the goal of improving Python's web presence. The thing that worries me is the perception that there are sides. We should all be working for Python's benefit here, in the knowledge that greater popularity for Python represents more opportunity for us. I don't expect people to be totally altruistic.
I agree that the consensus-building approach isn't working as well as I'd like. Maybe we need a bit more autocracy for a while - then at least responsibility for problems would be more readily identifiable!
@Steve, my solution is always more technology, dontcha know?
But seriously, what I don't understand is this: what is holding people back from contributing? There's a lot of goodwill in the Python community among the non-contributors, and I anecdotally hear about these legions who want to contribute. I see opportunities for personal contribution all the time, and I don't -- but that's because I don't have the time to do anything. Do I/we need to publicize these opportunities? What's holding them all back?
I really don't think the problem is a lack of things to fix, or a lack of direction -- i.e. complaints aren't going to help, much. We need more people actually doing something, anything. And we need the organization to take advantage of those people, when and if they show up.
I feel like I'm rambling.
I appreciate posts like this Steve, so keep them up.
The perception is that there are not volunteers. From where I sit there are plenty of people wanting to get their hands dirty, but the "old boy's club" is making them feel very unwelcome. The communication seems to be, "Help us as long as you filter everything through us." Part of volunteering is also being able to have some sort of participation in the decision process. That doesn't seem to be allowed. So you might want to start there.
No one from the PyPi community has ever said, "Jump in and help us make it better." Instead it's more like, "Send us code and we'll decide if we want to let it in or not." That will not get you far.
Instead of doing surveys around comments and such, just fix the f'n search capability. You know basic stuff. Show you're really interested in making it useful.
@Titus: I am sure the problem isn't a lack of things to fix. I am perplexed by the common remarks (see Empty's comments immediately after yours) that contributors are made to feel unwelcome.
I have spent some effort in the last year publicizing various opportunities, with mixed results. We could certainly use more developers, more writers, more testers, more webmasters ... the list goes on and on. But are people aware of these needs? Do they realize that they are suitable candidates?
You hit the nail on the head: we aren't properly organized to make new contributors feel welcome, and we don't offer them the support they need to reach productivity. I'd like to work out how to change that. Quite apart from anything else it could help our efforts to increase diversity.
@Empty: I think that is an important contribution. Am I a part of the "old boys' club"? Is this a real phenomenon, or are people waiting to be asked in the expectation that they will be slapped down if they just roll their sleeves up and start getting their hands dirty?
I'm not really sure that there is an identifiable "PyPi community", so there may not be much of a decision process either, though it sounds as though you have been rebuffed in the past. As far as I know the PyPi code is stored in a Subversion repository. I can easily look into what's required to get access to that repository if you'd like.
Maybe the obstacles are mostly imaginary. If not, once we have identified them we will be a least a part of the way towards removing them.
First, Steve, please don't quit. I appreciate everything you (and the other PSF board members) do.
To be honest, as a small, insignificant member of the community, I'd have no idea where to even begin helping out in the greater Python world. A lot of comments here make it sound like it's obvious. It's not. In five years of heavy Python use, Catherine's comment above is the first I've ever heard of "help@python.org" and I'd still have to go Google/Bing/Yahoo it just to know what it's for.
Beyond that, my perception from the trenches (if you will) is that there are indeed "sides" in the Python community. I've been privy to a couple of them, so I refuse to be told they don't exist. I consider myself a Python programmer, but the stigmata I've been given in the past is that I'm just a "Django guy" and was summarily dismissed as a result.
There are a couple things that jump out at me from reading through the comments. One: the "high-up" members of the community are tired and overworked, dedicating what they can to the language they enjoy using. Two: there's a massive Python user-base that aren't involved in day-to-day improvements. Three: everyone seems to agree there are things we need to improve on, even if we can't 100% agree with what they are. If it were up to me, I would say the PSF (or whoever it should be) should spend some time (hopefully paid) focusing on how to get more people involved.
As far as improving X (where X is PyPi/the bug tracker/any other web-based organization tool for Python's benefit), I don't personally see anything going anywhere. This is an area I'd be glad to step up in and yet might as well not bother, given the glorious infighting we do about the various web frameworks/libraries.
I hate to sound negative, because I love Python and I'm really not negative about the Python community at large. But there are portions of our community that are so entrenched, I don't feel like there's anything I could ever hope to accomplish there.
Steve, you might look to the Ubuntu project for inspiration. They organize various events to get the community involved such as open week, bug jams, hug days, ... as it helps potential volunteers where and how to contribute.
https://wiki.ubuntu.com/UbuntuOpenWeekhttps://wiki.ubuntu.com/UbuntuGlobalJam
https://wiki.ubuntu.com/UbuntuBugDay
This book might interest you as well:
http://www.jonobacon.org/2009/09/18/the-art-of-community-available-for-free-download/
http://www.artofcommunityonline.org/downloads/jonobacon-theartofcommunity-1ed.pdf
@Daniel: Thanks for writing. As individuals we are all small, insignificant members of the community, but the community is simply the aggregation of the individual members. Without each member the community would be less than it is.
I'm really sorry to hear about your experience as "just a Django guy". It seems to me that the same thing happens to the "Zope guys" and the "Plone guys". These technologies should be poster-children for the Python language, and yet I hear members of their communities don't feel like they belong in the Python world. I'd like to see much stronger bonds.
As PSF chairman it is easy to feel tired and overworked; and yet the most rewarding experiences I have had in the last year have been the meetings of local users groups that I have attended. Bruce Eckel described very well at EuroPython the positive aspects of being among Python users, and it's those aspects I'd like to encourage.
In so far as I am a part of the PSF "management" you can regard this as a focus on how to get more people involved. I think it's sufficiently important to take the risk of sounding negative to get to the root of this problem. If Python users feel excluded from the community then we need to do something about it. People need to feel welcome to be able to contribute effectively.
@Stani: Thanks very much. This demonstrates very well the benefits of a timed release cycle. I hope that the Python community will be able to put something similar together.
There have been discussions about having some sort of "community leadership summit" at PyCon, to get the leaders of local user groups together and share information of use to them all. Maybe the summit would be a suitable venue to discuss organizing the first "Python Community Day" or whatever it should be called.
As a programmer I hope someday I can do something to give back to the community.
Maybe a lot of people has willingness to help, but how?
I think there should be an easier entrance for people to feel part of the Python community.
Greatfully,
An insignificant Python user (that's not good at english)
A small cultural problem (in open source, and not at all specific to python) that I think would not be that difficult to fix is:
"if you don't like something, fix it" which @Titus mentioned.
What does this convey to the average user? Something like:
1. I am not going to fix your problem
2. I am not going to give you any pointers on how to fix it yourself
3. Go away
Open source projects are often staffed by volunteers and they certainly aren't *obligated* to do any work on behalf of users, but there are more diplomatic ways to say this.
Maybe: "Thanks for your feedback, but I'm not sure when I'll have time to look at this. If you do, the related source code is in this directory."
If you want people to get involved, then the tone of the community makes a lot of difference. Being helpful, even if that doesn't mean doing work for other people, goes a long way towards that.
To be fair, when I've interacted with the Python community I've actually had a good experience. I've even had small problems I reported almost immediately fixed. However, I've seen other projects where questions asked or problems reported on mailing lists routinely come back with RTFM, or "fix it yourself" in quite hostile sounding terms.
I have only been coding (in any form) for a couple of years and have found the python community very trustworthy, knowledgeable and safe (if a little set in their ways). I am not particularly young though, so I like the solidness.
The pypi stuff is very superficial to outsiders, especially as there are much much better ways of doing it i.e as Titus said a voting system for comments.
Making pypi an api would do and let the web community make it (I am sure if you asked the django guys they would let you use their voice/pony).
Documentation should have voted comments too which would sort out the documentation as user contributed stuff always ends up better.
PHP does this and its the best thing about the language. They managed to do it well as they are a web community. Python should let their web community do the same for web related issues. I don't use django personally but its a very undervalued resource and I am sure any other framework would jump on the chance to have their name on python home stuff.
The community focus needs to be more based around, "making cool stuff with python", then "what shall we do to improve python". The Ruby community has more this attitude which does it favors. There is a moratorium on the language now too, so it seems there is not much a bystander can do. Issues with packaging aside, I think people think that python is stable and does what it says. There are worries about concurrency and speed but the community does seem to be working at that.
In conclusion, let the web people control the web and let them show off (let the ponies fly).
I think "kindly" hit the nail on the head. Our current Web infrastructure seems handled by people like me (core developers) while it should be handled by people who are enthusiastic about Web stuff, and proactive in suggesting new functionalities/design/whatever.
I also hope you'll keep us updated about what you're planning to do to improve things, Steve :-) I think this is a very important issue (and I'd be glad to show moral support if needed!).
It's a flawed survey to be sure but given that the last four options all include giving the package owner the right to disallow comments that option really got 137+33+24+88=282 votes and seems to be the winning one.
Actually, why not just allow package owners to configure commenting and ratings so they can enable, disable, or make private either one independently?
You can't make everyone happy but that essentially provides all the options in the survey and some that weren't in the survey.
I realize it's probably easier said than done and have a lot of respect for the people that actually write the code.
Post a Comment