I've finally realized just what, exactly, so gets up my nose when I see organizations advertising for “rock star” programmers. To understand you need to cast your mind back (assuming it then existed) to 1971 (when I was in my first decade as a programmer) and Gerald Weinberg had just produced The Psychology of Computer Programming, in which he concluded that over-identification with the code one produced could, in interesting ways, be counter-productive.
He observed that programmers who were reluctant to share their code tended to hold on to false views of what might be causing issues rather longer than those who were open to review and discussion. My own development as a programmer was greatly aided by this approach, and at university a couple of close friends in particular discussed every aspect of the code we were creating.
Forty years later Weinberg's “egoless” approach, in which mistakes are accepted as inevitable and reviews are performed in a collegial way, remains the sanest way to produce code. Given that computer programming is fast becoming a mainstream activity it seems perverse to deliberately select for ego when seeking programming talent, since the inevitable shortfall in humility will ultimately work to undermine the rock star's programming skills.
When I think of the best programmers I know, the foremost characteristic they share is a modesty about their own achievements which others would do well to emulate. So, can we please dispel the myth of the “rock star” programmer? The best programmers can't be rock stars. Rock star egoism will stand in the way of developing your programming skills.
"Any sufficiently advanced technology is indistinguishable from magic" -- Arthur C Clarke
Musings from the technological frontier
November 30, 2014
July 8, 2014
Is Python Under Threat from the IRS?
Readers should remember that I no longer have any official standing in the Foundation other than my long-standing membership, but this post is informed by eight years on the board, over three of them as its chair.
Nicholas Tollervey (author and performer of A Programmer Pythonical) asked an interesting question on the Python Software Foundation's membership list today. Despite his misgivings it is one that will be of interest to many Python users, so I am grateful for his permission to quote the essence of it here.
The answer to the first question is absolutely not. The PSF is a long-standing 501(c)(3) non-profit in good standing. The recently-reported issues are all to do with current applications for non-profit status. It's fair to say that the IRS is applying scrutiny to such applications, but they are, after all, responsible for making sure that applications are genuine. As long as an existing non-profit makes the necessary returns and complies with all applicable laws, and as long as it continues to honor its requirement to maintain broad public support, there is no reason why the IRS would represent any kind of threat.
Some of those rejected have been advised to apply for a different form of non-profit status, others are reconsidering their options. Good legal advice is imperative when starting any such enterprise, and that requires a specialist. There is a feeling that the IRS might make better decisions if it could be better-informed about open source. From my limited knowledge of the recent cases I'd say that it's essential to ensure that your application has a sound basis in non-profit law. The IRS does not exist to help open source devotees to build a bright new world.
The answer to the second question is that loss of non-profit status would be a blow to the Foundation, but there's no reason why even that should be fatal to Python, though I very much doubt there is serious planning for such an eventuality. Ultimately the Foundation's the bylaws include a standard winding-up clause that requires asset transfer to a similar/suitable non-profit, so assets cannot be stripped even under those circumstances, including the ability to license Python distributions.
The developers could simply migrate to a different development base. They aren't directed or controlled in any way by the Foundation, which in the light of recent decisions it turns out is probably a good model. If the PSF were directing the development of the language there might have been a real risk of it being seen as no different from a software house or vendor, and it is at that point that doubts about non-profit status can be raised.
The Foundation's mission is ... therefore
Nicholas Tollervey (author and performer of A Programmer Pythonical) asked an interesting question on the Python Software Foundation's membership list today. Despite his misgivings it is one that will be of interest to many Python users, so I am grateful for his permission to quote the essence of it here.
I've noticed rumblings on the web about the IRS denying nonprofit/charitable status to various free software based foundations/organisations/legal entities similar to the PSF. In case you've missed it, here is a high level summary.
My questions are:
1) Is the PSF likely to be affected by this change of view from the IRS?
2) If so, do we have contingency for dealing with this (basically,
what are our options)?
The answer to the first question is absolutely not. The PSF is a long-standing 501(c)(3) non-profit in good standing. The recently-reported issues are all to do with current applications for non-profit status. It's fair to say that the IRS is applying scrutiny to such applications, but they are, after all, responsible for making sure that applications are genuine. As long as an existing non-profit makes the necessary returns and complies with all applicable laws, and as long as it continues to honor its requirement to maintain broad public support, there is no reason why the IRS would represent any kind of threat.
The IRS does not exist to help open source
devotees to build a bright new world
Some of those rejected have been advised to apply for a different form of non-profit status, others are reconsidering their options. Good legal advice is imperative when starting any such enterprise, and that requires a specialist. There is a feeling that the IRS might make better decisions if it could be better-informed about open source. From my limited knowledge of the recent cases I'd say that it's essential to ensure that your application has a sound basis in non-profit law. The IRS does not exist to help open source devotees to build a bright new world.
The answer to the second question is that loss of non-profit status would be a blow to the Foundation, but there's no reason why even that should be fatal to Python, though I very much doubt there is serious planning for such an eventuality. Ultimately the Foundation's the bylaws include a standard winding-up clause that requires asset transfer to a similar/suitable non-profit, so assets cannot be stripped even under those circumstances, including the ability to license Python distributions.
The developers could simply migrate to a different development base. They aren't directed or controlled in any way by the Foundation, which in the light of recent decisions it turns out is probably a good model. If the PSF were directing the development of the language there might have been a real risk of it being seen as no different from a software house or vendor, and it is at that point that doubts about non-profit status can be raised.
The Foundation's mission is ... therefore
genuinely educational and charitable
The Foundation's mission is to promote the growth of the language (check) and the development of the international Python community (check) and its mission is therefore genuinely educational and charitable. So even were it to apply again today I suspect the application might succeed. Fortunately, that doesn't have to be put to the test.
July 2, 2014
Closures Aren't Easy
Edit: This issue was reported (http://bugs.python.org/issue21904) and closed as "not a bug"
So a fairly simple, though somewhat contorted, workaround is to avoid multiple uses of the same non-local variable in different closures.
This blog was written as an IPython Notebook. Some of the implementation details it discusses are specific to CPython, the default implementation, and may work differently in other implementations. To keep the code clean, all necessary imports are made immediately below. I habitually import
print_function because I have become used to the Python 3 way of doing things, but this particular post was made using CPython 3.3.2.
In [1]:
from __future__ import print_function
from dis import dis
import sys
print(sys.version)
The Problem
In my Intermediate Python video series I discuss decorators, and point out that classes as well as functions can be decorated. As an example of this possibility I used a class decorator that wraps each of the class's methods in a function that prints out a message before calling the original method (or so I thought). Here is the relevant code.
In [2]:
def barking(cls):
for name in cls.__dict__:
if name.startswith("__"):
continue
func = getattr(cls, name)
def woofer(*args, **kw):
print("Woof")
return func(*args, **kw)
setattr(cls, name, woofer)
return cls
The essence of the decorator is to iterate over the class's
This seemed to work fine on a class with a single method.
__dict__, ignoring the so-called "dunder" names. Strictly we should perhaps also check that the name ends as well as begins with a double underscore, but it does not affect the example. Each method is then replaced with a wrapped version of itself that prints "Woof" before calling the original method.This seemed to work fine on a class with a single method.
In [3]:
@barking
class dog_1:
def shout(self):
print("hello from dog_1")
d1 = dog_1()
d1.shout()
I then defined a further class that inherited from
dog_1 with two additional methods, and decorated that. The inheritance is irrelevant: the significant fact is that two methods will be processed in the decorator loop, but it does demonstrate that superclass attributes do not appear in the subclass's __dict__.
In [4]:
@barking
class dog_3(dog_1):
def wag(self):
print("a dog_3 is happy")
def sniff(self):
print("a dog_3 is curious")
d3 = dog_3()
d3.wag(); d3.sniff(); d3.shout()
You have probably spotted the error already: calling both the
wag() and the sniff() methods gives the same result, which is to say that both wrapped methods in fact call the same original method. Unfortunately I missed this during development and production of the video code, but it was soon picked up by an eagle-eyed viewer who reported it via O'Reilly's web site.The Explanation
When a nested function makes references to variables in the containing function (non-local references) it is known as a closure. The interpreter has to ensure that the values referred to are still available after the call to the outer function terminates and the function call namespace is destroyed - otherwise, referenced objects could become garbage if the terminating function contained the only reference to the value. It does so by generating a cell object and storing the value in an element of the function's
__closure__ attribute, which is either None if the function does not refer to non-local cells or a tuple of non-local references.
The next two code blocks each define a function. The first is a regular function without non-local references, the second is a closure with a reference to the name
x from the enclosing function's namespace. You can see the difference in the bytecode reported by the dis module, and also that they return different results when created with and called with the same arguments.
In [5]:
def f_non_closure(x):
def inner(y):
return y
return inner
non_closure = f_non_closure(240)
dis(non_closure)
print("Value =", non_closure(128))
In [6]:
def f_closure(x):
def inner(y):
return x
return inner
closure = f_closure(240)
dis(closure)
print("Value =", closure(128))
You can clearly see the difference in the code. The non-closure uses a
LOAD_FAST operation that loads a local value from element 0 of the local namespace. The closure uses LOAD_DEREF, which loads a value from the function's __closure__ attribute. So let's take a look at the __closure__ of both functions.
In [7]:
print(non_closure.__closure__)
print(closure.__closure__)
Unsurprisingly the regular function does not have a value for its
_closure__ attribute. The closure, however, does - it is a tuple of cell objects, created by the interpreter. And if we want, we can obtain the values associated with the cells.
In [8]:
print(closure.__closure__[0].cell_contents)
This should make it obvious that the cell objects in the
__closure__ tuple are there so that values from the enclosing namespace remain available after that namespace has been destroyed. Those values herefore do not become garbage when the enclosing function terminates, returning the inner function (which is a closure). And the LOAD_DEREF 0 opcode simply loads the contents of the cell. It puts the function's __closure__[0].cell_contents onto the stack to be used (in this case) as a return value (and because it's written in C, it's much faster than the Python).
Let's complicate things a little by writing a function that returns a tuple of functions. The intention is that each function returned in the tuple should add successively larger numbers to its argument
x: the first one should return x+0, the second x+1, the third x+2 and so on. You find, however, that this does not happen.
In [9]:
def multiple_closures(n):
functions = []
for i in range(n):
def inner(x):
return x+i
functions.append(inner)
print(inner, "Returned", inner(10), inner.__closure__[0])
return functions
functions = multiple_closures(3)
print([f(10) for f in functions])
The output above might at first glance seem confusing. During creation of the functions inside
multiple_closures() they appear to work perfectly well. After returning from that function, however, they all return the same result when called with the same argument. We can find out why by examining the __closure__ of each function.
In [10]:
for f in functions:
print(f.__closure__[0])
The reason that the functions all give the same result is that they all use the same cell in their
__closure__. The interpreter assumes that since all functions refer to the same local variable they can all be represented by the same cell (despite the fact that the variable had different values at the different times it was used in different functions). Precisely the same thing is happening with the decorators in Intermediate Python.
If you read the code of the
barking() decorator carefully you can see that name, func and woofer are all names local to the barking() decorator function, and that func is used inside the inner function, making it a closure. Which means that all the methods end up referring to the last method processed, apparently in this case sniff().
In [11]:
print(d3.wag.__closure__[0].cell_contents.__name__) # In Python 2, use __func__
print(d3.sniff.__closure__[0].cell_contents.__name__) # In Python 2, use __func__
It now becomes obvious that the
func references in the inner woofer() function are all using the same local variable, which is represented by the same cell each time the loop body is executed. Hence, since a cell can only have a single value, they all refer to the same method.
In [12]:
print(d3.sniff.__closure__)
print(d3.wag.__closure__)
Is This a Bug?
I suspect this question is above my pay grade. It would certainly be nice if I could get this code to work as-is, but the simple fact is that at present it won't. Whether this is a bug I am happy to leave to the developers, so I will post an issue onbugs.python.org and see what they have to say. There's also the question of whether any current code is likely to be relying on this behavior (though I rather suspect not, given its unhelpful nature) - backwards compatibility should ideally not be broken.Workaround
The issue here is that different uses of the same non-local variable from a function will always reference the same cell, and no matter what the value was at the time it was referenced the cell always contains the final value of that variable.So a fairly simple, though somewhat contorted, workaround is to avoid multiple uses of the same non-local variable in different closures.
In [13]:
def isolated_closures(n):
functions = []
for i in range(n):
def wrapper(i=n):
def inner(x):
return x+i
return inner
f = wrapper(i)
functions.append(f)
print(f, "Returned", f(10), f.__closure__[0])
return functions
functions = isolated_closures(3)
print([f(10) for f in functions])
In the code above,
inner() is still a closure, but each time it is defined the definition takes plce in a different local namespace associated with a new call to wrapper(), and so each cell is a reference to a different local (to wrapper() - nonlocal to inner()) variable, and they do not collide with each other. Redefining the barking() decorator as follows works the same trick for that.
In [14]:
def barking(cls):
for name in cls.__dict__:
if name.startswith("__"):
continue
func = getattr(cls, name)
def wrapper(func=func):
def woofer(*args, **kw):
print("Woof")
return func(*args, **kw)
return woofer
setattr(cls, name, wrapper(func))
return cls
In [15]:
@barking
class dog_3(dog_1):
def wag(self):
print("a dog_3 is happy")
def sniff(self):
print("a dog_3 is curious")
d3 = dog_3()
d3.wag(); d3.sniff(); d3.shout()
This issue was quite fascinating to debug, and has led me to a much greater understanding of exactly how closures work in Python (though I still have to investigate some details). If I get time I may take a look at the CPython source to determine whether a fix can easily be made, but I wouldn't hold your breath. At this stage I have found the workaround I needed, and so will be able to respond to the viewer who raised it in the first place.
April 21, 2014
Neat Notebook Trick
I'm still trying to digest the little I saw of PyCon. Sadly I was pretty wiped out by the three-day recording session for Intermediate Python in California and then the intensive editing work that followed to help the amazing O'Reilly team get the whole thing ready a day before PyCon officially opened.
I also made the tactical (and, as it turned out, strategic) mistake of choosing to stay at the Hyatt in Montreal. This meant a considerable walk (for a gimpy old geezer such as myself) to the conference site, when the Palais des Congres is already intimidatingly large.
So the combination of exhaustion and knee pain meant I hardly got to see any talks (not totally unheard of) but that I also got very little time in the hallway track either. Probably the most upsetting absence was missing the presentation of Raymond Hettinger's Lifetime Achievement Award. As a PSF director I instituted the Community Service Awards, but these have never really been entirely appropriate for developers. This award makes it much clearer just how significant Raymond's contributions have been.
Because of the video releases I did spend some time of the O'Reilly stand, and signed away 25 free copies of the videos. I was also collecting names and addresses to distribute free copied of the Python Pocket Reference. If you filled out a form, you should receive your book within the next three weeks. We'll mail you with a more exact delivery date shortly.
But the real reason for this post is that I had the pleasure of meeting Fernando Perez, one of the leaders of the IPython project. He was excited to hear that the Intermediate Python notebooks are already available on Github, and when he realized the notebooks were all held in the same directory he showed me that if I dropped that URL into the Notebook Viewer site I would get a web page with links to viewable versions of the notebook. [Please note: they aren't currently optimally configured for reading, so it's still best to run the notebooks interactively, but in the absence of a local notebook server this will be a lot better than nothing. It will get better over time].
He also mentioned a couple of other wrinkles I hadn't picked up on, and we briefly discussed some of the interesting aspects of Notebooks being data structures.
The conversation was interesting enough that I plan to visit Berkeley soon to try and infiltrate my way into the documentation team and see if we can't make the whole system even easier to use and understand. One way or another, open source seems to be in my bloodstream.
I also made the tactical (and, as it turned out, strategic) mistake of choosing to stay at the Hyatt in Montreal. This meant a considerable walk (for a gimpy old geezer such as myself) to the conference site, when the Palais des Congres is already intimidatingly large.
So the combination of exhaustion and knee pain meant I hardly got to see any talks (not totally unheard of) but that I also got very little time in the hallway track either. Probably the most upsetting absence was missing the presentation of Raymond Hettinger's Lifetime Achievement Award. As a PSF director I instituted the Community Service Awards, but these have never really been entirely appropriate for developers. This award makes it much clearer just how significant Raymond's contributions have been.
Because of the video releases I did spend some time of the O'Reilly stand, and signed away 25 free copies of the videos. I was also collecting names and addresses to distribute free copied of the Python Pocket Reference. If you filled out a form, you should receive your book within the next three weeks. We'll mail you with a more exact delivery date shortly.
But the real reason for this post is that I had the pleasure of meeting Fernando Perez, one of the leaders of the IPython project. He was excited to hear that the Intermediate Python notebooks are already available on Github, and when he realized the notebooks were all held in the same directory he showed me that if I dropped that URL into the Notebook Viewer site I would get a web page with links to viewable versions of the notebook. [Please note: they aren't currently optimally configured for reading, so it's still best to run the notebooks interactively, but in the absence of a local notebook server this will be a lot better than nothing. It will get better over time].
He also mentioned a couple of other wrinkles I hadn't picked up on, and we briefly discussed some of the interesting aspects of Notebooks being data structures.
The conversation was interesting enough that I plan to visit Berkeley soon to try and infiltrate my way into the documentation team and see if we can't make the whole system even easier to use and understand. One way or another, open source seems to be in my bloodstream.
April 12, 2014
Intermediate Python: An Open Source Documentation Project
There is a huge demand for Python training materials, and there are many people who just don't have the spare cash to buy books or videos. That's one reason why, in conjunction with a new Intermediate Python video series I have just recorded for O'Reilly Media I am launching a new, open source, documentation project.
My intention in recording the videos was to produce a broad set of materials (the linked O'Reilly page contains a good summary of the contents). I figure that most Python programmers will receive excellent value for money even if they already know 75% of the content. Those to whom more is new will see a correspondingly greater benefit. But I was also keenly aware that many Python learners, particularly those in less-developed economies, would find the price of the videos prohibitive.
With O'Reilly's contractual approval the code that I used in the video modules, in IPython Notebooks, is going up on Github under a Creative Commons license [EDIT: The initial repository is now available and I very much look forward to hearing from readers and potential contributors - it's perfectly OK if you just want to read the notebooks, but any comments yuu have about your experiences will be read and responded to as time is available]. Some of it already contains markdown annotations among the code, other notebooks have little or no commentary. My intention is that ultimately the content will become more comprehensive than the videos, since I am using the video scripts as a starting point.
I hope that both learner programmers and experienced hands will help me turn it into a resource that groups and individuals all over the world can use to learn more about Python with no fees required. The current repository has to be brought up to date after a rapid spate of editing during the three-day recording session. It should go without saying that viewer input will be very welcome, since the most valuable opinions and information comes from those who have actually tried to use the videos to help them learn.
I hope this will also be a project that sees contributions from documentation professionals (and beginners they can help train), so I will be asking the WriteTheDocs NA team how we can lure some of those bright minds in.
Sadly it's unlikely I will be able to see their talented array of speakers as I will still be recovering from surgery. But a small party one evening or a brunch at the office might be possible. Knowing them it will likely involve sponsorship or beer. Or both. We shall see.
I think it's a worthwhile goal to have free intermediate-level Python sample code available, and I can't think of a better way for a relative beginner to get into an open source project. I also like the idea that two communities can come together over it and learn from each other. Suffice it to say, if there are enough people with a hundred bucks* in their pocket for a six-hour video training I am happy to use part of my share in the profits to support this project to some degree.
[DISCLOSURE: The author will receive a proportion of any profit from the O'Reilly Intermediate Python video series]
* This figure was plucked from the air before publication, and is still a good guideline, though as PyCon opened (Apr 11) a special deal was available on a package of both Jessica McKellar's Introduction to Python and my Intermediate Python.
My intention in recording the videos was to produce a broad set of materials (the linked O'Reilly page contains a good summary of the contents). I figure that most Python programmers will receive excellent value for money even if they already know 75% of the content. Those to whom more is new will see a correspondingly greater benefit. But I was also keenly aware that many Python learners, particularly those in less-developed economies, would find the price of the videos prohibitive.
With O'Reilly's contractual approval the code that I used in the video modules, in IPython Notebooks, is going up on Github under a Creative Commons license [EDIT: The initial repository is now available and I very much look forward to hearing from readers and potential contributors - it's perfectly OK if you just want to read the notebooks, but any comments yuu have about your experiences will be read and responded to as time is available]. Some of it already contains markdown annotations among the code, other notebooks have little or no commentary. My intention is that ultimately the content will become more comprehensive than the videos, since I am using the video scripts as a starting point.
I hope that both learner programmers and experienced hands will help me turn it into a resource that groups and individuals all over the world can use to learn more about Python with no fees required. The current repository has to be brought up to date after a rapid spate of editing during the three-day recording session. It should go without saying that viewer input will be very welcome, since the most valuable opinions and information comes from those who have actually tried to use the videos to help them learn.
I hope this will also be a project that sees contributions from documentation professionals (and beginners they can help train), so I will be asking the WriteTheDocs NA team how we can lure some of those bright minds in.
Sadly it's unlikely I will be able to see their talented array of speakers as I will still be recovering from surgery. But a small party one evening or a brunch at the office might be possible. Knowing them it will likely involve sponsorship or beer. Or both. We shall see.
I think it's a worthwhile goal to have free intermediate-level Python sample code available, and I can't think of a better way for a relative beginner to get into an open source project. I also like the idea that two communities can come together over it and learn from each other. Suffice it to say, if there are enough people with a hundred bucks* in their pocket for a six-hour video training I am happy to use part of my share in the profits to support this project to some degree.
[DISCLOSURE: The author will receive a proportion of any profit from the O'Reilly Intermediate Python video series]
* This figure was plucked from the air before publication, and is still a good guideline, though as PyCon opened (Apr 11) a special deal was available on a package of both Jessica McKellar's Introduction to Python and my Intermediate Python.
2 comments:
Labels:
community,
contribution,
documentaion,
open source,
oreilly,
pycon,
python,
training,
writethedocs
Labels:
community,
contribution,
documentaion,
open source,
oreilly,
pycon,
python,
training,
writethedocs
A Rap @hyatt Customer Service Request
It's 2am and the wireless is down
@Hyatt ... #pycon
That's why my face is wearing a frown
Even though I'm at ... #pycon
I love all these Canadians
And Montreal is cool
But don't you know how not to run a network
fool?
If I were a rapper
Then you'd have to call me Milton
Because frankly I get much better service
@Hilton
I'm a businessman myself
And I know we're hard to please
So kindly please allow me
To put you at your ease
Your people are delightful
And as helpful as the best
I want to help, not diss you
I'm not angry like the rest
The food is amazing
And the bar could be geek heaven
If only you weren't calling
For last orders at eleven
We're virtual and sleepless
So we need your help to live
And most of us are more than glad
To pay for what you give
But imagine you're away from home
And want to call your Mom
The Internet's our family
So you've just dropped a bomb
I've had my ups and downs with Hyatt
Over many years
But never felt before
That it should fall on other's ears
I run conferences, for Pete's sake
And I want to spend my money
If only I could reach someone
And I'm NOT being funny
PyCon is my baby
So I cherish it somewhat
But this has harshed my mellow
And just not helped a lot
We're bunch of simple geeks
Who get together every year
We aren't demanding, I don't think
Our simple needs are clear
I don't believe that I could run
Your enterprise right here
It's difficult, and operations
Aren't my thing I fear
So please, don't take this badly
But you've really disappointed
Which is why a kindly soul like me
Has made remarks so pointed
We will help you if we can
We know you pay a lot for bits
But I have to know if web sites
Are receiving any hits
You've cut me off, I'm blind
And so I hope there's nothing funky
Happening to my servers
While I'm sat here getting skunky
Enough, I've made my point
So I must stop before I'm rude
The Internet's my meat and drink
You've left me without food.
trying-to-help-while-disappointed-ly yr's - steve
@Hyatt ... #pycon
That's why my face is wearing a frown
Even though I'm at ... #pycon
I love all these Canadians
And Montreal is cool
But don't you know how not to run a network
fool?
If I were a rapper
Then you'd have to call me Milton
Because frankly I get much better service
@Hilton
I'm a businessman myself
And I know we're hard to please
So kindly please allow me
To put you at your ease
Your people are delightful
And as helpful as the best
I want to help, not diss you
I'm not angry like the rest
The food is amazing
And the bar could be geek heaven
If only you weren't calling
For last orders at eleven
We're virtual and sleepless
So we need your help to live
And most of us are more than glad
To pay for what you give
But imagine you're away from home
And want to call your Mom
The Internet's our family
So you've just dropped a bomb
I've had my ups and downs with Hyatt
Over many years
But never felt before
That it should fall on other's ears
I run conferences, for Pete's sake
And I want to spend my money
If only I could reach someone
And I'm NOT being funny
PyCon is my baby
So I cherish it somewhat
But this has harshed my mellow
And just not helped a lot
We're bunch of simple geeks
Who get together every year
We aren't demanding, I don't think
Our simple needs are clear
I don't believe that I could run
Your enterprise right here
It's difficult, and operations
Aren't my thing I fear
So please, don't take this badly
But you've really disappointed
Which is why a kindly soul like me
Has made remarks so pointed
We will help you if we can
We know you pay a lot for bits
But I have to know if web sites
Are receiving any hits
You've cut me off, I'm blind
And so I hope there's nothing funky
Happening to my servers
While I'm sat here getting skunky
Enough, I've made my point
So I must stop before I'm rude
The Internet's my meat and drink
You've left me without food.
trying-to-help-while-disappointed-ly yr's - steve
March 20, 2014
Social Media and Immortality
I suppose everyone who uses social networks like Facebook eventually comes up against the situation when that network presents a dead friend in some context that only makes sense about someone who is alive and actively maintaining a social media presence. I just did, again.
In this particular case it was triggered by the suggestion from LinkedIn that I might like to add a fellow Learning Tree instructor to my roster of friends. He died, quite young and to most of his colleagues' surprise, about fifteen months ago (if my memory can be relied upon, which I wouldn't necessarily recommend as a strategy). I've seen similar reports on Twitter from other friends.
Now, I'm just a guy who chose to eschew the corporate career ladder and work on small systems that do demonstrable good, so I freely admit that the young devops turks of today are able to develop far more capable systems that I could have conceived of at their age. That's just the nature of technological progress. At the same time, I have to wonder why nobody appears to have asked the question "Should we take special actions (or at least avoid taking regular ones) for users who haven't logged in in over a year?"
Do they have no business analysts? Must we geeks be responsible for avoiding even the most predictable social gaffes?
Sidebar: I once designed the database for a system that monitored the repayment of student support funds by those who had accepted assistance from the federal government to train in teaching disadvantaged students. There were certain valid reasons for deferring repayment (such as military service), and of course these deferrals had to be recorded. I remember feeling very satisfied that all I had to do was associate the null value with the deferral duration for "Dead" as the referral reason to have everything work perfectly well.
The answer to my question of two paragraphs above, by the way, would be “yes”. This will be the last time I give free advice to the social media companies, so Twitter, Facebook, LinkedIn, and the rest, I hope you can find some benefit in this advice. Anything further will cost you dearly. (I should be so lucky).
Quite separately from the above speculations on human frailty, I can't help wondering what kind of immortality a continued existence on these platforms represents (even though this will probably lead to hate mail from all kinds of people the concept offends). I had an email from Google a couple of days ago asking me to log in with a particular identity* within a month or have the account go inactive. That's a necessary second step to whatever palliative actions you choose to take when presenting the account to others. Google, for all their execrable support,** get that you have to log in now and again just to assert your continued existence.
It strikes me this is a reasonably humane way to proceed. If you want to keep someone's memory alive on a social media platform then you must know them at least well enough to log in to their account, after which it's basically your shrine to them if you want it to be. I really don't like to think about what kind of complications the lawyers will dream up about this, though. Otherwise, well, we are after all all born to die (Ray Kurzweil notwithstanding).
*Note to the Google identity nazis: no, of course I was joking, I only have one identity
** Hint re Google customer service: if you aren't paying you aren't a customer, so expecting service might seem presumptuous
In this particular case it was triggered by the suggestion from LinkedIn that I might like to add a fellow Learning Tree instructor to my roster of friends. He died, quite young and to most of his colleagues' surprise, about fifteen months ago (if my memory can be relied upon, which I wouldn't necessarily recommend as a strategy). I've seen similar reports on Twitter from other friends.
Now, I'm just a guy who chose to eschew the corporate career ladder and work on small systems that do demonstrable good, so I freely admit that the young devops turks of today are able to develop far more capable systems that I could have conceived of at their age. That's just the nature of technological progress. At the same time, I have to wonder why nobody appears to have asked the question "Should we take special actions (or at least avoid taking regular ones) for users who haven't logged in in over a year?"
Do they have no business analysts? Must we geeks be responsible for avoiding even the most predictable social gaffes?
Sidebar: I once designed the database for a system that monitored the repayment of student support funds by those who had accepted assistance from the federal government to train in teaching disadvantaged students. There were certain valid reasons for deferring repayment (such as military service), and of course these deferrals had to be recorded. I remember feeling very satisfied that all I had to do was associate the null value with the deferral duration for "Dead" as the referral reason to have everything work perfectly well.
The answer to my question of two paragraphs above, by the way, would be “yes”. This will be the last time I give free advice to the social media companies, so Twitter, Facebook, LinkedIn, and the rest, I hope you can find some benefit in this advice. Anything further will cost you dearly. (I should be so lucky).
Quite separately from the above speculations on human frailty, I can't help wondering what kind of immortality a continued existence on these platforms represents (even though this will probably lead to hate mail from all kinds of people the concept offends). I had an email from Google a couple of days ago asking me to log in with a particular identity* within a month or have the account go inactive. That's a necessary second step to whatever palliative actions you choose to take when presenting the account to others. Google, for all their execrable support,** get that you have to log in now and again just to assert your continued existence.
It strikes me this is a reasonably humane way to proceed. If you want to keep someone's memory alive on a social media platform then you must know them at least well enough to log in to their account, after which it's basically your shrine to them if you want it to be. I really don't like to think about what kind of complications the lawyers will dream up about this, though. Otherwise, well, we are after all all born to die (Ray Kurzweil notwithstanding).
*Note to the Google identity nazis: no, of course I was joking, I only have one identity
** Hint re Google customer service: if you aren't paying you aren't a customer, so expecting service might seem presumptuous
No comments:
Labels:
afterlife,
death,
humor,
media,
networks,
null,
philosophy,
social,
speculation
Labels:
afterlife,
death,
humor,
media,
networks,
null,
philosophy,
social,
speculation
January 9, 2014
Practical Python (1)
Note: this blog post is the first I am undertaking with the IPython Notebook. I am still playing with formatting and so on, so please bear with me if the content doesn't seem as easy to read as it should. The notebook itself can be found as a gist file on Github and you can alternatively view it using the online Notebook viewer.
I want to discuss a typical bit of Python, taken from a program sent me by a colleague (whether it's his code or someone else's I don't know, and it hardly matters). It's the kind of stuff we all do every day in Python, and despite the Zen of Python's advice that “there should be one, and preferably only one, obvious way to do it” there are many choices one could make that can impact the structure of the code.
This started out as a way to make the code more readable (I suspect it may have been written by somebody more accustomed to a language like C), but I thought it might be interesting to look at some timings as well.
In order to be able to run the code without providing various dependencies I have taken the liberty of defining a dummy Button function and various other “mock” objects to allow the code to run (they implement just enough to avoid exceptions being raised)*. This in turn means we can use IPython's %%timeit cell magic to determine whether my “improvements” actually help the execution speed.
Note that each timed cell is preceded by a garbage collection to try as far as possible to run the samples on a level playing field**.
In [1]:
import gc
class MockFrame():
def grid(self, row, column, sticky):
pass
mock_frame = MockFrame()
def Button(frame, text=None, fg=None, width=None, command=None, column=None, sticky=None):
return mock_frame
class Mock():
pass
self = Mock()
self.buttonRed, self.buttonBlue, self.buttonGreen, self.buttonBlack, self.buttonOpen = (None, )*5
f4 = Mock()
f4.columnconfigure = lambda c, weight: None
ALL = Mock()
The code in this next cell is extracted from the original code to avoid repetition - all loop implementations are written to use the same data.
In [2]:
button = ["Red", "Blue", "Green", "Black", "Open"]
color = ["red", "blue", "green", "black", "black"]
commands = [self.buttonRed, self.buttonBlue, self.buttonGreen,
self.buttonBlack, self.buttonOpen]
So here's the original piece of code:
In [3]:
g = gc.collect()
In [4]:
%%timeit
# Benchmark 1, the original code
for c in range(5):
f4.columnconfigure(c, weight=1)
Button(f4, text=button[c], fg=color[c], width=5,
command=commands[c]).grid(row=0, column=c, sticky=ALL)
You might suspect, as I did, that there are better ways to perform this loop.
The most obvious is simply to create a single list to iterate over, using unpacking assignment in the for loop to assign the individual elements to local variables. This certainly renders the loop body a little more readably. We do still need the column number, so we can use the enumerate() function to provide it.
In [5]:
g = gc.collect()
In [6]:
%%timeit
for c, (btn, col, cmd) in enumerate(zip(button, color, commands)):
f4.columnconfigure(c, weight=1)
Button(f4, text=btn, fg=col, width=5, command=cmd). \
grid(row=0, column=c, sticky=ALL)
pass
Unfortunately any speed advantage appears insignificant. These timings aren't very repeatable under the conditions I have run them, so really any difference is lost in the noise - what you see depends on the results when this notebook was run (and therefore also on which computer), and it would be unwise of me to make any predictions about the conditions under which you read it.
We can avoid the use of enumerate() by maintaining a loop counter, but from an esthetic point of view this is almost as bad (some would say worse) than iterating over the range of indices. In CPython it usually just comes out ahead, but at the cost of a certain amount of Pythonicity. It therefore makes the program a little less comprehensible.
In [7]:
g = gc.collect()
In [8]:
%%timeit
c = 0
for (btn, col, cmd) in zip(button, color, commands):
f4.columnconfigure(c, weight=1)
Button(f4, text=btn, fg=col, width=5, command=cmd). \
grid(row=0, column=c, sticky=ALL)
c += 1
pass
The next two cells repeat the same timings without the loop body, and this merely emphasises the speed gain of ditching the call to enumerate(). At this level of simplicity, though, it's difficult to tell how much optimization is taking place since the loop content is effectively null. I suspect PyPy would optimize this code out of existence. Who knows what CPython is actually measuring here.
In [9]:
g = gc.collect()
In [10]:
%%timeit
for c, (btn, col, cmd) in enumerate(zip(button, color, commands)):
pass
In [11]:
g = gc.collect()
In [12]:
%%timeit
c = 0
for btn, col, cmd in zip(button, color, commands):
pass
c += 1
Somewhat irritatingly, manual maintenance of an index variable appears to have a predictable slight edge over use of enumerate(), and naive programmers might therefore rush to convert all their code to this paradigm. Before they do so, though, they should consider that code's environment. In this particular example the whole piece of code is simply setup, executed once only at the start of the program execution as a GUI is being created. Optimization at this level woud not therefore be a sensible step: to optimize you should look first at the code inside the most deeply-nested and oft-executed loops.
If the timed code were to be executed billions of times inside two levels of nesting then one might, in production, consider using such an optimization if (and hopefully only if) there were a real need to extract every last ounce of speed from the hardware. In this case, since the program uses a graphical user interface and so user delays will use orders of magnitude more time than actual computing, it would be unwise to reduce the readability of the code, for which reason I prefer the enumerate()-based solution.
With many loops the body's processing time is likely to dominate in real cases, however, and that again supportus using enumerate(). If loop overhead accounts for 5% of each iteration and you reduce your loop control time by 30% you are still only reducing your total loop run time by 1.5%. So keep your program readable and Pythonically idiomatic.
* If you have a serious need for mock objects in testing, you really should look at the mock module, part of the standard library since Python 3.3. Thanks to Michael Foord for his valiant efforts. Please help him by not using mock in production.
** An interesting issue here. Originally I wrote the above code to create a new MockFrame object for each call to Button(), and I consistently saw the result of the second test as three orders of magnitude slower than the first (i.e. ms, not µs). It took me a while to understand why timeit was running so many iterations for such a long test, adding further to the elapsed time. It turned out the second test was paying the price of collecting the garbage from the first, and that without garbage collections in between runs the GC overhead would distort the timings.
January 3, 2014
Blip.tv Deletes Python Content
There's been some disturbance in the Python ecosphere because Blip.tv has removed a lot of Python content - for a long time, Next Day Video used Blip as their preferred hosting service (I don't know whether they still do or not, but after this I should hope not) and PyCon video was hosted there by default. According to their announcement
So I wrote to them to ask if it would be possible to get copies of the content they had made unavailable:After many years of being an open platform, we’re now taking our mission to bring the best original web series to our audience more seriously.
Their reply was unequivocal:I understand that many Python-related videos are no longer available on your service.This is to ask whether you can make the original media available to us for re-hosting, since there is definite demand for some of the video content you have removed.
So if you were thinking about using Blip's services, you might want to think again. They have just deleted all this stuff, as far as I can see without giving much notice to the people who posted it in the first place. They have made no friends in the Python world as a result, and I can only imagine who else they have pissed off with their apparently heavy-handed actions.Unfortunately this content is no longer available on Blip and we are not able to provide it to you. The original content owner may re-upload their source files to another hosting provider as an alternative.
Open Source and Money
David Heinemeier Hansson, a name well-known in the Ruby on Rails world, recently wrote a blog post entitled “The perils of mixing open source and money.” In it he argues that the ability of open source contributors to raise funding through sources like Kickstarter threatens open source, though I find his arguments unpersuasive.
First he suggests that fundraising represents a one-time “cashing in on goodwill earned,” whereas I suspect that if a funded project is successful that would increase the likelihood of receiving funding for future projects and wold actually increase the goodwill directed towards the fundraiser. Second he indirectly suggests that being compensated for writing software will lead to needless embellishment, whereas I should have thought that community pressures in any decent open source developer community would lead to negative code reviews and decisions not to include needless bloat.
Hansson then goes on to suggest that working for community donations causes people to work to keep the donations coming in rather than to improve the software. The fact that many Kickstarter software projects have apparently succeeded appears to make no difference to his opinion. Sadly it seems to me that in closing he reveals that the whole piece is indeed just opinion when he says
Either I misunderstood something or what Hansson wrote doesn't make sense. The fact of the matter is that the best open source projects don't include contributions because they have been funded, they include them because they are valuable to the project. As long as these values remain in place then the injection of money into open source projects is both desirable and useful.
First he suggests that fundraising represents a one-time “cashing in on goodwill earned,” whereas I suspect that if a funded project is successful that would increase the likelihood of receiving funding for future projects and wold actually increase the goodwill directed towards the fundraiser. Second he indirectly suggests that being compensated for writing software will lead to needless embellishment, whereas I should have thought that community pressures in any decent open source developer community would lead to negative code reviews and decisions not to include needless bloat.
Hansson then goes on to suggest that working for community donations causes people to work to keep the donations coming in rather than to improve the software. The fact that many Kickstarter software projects have apparently succeeded appears to make no difference to his opinion. Sadly it seems to me that in closing he reveals that the whole piece is indeed just opinion when he says
It's against this fantastic success of social norms that we should be extraordinary careful before we let market norms corrupt the ecosystem. Like a coral reef, it's more sensitive than you think, and it's how to underestimate the beauty that's unwittingly at stake. Please tread with care.Doesn't Hansson know that many people who work in open source do so principally because their employers pay them to do so? And yet their acceptance of the corporate shilling is apparently not in danger of perverting the course of open source development, while people with good ideas that others are prepared to fund apparently don't qualify to receive support because they put the whole ecosystem in danger.
Either I misunderstood something or what Hansson wrote doesn't make sense. The fact of the matter is that the best open source projects don't include contributions because they have been funded, they include them because they are valuable to the project. As long as these values remain in place then the injection of money into open source projects is both desirable and useful.
November 19, 2013
MacOS Migration: If I Have to Shave One More Yak ...
So I decided it was time to update my laptop, the current Macbook Air being three years old and having a tendency to forget that the screen is present. The triggering event was a minor spillage which stopped several crucial keys from working as the beer dried out (yes, I know ...).
Clearly the first thing to do was take the Air off the 'Net and back it up. Connecting the backup disk and starting a Time Machine backup saw it identify about 75,000 files of the 1,500,000 on y hard disk needed backing up. When the backup phase started it took about five minutes to get to "3k of 4.5GB backed up" and I realized I had a problem.
StackOverflow suggested I consider repairing the Time Machine disk. That's another hour I shall never see again, but at least Disk Utility gave the volume a clean bill of health, so I restarted the Time Machine backup and this time (hallelujah!) it ran. I then ran the Migration Assistant on the new (receiving) Mac and told it to restore my account, settings, applications - the whole caboodle. Since the Assistant kindly estimated this would take at least two hours and fifteen minutes I decided this would be an appropriate point at which to go for a couple of drinks with my buddy Kirby.
I found the Apple Migration Assistant to be quite friendly (disclaimer: I have always previously used manual migration procedures to switch computers and so cannot say whether Windows offers similar friendliness; if not, it should). When I returned about 90 minutes later I was delighted to find that the restore was complete (I hadn't waited around for Migration Assistant to upgrade its estimate), so I was able to log in to my newly-restored account on the updated Air.
My joy lasted as long as it took me to run a terminal session, when I saw the following delight:
Not exactly what you want to see when you log in, but at least indicating the the error was probably with my virtual environments. Fortunately it was quite easy to fix this daunting error message by doing a brew install python after uninstalling the previously installed version (thereby achieving an upgrade from 2.7.2 to 2.7.5). Since my account is set to prefer /usr/local/bin to /usr/bin it finds the updated python immediately. I then upgraded pip to the latest version and I appear to be good to go (though I am sure I shall find many other bumps in the road).
A very pleasant surprise was that the Mac knew about all my printers and was happy to try and use them for me. I won't know whether they work until I get back to the network to which they are connected, but if they do I will be both surprised and delighted. This is clearly superior technology where the user experience has been considered relatively carefully.
Unfortunately the user before me had already started using brew, so quite a lot of reowning was required to make my account the owner of essential locations before I could start to access the brew package again. Once I could start to think about doing that I discovered that brew update would not work because "The following untracked working tree files would be overwritten by merge". Thanks to this very helpful web page I updated by brew installation and then the update went ahead just fine. After which a brew upgrade decided that 35 packages needed updating (gulp!) and so I am currently waiting to see if the following packages upgrade without issues:
Clearly the first thing to do was take the Air off the 'Net and back it up. Connecting the backup disk and starting a Time Machine backup saw it identify about 75,000 files of the 1,500,000 on y hard disk needed backing up. When the backup phase started it took about five minutes to get to "3k of 4.5GB backed up" and I realized I had a problem.
StackOverflow suggested I consider repairing the Time Machine disk. That's another hour I shall never see again, but at least Disk Utility gave the volume a clean bill of health, so I restarted the Time Machine backup and this time (hallelujah!) it ran. I then ran the Migration Assistant on the new (receiving) Mac and told it to restore my account, settings, applications - the whole caboodle. Since the Assistant kindly estimated this would take at least two hours and fifteen minutes I decided this would be an appropriate point at which to go for a couple of drinks with my buddy Kirby.
I found the Apple Migration Assistant to be quite friendly (disclaimer: I have always previously used manual migration procedures to switch computers and so cannot say whether Windows offers similar friendliness; if not, it should). When I returned about 90 minutes later I was delighted to find that the restore was complete (I hadn't waited around for Migration Assistant to upgrade its estimate), so I was able to log in to my newly-restored account on the updated Air.
My joy lasted as long as it took me to run a terminal session, when I saw the following delight:
Last login: Tue Nov 19 01:27:18 on tty?? Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.4/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/local/Cellar/python/2.7.4/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/Library/Python/2.7/site-packages/virtualenvwrapper/hook_loader.py", line 16, infrom stevedore import ExtensionManager File "/Library/Python/2.7/site-packages/stevedore/__init__.py", line 3, in from .extension import ExtensionManager File "/Library/Python/2.7/site-packages/stevedore/extension.py", line 4, in import pkg_resources ImportError: No module named pkg_resources virtualenvwrapper.sh: There was a problem running the initialization hooks. If Python could not import the module virtualenvwrapper.hook_loader, check that virtualenv has been installed for VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python and that PATH is set properly. AirHead:~ sholden$
Not exactly what you want to see when you log in, but at least indicating the the error was probably with my virtual environments. Fortunately it was quite easy to fix this daunting error message by doing a brew install python after uninstalling the previously installed version (thereby achieving an upgrade from 2.7.2 to 2.7.5). Since my account is set to prefer /usr/local/bin to /usr/bin it finds the updated python immediately. I then upgraded pip to the latest version and I appear to be good to go (though I am sure I shall find many other bumps in the road).
A very pleasant surprise was that the Mac knew about all my printers and was happy to try and use them for me. I won't know whether they work until I get back to the network to which they are connected, but if they do I will be both surprised and delighted. This is clearly superior technology where the user experience has been considered relatively carefully.
Unfortunately the user before me had already started using brew, so quite a lot of reowning was required to make my account the owner of essential locations before I could start to access the brew package again. Once I could start to think about doing that I discovered that brew update would not work because "The following untracked working tree files would be overwritten by merge". Thanks to this very helpful web page I updated by brew installation and then the update went ahead just fine. After which a brew upgrade decided that 35 packages needed updating (gulp!) and so I am currently waiting to see if the following packages upgrade without issues:
ack 2.10, atk 2.10.0, cairo 1.12.16, cmake 2.8.12.1, erlang R16B02, fontconfig 2.11.0, gdk-pixbuf 2.30.1, gettext 0.18.3.1, gfortran 4.8.2, git 1.8.4.3, glib 2.38.2, gmp 5.1.3, gtk+ 2.24.22, harfbuzz 0.9.24, icu4c 52.1, libmemcached 1.0.17, libxml2 2.9.1, mercurial 2.8, mplayer 1.1.1, ncdu 1.10, nmap 6.40, pango 1.36.1, pixman 0.32.2, postgresql 9.3.1, pv 1.4.6, pypy 2.2.0, python 2.7.6, python3 3.3.2, redis 2.6.16, sphinx 2.1.3, sqlite 3.8.1, xz 5.0.5, zeromq 3.2.4I'll let you know how it goes! In the meantime, though my virtual environments still need some attention I am able to run Pythons 2 and 3 and the IPython Notebook. Kudos to Apple, this migration has left me much happier than I would have imagined.
November 5, 2013
What Do You Care?
I was saddened to discover from this blog post that Nadezhda Tolokonnikova is currently (and, most likely, call me paranoid if you will, deliberately) unaccounted for in the Russian prison system. I have never met this woman, but I know she has a family, and friends, who love her and need to know she will be kept safe. I also know that she tried, with her fellow band-members, to bring to the world's attention the fact that their homeland might not be ideal. For which they are all being punished.
If you won't just bring this one tiny injustice to your world's attention, what do you expect other people to do for you when you need help?
The Russian state needs to understand that unless it chooses to give up any pretense of listening to public opinion then it must get used to the fact that it too is being watched. If you can't find time to help Nadezhda, who should find time to help you?
Of course if you'd rather just write computer programs then this is just another annoying whiney post in a long series. Whose number is, by now, doubtless legion, sorry about that. But when you need to find your significant other in the TSA miscreants' repository don't come round asking me for help.
You think this could never happen? I so hope you are right.
If you won't just bring this one tiny injustice to your world's attention, what do you expect other people to do for you when you need help?
The Russian state needs to understand that unless it chooses to give up any pretense of listening to public opinion then it must get used to the fact that it too is being watched. If you can't find time to help Nadezhda, who should find time to help you?
Of course if you'd rather just write computer programs then this is just another annoying whiney post in a long series. Whose number is, by now, doubtless legion, sorry about that. But when you need to find your significant other in the TSA miscreants' repository don't come round asking me for help.
You think this could never happen? I so hope you are right.
August 21, 2013
Bradley Manning's Post-Sentencing Statement
This statement from Bradley Manning was read after hos 35-year sentence was passed by his lawyer, David Coombs.
The decisions that I made in 2010 were made out of a concern for my country and the world that we live in. Since the tragic events of 9/11, our country has been at war. We’ve been at war with an enemy that chooses not to meet us on any traditional battlefield, and due to this fact we’ve had to alter our methods of combating the risks posed to us and our way of life.
I initially agreed with these methods and chose to volunteer to help defend my country. It was not until I was in Iraq and reading secret military reports on a daily basis that I started to question the morality of what we were doing. It was at this time I realized in our efforts to meet this risk posed to us by the enemy, we have forgotten our humanity. We consciously elected to devalue human life both in Iraq and Afghanistan. When we engaged those that we perceived were the enemy, we sometimes killed innocent civilians. Whenever we killed innocent civilians, instead of accepting responsibility for our conduct, we elected to hide behind the veil of national security and classified information in order to avoid any public accountability.
In our zeal to kill the enemy, we internally debated the definition of torture. We held individuals at Guantanamo for years without due process. We inexplicably turned a blind eye to torture and executions by the Iraqi government. And we stomached countless other acts in the name of our war on terror.
Patriotism is often the cry extolled when morally questionable acts are advocated by those in power. When these cries of patriotism drown our any logically based intentions [unclear], it is usually an American soldier that is ordered to carry out some ill-conceived mission.
Our nation has had similar dark moments for the virtues of democracy—the Trail of Tears, the Dred Scott decision, McCarthyism, the Japanese-American internment camps—to name a few. I am confident that many of our actions since 9/11 will one day be viewed in a similar light.
As the late Howard Zinn once said, “There is not a flag large enough to cover the shame of killing innocent people.”
I understand that my actions violated the law, and I regret if my actions hurt anyone or harmed the United States. It was never my intention to hurt anyone. I only wanted to help people. When I chose to disclose classified information, I did so out of a love for my country and a sense of duty to others.
If you deny my request for a pardon, I will serve my time knowing that sometimes you have to pay a heavy price to live in a free society. I will gladly pay that price if it means we could have country that is truly conceived in liberty and dedicated to the proposition that all women and men are created equal.One might wish that President Obama could put up such a principled defense for his scurrilous conduct in handing over the security of the American people to secret courts and universal surveillance.
August 4, 2013
Getting Back to IT
Life has become much more interesting of late. With two or three interns in The Open Bastion office periodically I am revisiting basic software development practices to try and explain them. I am using it as an opportunity to bring my own skill-set up to date, and it's immense fun to actually get to grips with, even though quite a challenge in the face of intelligent (and sometimes daunting) questioning from the interns. We are all learning a lot, me included.
Because we want to be able to build flexible web systems we have decided to use django CMS as our base system. The editing interface may not represent the absolute zenith of design (though the 3.0 release, currently in Beta, looks rather better), but it appears to be a solid piece of technology and it is capable (when applied by a team including design skills) of producing attractive and easily-edited web content. Even with a relatively puny SQLite database and with the Django debug toolbar switched on you can see how people with little deep understanding of information technology could nevertheless use it to build their own web resources fairly quickly.
The original attraction of django-cms was its ability to maintain flexible content via its flexibly extensible plugin system. This is a valuable part of keeping content separate from presentation—and, of course, there's nothing to stop you writing plugins to use the same data as other Django code that's already part of your site. For a crowd of tech-heads the most difficult part will be the presentation and styling. Who knows, perhaps a design intern will want to get in on the ground floor.
Our first task was to get to grips with a solid but simple workflow. We have now settled on the simple but effective git branching model proposed in Vincent Driessen's excellent paper. Of course since everyone is learning git we are making all the usual mistakes and stumbling along at times. It's reassuring that whatever mistakes you do make are usually recoverable. Nowadays I sometimes even remember to checkout a new branch before committing development changes.
We are currently at the stage of creating a solid build and learning how to work with plugins. I have managed to extend the tables plugin to allow drop-down style selection (albeit with a somewhat inflexible mechanism) and gained much insight thereby. We still face the challenges of testing and deployment
Sadly now I have to go back to writing the code to create the slots for the DjangoCon schedule, but I hope to write more later.
Because we want to be able to build flexible web systems we have decided to use django CMS as our base system. The editing interface may not represent the absolute zenith of design (though the 3.0 release, currently in Beta, looks rather better), but it appears to be a solid piece of technology and it is capable (when applied by a team including design skills) of producing attractive and easily-edited web content. Even with a relatively puny SQLite database and with the Django debug toolbar switched on you can see how people with little deep understanding of information technology could nevertheless use it to build their own web resources fairly quickly.
The original attraction of django-cms was its ability to maintain flexible content via its flexibly extensible plugin system. This is a valuable part of keeping content separate from presentation—and, of course, there's nothing to stop you writing plugins to use the same data as other Django code that's already part of your site. For a crowd of tech-heads the most difficult part will be the presentation and styling. Who knows, perhaps a design intern will want to get in on the ground floor.
Our first task was to get to grips with a solid but simple workflow. We have now settled on the simple but effective git branching model proposed in Vincent Driessen's excellent paper. Of course since everyone is learning git we are making all the usual mistakes and stumbling along at times. It's reassuring that whatever mistakes you do make are usually recoverable. Nowadays I sometimes even remember to checkout a new branch before committing development changes.
We are currently at the stage of creating a solid build and learning how to work with plugins. I have managed to extend the tables plugin to allow drop-down style selection (albeit with a somewhat inflexible mechanism) and gained much insight thereby. We still face the challenges of testing and deployment
Sadly now I have to go back to writing the code to create the slots for the DjangoCon schedule, but I hope to write more later.
July 22, 2013
OSCON Community Leadership Summit
I've just come back enthused from having spent my entire weekend working.
Of course anyone who works in open source will recognize the effect: someone gives you a chance to learn from and inform a couple of hundred like-minded individuals and there goes the weekend. Didn't even go to the PyLadies informal Saturday gathering, normally a fixed point of the weekend when I'm at home (a sadly somewhat too rare occurrence still).
OSCON is celebrating its fifteenth anniversary this year, and four years ago Jono Bacon of Canonical engineered the first Community Leadership Summit. I participated in this fifth run more fully than in previous years, and have benefited from much collective wisdom.
As is my practice at conferences I spent a lot of time in the hallway track. I acquired this habit from years of organizing PyCon, when I would frequently be waylaid on my way to some session and end up having an amazingly interesting and informative discussion, meeting new people, broadening my knowledge of Python and the open source world generally. If I could hire all the people I know in the open source world (which I couldn't, even with unlimited capital, since many of them are pursuing their own dreams) I would be able to build an unbeatable distributed information systems engineering and operations team.
A couple of interesting facts appeared as the weekend went by. First of all, perhaps fifteen to twenty-five percent of those attending had no direct connection with the open source development world. Some of these people were enthusiastic users of open source projects, and others were community managers who have started to realize that the way the open source world uses a diverse distributed open toolkit to achieve its goals could be useful to them too.
Secondly, and perhaps largely as a result of the first, the open source crowd began to realize that as the community continues to become more diverse (yay, diversity!) we have to stop talking in jargon about things like "cloning git repositories" if we don't want these brave spirits who are here to learn about the social and organizational side of open source to be completely turned off by our inability to speak plain English*.
We might also need to consult with UX teams to ensure that our current technologies can be used as the basis of a layer that makes sense to people who don't have our intimate knowledge of the technologies but are nevertheless keen to learn and use our methods. That could represent a substantial challenge.
The lightning talks were uniformly inspiring. Jono was kind enough to say he really enjoyed mine, which was called Making Small Positive Differences. The tradition at the CLS is to set the talk times to accommodate everyone who signed up. I didn't know this, so like many of the other speakers I entered the room expecting a five-minute slot.
Fortunately I was fifth on the list, so I had time to go through my slides (did I mention there was no projector? another surprise, but what the hey, this is mostly an unconference) and extract the salient points into notes for a three-minute talk in my extremely convenient (and free) O'Reilly notebook.
Some of the gems of the lightnings for me included:
This last discussion raised many interesting aspects of how people working on project teams communicate, and we realized that training was one important way that people with diverse skills can learn to work together effectively and harmoniously.
All in all this was an excellent start to OSCON, and I feel more in touch with the hot-button issues of the collected open source communities as the convention proper begins (tutorials are the main activities today and tomorrow). I'm looking forward to a great week, and hope to round it off with a reprise of last year's OSCON Survivors' Breakfast. Drop me a line if you'd like an invite.
* No holier than thou here. The best I can do at short notice is:
Of course anyone who works in open source will recognize the effect: someone gives you a chance to learn from and inform a couple of hundred like-minded individuals and there goes the weekend. Didn't even go to the PyLadies informal Saturday gathering, normally a fixed point of the weekend when I'm at home (a sadly somewhat too rare occurrence still).
OSCON is celebrating its fifteenth anniversary this year, and four years ago Jono Bacon of Canonical engineered the first Community Leadership Summit. I participated in this fifth run more fully than in previous years, and have benefited from much collective wisdom.
As is my practice at conferences I spent a lot of time in the hallway track. I acquired this habit from years of organizing PyCon, when I would frequently be waylaid on my way to some session and end up having an amazingly interesting and informative discussion, meeting new people, broadening my knowledge of Python and the open source world generally. If I could hire all the people I know in the open source world (which I couldn't, even with unlimited capital, since many of them are pursuing their own dreams) I would be able to build an unbeatable distributed information systems engineering and operations team.
A couple of interesting facts appeared as the weekend went by. First of all, perhaps fifteen to twenty-five percent of those attending had no direct connection with the open source development world. Some of these people were enthusiastic users of open source projects, and others were community managers who have started to realize that the way the open source world uses a diverse distributed open toolkit to achieve its goals could be useful to them too.
Secondly, and perhaps largely as a result of the first, the open source crowd began to realize that as the community continues to become more diverse (yay, diversity!) we have to stop talking in jargon about things like "cloning git repositories" if we don't want these brave spirits who are here to learn about the social and organizational side of open source to be completely turned off by our inability to speak plain English*.
We might also need to consult with UX teams to ensure that our current technologies can be used as the basis of a layer that makes sense to people who don't have our intimate knowledge of the technologies but are nevertheless keen to learn and use our methods. That could represent a substantial challenge.
The lightning talks were uniformly inspiring. Jono was kind enough to say he really enjoyed mine, which was called Making Small Positive Differences. The tradition at the CLS is to set the talk times to accommodate everyone who signed up. I didn't know this, so like many of the other speakers I entered the room expecting a five-minute slot.
Fortunately I was fifth on the list, so I had time to go through my slides (did I mention there was no projector? another surprise, but what the hey, this is mostly an unconference) and extract the salient points into notes for a three-minute talk in my extremely convenient (and free) O'Reilly notebook.
Some of the gems of the lightnings for me included:
- Kevin Johnson's appeal to share our technology to the benefit of all
- Britta (?)'s suggestion that barriers to entry weren't always a bad thing, with examples of how they could work in a community's favor (without discouraging diversity?)
- Josh Hibbet's talk about CityCamp, which could become a nationwide (worldwide?) series of events to promote citizen participation and government transparency
- Gribble, an IRC bot built by SourceForge to be friendly and welcoming to new IRC community members
- Selena Deckleman's invitation to go and speak in a K-12 classroom
- Aaron Wolf's promotion of the Snowdrift Co-op
- A talk by someone [whose name I need to research so I can introduce him to a colleague with a keen interest in such matters] who has developed a way (if I understood correctly) to codify the law and represent legal entities engaging in contractually-bound mutual obligations.
This last discussion raised many interesting aspects of how people working on project teams communicate, and we realized that training was one important way that people with diverse skills can learn to work together effectively and harmoniously.
All in all this was an excellent start to OSCON, and I feel more in touch with the hot-button issues of the collected open source communities as the convention proper begins (tutorials are the main activities today and tomorrow). I'm looking forward to a great week, and hope to round it off with a reprise of last year's OSCON Survivors' Breakfast. Drop me a line if you'd like an invite.
* No holier than thou here. The best I can do at short notice is:
If you just need to use project X, go to [some web page] and click the "Install" button. Then follow the instructions on that page to make sure the installation has succeeded, and the follow the [tutorial web page] or browse [web search for currently best-rated and/or most popular blog posts about Projct X]
If you are a project X user who finds it unsatisfactory in some way we would love to hear from you. Only by being keenly attentive to feedback from our dedicated users can we ensure that project X continues to become more useful to a wider range of people every time.
If you would like project X to do things it currently doesn't please let us have your ideas on [ideas@projectX.hostingsite.example.com]. We will always attempt to acknowledge input and respond to feedback.
If you would like to help project X become more capable then please ask us how you can support the open source communities who built and help to maintain it. We are always happy to welcome new project members, who can provide the one thing without which open source could not be possible: the willingness to build software to raise the tide and level the playing field [see what I just did there].
By creating new infrastructure open to all we hope to improve humanity's lot more effectively. Project X is available under the [link to OSI-approved license] and its documentation is available under the [some sort of Creative Commons license] on Read The Docs.
Subscribe to:
Posts (Atom)