June 27, 2006

on Leadership - tech teams and the RTFM factor

Leadership is not measured by titles, or by pay or by department sizes. Nor is it measured by columns of ink in a sunday rag. Quite the reverse, really -- when some journo starts talking about leadership it's generally a proxy for Company's stock price, PR budget and journalists' ignorance or lack of inspiration.

To get to today's point we have to build up a picture of structure - what it means to be part of a team. Let's talk about people that are team members, the vast majority, even though we are writing to leaders today, whatever they are.

Consider Alice, a new, junior and very inexperienced programmer. She attempts to establish territory in an area, her first area of deep knowledge. This is a natural desire as it gets a foot on the ground, from which she can start to contribute.

In a complex engineering field like Internet, we have lots of different territories: systems programmers (where I hale from), applications programmers, graphical programmers, graphical artists, ... etc etc. Generally, a programmer dives into one of these areas and tries to make it their profession - their specialty. But in doing that, they inevitably find that this is at the cost of the other areas.

I for example know lots about systems programming, but nothing about graphical programming. Indeed, it's worse than that, because within systems programming there are dozens of areas (crypto, protocols, device drivers, OS, client-server, database, accounting, languages, real-time, capabilities, ...), and I imagine the same is true of graphical programming, if I ever bothered to ask.

How then does our new programmer thrive and learn in her chosen specialty? Traditionally she does it by herself. Alice learns. Swots, reads, surfs, codes up some stuff, etc etc.

What she doesn't do is ask someone to teach her. Bob, who may be the world's expert in GUI programming, for example may be sitting next to her and have all the knowledge she needs. But, if Alice asks "how do I create a widgit?" she'll be told *RTFM* -- read the f&&king manual. The best Alice can do is talk to Dave who happens to be at her level, and in her specialty. Dave also doesn't know how to create a widgit, but at least they can enjoy their conversation about it.

The reason for this is perverse: in order for Alice to deserve to be given that help from Bob, she has to prove that she is capable of using and appreciating that information. There's no point in passing on that knowledge to someone who isn't capable of using it! But the only way for Alice to show she can acquire that knowledge is to do it -- to acquire the knowlege.

This chicken and egg situation explains why computing people are so apparently antagonistic to each other if they arrive from different levels of knowledge. By asking for something you should be able to get yourself, you are revealling yourself as incapable of figuring it out. In contrast, when you know, and someone asks you some small informational question, you are bound by the arcane unwritten rules of the league of extraordinary programmers to tell that person to RTFM. It's an unfortunate truth, an economic hard reality that self-learning is more effective that teaching in the computing world.

So we have a communication issue up and down the levels. Communication between peers at the same level is good, and I guess it has to be otherwise we'd all die of loneliness. But now couple this situation to the above - the diversity of specialties. (We are getting closer to the point now.)

When I come in with all the knowledge in some specialty (today I should be working on datagram security protocols) and talk to some other expert, Carol, in some other field (say, GUIs for email clients), I have a problem. My brain is steeped, literally soaked in RTFM juice. So when Carol asks how to do a cryptoblahblah operation, my kneejerk reaction is to tell her to RTFM. And, even if I can see that she is in the wrong specialty, and she can't possibly be expected to know this or aquire this information, *and* I manage to quell the quivering knees by tying myself in knots of politeness, I still can't help her by explaining it to her, because ... I don't know how!

As a specialist I've spent my entire life reading, learning, coding, doing great things, but all of this has been done by myself. I've never had to teach anything to anyone, and so I simply don't know how to do it. Carol of course is exactly the same, except she's more colourful at it.

So now we see how complex heterogeneous teams have a real problem that homogeneous teams do not have -- and hence a real need for leadership. The more diverse the people are, the more that they lack a common language.

Imagine it like silos -- language frequently used in the financial world. In an OS project like Linux, the teams are all quite homogoneous. All the people in the kernel team are kernel hackers, pretty much, and the Unix kernel was famously designed to be comprehensible by one talented person (the late great Professor John Lions made the point that 10k lines of code, which was what v6 kernel was, could just about fit in a person's brain).

But over in the heterogeneous world of say Firefox, we likely have lots of silos: graphics, layout, protocol, systems interfaces, plugins, crypto, certs, human languages, computer languages ... and these are all completely distinct specialties. Probably every area alone exceeds John Lions' 10k brain limit.

Now look at a leader's problem. You have a lot of people who know a lot of stuff, are convinced that they know a lot of good stuff, yet they know very little of each other's areas, and what's worse, they lack the ability or skills to communicate. Even if they wanted to, they cannot. A nightmare!

A leader in this field then needs exceptional communication skills, to make up for the lack within the team. She has to bring together two or more experts who are both desperately trying to yell as loud as possible "RTFM!" at each other. Unto the breach, she goes, and like as not, they'll both turn and attack her, because she reveals that she doesn't know enough about *either* speciality.

So, it is no small wonder that these teams are hard, and that there are very few leaders. Who'd want the job? It's the worst of all jobs: you get RTFM'd by all your team members, when they are not RTFMing each other, and because you don't get seen as a specialist, you can't even stand on your turf and defend that. It's lose-lose-lose all the way, and in a world where the star is the loudest, the brightest, the most colourful, you also end up getting the lowest pay.

It's worse than that even, in the open source world, as we haven't even got the facade of professionalism and pay to smooth over the gruesome truth. But we've got to the essential point - leadership in the heterogenous team situation is hugely about communication. Massively. And, we are not talking "good communication skills," we are talking exceptional, gifted skills. These are lion-tamer grade, capable of making hungry carnivores purr with pleasure, quelling riots with a look, redirecting warring armies into cooperative ventures.

So, you ask, how then do we acquire these skills? RTFM! The only problem is, the manual for these skills hasn't been written. I am (ahem) writing it and will no doubt get around to it some day, but as you know, I am congenitally impaired when it comes to explaining these things. I've never had to, you see.

Posted by iang at June 27, 2006 05:01 PM | TrackBack

The problem is actually that people ask the wrong question. They tend to ask, "How do I do xxxx?" when they really need to ask "Where can I learn to do xxxx?". Usually someone shouts RTFM when the answer is something that a simple google search could answer so people tend to get annoyed by people not doing basic research. I don't mind helping people learn. What I do mind is people wanting to be spoonfed.

Posted by: Wofl at June 28, 2006 03:27 PM

I've been re-reading "Agile Software Development" by Alistair Cockburn and two relevant things from this book stick out.

1. As a programmer gains experience in a domain he/she moves through three levels of understanding. The first is where the only things done are by rote, the second is where some experiments are undertaken and variations are understood, the third is where variations are almost invisible and the most appropriate variation appears without any real mental energy being expended.

2. People that work together or who come from very similar backgrounds/subcultures form their own private languages that allow them to communicate in a kind of shorthand.

I suppose that these two effects make it appear to a level three expert in systems programming that someone asking a question using the wrong terminology and seeking a level one answer is somewhat below par and simply not making sense. Level three people find it very difficult to give level one answers. It takes awareness that the different levels of comprehension exist and and then there is the problem of forcing oneself to operate at such a naive/literal level.

Hard stuff. I'm looking forward to the rest of the series.

Posted by: Michael at July 3, 2006 06:26 AM

this is a slightly different issue ... it is a lot easier to answer the question when it is a level 1 question ... when a level 1 person asks about some level 3 issue ... the person asking the question frequently doesn't have the context to understand any answer. i've periodically used the analogy of a five year old asking a question about some issue involving graduate school mathematics or physics.

there is also issue of whether they are trained professionals or somebody that has no professional experience or training in the area. a side issue is a lot of system programming being much more like a craft than a science (need long years of guild apprenticeship)

periodically when you attempt to gracefully explain to a grown person that they lack the context to understand the answer ... you get a response that claims their lack of understanding is because you aren't capable of explaining the answer (its your fault not their fault).

slightly related, in various usenet news groups there is sporadic antagonism ... not so much about the level 1 questions ... but when the questions appear to be blatent homework. in comp.arch (computer architecture) there use to be some correlation with homework questions and the start of a semester (entering freshman having their 1st exposure to online computing) ... but now it seems to be randomly distributed year around.

and of course, one of the ultimate authorities on "agile" was boyd

Posted by: Lynn Wheeler at July 3, 2006 11:16 AM

Which raises an interesting question. If there is a vast difference between level 1 and level 3 expertises, and I don't suppose that is so controversial, then why is it that in computing sciences in general, this isn't well understood and isn't well accepted? Not a day goes past when I come up against someone with a month's experience in Ruby or REST or Ajax or something equally current and hot, and find that this encourages them to assume and demand equality in knowledge. It seems like it is almost the opening gambit: "I'm hot in this month's flavour, if you're not, you already missed the boat, you're so past it, dude!"

At first blush, the notion that the young turks can put together fast hard apps with these tools is great, and does deserve the credit. But there are too few examples of success from what I've seen to make that a reliable guide. For the most part, these great tools are 99% bluster; after the hype has become boring, the hard work is still left to do.

At second blush, it might be a reflection of silver bullets. The old arts tended to solve this with a guild or professional structure approach. But that hasn't worked for computing for various reasons.

Posted by: Iang (The Market for Silver Bullets) at July 3, 2006 01:53 PM

to some extent guilds persisted over long period of time in relatively stable (stagnic) environments.

in rapidly changing and expanding domain ... a 20 year (or even five year) apprenticeship may leave you with a lot of obsolete skills.

some number of the tools, SDKs and development environments from the 90s ... especially object-oriented stuff ... where heavily laced with GUI capabilities that allowed toy demos to be turned out quickly and cheaply ... but rarely had any significant support for hard-core industrial strength data processing (and there frequently was a large gap between the toy demos and real deployable operations ... other than using the toy demos for entertainment situations akin to games).

one example was the taligent stuff ... we had done a one week JAD (with dozen people from taligent) look and came up with a 30 percent hit to existing taligent base plus 30 percent new code ... in order to add any appreciable support for industrial strength data processing.

finally, in a rapidly changing and expanding domain ... the demand for "experts" can far exceed the supply ... and so many have to make do with what they can get.

this can lead to chicken and egg situation ... if the demand for experts far exceeds the supply, vendors will frequently try to adapt their products to the skill level that is available ... trying to make do w/o requiring real experts.

Posted by: Lynn Wheeler at July 3, 2006 03:34 PM
Post a comment

Remember personal info?

Hit preview to see your comment as it would be displayed.