The week after I posted my blog on integer division I was asked for help on a problem involving linearising a sensor output. Part of the answer turned out to involve multiplying a function of the output value by some weird factor like 211/255. Well, given that I'd just written exactly the code needed to handle this sort of strangeness, this turned out to take approximately ten minutes of my day. Result.
Of course this approach only occurred to me because I'd so recently been playing with integer division code, and so made the obvious connection. Maybe given time, if I hadn't been handed the answer by the universe immediately beforehand, I would have come up with an equally valid alternative approach. But for the life of me, I can't think of a neater solution to this particular problem.
A second example of the power of autodidacticism (not as dirty as it sounds) came about the following week when I was looking at how to add new events to a client-server messaging system. Having a global enumeration of all possible events works fine, but means that you have to rebuild every darned application every time you add a new enumeration. Well, I'm something of a compulsive purchaser of programming books, and one evening I was idly flicking through the excellent Game Coding Complete when I came upon an extremely neat solution. You calculate a hash of a descriptive string, and use that as the event enumeration. For instance, if you have an event to set the gain of an amplifier, you calculate the enumeration as hash( "set_amplifier_gain" ). Store the string and enumeration in an object that gets passed around the system. Point your debugger at an event, and you see a clear text description of what it is. There's a bit of housekeeping to ensure that two strings don't accidentally generate the same hash value, but the approach is lightweight, and lets you arbitrarily add new events while only rebuilding applications that actually need to know about them.
So: two closely-spaced incidents. In one, I have a neat solution, and almost immediately stumble upon a matching problem. In the other, I have a trivial-but-irritating problem, and almost immediately stumble upon a neat answer. Both cases involved looking outside the scope of my everyday activities to add some tools to my grab-bag of solutions.
It's what Stephen Covey refers to as sharpening the saw, which is a great analogy, and resonates conveniently with the tools metaphor - we all have a bunch of tools at our disposal, and it behooves us to care for them. It also pays dividends to get new tools, upgrade old ones, or simply window-shop to check out what's available.
As the always-readable Rands wrote, "We see the world how we want. A carpenter sees all problems as a nail. I see problems as finite state machines." Well, amen to that, my brother. (Hint: a state machine may not always be the best answer, but it's almost always an answer.)
So: not exactly a profound revelation, just a gentle slap upside the head from fate, reminding me that it's always worth spending time learning new techniques, following interesting blogs and articles, and keeping a weather eye on seemingly unrelated spheres of interest. Even if the number of times you extract a true nugget is small, it's not vanishingly so, and the return on investment can be enormous.
Thursday, 12 November 2009
Sunday, 21 June 2009
Division of Integers by Constants
Nigel Jones recently wrote a blog entry that very neatly summarises an article called 'Reciprocal Multiplication, a tutorial', by Douglas Jones.
The (Douglas) Jones article explains how to use reciprocal multiplication to perform division - for instance, if you want to divide by 10, this is the same as multiplying by 1/10. However, by doing everything in fixed point arithmetic, you can avoid the computational overhead of invoking your compiler's division routine. The result is smaller, faster code that has exactly the same accuracy as the compiler-supplied result. Anyone writing PC code may not appreciate this, but in the embedded world this sort of approach is vitally important in terms of both code size and run time.
The original article is quite long and detailed, and despite having stumbled upon it a couple of times in the past, I'd never got round to actually reading it. And so (Nigel) Jones's article came as a very welcome abstract, boiling it down as he does to a couple of simple algorithms - if you want to divide by 'x', here's what to do.
At the end of his post, (Nigel) Jones said 'If someone has too much time on their hands and would care to write a program to compute the values for all integer divisors, then I'd be happy to post the results for everyone to use.' Well, I don't know about having too much time, but I do enjoy a lunchtime programming exercise, so I sent him a text file containing the coefficients to perform integer division for all 16-bit unsigned numbers from 3 to 32768. I also asked if he would mind if I posted the source code here, in case it was of use to anyone.
He responded with an excellent suggestion - how about generating a header file containing macros with the appropriate coefficients for all divisors? Anyone wanting to use these algorithms could then simply include the header file, and call the relevant macro to perform the required division.
I've followed this suggestion, so please follow the links to check out:
The (Douglas) Jones article explains how to use reciprocal multiplication to perform division - for instance, if you want to divide by 10, this is the same as multiplying by 1/10. However, by doing everything in fixed point arithmetic, you can avoid the computational overhead of invoking your compiler's division routine. The result is smaller, faster code that has exactly the same accuracy as the compiler-supplied result. Anyone writing PC code may not appreciate this, but in the embedded world this sort of approach is vitally important in terms of both code size and run time.
The original article is quite long and detailed, and despite having stumbled upon it a couple of times in the past, I'd never got round to actually reading it. And so (Nigel) Jones's article came as a very welcome abstract, boiling it down as he does to a couple of simple algorithms - if you want to divide by 'x', here's what to do.
At the end of his post, (Nigel) Jones said 'If someone has too much time on their hands and would care to write a program to compute the values for all integer divisors, then I'd be happy to post the results for everyone to use.' Well, I don't know about having too much time, but I do enjoy a lunchtime programming exercise, so I sent him a text file containing the coefficients to perform integer division for all 16-bit unsigned numbers from 3 to 32768. I also asked if he would mind if I posted the source code here, in case it was of use to anyone.
He responded with an excellent suggestion - how about generating a header file containing macros with the appropriate coefficients for all divisors? Anyone wanting to use these algorithms could then simply include the header file, and call the relevant macro to perform the required division.
I've followed this suggestion, so please follow the links to check out:
- a header file containing the coefficients for unsigned 16-bit division
- a header file containing the coefficients for unsigned 8-bit division
- a command-line program generating the coefficients for unsigned 16-bit division [source][exe]
- a command-line program generating the coefficients for unsigned 8-bit division [source][exe]
Monday, 1 June 2009
Profound Advice: Learn a Scripting Language
This is another entry in an intermittent series in which I arrogantly pontificate on what constitutes profound advice in the world of embedded software. Actually, I'm unnecessarily narrowing my horizons here, I feel quite sure I could be arrogant in a much larger sphere.
So, here we go again. This time I would urge neophytes to learn a scripting language.
These may not be directly useful in terms of running on a little 8-bitter, but they are unbelievably useful in a range of other applications.
The advantage of scripting languages is that they're... well, scripting languages. They act like super batch files letting you leverage the power of other programs to get a job done. They tend to be interpreted rather than compiled, and so are (a) really fast to write, since they sidestep the edit-compile-link-run paradigm, and (b) relatively slow to run, since they've sidestepped the edit-compile-link-run paradigm. But with the power of modern desktops, performance really isn't a huge concern with the applications I'm interested in.
At a more detailed level, they also tend to support dynamic typing, which lets you blithely ignore whether the variable you've just created is a string, an integer, or whatever, until you start using it in a context that defines what it has to be. This is both a good and a bad thing, as what the interpreter thinks it is doesn't always tally with what you think it is.
Yes, you could create an executable that would run much faster, as it's not interpreted. But for sheer speed of get-in-there-and-hack-out-something-that-works, you can't beat a scripting language.
But before I go blindly hacking away into the undergrowth of what a language is or isn't, let's try to steer the conversation back on track.
The scripting applications I find myself revisiting time and again are (1) analyzing log files, and (2) generating code.
Dumping data to disk on a PC hooked up to an embedded system is part and parcel of the daily life of a firmware engineer. But when the system's been running for several days in an environmental chamber, or for months in a remote data acquisition application, these log files can be enormous.
The chances are you're only interested in a tiny fraction of the logged data, or you're looking for a particular event, or you want to change its format slightly, or... something else. Maybe you need to break it up into chunks for analysis. Maybe you only want to see every line in which the fourth CSV field is greater than 10. Maybe you want to reformat the data for input to another program. For whatever reason, you've got a bunch of data on a disk, and you want to pull the data needle out of the multi-megabyte haystack.
The script to do this will typically boil down to a few lines containing regular expressions. Pipe the data into the script, and out to a text file, and you're done.
Scripting languages are also a natural for code generation. Write a script to generate big chunks of your firmware, which are then compiled as normal. Any type of code containing lots of variations on a theme is a natural for this approach - good candidates are communications handlers and state machines. Give the script a list of the UART commands to be handled, or the FSM states, and let it build all of the scaffolding code in a loop over the list contents. You follow along afterwards and fill in the blanks detailing, for instance, how to handle the parameters of a specific command.
You could of course make your script sophisticated enough to accept details like the command parameters, and then generate the handling code too. But there's always the danger of spending too much time developing the script instead of doing actual useful work - as always, your mileage will vary, you need to be pragmatic, and you need to make realistic trade-offs.
My weapon of choice as a scripting language is Perl. The reason for this is mainly historical, as I've been using it for years now. I can't even remember why I first picked it up, but would guess that I wanted to adapt something that almost-but-not-quite did what I wanted, and it happened to be in Perl.
My relationship with Perl is somewhat ambivalent. I find it an enormous sprawling mess of a language, both immensely powerful and willfully obtuse. In fairness I don't spend enough time with it to make it really sing, and tend to rely on the Camel, the Cookbook, and my collection of previous scripts to bash together what I need at any given time.
The Perl apostles trumpet its approach of There's More Than One Way To Do It (TMTOWTDI) as a strength, but I find it muddies the waters; ask three Perl coders to write some code, and they'll do it in at least six different ways.
However, there's no denying the power of the language. Most scripts boil down to relatively few lines of actual code, and are amazingly compact compared to the functionally equivalent C/C++ code.
There are of course scads of scripting languages; Ruby and Python seem to be very much à la mode. Just pick one, preferably one that somebody near your desk can help out with when you get stuck, and dive in.
So, here we go again. This time I would urge neophytes to learn a scripting language.
These may not be directly useful in terms of running on a little 8-bitter, but they are unbelievably useful in a range of other applications.
The advantage of scripting languages is that they're... well, scripting languages. They act like super batch files letting you leverage the power of other programs to get a job done. They tend to be interpreted rather than compiled, and so are (a) really fast to write, since they sidestep the edit-compile-link-run paradigm, and (b) relatively slow to run, since they've sidestepped the edit-compile-link-run paradigm. But with the power of modern desktops, performance really isn't a huge concern with the applications I'm interested in.
At a more detailed level, they also tend to support dynamic typing, which lets you blithely ignore whether the variable you've just created is a string, an integer, or whatever, until you start using it in a context that defines what it has to be. This is both a good and a bad thing, as what the interpreter thinks it is doesn't always tally with what you think it is.
Yes, you could create an executable that would run much faster, as it's not interpreted. But for sheer speed of get-in-there-and-hack-out-something-that-works, you can't beat a scripting language.
But before I go blindly hacking away into the undergrowth of what a language is or isn't, let's try to steer the conversation back on track.
The scripting applications I find myself revisiting time and again are (1) analyzing log files, and (2) generating code.
Dumping data to disk on a PC hooked up to an embedded system is part and parcel of the daily life of a firmware engineer. But when the system's been running for several days in an environmental chamber, or for months in a remote data acquisition application, these log files can be enormous.
The chances are you're only interested in a tiny fraction of the logged data, or you're looking for a particular event, or you want to change its format slightly, or... something else. Maybe you need to break it up into chunks for analysis. Maybe you only want to see every line in which the fourth CSV field is greater than 10. Maybe you want to reformat the data for input to another program. For whatever reason, you've got a bunch of data on a disk, and you want to pull the data needle out of the multi-megabyte haystack.
The script to do this will typically boil down to a few lines containing regular expressions. Pipe the data into the script, and out to a text file, and you're done.
Scripting languages are also a natural for code generation. Write a script to generate big chunks of your firmware, which are then compiled as normal. Any type of code containing lots of variations on a theme is a natural for this approach - good candidates are communications handlers and state machines. Give the script a list of the UART commands to be handled, or the FSM states, and let it build all of the scaffolding code in a loop over the list contents. You follow along afterwards and fill in the blanks detailing, for instance, how to handle the parameters of a specific command.
You could of course make your script sophisticated enough to accept details like the command parameters, and then generate the handling code too. But there's always the danger of spending too much time developing the script instead of doing actual useful work - as always, your mileage will vary, you need to be pragmatic, and you need to make realistic trade-offs.
My weapon of choice as a scripting language is Perl. The reason for this is mainly historical, as I've been using it for years now. I can't even remember why I first picked it up, but would guess that I wanted to adapt something that almost-but-not-quite did what I wanted, and it happened to be in Perl.
My relationship with Perl is somewhat ambivalent. I find it an enormous sprawling mess of a language, both immensely powerful and willfully obtuse. In fairness I don't spend enough time with it to make it really sing, and tend to rely on the Camel, the Cookbook, and my collection of previous scripts to bash together what I need at any given time.
The Perl apostles trumpet its approach of There's More Than One Way To Do It (TMTOWTDI) as a strength, but I find it muddies the waters; ask three Perl coders to write some code, and they'll do it in at least six different ways.
However, there's no denying the power of the language. Most scripts boil down to relatively few lines of actual code, and are amazingly compact compared to the functionally equivalent C/C++ code.
There are of course scads of scripting languages; Ruby and Python seem to be very much à la mode. Just pick one, preferably one that somebody near your desk can help out with when you get stuck, and dive in.
Tuesday, 12 May 2009
Profound Advice: Do It Right the Third Time
This is one of series of posts on the subject of profound advice, in which I metaphorically travel back in time to meet my younger self, and give that sad git some hard-earned pearls of wisdom. Which, knowing me, I will completely ignore. Or did ignore. Or something.
This week marks the 40th anniversary of the release of 'The Italian Job'. This most quintessentially British of all crime capers stars Michael Caine as Charlie Croker, modish wide boy with an eye for the main chance.
Noel Coward is magisterial as Mr. Bridger, the uber-royalist crime lord running the underworld from his prison cell.
Even Benny Hill is watchable as the lascivious computer expert, with a penchant for the larger lady, charged with bringing Turin to a standstill by overriding the traffic control system.
And of course it gave the world the essential Michael Caine quote, the one guaranteed to be invoked by generations of impressionists.
The 2003 remake with Mark Wahlberg was a perfectly acceptable crime movie, but in the UK at least was damned by association with the original - how dare anybody remake such a classic movie? My wife wouldn't even let me bring it into the house until last year.
It's something of a truism in engineering that the best time to design your solution to a given problem is just after the project's finished. At this point you've met the pitfalls and pratfalls awaiting you, you've worked around them, and you're fully aware of the implications of your early decisions. Knowing all this, and with the benefit of 20-20 hindsight, it would be relatively trivial to re-do the project.
A depressingly common approach to design is the big bang approach, AKA BUFD (big up-front design). Given a problem, you attempt to construct an all-singing, all-dancing solution that will stand forever as a testament to your genius. You try to build a framework that will solve not only your immediate problem, but a whole class of similar problems that may or may not arise in the future. Which is a fine and worthy goal, except that it's almost certainly a colossal waste of time and effort. It's certainly not shipping code. You don't know the future - if you did you would be betting on horses, not cranking code. So admit your temporal limitations and code for what you need right now, not what you may need in some conjectural future.
My first attempt at solving a problem tends to involve a lot of hardwired assumptions - e.g., the demo code only works if the microcontroller is at a specific address on the I2C bus, has a fixed memory map, and is running a specific version of the firmware.
Which works fine a surprising amount of the time. If I have to revisit the code - for instance, the firmware gets updated - I'll add a bit more intelligence to the code to handle this new situation.
Finally, if I have to revisit the code yet again, I'll make more of an effort to do things 'right'. At this point I've already solved the basic problem twice, so I've got a pretty good handle on how to do this. I know what works and what doesn't. I can devote some effort to developing a more generic solution with the full knowledge of what functionality is likely to actually be needed - because it was actually needed. I can develop a more modular code layout that lends itself to future maintenance work. If I'd tried to do this from the get-go, I would needlessly have locked myself into what I thought was the correct architecture on day one. And it would have taken a lot longer to build the generic-but-wrong solution than it took to build my hardwired-but-quick-to-get-going solution.
As usual I thought I had independently invented here a whole new concept that would revolutionise the world, a concept I termed 'do it right the third time'. I also toyed with the phrase 'speculative complexity', which pleases me, as it captures something of the unnecessary intricacy many people seem to find so attractive.
However, it turns out that lots of other people have had exactly the same idea; the extreme-ists call it 'you ain't gonna need it', which is admittedly much cooler.
Don't go for a big bang when you're only supposed to blow the bloody doors off.
This week marks the 40th anniversary of the release of 'The Italian Job'. This most quintessentially British of all crime capers stars Michael Caine as Charlie Croker, modish wide boy with an eye for the main chance.
Noel Coward is magisterial as Mr. Bridger, the uber-royalist crime lord running the underworld from his prison cell.
Even Benny Hill is watchable as the lascivious computer expert, with a penchant for the larger lady, charged with bringing Turin to a standstill by overriding the traffic control system.
And of course it gave the world the essential Michael Caine quote, the one guaranteed to be invoked by generations of impressionists.
The 2003 remake with Mark Wahlberg was a perfectly acceptable crime movie, but in the UK at least was damned by association with the original - how dare anybody remake such a classic movie? My wife wouldn't even let me bring it into the house until last year.
It's something of a truism in engineering that the best time to design your solution to a given problem is just after the project's finished. At this point you've met the pitfalls and pratfalls awaiting you, you've worked around them, and you're fully aware of the implications of your early decisions. Knowing all this, and with the benefit of 20-20 hindsight, it would be relatively trivial to re-do the project.
A depressingly common approach to design is the big bang approach, AKA BUFD (big up-front design). Given a problem, you attempt to construct an all-singing, all-dancing solution that will stand forever as a testament to your genius. You try to build a framework that will solve not only your immediate problem, but a whole class of similar problems that may or may not arise in the future. Which is a fine and worthy goal, except that it's almost certainly a colossal waste of time and effort. It's certainly not shipping code. You don't know the future - if you did you would be betting on horses, not cranking code. So admit your temporal limitations and code for what you need right now, not what you may need in some conjectural future.
My first attempt at solving a problem tends to involve a lot of hardwired assumptions - e.g., the demo code only works if the microcontroller is at a specific address on the I2C bus, has a fixed memory map, and is running a specific version of the firmware.
Which works fine a surprising amount of the time. If I have to revisit the code - for instance, the firmware gets updated - I'll add a bit more intelligence to the code to handle this new situation.
Finally, if I have to revisit the code yet again, I'll make more of an effort to do things 'right'. At this point I've already solved the basic problem twice, so I've got a pretty good handle on how to do this. I know what works and what doesn't. I can devote some effort to developing a more generic solution with the full knowledge of what functionality is likely to actually be needed - because it was actually needed. I can develop a more modular code layout that lends itself to future maintenance work. If I'd tried to do this from the get-go, I would needlessly have locked myself into what I thought was the correct architecture on day one. And it would have taken a lot longer to build the generic-but-wrong solution than it took to build my hardwired-but-quick-to-get-going solution.
As usual I thought I had independently invented here a whole new concept that would revolutionise the world, a concept I termed 'do it right the third time'. I also toyed with the phrase 'speculative complexity', which pleases me, as it captures something of the unnecessary intricacy many people seem to find so attractive.
However, it turns out that lots of other people have had exactly the same idea; the extreme-ists call it 'you ain't gonna need it', which is admittedly much cooler.
Don't go for a big bang when you're only supposed to blow the bloody doors off.
Thursday, 7 May 2009
The Tom (not Peters) Guide to Man Management
Last weekend was a bank holiday here in the UK, and so, as is our wont, on Friday evening we hopped in the car and headed off to the New Forest to go camping. We're ideally positioned for this, close enough to get out into the heart of the forest in 30 or 40 minutes (as long as we can dodge the bank holiday traffic). It means we can do a full day at work, but be pitched up with the dinner on and the first beer (always the best) in hand by 8 or 9 o'clock.
One of our favourite haunts is a site called Tom's Field. This is a field, owned by a bloke called Tom - the clue's in the name. It's low key, the facilities are basic but clean and well-maintained, and Tom and the loose confederacy who seem to run the place are friendly and helpful. Plus, of course, it's in the New Forest, with easy access to loads of walking, cycling, and pub lunches.
We had an excellent weekend, and by Monday lunchtime were packing up ready to go home and start preparing for the week ahead. Along with everybody else on-site.
Now, Tom owns a thing called a Gator. It's a little quad bike/tractor thingy that he uses for moving stuff around the site. The sort of thing you see farmers and groundsmen using.
On the Monday morning Tom went round the site telling all of the kids that if they put all the rubbish in the bins provided by the gate, they could have a ride on the back of the Gator.
This is genius. Sheer genius.
With this simple act Tom unleashed a veritable Tsunami of wombles who swept across the site like locusts, picking it clean of every last scrap of litter. In return for ten minutes of taking kids for joyrides, Tom saved himself several hours of tedious work.
I'm not entirely sure how to link this little story back into coding, the nominal topic of this blog. Somebody with more perspicacity and insight than me could doubtless create a compelling analogy to empowering the inherent synergy in a cross-functional team to achieve a win-win scenario. To using the carrot rather than the stick. To envisaging the desired outcome of a project and setting in motion the train of events necessary to manifest that version of reality.
Me, I'm just impressed that Tom gets his field cleaned for him every weekend. Tom, I salute you.
One of our favourite haunts is a site called Tom's Field. This is a field, owned by a bloke called Tom - the clue's in the name. It's low key, the facilities are basic but clean and well-maintained, and Tom and the loose confederacy who seem to run the place are friendly and helpful. Plus, of course, it's in the New Forest, with easy access to loads of walking, cycling, and pub lunches.
We had an excellent weekend, and by Monday lunchtime were packing up ready to go home and start preparing for the week ahead. Along with everybody else on-site.
Now, Tom owns a thing called a Gator. It's a little quad bike/tractor thingy that he uses for moving stuff around the site. The sort of thing you see farmers and groundsmen using.
On the Monday morning Tom went round the site telling all of the kids that if they put all the rubbish in the bins provided by the gate, they could have a ride on the back of the Gator.
This is genius. Sheer genius.
With this simple act Tom unleashed a veritable Tsunami of wombles who swept across the site like locusts, picking it clean of every last scrap of litter. In return for ten minutes of taking kids for joyrides, Tom saved himself several hours of tedious work.
I'm not entirely sure how to link this little story back into coding, the nominal topic of this blog. Somebody with more perspicacity and insight than me could doubtless create a compelling analogy to empowering the inherent synergy in a cross-functional team to achieve a win-win scenario. To using the carrot rather than the stick. To envisaging the desired outcome of a project and setting in motion the train of events necessary to manifest that version of reality.
Me, I'm just impressed that Tom gets his field cleaned for him every weekend. Tom, I salute you.
Sunday, 26 April 2009
The Most Profound Advice You Will Ever Receive
Strangely enough, I received the most profound piece of advice I have ever heard in a pub out in the wilds of Wales, while on a stag do. We'd spent the day quad biking, shooting, and engaged in other equally testosterone-fuelled activities. We'd had our curries, and were sinking a few beers before the stripper-that-the-groom-didn't-know-about-but-fully-expected was due to arrive.
I got talking to one of the guys, and the talk turned to babies. My wife was pregnant, and I expressed some concerns about my ability to deal with the immense responsibility of caring for and raising a whole new human being.
"Don't worry about it." came the booze-fuelled reply from the father of three. "You already know everything you need to know. If they're hungry, feed them. If they're cold, warm them up. If they're too hot, cool them down. You'll be fine."
At that point an approving roar announced the arrival of a comely lass clad in a nurse's uniform, and we all lost interest in the conversation.
But the guy was absolutely right - common sense will take you a very long way as a new parent. And that it why I have no hesitation in awarding this particular piece of advice the most profound I have ever received.
---
Last year I was chatting to a new father-to-be, and he asked me for advice on dealing with a new baby, as he was freaking out about the responsibility etc etc as mentioned above. I furrowed my brow, drew myself up to my full height, pretended to be thinking thoughts of the utmost profundity, and passed on verbatim the advice given to me.
He didn't believe me, of course, but I hope that in the fullness of time he'll appreciate my astuteness.
This got me thinking about what advice I would give a newbie engineer, in the hypothetical situation where such a person were to ask me for advice on programming. I say 'hypothetical' because (a) nobody's ever asked me for advice, and (b) as a newbie, I was so self-confident and arrogant that I rarely asked for advice. I fully expect the new generation(s) of engineers to be exactly the same. Indeed, I found that the people who tried to give me advice were almost without exception fine instances of a combination of arrogance and incompetence. My favourite piece of expert advice is still that function prototypes are a waste of time, so don't bother with them. Absolutely priceless, you can't make this sort of thing up.
Instead I just wrote a whole bunch of dreadful code, slowly and painfully worked out what worked and what didn't, and over the years developed an approach that now serves me well.
So, if asked for advice, here is what I would say. These will form the basis of a series of blog articles over the next few weeks/months, depending on how often I find the time to sit down with my netbook and a few glasses of wine.
1. Use version control.
2. Use unit testing.
3. Do it right the third time.
4. A quick wrong decision is often better than a slow right decision.
5. When in doubt, use brute force.
6. Fail fast and hard.
7. Learn a scripting language.
8. Use static analysis.
I got talking to one of the guys, and the talk turned to babies. My wife was pregnant, and I expressed some concerns about my ability to deal with the immense responsibility of caring for and raising a whole new human being.
"Don't worry about it." came the booze-fuelled reply from the father of three. "You already know everything you need to know. If they're hungry, feed them. If they're cold, warm them up. If they're too hot, cool them down. You'll be fine."
At that point an approving roar announced the arrival of a comely lass clad in a nurse's uniform, and we all lost interest in the conversation.
But the guy was absolutely right - common sense will take you a very long way as a new parent. And that it why I have no hesitation in awarding this particular piece of advice the most profound I have ever received.
---
Last year I was chatting to a new father-to-be, and he asked me for advice on dealing with a new baby, as he was freaking out about the responsibility etc etc as mentioned above. I furrowed my brow, drew myself up to my full height, pretended to be thinking thoughts of the utmost profundity, and passed on verbatim the advice given to me.
He didn't believe me, of course, but I hope that in the fullness of time he'll appreciate my astuteness.
This got me thinking about what advice I would give a newbie engineer, in the hypothetical situation where such a person were to ask me for advice on programming. I say 'hypothetical' because (a) nobody's ever asked me for advice, and (b) as a newbie, I was so self-confident and arrogant that I rarely asked for advice. I fully expect the new generation(s) of engineers to be exactly the same. Indeed, I found that the people who tried to give me advice were almost without exception fine instances of a combination of arrogance and incompetence. My favourite piece of expert advice is still that function prototypes are a waste of time, so don't bother with them. Absolutely priceless, you can't make this sort of thing up.
Instead I just wrote a whole bunch of dreadful code, slowly and painfully worked out what worked and what didn't, and over the years developed an approach that now serves me well.
So, if asked for advice, here is what I would say. These will form the basis of a series of blog articles over the next few weeks/months, depending on how often I find the time to sit down with my netbook and a few glasses of wine.
1. Use version control.
2. Use unit testing.
3. Do it right the third time.
4. A quick wrong decision is often better than a slow right decision.
5. When in doubt, use brute force.
6. Fail fast and hard.
7. Learn a scripting language.
8. Use static analysis.
Sunday, 12 April 2009
Unconscious Programming
If you've ever spent any time trying to learn a new skill you may have noticed that you go through a well-defined sequence of steps. Learning theorists tell us that this sequence is independent of the skill, i.e., it applies whether you're learning to play the piano, ski, paint, or whatever.
Having been through this loop a few times myself, the explanation I like best is laid out in Gestalt learning theory (aka the four stages of competence). This tells us that we go through the following steps when learning a skill (caveat: I was told this by a bloke in a pub, and haven't bothered to research it in any depth whatsoever, but I have found it to be true):
1. Unconscious incompetence. You're rubbish at your new skill, but are happily ignorant of the fact.
2. Conscious incompetence. Having banged away at it for a while, you start to appreciate the full depth and width of your ignorance and inability. I vividly remember listening to some Jimi Hendrix after having been going to guitar lessons for a year or so, and being completely blown away by how freakin' GOOD the guy was. Believe me, if you've never tried to play the guitar, you have no idea.
3. Conscious competence. You can do it, but you need to concentrate on it. It's a bit of an effort, but it's all starting to come together.
4. Unconscious competence. Everything's effortless. Doing everything properly is just part of your DNA, you've got to the point where doing it any way other than the simplest, most direct way is just an alien concept that makes no sense. I love to watch people at this level, whether playing instruments, turning wood, building a wall, or whatever, I have complete respect for anyone who's really good at what they do. Especially stuff I've had a go at, and so have some idea of how complicated what they're doing is.
Jeff Atwood recently blogged about the eight levels of programmers, with the Knuths et al of this world sat at the top of his pyramid. And rightly so. But I think there's a lot of mileage in the Gestalt model. I can see my arc of progress through it in a few areas I've dabbled in. I guess the main problem with it is that you can't really judge your own position in it. After all, how do you know when you push on through from unconscious incompetence to conscious incompetence? How do you know that you're fully appreciating how bad you are at[insert area of interest]? These are judgement calls that can only be made months or years after the fact.
I also have a sneaking feeling that the whole cycle is exactly that, a cycle, where you get to some level of mastery, and then start to realise that, actually, there's a whole new series of levels out there that you were previously unaware of. And at that point, you've just gone back through to conscious incompetence again.
I suspect that programming is a particularly fertile area for this sort of progression. There's a seemingly never-ending progression of bright and shiny new things to play with - say hello design patterns, extreme programming, generic programming, OOP, UML, ....
I know that I'm easily distracted by such baubles. After mainlining on the Gang of Four I spent at least a year in which my code was full of Singletons, Observers, Factorys, etc. None of which improved the code in any meaningful way, and in most cases actively made it harder to maintain for those following in my footsteps (mental note: must send a box of chocs to those poor sods). Which isn't to say that such things don't have a place, just that they should be in my code on merit rather than on a desire to play with some fancy notion I've read about.
Over the last few years my code has become noticeably leaner and simpler, and I think this is a good thing. I've also been working with resource-constrained microcontrollers for the last few years, rather than desktops which to all intents and purposes are infinitely fast and have infinite memory (I know, I know, I'm exaggerating for comic effect, but after you've spent a couple of days trying to save 100 bytes so that you can fit a program in a 4K chip, you get a totally different perspective on things).
I like to think that I passed through one level of unconscious competence (that I now think of as smug complacency) and am now nearing the end of another cycle. Which leaves me wondering where I'll go from here. What will I think of my current self in two years?
Having been through this loop a few times myself, the explanation I like best is laid out in Gestalt learning theory (aka the four stages of competence). This tells us that we go through the following steps when learning a skill (caveat: I was told this by a bloke in a pub, and haven't bothered to research it in any depth whatsoever, but I have found it to be true):
1. Unconscious incompetence. You're rubbish at your new skill, but are happily ignorant of the fact.
2. Conscious incompetence. Having banged away at it for a while, you start to appreciate the full depth and width of your ignorance and inability. I vividly remember listening to some Jimi Hendrix after having been going to guitar lessons for a year or so, and being completely blown away by how freakin' GOOD the guy was. Believe me, if you've never tried to play the guitar, you have no idea.
3. Conscious competence. You can do it, but you need to concentrate on it. It's a bit of an effort, but it's all starting to come together.
4. Unconscious competence. Everything's effortless. Doing everything properly is just part of your DNA, you've got to the point where doing it any way other than the simplest, most direct way is just an alien concept that makes no sense. I love to watch people at this level, whether playing instruments, turning wood, building a wall, or whatever, I have complete respect for anyone who's really good at what they do. Especially stuff I've had a go at, and so have some idea of how complicated what they're doing is.
Jeff Atwood recently blogged about the eight levels of programmers, with the Knuths et al of this world sat at the top of his pyramid. And rightly so. But I think there's a lot of mileage in the Gestalt model. I can see my arc of progress through it in a few areas I've dabbled in. I guess the main problem with it is that you can't really judge your own position in it. After all, how do you know when you push on through from unconscious incompetence to conscious incompetence? How do you know that you're fully appreciating how bad you are at
I also have a sneaking feeling that the whole cycle is exactly that, a cycle, where you get to some level of mastery, and then start to realise that, actually, there's a whole new series of levels out there that you were previously unaware of. And at that point, you've just gone back through to conscious incompetence again.
I suspect that programming is a particularly fertile area for this sort of progression. There's a seemingly never-ending progression of bright and shiny new things to play with - say hello design patterns, extreme programming, generic programming, OOP, UML, ....
I know that I'm easily distracted by such baubles. After mainlining on the Gang of Four I spent at least a year in which my code was full of Singletons, Observers, Factorys, etc. None of which improved the code in any meaningful way, and in most cases actively made it harder to maintain for those following in my footsteps (mental note: must send a box of chocs to those poor sods). Which isn't to say that such things don't have a place, just that they should be in my code on merit rather than on a desire to play with some fancy notion I've read about.
Over the last few years my code has become noticeably leaner and simpler, and I think this is a good thing. I've also been working with resource-constrained microcontrollers for the last few years, rather than desktops which to all intents and purposes are infinitely fast and have infinite memory (I know, I know, I'm exaggerating for comic effect, but after you've spent a couple of days trying to save 100 bytes so that you can fit a program in a 4K chip, you get a totally different perspective on things).
I like to think that I passed through one level of unconscious competence (that I now think of as smug complacency) and am now nearing the end of another cycle. Which leaves me wondering where I'll go from here. What will I think of my current self in two years?
Subscribe to:
Posts (Atom)
