Code Review

On The Naming of Things

2011-06-12T14:39:00.001+01:00

Naming things is hard.

Really hard.

Even the Lord delegated the responsibility of naming the animals to Adam :)

I have recently been involved in some new projects. In general projects need names, and preferably something catchier than 'Project June 2011 #1'. Occasionally there will be a semi-autocratic statement from on high defining a naming convention. For a while we had projects named after local villages, for instance. But such initiatives inevitably get forgotten or ignored after a few iterations. Engineers seem to resent any such imposition and prefer to come up with their own ideas, a natural consequence of the pure, undiluted Not Invented Here coursing through their veins.

It's something of a relief to see that our customers, many of whom are major manufacturing entities, seem to suffer from the same problem. We work on projects named after mountains, rivers, planets, moons, gods (a veritable panoply of pantheons), and a whole bunch of names that don't make a lot of sense. [Such names may make more sense if you're from the relevant country, of course. This may simply be a lack of local knowledge on my part.] But consecutive projects from the same company typically have completely unrelated names.

Big companies also use project names as a sleight of hand. If the name 'Project Thor' slips out or is leaked, there's nothing to tie that with the imminent release of a new version of some trendy smartphone. There are also occasional stories of companies deliberately creating dummy project names in an effort to track down the source of previous leaks. If corporate group A is told of dummy project Alpha, group B is told of dummy project Beta, and the tech websites start humming with rumours of the forthcoming awesomeness that is project Alpha, well, somebody in group A is going to be having a chat with HR.

But I die a little every time I get involved in a conversation or email thread about what to call a new project. It inevitably results in a long and pointless discussion that generates more heat than light. Everyone has their own ideas, typically based on showing how clever/well read/hip/funny they are, and tends to think that other people's ideas suck. Interestingly it's relatively easy to get concensus on the suckiness of an idea, as opposed to agreement that it's good. In general I'm quite happy to be just given a name to use, even if it's self-evidently sucky, as it's much easier than having to come up with a good idea myself, or to engage in the dreaded Naming Discussion.

The nature of my company's business is that we tend to work either on customer projects, which come complete with their own name, or on our own firmware. For the latter we have a defined naming convention based on the capabilities of the silicon/firmware combo. As a result it's actually quite rare for us to have to have a completely standalone initiative needing its own moniker.

The one project name I'm personally proud of is Hawkeye. This is the name I gave to a tool I wrote to graphically display debug data in real time as it is received from one of our embedded chips. However the first few builds of this (long before I let anyone else see it) were called Alethiometer. The name comes from the His Dark Materials trilogy by Philip Pullman, and refers to a device that always tells the truth. This seemed apt, and also served to show how clever/well read I am. But even then I knew that it needed a better name. I found it entirely by accident when I caught some Wimbledon coverage on TV. Hawkeye is the name of the instant replay system used to adjudicate when a player contests a line call. The name had the right air of exacting vigilance and constant awareness I was after, and was sufficiently removed from the target domain to avoid any confusion with my tool.

More recently one of my colleagues has been struggling to come up with a name for a web-based tool to automate some of our design processes. He came up with a series of names, all of which were rejected internally. And so the project has been known at various stages in its lifecycle as Trinity, Twoflower, Rincewind, and Dubery. The last was suggested by management, and was instantly rejected by the engineer concerned, who countered with Aegis. This has been deemed an acceptable compromise, as it is a real word, sounds cool, and although nobody's really sure what it means, it has a pleasing air of classical scholasticism about it.

The issue of project naming and constructive dissembling doesn't just apply to paranoid tech companies either. The movie industry frequently assigns dummy names to nascent projects to preserve some secrecy about forthcoming releases. For instance Steven Spielberg once put out a call sheet containing disinformation about a shoot-out between aliens and cowboys. Which was funny, except that it was instantly all over the internet, picked up momentum, and in a meta-twist of cosmic proportions the movie "Cowboys and Aliens" is about to be a major summer release starring Daniel Craig and Harrison Ford. And I for one will go and see it based purely on the title - which is genius.

England In Despair

2010-07-01T22:15:00.001+01:00

Well, it gives me no pleasure to have been proved right in my earlier predictions - on Sunday the English football team were indeed humiliated and sent home by a clearly superior Germany. At least Germany had the grace to inflict a heavy defeat during normal time, without it having to go to penalties. Indeed, at 4-1 it was England's heaviest ever World Cup defeat.

Yes, Lampard's goal was disallowed, and maybe in some alternate universe it proved a turning point, inspiring England on to victory. But in this 'verse Germany weathered some pretty sustained pressure and then simply ran the length of the pitch and scored. Twice. The disallowed goal will give the disaffected fans something to whine about for the next few years (alright, for many, many years), but it finally makes up for England's disputed goal against Germany in the 1966 final.

Overall it was a lacklustre campaign. This wasn't helped by the fact that the British press are both vicious and fickle, and having whipped up national hopes (as always) to near-hysterical levels, will now turn on the team, looking for blood. The goalkeeping error against the USA was instantly (and rather brilliantly) dubbed 'The Hand of Clod', and Monday's headlines proclaimed 'Rout of Africa' and 'Fritz All Over Now'. The fate of Fabio Capello's stint as boss must now be in the balance.

So what's a nation to do? Why, turn to tennis, of course. Andy Murray's through to the Wimbledon semi finals tomorrow. He's Scottish, but that's near enough for the English papers, who conveniently refer to him as British while he's winning. And the cricket's going rather well, although the prospect of a whitewash in the current one day series has been spoiled by Australia rather inconveniently winning the fourth test to bring the score to 3-1.

Oh, one more thing. My wife has asked me to point out that, despite my previous snide comments, she does indeed understand the football offside rule. And she gave a very convincing demonstration of this using the kitchen table, assorted condiments for players, and a handy hard boiled egg for a ball. I most humbly apologise.

Enddianness and Neddiness

2010-07-01T21:41:00.001+01:00

This is not an article about byte ordering.

Nor is it a discussion of the correct way to eat your eggs.

Rather, it is an exposition on the inability of the human brain to grasp the implications of statistics.

I'm writing this on a Saturday evening. On Friday night, the last thing I did before leaving the office was to start a stress test running on one of my chip designs. It's testing the chip communications. I'm testing that under no circumstances will the chip lock up, or fail to respond correctly to communications. I'm running the test because previous designs on other silicon have exhibited failure modes in which it was possible for the chip to stop responding to communications.

On Monday morning the first thing I will do on going into the office will be to see if the LED attached to the chip is still blinking, indicating that the test is still running, and no failures have occurred. I've been running this same test every night and every weekend for about three weeks, and seen no failures yet. Does this mean that the communications in this chip is bulletproof? No, it does not.

The highly recommended book The Black Swan, by Nassim Nicholas Taleb, talks about the nature of randomness, and in particular what the author dubs Black Swans - events deemed to be of very low likelihood, that have a devastating impact, and that can be explained away after the fact by experts with a little judicious hindsight. (In this podcast, Taleb describes this as "a retrospective ability to weave a causal link", which is a rather wonderful turn of phrase). These would be the self same experts who were completely blindsided by the events themselves, but who can confidently explain them away after the fact. Events such as the two World Wars and the various economic disasters of the last century or so are all Black Swans.

Taleb, in both this book and the earlier Fooled by Randomness, pours withering (and often entertaining) scorn upon such experts. Economists, stock traders, MBAs, and indeed anyone having the temerity to make predictions about the future based on past trends, are all mercilessly ridiculed. (He makes a few honourable exceptions - Karl Popper, George Soros, and Benoit Mandelbrot have all earned his respect.)

In one chapter of The Black Swan Taleb discusses the nature of human fallibility. In it he talks about the medical acronym NED (No Evidence of Disease). Apparently this is written on a patient's records after some tests have been run, and no sign of any malignancy or unusual activity has been found.

What, Taleb points out, doctors will never write is END - Evidence of No Disease. That is, they will happily say that they did not see any sign of a problem, but they will never say that there is no problem.

As engineers, it behooves us to adopt the same approach. If you have been working as an engineer for any length of time, you will have been caught out by the apparent absence of any problems in just the same way that I have in the past. Just because we have run tests for a period of time and seen no bugs, this does not mean that there are no bugs - it just means that we have not seen any. But even though I'm now on the lookout for feathery portents of doom, I'm still really hoping that that LED will be blinking on Monday morning.

England in Bloom

2010-06-10T21:32:00.001+01:00

[Disclaimer: this post is completely unrelated to software, firmware, hardware, engineering or managerial practices, and in truth doesn’t really belong on this site. Normal service will be resumed in due course.]

A quadrennial crop of Saint George's crosses are flowering across the nation. Obviously every pub, white van, and lager lout seems to be bedecked with the emblem, but so are a surprising percentage of ordinary houses, shops, and cars. The reason of course is the forthcoming football World Cup.

A quick note to any American passers-by: when I say 'football', I mean football - what you would call soccer. What you call football is in fact American Football, which I am assured by sports-minded people is completely different. I'm told this is similar to rugby, but they stop every time anything exciting looks like happening. Also, when I say 'World', I actually mean the entire World - calling something a 'World Series' when it's only you that plays in the competition is charmingly parochial, but displays a shocking failure to grasp the basics of cartography. Thank you for your time and attention ;)

Yes, the World Cup is coming, and even I have noticed it. I don't watch television, don't read newspapers, and assiduously avoid news web sites. But in this country the sport is so insidious, pervasive, and all-embracing, that even I know Beckham's not playing this time round, thereby dashing his hopes of playing in four consecutive World Cups, Rio Ferdinand is out with an injury, and Wayne Rooney was recently carded for swearing. It's some form of weird cultural/sporting osmosis, I think.

I generally actively dislike watching sports, preferring to take part myself. But there's something about the camaraderie and group experience of watching a high profile, high stakes match with complete strangers in a packed room that breaks down the normal barriers and leaves you high on the atmosphere of the situation.

I vividly remember seeing Beckham sent off, Gazza in tears, and the hand of God. (This last, in this country at least, shamefully overshadows the simply breath-taking other goal that Maradona scored in the same game - still one of the most awesome pieces of skill you will ever see on a football pitch. I recently heard an interview with Gary Lineker, who was playing that day, in which he said that he wanted to simply stand up and applaud this goal at the time.)

It was before my time, but Kenneth Wolstenholme's commentary on the closing moments of the 1966 World Cup ('they think it's all over…') is completely legendary here. I suspect that it's actually taught in schools. Although any discussion about this game with German colleagues has inevitably ended in a dissection of England's controversial third goal...

In any case, if you can't stand football, good luck for the next month. There's normally a few places around here advertising themselves as footy-free zones during World Cups and European Championships, but this time round they seem conspicuous by their absence.

English World Cup campaigns generally follow a well-worn path. They start off with boundless enthusiasm and wildly optimistic self-belief on the part of the entire nation, to be followed inevitably by gut-wrenching disappointment, ignominy, and defeat. Generally featuring Argentina or Germany, and probably a penalty shoot-out for good measure. Yet the fire of hope burns forever bright.

I for one am looking forward to it. Come 7.30 on Saturday evening when England kick off against the USA, I'll have found a pub or friend with a TV, will be clutching a pint, and will be loudly expressing my concerns with the manager's strategic choices, something about which I know absolutely nothing. I do know I will be trying to explain the offside rule to my wife YET AGAIN.

UPDATE: In the office sweepstake, I've managed to pick England as my team. I tried to explain to the girl running it that I'm such a dead cert to win that she might as well hand over my winnings now, but unfortunately she too has known the heartbreak of following the Three Lions, and politely told me to naff off.

Monitor Functions in the IAR Compilers for AVR

2010-04-28T21:47:00.003+01:00

In AVR chips, game-changing actions like changing the clock prescaler or setting the watchdog are so important that they are afforded a protection mechanism all of their own. Timed write sequences are used to prevent accidental or rogue writes from fundamentally messing with your day.

These require you to perform a sequence of write operations within a defined time period - typically you need to do two writes within four clock cycles.

Care must be taken when performing such a write sequence that no other operations could take place that would violate the required timing. Typical candidates for such mischievous behaviour are interrupt service routines, which have an uncanny knack of firing at the most inconvenient times.

The obvious way to ensure that your timed write sequence is interrupt-safe is to disable interrupts around it, like so:

void do_timed_write_operation( void )
{
__disable_interrupt();
// timed write sequence goes here
__enable interrupt();
}

Unfortunately this contains a nasty little gotcha: if you call this function from code in which interrupts are disabled, they will be enabled on return from the function. This is almost certainly not what is intended, and can lead to late night debugging sessions.

A safer approach is to save the interrupt state on function entry, disable interrupts, do your business, and then put the interrupts back the way you found them:

void do_timed_write_operation_2( void )
{
 uint8_t interrupt_state = __save_interrupt();
 __disable_interrupt();
 // timed write sequence goes here
 __restore_interrupt( interrupt_state );
}

This approach is so common and idiomatic that the IAR compilers for AVR support it directly with the __monitor keyword. The following function is equivalent to the previous one:

__monitor void do_timed_write_operation_3( void )
{
  // timed write sequence goes here
}

Unfortunately declaring a monitor function involves a code overhead - after all, even though you can't see it explicitly in the C, a quick squint at the list files will show that the interrupts are still being saved, disabled, and restored.

A saving grace here is that most such Big Important Things tend to happen during system initialisation, when interrupts are disabled anyway. In such circumstances you can perform your timed writes au naturel, without any interrupt protection. However you really need to convince yourself that it is safe to do so. It would be a brave man indeed who called the first function above once the system was up and running without spending a very long time looking at possible consequences.

In general I would suggest that all timed write sequences be protected by default. Such protection can be removed if you're convinced that it's safe to do so - and of course if you document this assumption in the comments.

File Under Easy Listening

2010-04-21T22:07:00.001+01:00

This is an excellent podcast by David Heinemeier Hansson (DHH) on Legacy Software. It's a recording from a talk at RailsConf Europe, and the podcast (i.e., audio-only) format doesn't really work in the last third of the talk, in which he's refactoring some code on stage. However, the gist of his argument is completely sound - if you wince in embarrassment/pain/surprise when you look at code you wrote a few years ago, it's nothing to be ashamed of. The code hasn't got worse - you've got better. In the interim you've learned new techniques and idioms, and the fact that the earlier you wasn't favoured with your current clear-sighted and visionary approach to writing code shows that you've progressed.

[The corollary of this is that the future you will be embarrassed/pained/surprised by what the current clear-sighted and visionary you is doing today, but what the hey, just appreciate the step change from yesteryear.]

Also highly recommended is this appearance by DHH on Jason Calacanis's This Week In Startups podcast. Two successful and highly opinionated people with entirely different takes on business, money, and work slug it out. Who's right? They both are, of course. They've figured out what works for them, and good luck to them. You don't have to agree wholeheartedly with either to enjoy the debate.

Joel Spolsky and the Death of Blogging

2010-03-28T16:05:00.001+01:00

Warning: this article contains meta material. If you are easily offended by blogging about blogging, please stop reading now.

Still here? Good, let's crack on then.

I personally am not a fan of metablogs - I read blogs for interest, entertainment, and education, and am not particularly interested in reading the authors' pontifications on the state of their respective navels. One saving grace for blogs in this respect is that they are less susceptible to the curse of self-reference than tech podcasts. These seem without exception to descend into an analysis of the microphones, headphones, software, and patch cables being used. I suspect this is largely a by-product of putting geeks in close proximity to, you know, geeky stuff. Perhaps blogging is slightly less prone to this phenomenon as it features fewer tools - after all, you just need a text editor.

And so it was with some surprise that I found myself writing this. What prompted it was the recent announcement by Joel Spolsky that he's going to quit (actually, by the time this goes up it will be 'has quit') blogging. He made the announcement in an Inc Magazine article, and his blog entry on distributed version control looks like being his last.

In case your universe hasn't intersected that of Joel's, he runs Fog Creek Software in New York. They have several products in their portfolio, but I think it's fair to say that their FogBugz bug tracking application is their main calling card. Joel has also run the Joel on Software blog for some time, and is a well known pundit and commentator on all things software. More recently he co-founded the Stack Overflow website, a Q and A website for programmers that has very rapidly gained a huge chunk of mindshare.

In his valedictory Inc Magazine article Joel states amongst other things that blogging is just too narrowly focused to be effective advertising for his company. Which makes perfect sense if you're using it as a source of potential revenue.

In his penultimate Joel on Software article Joel lists some of the subject articles that have been written several kajillion times, by legions of pundits exploiting the whole democratisation of opinion made possible by the explosion of the Internet. Which makes perfect sense if what you want to do is something truly original, especially in the narrowly-defined genre of geekerature.

All of which made my pause and consider - why am I doing this? If someone as well known as Joel has had enough, doesn't see the point, and is getting out, why should I bother? Why expend time and mental effort re-visiting subjects that other people have already addressed, more incisively and profoundly than I can?

Well:

I enjoy the act of writing. I take pleasure in jotting down fleeting ideas or comments, and letting them germinate until they spring forth as fully-fledged articles. It forces you to articulate and justify vaguely-held opinions and beliefs. Under the weight of such scrutiny some ideas have self-imploded and got thrown out as blog fodder, but that doesn't diminish the importance of the process one whit. In fact, if anything it increases it.
Joel's been doing this a lot longer than I have. He's been there, done that, got the T-shirt, and it's time to move on. Actually, he must have a whole wardrobe full of T-shirts. For me the world of blogging is still new and shiny and pretty. I'm enjoying it, so why stop?
There's a vague chance that some of my ramblings may be of some use to somebody, somewhere, some when. Goodness knows I've learnt plenty from other people's blogs, and in some sense it's a little bit of repayment of accumulated karmic debt.
The articles act as a reference of techniques and ideas that I can easily find in the future. If I want to do some fixed point arithmetic, or generate some random numbers, I've got ready-to-go code snippets and links back to original source material. Of course I haven't actually written all that much yet, but I've already had reason to dip back into the archives a few times. I've got a stack of blog ideas and sketched-out articles, so this blog will hopefully turn into a useful resource for the future me.
It is also of course partly self-aggrandising/self-serving. The Internet has zeroed the cost of vanity publishing, so I may as well put something out there. Something I can link to at some point in the future to flesh out my CV a bit. Something that shows that I haven't spent the last several years just fulfilling a mandate as a corporate drone, but that I also think about and care about what I do.

My apologies for the brief digression into self-analysis. Relatively normal service will be resumed in due course.

Backing Up Your Development Environment

2010-02-28T15:29:00.001Z

Back in the good old bad old days, backing up an embedded project was simple. You put the source files and makefile on a floppy disk. And for good measure, you could put a copy of the compiler, assembler, linker, and any other tools on the floppy too. The complete toolchain needed to recreate the shipped code, neatly wrapped up in its own little time capsule.

And that toolchain is an important detail. The source code on its own is not enough to recreate a build, you have to factor in the tools too. The output from a compiler can vary by a surprising amount between versions.

These days things are not so straightforward. The compiler, assembler, and linker still exist, but they generally lurk behind the comforting facade of a glossy integrated development environment (IDE). You don't need to figure out the arcane invocations needed to convince the command line tools to do your bidding, you just tick some checkboxes in an options dialog. And I for one don't have a problem with this, I'm perfectly happy for such details to be sorted out behind the scenes.

However this ease of use can be a double-edged sword. That sparkly shiny IDE is likely a significant chunk of code, and will be squirreling away information in all sorts of nooks and crannies on your PC - in the registry, in initialisation files, in configuration files, and down in the depths of your Documents and Settings folder.

Add in some vendor patch releases and user modifications, and suddenly keeping a snapshot of the state of your tools starts getting tricky. Not only do you need to know that the code was built with v1.2.3.4 of the Acme Super Awesome IDE, but also that at that point you'd applied three vendor patches, and hand-modified a chip configuration file that you'd found a bug in. You'd reported it to Acme MegaCorp technical support, but there hadn't been a bug fix release yet.

Now what do you do?

My current solution is to store a simple text file called 'build instructions.txt' or similar with each project. It lists the IDE version, any applied patches, and any funnies such as the aforementioned modified chip configuration file. This approach is not exactly high tech or complicated, but works fine.

I can't help thinking that there must be a better way though - this is still a bit handraulic and error-prone. I suggested to our IT guy that we could create a virtual machine (VM) image containing a bare bones XP install, plus the IDE, plus any 'non-standard' files (patches and the like). That way we could simply specify which VM image to use with the source and project files hauled out of version control. But that made him go a bit green and start shaking. He was mumbling about licensing issues with that many Windows images.

So - if anybody knows of a better way of doing this, do please let me know in the comments.

Zero Tolerance

2010-02-14T15:21:00.001Z

This week I picked up some code from a colleague for review. To me, part and parcel of the review process is rebuilding the code - I want to check that I get the same hex (i.e., output) file as has been checked in to our version control system for release. And sure enough, this time I did.

But I also got 16 warning messages. The compiler spat out a variety of messages, including warnings of integer truncation, and what looked like notes the developer had written to himself - "This could be done faster with pointers" and the like.

What did this mean? Was I using a different compiler version than the developer? Was my version smarter about flagging potential coding errors? It seemed unlikely, as I got the same hex file. This almost certainly indicated that we were using the same compiler. Also, there was the matter or all those notes-to-self. Those were clearly there for a reason. How did he know it would be faster with pointers? Presumably he'd coded that up to test it, as otherwise he couldn't know that it would be faster? If he had, and it was, why hadn't he used it?

I spent a while scratching my head over this before going to see the developer. "Oh yeah, I had a look at those, the real warnings are all fine. Nothing to worry about. And the others are just reminders to myself.". And the comments about pointers and the like? "It's always faster to do that code with pointers, I just didn't have the time. That's just a reminder in case I look at this again."

Well, that was fine for him, as he'd written the code. But at some point somebody else was going to look at it. Maybe a customer would report a bug, and a colleague would be assigned to fixing it. Maybe we'd want to add some new features. Maybe we'd want to use this code as the basis for another product with similar functionality. Or maybe, as in this case, a reviewer would pick up the code.

In all of these scenarios, Joe Bloggs engineer would pick up some allegedly working code, hit build, be faced with a screenful of warnings, and freeze like a rabbit trapped in headlights. Hampered as he would be by a lack of omnipotence, he wouldn't know that all the warnings are benign, and that a colleague knows a better way to write his own code.

The warnings might indicate real problems. They might be the compiler being picky. But our hypothetical engineer is going to have to assume that each is a land mine requiring individual attention and possible defusing. All of which could have been avoided if the original developer had simply added some casts or whatever else was needed to placate the compiler.

The only acceptable number of warnings in a build is zero. Anything else is going to waste people's time. Any cost saving to the developer in not dealing with the issues as they crop up will be more than offset by the time wasted by a colleague (or possibly themselves) further down the line.

Possibly more importantly, a developer is fooling themselves if they think they can live with a raft of warning messages.

If you have 16 warnings every time you build, you are much less likely to notice the silent arrival of a seventeenth, this one a genuine harbinger of doom. If however your build has always been clean, and is suddenly polluted by a warning - well, you'll see that.

This is indicative of a professional attitude to development, as well as saving time in the long run. If all warnings are examined and addressed as they surface, the ongoing code quality and workmanship are high. If attending to such items is left as a last-minute operation before release, or even worse pencilled in for the often mythical post-release clean-up, then it will either (a) be a huge job, or (b) not get done.

The same comments apply to writing code that is MISRA-compliant, or that gets a clean bill of health from Lint. It's always faster and easier to do this as you go, rather than banging a keyboard for several days and then trying to impose some semblance of quality as an afterthought.

We are what we repeatedly do. Excellence, then, is not an act, but a habit. (I didn't say that, Aristotle did. And according to Wikipedia, it's actually a misattribution. But as misattributions go, it's got to be one of the best.)

Generating Random Numbers using LFSRs

2010-02-07T15:58:00.003Z

On a recent project I needed a simple random number generator. It didn't need to be anything fancy, nothing mathematically pure or statistically valid, just a quick'n'dirty little generator that would spit out numbers that looked random to the casual eye.

In a previous incarnation I worked for a mobile radio consultancy. One of the things I came across in that particular life was the concept of Pseudo Random Binary Sequences (PRBSs).

These are sequences of 0s and 1s that are generated deterministically, yet are random enough for many practical purposes. They are used extensively in communications to spread the bandwidth occupied by a signal. This makes the signal more immune to jamming, as the signal is smeared across a bigger chunk of spectrum than would otherwise be the case. It is also more difficult to intercept, since any receiver must know and be synchronised to the spreading sequence at the transmitter. A more subtle usage is that multiple signals can be spread across the same spectrum by multiple PRBSs, yet be teased apart at multiple receivers by correlation with the correct individual spreading sequences.

PRBSs are typically generated using Linear Feedback Shift Registers (LFSRs). These are cheap and easy to build in hardware, and easy to code in software.

At any time, a function of the LFSR contents is generated as the output bit value. This is fed back into the shift register as the next input. The output bit sequence is essentially random, as is the sequence of states formed by the LFSR contents.

The 'function of the LFSR contents' mentioned above is called a generator polynomial. It determines which LFSR cells are combined and fed back to form the LFSR input for the next state.

For instance, the Wikipedia article on LFSRs lists the following generator polynomial for an 8-bit PRBS:

x^8 + x^6 + x^5 + x^4 + 1

We now need to know how to convert this into code.

As we are generating an 8-bit polynomial, we start off with the 8-bit unsigned value 0x00u as our feedback function. If a power of x, say m, is used in the polynomial, assign a 1 to polynomial bit (m-1). The +1 in the polynomial corresponds to the output bit, and does not appear in the feedback settings. Here we have an 8-bit generator polynomial with bits 7, 5, 4, and 3 set, giving a value of 0xB8u.

Given this, and also using the example code in the Wikipedia article, we can now write a function that returns the next value in a pseudo-random 8-bit sequence.

#define LFSR_8_INITIAL_VALUE 0x01u
#define LFSR_8_POLYNOMIAL    0xB8u

uint8_t next_lfsr_8( void )
{
 /* seed LFSR */
 static uint8_t lfsr = LFSR_8_INITIAL_VALUE;

 /* get LFSR LSB */
 uint8_t lsb = (uint8_t)( lfsr & 0x01u );

 /* shift LFSR contents */
 lfsr = (uint8_t)( lfsr >> 1u );

 /* toggle feedback taps if we output a 1 */
 if( 1 == lsb )
 {
    lfsr ^= LFSR_8_POLYNOMIAL;
 }

 return lfsr;
}

An n-bit PRBS is said to be maximal length if the number of distinct output values it generates is 2^n - 1. That is, a maximal length 8-bit PRBS consists of 255 values. Our generator polynomial is indeed maximal length, and so the values returned will start to repeat after 255 calls.

In the above code I have set the initial value of the LFSR to 0x01u. The LFSR will generate a fixed sequence of output values, and the initial value merely determines where in the sequence the reported values will start.

Note that because the feedback taps uses the XOR operator, the LFSR will get stuck in the all-zeros state if it ever gets there. Care must be taken that an LFSR is not initialised with this value.

The corresponding 16-bit function is as follows.

#define LFSR_16_INITIAL_VALUE 0x0001u

#define LFSR_16_POLYNOMIAL 0xB400u

uint16_t next_lfsr_16( void )
{
 /* seed LFSR */
 static uint16_t lfsr = LFSR_16_INITIAL_VALUE;

 /* get LFSR LSB */
 uint8_t lsb = (uint8_t)( lfsr & 0x0001u );

 /* shift LFSR contents */
 lfsr = (uint16_t)( lfsr >> 1u );

 /* toggle feedback taps if we output a 1 */
 if( 1 == lsb )
 {
    lfsr ^= LFSR_16_POLYNOMIAL;
 }

 return lfsr;
}

Using the above approach, you can generate PRBSs of arbitrary length, given tables of generator polynomials (your search engine of choice will unearth plenty).

What's In Your Toolbox?

2009-12-20T18:25:00.001Z

This article started out as a simple list of the tools and applications on my work PC. This was to be both an audit of the current state of the union on my PC, and a checklist for myself for when my PC gets upgraded or dies (note that there is no 'if' in the preceding statement). I was interested in not just the full-blown applications that appear in the Start menu, but also the little scripts, batch files, and utilities that tend to accrete on our PCs, the small-but-useful stuff that doesn't get much mind share.

The main reason stuff is on my PC is to help me develop, maintain, or review code, but some other bits and pieces have snuck on too, for instance applications that help me write documentation, or prepare a presentation. I try to keep my PC fluff-free, and any application that doesn't cut the mustard is ruthlessly excised. If it's listed here, I use it, and find it useful.

Programmers are notoriously obsessive about their choice of tools, and love agonising over them in minute detail. Just type 'best text editor' into Stack Overflow and watch your screen melt. I suspect engineers are comparatively pragmatic, but there's no harm in taking a lead from our non-soldering brethren.

This list does not by any means constitute the best of breed. It's just bits and pieces I've accumulated in my travels. Sometimes I stumble upon gems through books, blogs, or magazines, sometimes I go looking for a particular functionality I'm sure somebody must have built, and sometimes people recommend programs. Whatever. If you see something here that looks interesting, but would like to see some alternatives, Scott Hanselman maintains an awesomely huge tools über-list, and I highly recommend that you at least skim it.

One tip: learn to use your tools. RTFM, and learn the keyboard shortcuts. I've found over the years that many of the tools I've been using are more powerful than I realised, but you need to delve to find this out. And using keyboard shortcuts is a no-brainer - even if they only save a couple of seconds over the equivalent operation with a mouse, the compound interest on this saving over the course of a year is enormous. [Related pet peeve - I find it almost physically painful to watch somebody doing something like copying and pasting text between two documents using the mouse. They carefully highlight the text of interest, then go to Edit|Copy, then press the toolbar icon for the destination document, carefully place the cursor at the appropriate point, and then select Edit|Paste. And in my head I'm shouting "Double click! Ctrl-C! Alt-tab! Ctrl-V!". But maybe that's just me.]

Don't be afraid to splash your own cash. Even if the department budget won't stretch to buying everybody a decent text editor, spending a few quid of your own hard-earned on decent tools is well worth the investment. I've a friend who's a car mechanic, and has spent thousands of pounds on stocking a decent toolbox over the years - I don't see why supposedly professional engineers should shy away from investing in their own careers in like manner. That said, many of the tools listed below are free.

Rules for inclusion: I'm interested in applications that may be of interest to the engineer on the Clapham omnibus, and so haven't listed project-specific tools like compilers and vendor IDEs. If you need such tools, you'll have them. I've also skipped over staples like Microsoft Office - such applications are likely to be on the default engineer PC install. Again, if you need them, you'll have them. Tools are listed in alphabetical order. And lastly, these are all Windows applications, because that's where I live. Some may have ports to other platforms, and there are almost certainly functional equivalents for Linux, OS X, etc.

7-zip: Everybody needs a compression tool. This one is free, and handles all of the formats I've thrown at it over the years (including things like *nix tarballs). It can also generate password-encrypted zip files, which is neat for sending confidential documents or source code to offsite contacts.

Absolute Uninstaller: The clue's in the name - this uninstalls programs. But this free program lists installed programs much more quickly than control panel, removes programs more completely, and can export a list of all installed programs. In truth this is my main use for this program - generating a weekly list of all installed programs as a snapshot for future reference.

Agent Ransack: This is just insanely useful. It's a free search program allowing you to recursively search folders for files containing specified text. Both the filenames and the search text can be regular expressions. When installed, "Agent Ransack..." appears on the right-click context menu in Windows Explorer, allowing you to specify the root folder for your search. I have this up and running pretty much all the time I'm developing or examining code.

Beyond Compare: Again, everybody needs a file differencing program. This one lets you compare files or folders in text or binary mode. Text comparisons can ignore whitespace-only differences, or 'trivial' changes such as source code comments. Changes between two files or folders can be copied across in either direction. Folders can be synchronised, FTP is supported, and there's a whole bunch of options I haven't even looked at. Heeding my own advice above, I really need to revisit this and RTFM. Again, the tool integrates with Windows Explorer, appearing on the right-click menu to let you choose the 'from' and 'to' files/folders for a comparison.

Bug Shooting: Before I installed this free screenshot utility, I wouldn't have believed how much I would use it. It can be run by clicking on its icon in the system tray, or by pressing one of the hot key combinations. It can capture either the whole screen, or just a specified section, and can do so immediately or after a timed interval. Once captured, you can crop the image, highlight an area, write text on the image, and so on. It was originally developed for posting screenshots to bug tracking systems, but I use it for writing documentation, FAQs on our in-house wiki, and a bunch of other stuff that wouldn't have occurred to me before I'd installed it.

CCCC: This one is fairly hard-core geeky. It's an open source application based, I believe, on a doctoral thesis, that analyses C/C++ source code and spits out various metrics including line of code, line of comments, and McCabe complexity metric. I use these as a general basis for comparison of bodies of source code - is this application inherently more complicated than this other one? It also highlights complexity hotspots worth investigating in a bit more detail.

CCleaner: This free program removes unused files to free up space by clearing your recycle bin, Windows temporary and log files, browsing history, and so on. It also contains a registry cleaner for removing unused and old entries.

Clean Sources Plus: This is a neat little freebie utility from Jeff Atwood. It gives you a right-click menu item in Windows Explorer (yes, I'm a sucker for these) that lets you delete a user-specified set of files and folders in and under the clicked-on folder. So, for instance, you can remove all object files in your development folder before checking your code into version control. It also gives you the option to zip the folder - very handy when you want to upload or email some code.

Clipname: Another freebie that rapidly became indispensable. Right-click on one or more files or folders, and you get the option to save their names to the clipboard for copying into other applications. You can save just their names, or their full paths including all folder information, and can do so in the URL-encoded format needed for hyperlinks. Very, very helpful.

CmdHere Powertoy: This time the freebie is from Microsoft itself. Right-click on a folder (are you spotting a common theme here?) to get the option to open a command window pointing at the selected folder. Very useful when you want to jump in and check the output from a batch file, or quickly run a program with particular command line options.

DebugView: The first but by no means the last appearance for the Sysinternals tools on this list. This program captures debug output locally, or across a TCP/IP network. In my case I've only ever used it to capture OutputDebugString() calls in my PC code, but apparently it also supports kernel mode debugging.

Editplus: a decent text editor is absolutely essential. When developing you almost certainly spend a huge chunk of your working day in a text editor, and so should be using a tool that helps you as much as possible. For me, the minimum requirements of a text editor include language-specific syntax highlighting, the ability to record and play back keystroke macros, and regular expression searching. Failure to provide any of these results in me immediately losing interest. For historical reasons my weapon of choice here is Editplus, which is small, cheap, and powerful. BTW, one of my standard interview questions is to ask the candidate what their favourite editor is. I'm not really interested in their answer, so much as in if they have one. If not, they're not thinking about what they're doing.

EverNote: I realise that I'm only using a tiny percentage of the functionality of this application. I basically use it as a journal/logbook, writing up notes as I work, and pasting in code snippets, scope shots, and photos as appropriate. These are all then available for searching when, in six months, I want to quickly find all my notes on a given project.

Fences: This one is still in the stage I like to call "trying-it-out-to-see-if-I-like-it-and-or-find-it-useful". It lets you create areas (that it calls 'fences') on your desktop, in which you can drop files or shortcuts. I personally try to keep my desktop empty, and use a hotkey application launcher to start up most of my programs, but this has potential. I've been using it to create project fences containing links to the datasheet for the chip I'm working with, the project demo software, and so on. Fences feels stable and would certainly be worth looking at if you tend to use your desktop as a holding area.

Foxit Reader: Smaller, faster, and leaner than Adobe Reader.

Google Desktop Search: This brings the power of the mighty Google to your desktop. Once installed it starts indexing all of the files on your PC. I've got it running in the system tray, and by typing in, for instance, the name of a project I've worked on, it will find all files containing that name. And that includes not just text files, but also Word, Excel, PDFs, emails, etc.

Janitor Script: I have a folder on my PC where I put all downloads, email attachments, and things I only want to deal with on a temporary basis. This little script runs every lunch time and deletes everything older than a week from the folder. Neat.

Keepass: Keep all your passwords in a single database with a single password to remember. I use this for not only all of the web sites demanding I have a password to identify myself, but also for the passwords I've assigned to users on some of our internal tools.

Launchy: This is just mind-bogglingly, consciousness-expandingly useful. It's a hotkey application launcher that has totally removed my need to use the Start menu, shortcuts, or just about any other method of running a program or opening a commonly accessed file. I press ALT-space, type in the first few characters of the application I'm interested in, it auto-suggests the most likely candidate based on my previous invocations, and I hit enter to run the program or open the file. Or, in many cases, I can supply further information. For instance, to search for a book on Amazon, I type in 'am' to get Launchy to suggest 'Amazon', then hit tab, followed by the name of the book I'm interested in. It then launches my browser and displays the Amazon search results. I can also create emails, add items to my web-based to-do list, and append text to files on my PC. If you haven't tried a hotkey launcher, give them a go, they'll change your life.

PDFCreator: Install this freebie to have a PDF output option appear in your print menu. This lets you create PDFs straight from any Windows application that lets you print. Neat.

Paint.NET: Free and increasingly-powerful image and photo editing software. I tend to use it for really simple things like annotating photos, but there are a wealth of features in there.

PC-Lint: If you're writing C or C++, and don't have this, or a functional equivalent, go buy it. Now. Seriously. This static analysis tool remorselessly examines all of your code, and pedantically flags every potential violation of the rules of the language. You can also configure it to perform MISRA compliance checking, so consider it as two-for-the-price-of-one. It's remarkably flexible (for which read 'difficult to configure to get it going'), but well worth the effort. Hook it in to your development system so that it runs every time you build your application, and you'll never look back.

Portmon: YASG (Yet Another Sysinternals Gem). I stumbled across this when debugging a serial port application I'd written. It lets you capture all reads and writes to/from specified serial and parallel ports. Absolutely essential if you're playing with such items. Also useful if you want to figure out the interface protocol for a bit of third-party kit. There used to be similar applications for file activity (filemon) and registry activity (regmon), but these have now been superseded by a process monitor (procmon).

Process Explorer: YASG (see above). Task Manager on steroids. See unbelievably detailed information on what's going on in your PC. I've no idea what a lot of the displayed data is, but what I have been able to do with this in the past is use it to check for memory leaks in my PC code. Select a program, right click to select 'Properties...', and switch to the 'Performance Graph' tab to see the application memory usage - if it continually ramps up, you've got a problem.

RegexBuddy: OK, this is pretty geeky too, but in a good way. I bought this following Jeff Atwood's enthusiastic endorsement, and find that it excels in its (admittedly rather narrow) sphere of application. As previously mentioned, one of my main uses for script languages is searching for data in log files. Well, with RegexBuddy, I can copy some of a log file into a 'test' window and try out regular expressions that are evaluated on the fly, highlighting matches as I type. When I'm happy with my expression I can copy and paste it into a fully-fledged script and I'm good to go. Similarly, I can copy sections of code into the test window, and develop regular expression searches to be pasted into the aforementioned Agent Ransack.

Source Code Spell Checker: As I’ve mentioned before, I have a fairly dogmatic approach to spelling and grammar in source code. I run this tool on all of my own code, and on code I'm reviewing, and fix or flag all transgressions. Cheap, and does what it says on the tin.

Source Navigator: This is an open source code analysis tool. It can generate function call trees, variable cross reference tables, class hierarchies, etc. Extremely handy in getting you up and running on a glob of code you haven't seen before.

SyncToy: Another freebie from Redmond. This one lets you synchronise folders, which is very useful when, for instance, keeping a snapshot of your source code on a USB stick or network drive.

TreeSize: Right-click a folder to see a graphical representation of the size of it and all folders under it. Great when trying to track down what's eating all of the space on your hard drive/USB stick.

ZoomIt: YASG (see above). Lets you zoom in and scroll around on your PC screen, highlight areas, and scribble notes on it. Invaluable during presentations or training sessions.

Synchronicity or Serendipity?

2009-11-12T22:03:00.003Z

The week after I posted my blog on integer division I was asked for help on a problem involving linearising a sensor output. Part of the answer turned out to involve multiplying a function of the output value by some weird factor like 211/255. Well, given that I'd just written exactly the code needed to handle this sort of strangeness, this turned out to take approximately ten minutes of my day. Result.

Of course this approach only occurred to me because I'd so recently been playing with integer division code, and so made the obvious connection. Maybe given time, if I hadn't been handed the answer by the universe immediately beforehand, I would have come up with an equally valid alternative approach. But for the life of me, I can't think of a neater solution to this particular problem.

A second example of the power of autodidacticism (not as dirty as it sounds) came about the following week when I was looking at how to add new events to a client-server messaging system. Having a global enumeration of all possible events works fine, but means that you have to rebuild every darned application every time you add a new enumeration. Well, I'm something of a compulsive purchaser of programming books, and one evening I was idly flicking through the excellent Game Coding Complete when I came upon an extremely neat solution. You calculate a hash of a descriptive string, and use that as the event enumeration. For instance, if you have an event to set the gain of an amplifier, you calculate the enumeration as hash( "set_amplifier_gain" ). Store the string and enumeration in an object that gets passed around the system. Point your debugger at an event, and you see a clear text description of what it is. There's a bit of housekeeping to ensure that two strings don't accidentally generate the same hash value, but the approach is lightweight, and lets you arbitrarily add new events while only rebuilding applications that actually need to know about them.

So: two closely-spaced incidents. In one, I have a neat solution, and almost immediately stumble upon a matching problem. In the other, I have a trivial-but-irritating problem, and almost immediately stumble upon a neat answer. Both cases involved looking outside the scope of my everyday activities to add some tools to my grab-bag of solutions.

It's what Stephen Covey refers to as sharpening the saw, which is a great analogy, and resonates conveniently with the tools metaphor - we all have a bunch of tools at our disposal, and it behooves us to care for them. It also pays dividends to get new tools, upgrade old ones, or simply window-shop to check out what's available.

As the always-readable Rands wrote, "We see the world how we want. A carpenter sees all problems as a nail. I see problems as finite state machines." Well, amen to that, my brother. (Hint: a state machine may not always be the best answer, but it's almost always an answer.)

So: not exactly a profound revelation, just a gentle slap upside the head from fate, reminding me that it's always worth spending time learning new techniques, following interesting blogs and articles, and keeping a weather eye on seemingly unrelated spheres of interest. Even if the number of times you extract a true nugget is small, it's not vanishingly so, and the return on investment can be enormous.

Division of Integers by Constants

2009-06-21T21:13:00.003+01:00

Nigel Jones recently wrote a blog entry that very neatly summarises an article called 'Reciprocal Multiplication, a tutorial', by Douglas Jones.

The (Douglas) Jones article explains how to use reciprocal multiplication to perform division - for instance, if you want to divide by 10, this is the same as multiplying by 1/10. However, by doing everything in fixed point arithmetic, you can avoid the computational overhead of invoking your compiler's division routine. The result is smaller, faster code that has exactly the same accuracy as the compiler-supplied result. Anyone writing PC code may not appreciate this, but in the embedded world this sort of approach is vitally important in terms of both code size and run time.

The original article is quite long and detailed, and despite having stumbled upon it a couple of times in the past, I'd never got round to actually reading it. And so (Nigel) Jones's article came as a very welcome abstract, boiling it down as he does to a couple of simple algorithms - if you want to divide by 'x', here's what to do.

At the end of his post, (Nigel) Jones said 'If someone has too much time on their hands and would care to write a program to compute the values for all integer divisors, then I'd be happy to post the results for everyone to use.' Well, I don't know about having too much time, but I do enjoy a lunchtime programming exercise, so I sent him a text file containing the coefficients to perform integer division for all 16-bit unsigned numbers from 3 to 32768. I also asked if he would mind if I posted the source code here, in case it was of use to anyone.

He responded with an excellent suggestion - how about generating a header file containing macros with the appropriate coefficients for all divisors? Anyone wanting to use these algorithms could then simply include the header file, and call the relevant macro to perform the required division.

I've followed this suggestion, so please follow the links to check out:

a header file containing the coefficients for unsigned 16-bit division
a header file containing the coefficients for unsigned 8-bit division
a command-line program generating the coefficients for unsigned 16-bit division [source][exe]
a command-line program generating the coefficients for unsigned 8-bit division [source][exe]

The programs were written in Borland Builder, but are in ANSI C, and so should run on any platform with appropriate tweaks to the uintX_t typedefs. They accept floating-point arguments, and so can generate the coefficients for division by π, sqrt(2), etc. They will also report the number of errors found during an exhaustive search of all divisors, and the maximum error found.

Profound Advice: Learn a Scripting Language

2009-06-01T22:19:00.004+01:00

This is another entry in an intermittent series in which I arrogantly pontificate on what constitutes profound advice in the world of embedded software. Actually, I'm unnecessarily narrowing my horizons here, I feel quite sure I could be arrogant in a much larger sphere.

So, here we go again. This time I would urge neophytes to learn a scripting language.

These may not be directly useful in terms of running on a little 8-bitter, but they are unbelievably useful in a range of other applications.

The advantage of scripting languages is that they're... well, scripting languages. They act like super batch files letting you leverage the power of other programs to get a job done. They tend to be interpreted rather than compiled, and so are (a) really fast to write, since they sidestep the edit-compile-link-run paradigm, and (b) relatively slow to run, since they've sidestepped the edit-compile-link-run paradigm. But with the power of modern desktops, performance really isn't a huge concern with the applications I'm interested in.

At a more detailed level, they also tend to support dynamic typing, which lets you blithely ignore whether the variable you've just created is a string, an integer, or whatever, until you start using it in a context that defines what it has to be. This is both a good and a bad thing, as what the interpreter thinks it is doesn't always tally with what you think it is.

Yes, you could create an executable that would run much faster, as it's not interpreted. But for sheer speed of get-in-there-and-hack-out-something-that-works, you can't beat a scripting language.

But before I go blindly hacking away into the undergrowth of what a language is or isn't, let's try to steer the conversation back on track.

The scripting applications I find myself revisiting time and again are (1) analyzing log files, and (2) generating code.

Dumping data to disk on a PC hooked up to an embedded system is part and parcel of the daily life of a firmware engineer. But when the system's been running for several days in an environmental chamber, or for months in a remote data acquisition application, these log files can be enormous.

The chances are you're only interested in a tiny fraction of the logged data, or you're looking for a particular event, or you want to change its format slightly, or... something else. Maybe you need to break it up into chunks for analysis. Maybe you only want to see every line in which the fourth CSV field is greater than 10. Maybe you want to reformat the data for input to another program. For whatever reason, you've got a bunch of data on a disk, and you want to pull the data needle out of the multi-megabyte haystack.

The script to do this will typically boil down to a few lines containing regular expressions. Pipe the data into the script, and out to a text file, and you're done.

Scripting languages are also a natural for code generation. Write a script to generate big chunks of your firmware, which are then compiled as normal. Any type of code containing lots of variations on a theme is a natural for this approach - good candidates are communications handlers and state machines. Give the script a list of the UART commands to be handled, or the FSM states, and let it build all of the scaffolding code in a loop over the list contents. You follow along afterwards and fill in the blanks detailing, for instance, how to handle the parameters of a specific command.

You could of course make your script sophisticated enough to accept details like the command parameters, and then generate the handling code too. But there's always the danger of spending too much time developing the script instead of doing actual useful work - as always, your mileage will vary, you need to be pragmatic, and you need to make realistic trade-offs.

My weapon of choice as a scripting language is Perl. The reason for this is mainly historical, as I've been using it for years now. I can't even remember why I first picked it up, but would guess that I wanted to adapt something that almost-but-not-quite did what I wanted, and it happened to be in Perl.

My relationship with Perl is somewhat ambivalent. I find it an enormous sprawling mess of a language, both immensely powerful and willfully obtuse. In fairness I don't spend enough time with it to make it really sing, and tend to rely on the Camel, the Cookbook, and my collection of previous scripts to bash together what I need at any given time.

The Perl apostles trumpet its approach of There's More Than One Way To Do It (TMTOWTDI) as a strength, but I find it muddies the waters; ask three Perl coders to write some code, and they'll do it in at least six different ways.

However, there's no denying the power of the language. Most scripts boil down to relatively few lines of actual code, and are amazingly compact compared to the functionally equivalent C/C++ code.

There are of course scads of scripting languages; Ruby and Python seem to be very much à la mode. Just pick one, preferably one that somebody near your desk can help out with when you get stuck, and dive in.

Profound Advice: Do It Right the Third Time

2009-05-12T22:29:00.004+01:00

This is one of series of posts on the subject of profound advice, in which I metaphorically travel back in time to meet my younger self, and give that sad git some hard-earned pearls of wisdom. Which, knowing me, I will completely ignore. Or did ignore. Or something.

This week marks the 40th anniversary of the release of 'The Italian Job'. This most quintessentially British of all crime capers stars Michael Caine as Charlie Croker, modish wide boy with an eye for the main chance.

Noel Coward is magisterial as Mr. Bridger, the uber-royalist crime lord running the underworld from his prison cell.

Even Benny Hill is watchable as the lascivious computer expert, with a penchant for the larger lady, charged with bringing Turin to a standstill by overriding the traffic control system.

And of course it gave the world the essential Michael Caine quote, the one guaranteed to be invoked by generations of impressionists.

The 2003 remake with Mark Wahlberg was a perfectly acceptable crime movie, but in the UK at least was damned by association with the original - how dare anybody remake such a classic movie? My wife wouldn't even let me bring it into the house until last year.

It's something of a truism in engineering that the best time to design your solution to a given problem is just after the project's finished. At this point you've met the pitfalls and pratfalls awaiting you, you've worked around them, and you're fully aware of the implications of your early decisions. Knowing all this, and with the benefit of 20-20 hindsight, it would be relatively trivial to re-do the project.

A depressingly common approach to design is the big bang approach, AKA BUFD (big up-front design). Given a problem, you attempt to construct an all-singing, all-dancing solution that will stand forever as a testament to your genius. You try to build a framework that will solve not only your immediate problem, but a whole class of similar problems that may or may not arise in the future. Which is a fine and worthy goal, except that it's almost certainly a colossal waste of time and effort. It's certainly not shipping code. You don't know the future - if you did you would be betting on horses, not cranking code. So admit your temporal limitations and code for what you need right now, not what you may need in some conjectural future.

My first attempt at solving a problem tends to involve a lot of hardwired assumptions - e.g., the demo code only works if the microcontroller is at a specific address on the I2C bus, has a fixed memory map, and is running a specific version of the firmware.

Which works fine a surprising amount of the time. If I have to revisit the code - for instance, the firmware gets updated - I'll add a bit more intelligence to the code to handle this new situation.

Finally, if I have to revisit the code yet again, I'll make more of an effort to do things 'right'. At this point I've already solved the basic problem twice, so I've got a pretty good handle on how to do this. I know what works and what doesn't. I can devote some effort to developing a more generic solution with the full knowledge of what functionality is likely to actually be needed - because it was actually needed. I can develop a more modular code layout that lends itself to future maintenance work. If I'd tried to do this from the get-go, I would needlessly have locked myself into what I thought was the correct architecture on day one. And it would have taken a lot longer to build the generic-but-wrong solution than it took to build my hardwired-but-quick-to-get-going solution.

As usual I thought I had independently invented here a whole new concept that would revolutionise the world, a concept I termed 'do it right the third time'. I also toyed with the phrase 'speculative complexity', which pleases me, as it captures something of the unnecessary intricacy many people seem to find so attractive.

However, it turns out that lots of other people have had exactly the same idea; the extreme-ists call it 'you ain't gonna need it', which is admittedly much cooler.

Don't go for a big bang when you're only supposed to blow the bloody doors off.

The Tom (not Peters) Guide to Man Management

2009-05-07T21:20:00.003+01:00

Last weekend was a bank holiday here in the UK, and so, as is our wont, on Friday evening we hopped in the car and headed off to the New Forest to go camping. We're ideally positioned for this, close enough to get out into the heart of the forest in 30 or 40 minutes (as long as we can dodge the bank holiday traffic). It means we can do a full day at work, but be pitched up with the dinner on and the first beer (always the best) in hand by 8 or 9 o'clock.

One of our favourite haunts is a site called Tom's Field. This is a field, owned by a bloke called Tom - the clue's in the name. It's low key, the facilities are basic but clean and well-maintained, and Tom and the loose confederacy who seem to run the place are friendly and helpful. Plus, of course, it's in the New Forest, with easy access to loads of walking, cycling, and pub lunches.

We had an excellent weekend, and by Monday lunchtime were packing up ready to go home and start preparing for the week ahead. Along with everybody else on-site.

Now, Tom owns a thing called a Gator. It's a little quad bike/tractor thingy that he uses for moving stuff around the site. The sort of thing you see farmers and groundsmen using.

On the Monday morning Tom went round the site telling all of the kids that if they put all the rubbish in the bins provided by the gate, they could have a ride on the back of the Gator.

This is genius. Sheer genius.

With this simple act Tom unleashed a veritable Tsunami of wombles who swept across the site like locusts, picking it clean of every last scrap of litter. In return for ten minutes of taking kids for joyrides, Tom saved himself several hours of tedious work.

I'm not entirely sure how to link this little story back into coding, the nominal topic of this blog. Somebody with more perspicacity and insight than me could doubtless create a compelling analogy to empowering the inherent synergy in a cross-functional team to achieve a win-win scenario. To using the carrot rather than the stick. To envisaging the desired outcome of a project and setting in motion the train of events necessary to manifest that version of reality.

Me, I'm just impressed that Tom gets his field cleaned for him every weekend. Tom, I salute you.

The Most Profound Advice You Will Ever Receive

2009-04-26T21:36:00.007+01:00

Strangely enough, I received the most profound piece of advice I have ever heard in a pub out in the wilds of Wales, while on a stag do. We'd spent the day quad biking, shooting, and engaged in other equally testosterone-fuelled activities. We'd had our curries, and were sinking a few beers before the stripper-that-the-groom-didn't-know-about-but-fully-expected was due to arrive.

I got talking to one of the guys, and the talk turned to babies. My wife was pregnant, and I expressed some concerns about my ability to deal with the immense responsibility of caring for and raising a whole new human being.

"Don't worry about it." came the booze-fuelled reply from the father of three. "You already know everything you need to know. If they're hungry, feed them. If they're cold, warm them up. If they're too hot, cool them down. You'll be fine."

At that point an approving roar announced the arrival of a comely lass clad in a nurse's uniform, and we all lost interest in the conversation.

But the guy was absolutely right - common sense will take you a very long way as a new parent. And that it why I have no hesitation in awarding this particular piece of advice the most profound I have ever received.

---

Last year I was chatting to a new father-to-be, and he asked me for advice on dealing with a new baby, as he was freaking out about the responsibility etc etc as mentioned above. I furrowed my brow, drew myself up to my full height, pretended to be thinking thoughts of the utmost profundity, and passed on verbatim the advice given to me.

He didn't believe me, of course, but I hope that in the fullness of time he'll appreciate my astuteness.

This got me thinking about what advice I would give a newbie engineer, in the hypothetical situation where such a person were to ask me for advice on programming. I say 'hypothetical' because (a) nobody's ever asked me for advice, and (b) as a newbie, I was so self-confident and arrogant that I rarely asked for advice. I fully expect the new generation(s) of engineers to be exactly the same. Indeed, I found that the people who tried to give me advice were almost without exception fine instances of a combination of arrogance and incompetence. My favourite piece of expert advice is still that function prototypes are a waste of time, so don't bother with them. Absolutely priceless, you can't make this sort of thing up.

Instead I just wrote a whole bunch of dreadful code, slowly and painfully worked out what worked and what didn't, and over the years developed an approach that now serves me well.

So, if asked for advice, here is what I would say. These will form the basis of a series of blog articles over the next few weeks/months, depending on how often I find the time to sit down with my netbook and a few glasses of wine.

1. Use version control.
2. Use unit testing.
3. D o it right the third time.
4. A quick wrong decision is often better than a slow right decision.
5. When in doubt, use brute force.
6. Fail fast and hard.
7. Learn a scripting language.
8. Use static analysis.

Unconscious Programming

2009-04-12T21:21:00.001+01:00

If you've ever spent any time trying to learn a new skill you may have noticed that you go through a well-defined sequence of steps. Learning theorists tell us that this sequence is independent of the skill, i.e., it applies whether you're learning to play the piano, ski, paint, or whatever.

Having been through this loop a few times myself, the explanation I like best is laid out in Gestalt learning theory (aka the four stages of competence). This tells us that we go through the following steps when learning a skill (caveat: I was told this by a bloke in a pub, and haven't bothered to research it in any depth whatsoever, but I have found it to be true):

1. Unconscious incompetence. You're rubbish at your new skill, but are happily ignorant of the fact.
2. Conscious incompetence. Having banged away at it for a while, you start to appreciate the full depth and width of your ignorance and inability. I vividly remember listening to some Jimi Hendrix after having been going to guitar lessons for a year or so, and being completely blown away by how freakin' GOOD the guy was. Believe me, if you've never tried to play the guitar, you have no idea.
3. Conscious competence. You can do it, but you need to concentrate on it. It's a bit of an effort, but it's all starting to come together.
4. Unconscious competence. Everything's effortless. Doing everything properly is just part of your DNA, you've got to the point where doing it any way other than the simplest, most direct way is just an alien concept that makes no sense. I love to watch people at this level, whether playing instruments, turning wood, building a wall, or whatever, I have complete respect for anyone who's really good at what they do. Especially stuff I've had a go at, and so have some idea of how complicated what they're doing is.

Jeff Atwood recently blogged about the eight levels of programmers, with the Knuths et al of this world sat at the top of his pyramid. And rightly so. But I think there's a lot of mileage in the Gestalt model. I can see my arc of progress through it in a few areas I've dabbled in. I guess the main problem with it is that you can't really judge your own position in it. After all, how do you know when you push on through from unconscious incompetence to conscious incompetence? How do you know that you're fully appreciating how bad you are at [insert area of interest]? These are judgement calls that can only be made months or years after the fact.

I also have a sneaking feeling that the whole cycle is exactly that, a cycle, where you get to some level of mastery, and then start to realise that, actually, there's a whole new series of levels out there that you were previously unaware of. And at that point, you've just gone back through to conscious incompetence again.

I suspect that programming is a particularly fertile area for this sort of progression. There's a seemingly never-ending progression of bright and shiny new things to play with - say hello design patterns, extreme programming, generic programming, OOP, UML, ....

I know that I'm easily distracted by such baubles. After mainlining on the Gang of Four I spent at least a year in which my code was full of Singletons, Observers, Factorys, etc. None of which improved the code in any meaningful way, and in most cases actively made it harder to maintain for those following in my footsteps (mental note: must send a box of chocs to those poor sods). Which isn't to say that such things don't have a place, just that they should be in my code on merit rather than on a desire to play with some fancy notion I've read about.

Over the last few years my code has become noticeably leaner and simpler, and I think this is a good thing. I've also been working with resource-constrained microcontrollers for the last few years, rather than desktops which to all intents and purposes are infinitely fast and have infinite memory (I know, I know, I'm exaggerating for comic effect, but after you've spent a couple of days trying to save 100 bytes so that you can fit a program in a 4K chip, you get a totally different perspective on things).

I like to think that I passed through one level of unconscious competence (that I now think of as smug complacency) and am now nearing the end of another cycle. Which leaves me wondering where I'll go from here. What will I think of my current self in two years?

Not all assignments are equal: = != ==

2009-01-26T19:56:00.000Z

C, for all its brevity, is a minefield for the unwary.

One of the more irritating gotchas in the lexicon is that:

if( x = 10 )

is perfectly correct C, and will compile and run perfectly happily. The preprocessor will not squint at it, the compiler will look the other way, the linker will care not a jot. And yet it is almost certainly wrong, as it is an assignment rather than a test for equality (that is, it sets x to 10, rather than testing if x is already equal to 10).

What was probably intended here was:

if( x == 10 )

Grizzled veterans (by which I mean those who've been bitten often enough by this particular gotcha) avoid the whole issue by writing:

if( 10 == x )

This is a neat approach, as this code will not compile if you accidentally try to perform an assignment:

if( 10 = x ) // compiler error

This is a good sign in an interview candidate, as it displays both an awareness of the problem, the natural cunning to work around it, and the self-awareness to admit that they won't always get it right.

I still fall prey to this occasionally, despite being fully aware of the issue. Perhaps once every two years, while tracing some random weirdness in my code, I'll be stepping through the source in the debugger when I see the dreaded accidental assignment of doom. At this point I generally curse, make a mental note to kick the cat when I get home, and quickly fix the offending line before anyone can see it.

The problem is that the part of my brain that does maths rebels at the 'backwards' statement 'if 10 equals x'. It just looks ugly and wrong, despite being the 'smart' thing to do. I do actually do it, but I have to force myself to, and it always feels a bit clunky. And because it feels a bit clunky, I sometimes submit to my natural tendency to write it the other way round.

In my daily work I tend to rapidly context switch between C, C++, Delphi, and Perl. Delphi is particularly nasty in this respect, as in it '=' is the test for equality. I suspect that I generally make this mistake on switching from Delphi to C/C++.

Why You Should Spell Check Your Comments

2009-01-20T22:05:00.000Z

A couple of years ago we had our boiler replaced. We had the incumbent traditional boiler replaced with a combi boiler, which has behaved (more or less) impeccably ever since.

This is a fairly substantial operation, involving the removal of the previous boiler, hot water tank, expansion cistern, etc, and the installation of the new boiler itself. Our guy recommended putting the new boiler where the old hot water tank had been, in the airing cupboard in the bathroom.

I called our mate Gav to explain all of this. Now Gav is a top bloke; likes a good curry, and extremely partial to a real ale. But more pertinently, he's a plumber and gas fitter. Unfortunately he doesn't live locally or I'd have had him in to do the work like a shot. But needs must. Anyway, I explained what we were having done, and asked him for any pointers. Tips of the trade, that sort of thing. And he said I should check what the fitter did with the pipes leading to the old hot water tank. These pipes would now be unused, and could be cut back and capped. "He should cut them right back to the floor or ceiling", declared Gav. "They should be right out of the way."

Eh? These pipes would be completely unused, hidden right at the back of a cupboard where only we might ever see them, and even then, only if we went looking for them. What was the point of cutting them right back?

"It shows he knows what he's doing, and cares about how he does it. He wants his work to be neat and tidy, even if no-one will ever see it. It's the sign of a craftsman."

And sure enough, Mike the fitter cut the pipes right back. Even though there's no particular functional reason for doing so. And I still notice this every time I open the cupboard.

---

When I review some code for a colleague, the very first thing I do is run a spell check. On both the code and the comments. And every time, it throws up a whole bunch of misspellings.

Now, the tool chain doesn't give a monkey's about how the code and/or the comments are spelled. The pre-processor blithely rips out all of the comments, and as long as the engineer spells consistently, whether well or badly, the compiler will be perfectly happy with the source code. But I care, and you should too. I also care about grammar, and about code layout. Why?

Engineering is a global discipline. For many of my colleagues, English is a second or third language. I work for a global company, where people on pretty much any continent may pick up some code you've written. It's difficult enough to understand another person's code without making things even worse by mangling the English syntax as well as the code. Parsing that mess is just a waste of brain cycles.
It's indicative of a professional approach. It just smacks of quality. I know of no objective research to back me up, but experience tells me that well-spelled, carefully laid-out code has fewer bugs than sloppily written and maintained code. Of course, this doesn't mean that developers can improve their code by having a dictionary to hand while programming. It's just that there's a very strong correlation between spelling, layout, and quality. Someone who cares enough to take the time to spell their comments correctly is more likely to take the time to check their code for common gotchas such as array overruns. Go figure.
For me personally, it's as good a way as any of randomly vectoring into a fresh chunk of code. When a fresh glob of steaming source lands on my desk, I have to start dissecting it somehow. And when my spell check throws up a query, I go look at the offending file. As often as not I'll notice something suspect about the code in the immediate vicinity of the typo.

I don't religiously flag all typos. I recently came across a comment describing 'burst initialisation' which had somehow been transcribed as 'bust initialisation'. That was just such a beautiful concept that I had to leave it in.