Documentation

People say they hate writing documentation, but what they really hate is Word. And even Word would be okay if no one cared about formatting. Once you have to conform to these corporate styles things get so awkward - oh, you used 11 point font instead of 10, your margins are .05" off, you can't use a table here because it doesn't justify correctly. I've been in peer reviews where the only comments people have are formatting (and spelling errors). It's such an anti-pattern.

Wouldn't it be much better if documentation were like wikis? Where anyone can find the document they want to edit? Where all you need is a web browser to edit it and it's just text? Sure you have to use *'s instead of bullets maybe, or -'s or something, but have you looked lately how many different options you have for bullets in Word? It's insane. I'd rather have one ugly bullet.

So sure wikis are simplistic, but they're straightforward and you get to focus on writing instead of margins. But they don't work for real engineering, right? Real engineering documents are version-controlled, have complicated title pages, fancy diagrams and backgrounds that say things like 'UNCONTROLLED'. Wikis couldn't ever.... or could they?

Step one: version control. Github now has wikis. But Github doesn't just do wikis - anyone can do wikis. Github does version-controlled wikis. Wikis are written in text-based markup: typically Markdown, MediaWiki, etc. But they're all text - just text. Github saves each page you create in the wiki as a text file in a repository separate from the project you're working on. The only non-ideal thing about this whole setup is that all of the wiki pages are stored in one directory - no structure at all. So if you want to create a block diagram for a sub-assembly in a sub-directory you'll have to figure out how to store that information somewhere. I'm considering storing the directory information in the name somehow, but this may be a bit unwieldy.

So you'll end up with a bunch of text files with odd markup stored in a repository separate from your project. Surely there must be a way to take these text files, written with special markup, and turn them into something (dare I say) pretty? Well of course there is - Github takes the text files and creates web pages doesn't it? So yes, it can be done and it will be done. There's an open source program called Pandoc that describes itself as a swiss army knife for transforming markup formats. If you look at the list it can exchange between a lot of formats. Very neat, very useful. Now instead of text files you can get PDFs or... DocBook.

Now with the PDFs you get PDFs that look like nice, printable versions of web pages. Basic but serviceable. But engineering documents from real engineering companies don't just look serviceable - they look complicated. They're full of revision history blocks, referenced documents, government standards and the aforementioned 'UNCONTROLLED' backdrops. You can still do this in this approach but you need a lot more finesse. Enter DocBook. DocBook is used to create... books. You know all of those programming books with different animals on the front? Like this one? If my history is correct, they're all written in DocBook and in fact O'Reilly invented DocBook so they could write their books easier. That's why the all pretty much look the same. That and I guess those folks are boring.

The great thing about DocBook is that it's customizable. The input files are just XML, but the output is usually PDF - just print it off, bind it, draw a fish on the front and you've got a book. Or, if you want an engineering document, you describe some table layouts for revision history, title page, etc, fill out that information in your XML file, transform it and then you've got an engineering document. True, that will be a LOT of work, but so is trying to use Word to do the same thing. Best of all, Git is version control, so your revision history is built-in: you can parse Git commit logs to fill out the revision history section. If your referenced documents are in version control (which would be a good idea) then you can link right to them. And DocBook has all sorts of other neat features built in: automatic table of contents creation, automatic figure referencing with hotlinks, you name it. It's worth looking in to.

Text is great, yes, but we all scream for graphics. The Github wikis can reference documents from your project repository on the Github wiki, so including graphics in the online wiki is not really a problem, but what about on locally-produced PDFs? This might bet hairy. Pandoc has a different format for specifying image links than the Github wiki has. Luckily Pandoc is an open-source project so you can modify it to your heart's content if you so like. I might just figure out something else. So the workflow looks like this:

1. Draw your tables, graphics, etc in whatever program you use locally.
2. Use command-line tools (as part of a makefile) to export the local graphics to a GIF or JPG format so they can be included in your documentation.
3. Save the newly-exported graphics in a common area of your project repository.
4. Commit your changes to Github.
5. Write your documentation in a Github wiki and reference the graphics you just committed. This will produce easily-accessible online documentation.
6. Retrieve the wiki changes from Github to your local wiki repository.
7. Modify the local copies of the wikis to allow Pandoc to run on them seamlessly.
8. Run Pandoc on the wiki text files to create either PDF output or DocBook output and copy it to the correct place in your project repository directory structure
9. If you just want PDFs, you're done. If you created DocBook output then there will be another step to distill the DocBook to PDF after running it through all of your custom stylesheets.
10. Commit your changes and you're done

Tada! You have professional-looking PDF documentation derived from a wiki and various graphics. And what's great is that most of these steps are automated once you set up the makefiles. The only non-automated steps are actually writing the documentation, making the graphics and creating the stylesheets. Aren't you happy?

Tool Vs. Patterns

It's hiring time where I work - mostly new grads. That means lots of confusion. The disparity between what new grads expect and what actually happens in industry is sometimes wide.

For instance, my company might say 'We're looking for people with knowledge of VHDL to program FPGAs'. Plenty of students take a VHDL class or two - so you'd think that'd be a good fit, yes?

Yes... and no. The tool is not the end product. Whatever is made will be made with VHDL, true, but the end product will not materially depend on VHDL being the tool used to create it. In fact you could use Verilog to achieve the same, or even schematic capture. Or even step back a bit: digital logic is fundamentally NAND gates or NOR gates. You can make any digital device from NAND gates - everything from a half adder to an iPod. But you won't get a job anywhere just because you memorized the pinout of an SN7400 IC - the important part is what you're creating moreso than how. Fundamentally, being familiar with common patterns used in your field of choice is the mark of a good developer moreso than a strict adherence to syntax.

I'll illustrate with an anti-example that's more in my line. Let's talk C code. 'But it compiles!' has been long the defensive cry raised from many a young coder to defend their petty messes. Their unspoken assumption is that adherence to syntax is the sole qualification of a good C coder. This is patently untrue and a prime example is the anti-pattern of placing all of your code into main(). We've all done it - in fact it's the first thing you learn in class because they haven't taught you function calls yet. But is it a suggested practice? Not in the slightest. It produces code that's difficult to read, difficult to control (requiring the use of many gotos (warning: considered harmful) and just in general a pain. Of course the proper way to write code is the use of function calls to compartmentalize functionality and efficiently reuse it. The only people who deny this are defensive greenhorns more concerned about compiler errors than writing maintainable, readable and efficient code. Knowing syntax is a necessary condition for being a good programmer/engineer/developer, but certainly not a sufficient condition.

What about a straight example of why you should pay more attention to patterns than syntax? Something that looks awful but is brilliant, effective and clean. I present to you Duff's Device:

send(to, from, count)
register short *to, *from;
register count;
{
register n=(count+7)/8;
switch(count%8){
case 0: do{ *to = *from++;
case 7: *to = *from++;
case 6: *to = *from++;
case 5: *to = *from++;
case 4: *to = *from++;
case 3: *to = *from++;
case 2: *to = *from++;
case 1: *to = *from++;
}while(--n>0);
}
}

(Code formatting is always iffy on this blog so you'll have to forgive me.)

What manner of insanity is this? It's truly mind-breaking when you first look at it. So much so that when I show it to new grads/students the first words out of their mouths are more often than not 'Well that won't compile'.

And at this point I get an evil, evil smile on my face. They've shown their hand - more concerned with syntax than patterns. I assure them that it does and the response follows 'But it's not right!' Oh but it compiles! So it must be right, no? Oh the joy I have at this point - I've defeated a mere child in a battle of wits (Ok, I'm not a great person). None of them attempt to figure out what's going on - which is sad. Because once you understand the purpose and function of this code you recognize at least two patterns right off:

*to = *from++

This is your basic memory copy with a twist - the destination address isn't incremented. If you're really smart and had a focus on embedded systems you might realize that's because it's meant for a memory-mapped I/O register so the address won't change between writes.

switch(count%8){
case 0: do{ *to = *from++;
case 7: *to = *from++;
case 6: *to = *from++;
case 5: *to = *from++;
case 4: *to = *from++;
case 3: *to = *from++;
case 2: *to = *from++;
case 1: *to = *from++;

You might recognize this as loop unrolling. Loop unrolling is used to minimize the overhead incurred from jumping around in a loop: instead of doing one thing n times you do four things n/4 times. Useful.

Go ahead and read the full explanation of Duff's Device. If you can follow, you'll see why it's an amazing piece of code and why it vindicates the idea of patterns. Tom Duff understood assembly programming and knew how to implement loop unrolling in it - he was familiar with the pattern. He wanted to do it in C but couldn't think of a straightforward way so he abused the hell out of C syntax to make it work. It makes you want to cry for joy and in pain because it looks like that poor, efficient program is being tortured to death.

This is why patterns are more important than syntax. Patterns are like tools: if you're working too hard, chances are there's tool to help you be lazy. And new engineers usually work way too hard - most often at reinventing the wheel. They'll figure out some complicated method of ensuring single-access to a variable because no one taught them about semaphores. Oops. Patterns help us write efficient and correct code because they are distilled knowledge - the lessons of programmers before us given form. It pays great dividends to know about them.

That's what 'x years of y experience' on a resume means: I know a lot of patterns, I have a lot of tools in my toolbox, I won't make stupid mistakes because I'm past that. If knowing syntax is intelligence then knowing patterns is wisdom (go D&D!). If you don't have both you will never be as awesome a programmer as Tom Duff.

Offering Advice

I spend an inordinate amount of my free time on a site called Chiphacker. It's a Stack Overflow for electronics, embedded software and general EE nerdiness. Sometimes you get wonderful questions like this one:

Hey guys, I'm trying to properly bias the LM34476U chip (datasheet here) for operation in the saturated MODE. However, I want to be able to dynamically change the bias to on mode when my signal source changes. I've sketched out a design using the MAX4640AB 4:1 multiplexer that will switch out bias resistors but I'm getting a split second where both resistors are conducting and the feedback causes ringing in my output that I can't get rid of. Can anyone recommend methods to reduce the bandwidth of the LM3447 to eliminate the ringing? I've checked the app notes but the method they suggest there doesn't apply directly to my circuit topology and I don't know where to connect the feedback capacitor with my setup.

Wow, just wow. It's questions like that that I open up my circuits text book for.
Literally - check out this question here. I was even nice!

But sometimes there are questions asked that... spawn blog posts. That spawn stand up comedy routines ('Hey guys did you hear about the guy that asked a stupid question on the internet?'). Questions asked by people with obviously little knowledge of engineering, analog design, programming, digital logic, etc. Out of respect for the guilty I will not post one of the questions here but instead show you a creation of my own twisted mind that makes me approximately as angry as the real thing:

Hey guys, I've just had a great idea that I'd like to follow up on but I don't know where to start so I was hoping you could give me some pointers.
It seems to me that a lot of disabled people have problems with picking things up, so I want to build a robot that will roll around and pick up the things that they need picked up and you can control it with voice commands.
I've got a lot of software experience with VB6 and I've been reading up on the PIC so I think I can write the code for it when I find an implementation of the .NET framework for the PIC.
What I don't know about is the motors and stuff and the voice control maybe. I need the robot to roll around with wheels and its hand has to be just like a persons hand so it can pick everything up.
It also needs to know what the words for things are and it should probably understand if you point at something too.

My cousin is disabled and he has such a hard time... I really want to help him and think this is a good idea. Do any of you guys know a website or book that can help me figure this stuff out?

Thanks guys, I know together we can work this out!

This just makes me angry. Doesn't it make you angry? The first thing that pops up in my head is that this isn't really a question. If you had to distill it it wouldn't end up as 'Is this possible?' or 'What will be involved in doing this?' but instead 'Tell me how to do this'. It's like saying 'I want to get into Heaven - tell me how to do this.' Man, you'd be in for quite a discussion let me tell you....

But what are the specific ways I hate this question: Oh, let me count them:

Hey guys, I've just had a great idea that I'd like to follow up on but I don't know where to start so I was hoping you could give me some pointers.

First off, I hate it when you use any honorifics ('Dear sirs, I am many problems with this software having! For much to help please!') or really, any form of address. Don't call me 'guys', don't pretend like you know me unless you post regularly. Yes, I do check. Secondly the phrases 'I don't know where to start' and 'give me some pointers' indicate this person has no idea how to do what he wants to do. At all. Not even close. When the last time you had a great idea didn't you at least have some clue how it might be done? You think 'I want to know when I should stop pulling my car into the garage' and then immediately assume that this is a question you shouldn't even consider - that you should go consult a professional? Even if you start thinking about it and conclude you don't have enough specific knowledge to formulate a final solution that at least leaves you with some specific questions to ask. If for some reason you decided it needed to be electronic you'll quickly reach the question 'How can I measure distance electronically?' Then you ask THAT question to the professionals. Saying 'I don't know where to start' means you didn't bother to think through it even a little or you honestly don't understand what you want.

It seems to me that a lot of disabled people have problems with picking things up, so I want to build a robot that will roll around and pick up the things that they need picked up and you can control it with voice commands.

This boggles my mind. Do I want to help disabled people? Yes. Do I understand some of the issues that may be affecting them? Yes - apparently they 'have problems with picking things up'. Could I design an apparatus that might help disabled people pick things up - via voice commands? Perhaps - but it would take years. You see, I'm a simple embedded software engineer (before this I was a controls engineer - before that an electrical engineer and before that? Farmer) and the request you just made involves at least four different fields of study (Mechanical engineering, embedded software development, computer science with a specialty in artificial intelligence, digital signal processing, analog design - all of the top of my head) and would require someone with a college education in each. I'd probably need one PhD as well - just for good measure. It's not as if I could do wonderful things to help the disabled but I just get drunk instead. This is HARD.

I've got a lot of software experience with VB6 and I've been reading up on the PIC so I think I can write the code for it when I find an implementation of the .NET framework for the PIC.

Okay... this means you don't understand any of the potential tools you might be working with. Do you guys know what the PIC is? An 8-bit microcontroller that can't find its ass with both hands. Actually, scratch that - the PIC only implements one hand in hardware and requires you to emulate the other hand in software if you want to access it. This thing is dumb as sin. It most certainly doesn't have an implementation of .NET and even if it did the first thing he'd probably try to do is make 'Hello World' using console.writeline or whatever prissy function they use because they're not man enough to call it printf. You have to work with bits when you use this thing. Software people don't even know what bits are anymore - this guy is way out of his league.

What I don't know about is the motors and stuff and the voice control maybe. I need the robot to roll around with wheels and its hand has to be just like a persons hand so it can pick everything up.
It also needs to know what the words for things are and it should probably understand if you point at something too.

Oh okay, what you'll admit you don't know about is maybe 3/4's of the entire thing? So at the very least it's only ten years of school? Do people seriously think that these matters are trivial? Many legitimate college graduates are simply lost as far as practical skills in their first job and even then a mechanical engineer (for instance) isn't a machinist - he may be able to design the mechanical parts but he can't make them. You need someone who works with his hands and hopefully still has all of his fingers - that means he's good! And I'm not even going to get into how difficult it is to 'make it understand words'. Suffice it to say the most advanced processing hardware in the world takes years to learn a language and usually doesn't learn it correctly anyhow.

My cousin is disabled and he has such a hard time... I really want to help him and think this is a good idea. Do any of you guys know a website or book that can help me figure this stuff out?

Thanks guys, I know together we can work this out!

Oh of course - a book! Or website! All he needs is a website - one that tells him EXACTLY WHAT TO DO. Every step from A to Z to make this magical device that barely exists in his mind because he hasn't thought it through very much. Yes, I wrote that webpage - it's on Geocities. I dare you to find it. Face it - even if such a thing exists someone's selling it and they're not going to tell you how to make it. They're not going to document it completely and put it on the web for you to reproduce so you can not pay them anything. People like their effort to be rewarded - and this is effort

People will try to defend this person. 'But Angry', they'll start, 'he just wants to know if it's possible!' Screw that - everything is possible. Period. It all depends how much you want to pay. And I can't tell you how much it will cost. Do you realize that real companies have people who are paid - full time, real money - to estimate how much jobs will cost? And (call me cynical) when you tell him that the price for development of this miracle is in the millions he'll say 'I don't understand why it needs to cost so much!!! You can't be for real!' You see, he has already demonstrated that he lacks the ability and knowledge necessary to judge the difficulty of the ideas he presents. Remember that as far as he's concerned he simply hasn't found the right website to tell him how to do it. After that it's easy.

'He just wants to know what it will take!'

Seriously? Remember how he can't tell whether he's asking for something reasonable or something insane? When I tell him it's something insane guess what? That's not the answer he's looking for. He has every reason to ignore me and perhaps ridicule me. I can't say for sure whether he has a rigid mindset that won't listen to reason but I am saying for certain he has no facility to judge whether something is reasonable or not. So why even ask? Why not ask 'Am I way off base here?' instead of 'Show me how to do this'.

'You should at least be nice to him'

Well... I could. It's true. I don't have to do anything - so many questions simply remain unanswered, bereft and eventually die in obscurity. It feels so fulfilling to see no answers, no comments, no nothing on a question months after it's been asked. But I have my limits. I am the Angry EE for a reason. And that reason is that some things I just can't let go. On the internet anyhow - where all I have to do is type. Otherwise? Waaaay too lazy. But I will not refrain from essentially answering his question in the only way one paragraph can. I am not on retainer for more than one paragraph - no one on stackexchange sites should be. It will be pithy, it will be a little acerbic, but it will be right. If you choose not to believe it, I will follow up with comments that make fun of you more and more.

The moral of the story is to do work for yourself. Do some, any! Then you can ask more specific questions - questions which I will be less hesitant to answer than blanket questions about topics you obviously don't understand and haven't thought about. I answer lots of questions on this site and there's plenty of good ones which means that plenty of people don't expect me to do their homework for them. I like those people. Yeah, don't expect me to do your homework. Homey don't play dat.

(Does anyone get that reference?)