The Angry EE

Labview is Actually Not All That Bad

2011-12-14T08:31:00.000-08:00

In stark contrast to the flurry of epithets I hurl at Labview on a daily basis at work I will now say good things about it - because it deserves it!

The blocks are actually pretty old school. If you get stuck trying to decode a string in Labview your tools look surprisingly like what you'd expect from the C standard library. Granted - you don't need to worry about NULL termination or anything, but most of the basic building blocks (like string subset, search and replace, etc) promote code that acts (in form) like old-style C code. It's not like writing the routines yourself (iterating through each character, comparing, discarding, buffering, etc.) but the solution you come up with to manipulate strings will probably feel like an old-style C solution does. I also like the way the control structures work - you get nice features like shift registers (that teach you about instantiating a variable before the loop) or straight-through tunnels that just take the last value from the loop and pass it out. You can make complicated while or for loops with conditional terminations or conditional continuances. These are very welcome forms to have present. There's a lot of nice tools and abilities in Labview that don't feel new and gimmicky, but instead old and tried.

Second, while most Labview VIs are just a mess plain and simple what I've found is that a messy VI usually means messy badly-organized code. A clean, well-organized VI means you've created sub-VIs in the right places, organized the right data into clusters, used the right type of loop, etc. If your VI is clean and organized, chances are your code is too. Thus, by seeking to visually clean up your block diagram you can actually write good code. Of course, the opposite is not true: you can have great code that still looks like a mess. Because you just can't get around that in Labview sometimes. But chances are if the block diagram looks good the reason is because of a highly-organized coding mind behind it. It helps people learn how to organize code by presenting it as a visual problem instead of an esoteric abstract one - and many people are simply visual learners.

Third, I'm ecstatic as hell that whenever I Google anything related to Labview I come up with an answer - period. Someone has tried to do what I'm doing or for some insane reason National Instruments make a guide on how to manipulate TestStand sequences programmatically from Labview. Quite simply put I spent hours trying to figure out a problem on my own and when I get the bright idea to Google it the answers is invariably THERE. Done in five minutes. At this point in my career I'm a little more focused on results than banging my head against a wall to 'learn' so that feels really good. I have no idea why all of this support is out there but it is and it makes me happy.

Fourth, Labview is actually free of a lot of the object-oriented crap that plagues many trendy languages today. Yes, Labview is actually a bit trendy by itself, but (and this impresses me because the more I think about it the more I realize it's true) Labview is actually pretty old-school. Some old-school hard core stuff you can't do (like function pointers - but then only kinda) but I'm pleased that I haven't seen the term 'inheritance' once when dealing with Labview. True, all the things that make me hate C++ might be in there but I haven't been forced to deal with it and I haven't seen anyone else's work that deals with it either.

Don't get me wrong - I still find plenty to hate about it (and I may get to that later) but the more I consider it Labview feels like C but graphical. And that's not terrible.

Good Coding Practices #...?

2011-10-05T19:01:00.000-07:00

I have a semi-ongoing series in good coding practices... to the extent that I ever update my blog anyhow. Lately I've taken up work on an iTunes Android Sync tool. Is anyone else amazed that there are very few good programs that will sync iTunes playlists to an Android device? There are some out there, certainly but for some reason each of them tends to have one or two major flaws: way too slow, randomly renames your songs to the titles of different songs, costs money - the usual complaints. So I looked into it and it turns out that iTunes maintains an XML version of its library information file. But then I discovered that the Android playlists are stored in a highly technical format called ASCII-encoded text files. Let me tell you, it took forever to crack that puppy.

Well when all you have is a hammer every problem looks like a nail. When you have Python every problem looks... easy. So I decided to make a Python program to:

1) Read the XML library file.
2) Figure out what playlists are in there
3) Figure out what songs are in those playlists
4) Figure out where those songs are
5) Make Android playlists from the iTunes Playlists
6) Copy the playlists and music files to the Android device
7) DANCE!

So at first I tried using my favorite parser - SGML parser. But it turned out that SGML parser doesn't handle truncated tags. You know - the ones with nothing in them? With only a start tag that has a / in it and then it's done? Yeah, those. So I had to switch to expat which isn't so bad either.

But enough of that! I'm going to show you what I did that's a good coding practice. The iTunes XML file has several parts in it: a general section that describes the library, a tracks section that describes each track and assigns it a unique ID, a playlists section that describes the playlist and lists the unique track IDs in the playlist.

I wanted to start off by parsing all the goodness of the general library section and ignore the rest while at the same time planning ahead so I would... be able to figure out where to put the code to parse the rest of it as well. To that end I present a random code snippet:

def handle_data(self,text):

if self.current_tag == KEY_TAG:
self.current_key = str(text)
print "Key: " + text
elif self.current_tag == INTEGER_TAG or self.current_tag == STRING_TAG or self.current_tag == DATE_TAG or self.current_tag==TRUE_TAG:
if self.current_parent == LIBRARY_KEY:
if self.current_key in libraryKeyList:
print self.current_tag + "=" + text
self.tempDict[self.current_key] = text

elif self.current_parent == TRACKS_KEY:
pass
elif self.current_parent == PLAYLISTS_KEY:
pass
elif self.current_parent == TRACK_KEY:
pass
elif self.current_parent == PLAYLIST_KEY:
pass
else:
pass

self.current_key = ""

Some explanation: this function handles data inside of tags. It handles key tags specially, but handles tags that contain data (integer, string, date, etc) differently still depending on which section they reside in. So you can see I've written the code that handles the data in the library section but left out handling data in all the other sections. But this is by design: if I wasn't planning ahead I wouldn't have put the if statement that checks what the parent is in that function. I would just have put the code that handles data for the library section without verifying that I was still in the library section - and then it would have handled a whole lot of data in the rest of the file.

By putting the parent key check in there and explicitly listing the different situations that I want to code for I'm doing two things. First, I'm specifying the exact situation I expect this code to run in - putting my assumptions right out there in the code. Second I specify all of the other situations that I haven't yet coded for but want to in the future. I'm using the code to inform myself (in the future) that I need to put code there that does something different. That's the good coding standard.

It can be used in a variety of languages. In Python use the above form but make sure that you put the pass statement in an empty case - otherwise it gets angry. In C you can use #warning directives to produce a warning when you know you'll have to write some code but just haven't yet. Like '#warning Will Robinson, you didn't handle the default case!'

On Excellence

2011-07-03T07:25:00.001-07:00

I fancy myself pretty good at C. Not great, but pretty good. I can find my way around source code, I can write from scratch, I can debug with the most average of them. I'm handy in a variety of ways and I eat source code for breakfast.

This didn't happen in college. This didn't happen in high school. This didn't happen in freaking grade school. I have been programming since I was 6. I started off with AppleSoft BASIC on an Apple IIc knockoff (Laser 128c was the correct answer for those of you playing the home game). After AppleSoft BASIC it was GW-BASIC on the 8088 and QuickBASIC on the 286 and up. But roundabouts high school I decided I had to learn C - because that was the language that grownups used. So I bought a C book (the right one as it turns out - if you want to learn C get this book first), downloaded DJGPP and got to work!

I'm having a computer weekend (putting a computer whose hard drive failed back into working order) so I'm going through old files looking for utilities and just plain reminiscing. I decided to see what my old C code looked like.

Oh God, it's awful.

It's terrible. Here's an example (with some helpful comments from future Steve):

i=-1;

fseek (readfile,0,SEEK_END); //Set starting point to end
size = ftell(readfile); //Find file size
fseek (readfile,0,SEEK_SET); //Set starting point to start

//FS - Seriously? Is there no better way of getting the size of a file?

//FS - Oh god, who gave me malloc?

readfiledata = malloc(size); //Allocate memory for char

printf("Filename is %s\n", argv[1]);
printf("Size is %d\n", size);
printf("Copied %s to steve.tmp\n", argv[1]);

//FS - WHAT?! WHAT?! Index of -1?

i=-1;

do
{
//FS - SERIOUSLY!? Is there no easier way to get all the data in an array? Did you not look?

readfiledata[i] = fgetc(readfile); //FS - OH GOD YOU ACCESSED INDEX -1 OF AN ARRAY!
i++;
}
while(!feof(readfile));

In case you were wondering, I used malloc() and, no, there is no corresponding free() call. I relied on the fact that the OS would free the memory once the program exited. There were variables defined in .h files (no, not extern defines, plain old defines). There were what should have been arrays of constant strings were 'initialized' using sprintf (copy constant to string) rather than just initializing them when the array was defined (as any normal person would do).

And the best part is that the whole program I made basically amounted to a regular expressions parser. All I needed to do was remove images and other formatting from HTML files so they'd be easier to print off and use less ink. That could have been done a lot easier.

This was 1999. So, becoming average takes at least 12 years of constant use of a skill - and I still screw up. Looking at this I can see a lot of myself in new grads coming out of college - the same mistakes, the same assumptions, the same basic design assumptions that end up making bad code (even if it runs). It falls in line with Malcom Gladwell as he writes in Outliers: if you do something for 10,000 hours you'll be great at it. It's not necessarily innate skill, it's practice, practice, practice. It's why I'm a professional programmer and not a professional trombone player - I just code a whole lot more than I play trombone.

So knowing this I can see where new grads come from - hey, they haven't had 10,000 hours of programming, they probably haven't had 10,000 hours of anything engineering related from their college experience. 10,000 hours is 3.5 years at 8 hours a day and that's just for one skill. Engineering is a whole plethora and if you don't know where your career is taking you, why bother practicing one skill over another?

That's my strawman argument - I don't agree with it. My question is - if you know you won't have you 10,000 hours in whatever you want to be good at by the time you graduate college (and by extension, be able to show some really awesome work to a prospective employer) why didn't you start earlier? Did you plan to be average? To be right in the middle of the pack, to not stand out? To be, essentially, replaceable by any other member of your graduating class? Did you plan to go out into the job market and have a big corporation tell you what you should be good at instead of deciding it yourself? Didn't you get into engineering for a reason?

I see people on both sides of this question and you can tell them apart right away in an interview. The people who don't know why they're engineers - the ones who didn't start early excelling at something they loved and wanted to do - come to a job interview and basically want you to tell them what kind of career they should have. They hit the middle of the road for all of their classes - probably picked whatever electives were the most popular because they didn't really care about the difference, didn't have an opinion on what they wanted, didn't find anything particularly exciting and just followed their friends into a class. Their senior projects were whatever they were assigned and they just sort of did them. They don't speak about them with passion, they just wanted to graduate and they needed a project, so they did it. And they're not dumb - a lot of them have 4.0 GPAs for what its worth. But since they didn't know what they wanted to do they never got in-depth on anything. They never really put together the pieces that every single topic in electrical engineering is inexorably linked to every other. Analog circuits mean differential equations, considerations of bandwidth, frequency response, frequency content of waveforms, Fourier Series, linear algebra, matrix equations and any other number of fields of study. The coursework isn't a checklist, it's a symphony of learning. But if you don't have passion it is just a checklist. Mom and dad want you to be an engineer. You're smart, so you do well in classes and you graduate with a high GPA. You go for a job at a big corporation and they grind you into whatever kind of employee/engineer they want. Yay for you - you're average.

But the ones who have passion and drive and love what they've done - they stand out immediately as well. They had a definite plan when they went to college. They'll tell you how they took apart TVs when they were a kid (good Lord it's dangerous - let your kids do it but make sure those capacitors are discharged) or how they wrote dumb little computer games in Visual Basic to entertain themselves. They just won't shut up about their senior project or whatever personal projects they have if they haven't started their senior project. Their eyes light up when you suggest ways they could improve their project ('Ooooh my God... I wish we could go back and work on that more - I'd make it so much better!') or they kept working on it themselves after they graduated. They've learned weird programming languages for fun. In essence, they love what they do and they just won't stop doing it. They don't ask you for direction, they tell you - I'm this kind of engineer and I love doing it, do you need me? And the answer is usually yes, we need you.

So essentially the choice is yours: You can be average or excellent. There is certainly a long road between the two, but you have the choice to take the journey and practice practice practice. And if you find out that whatever you had your eye on doesn't really interest you then fine - move your target, pick something else. Excel at something. Hell, maybe you'll have 10,000 hours of random junk you've practiced. That's okay - it makes you an excellent generalist. Don't just sit around and play video games - ply your craft. I guarantee there's a payoff even though it's a long way down the road. Yes, a very long way. But it's worth it.

Fathers Day

2011-06-20T07:26:00.000-07:00

I just read a nice article about someone whose father taught him how to build, use tools, make things. Reading that, it occurs to me that I've never heard a 'My father taught my how to code and I'm grateful for it.' story. It seems like everyone who codes well has been mostly self-taught - a loner. I wouldn't have wanted it then, but looking back now I would have liked to have been taught something so wonderful by my father. I hope we have these stories someday.

That is all.

Documentation

2011-05-25T18:37:00.001-07:00

People say they hate writing documentation, but what they really hate is Word. And even Word would be okay if no one cared about formatting. Once you have to conform to these corporate styles things get so awkward - oh, you used 11 point font instead of 10, your margins are .05" off, you can't use a table here because it doesn't justify correctly. I've been in peer reviews where the only comments people have are formatting (and spelling errors). It's such an anti-pattern.

Wouldn't it be much better if documentation were like wikis? Where anyone can find the document they want to edit? Where all you need is a web browser to edit it and it's just text? Sure you have to use *'s instead of bullets maybe, or -'s or something, but have you looked lately how many different options you have for bullets in Word? It's insane. I'd rather have one ugly bullet.

So sure wikis are simplistic, but they're straightforward and you get to focus on writing instead of margins. But they don't work for real engineering, right? Real engineering documents are version-controlled, have complicated title pages, fancy diagrams and backgrounds that say things like 'UNCONTROLLED'. Wikis couldn't ever.... or could they?

Step one: version control. Github now has wikis. But Github doesn't just do wikis - anyone can do wikis. Github does version-controlled wikis. Wikis are written in text-based markup: typically Markdown, MediaWiki, etc. But they're all text - just text. Github saves each page you create in the wiki as a text file in a repository separate from the project you're working on. The only non-ideal thing about this whole setup is that all of the wiki pages are stored in one directory - no structure at all. So if you want to create a block diagram for a sub-assembly in a sub-directory you'll have to figure out how to store that information somewhere. I'm considering storing the directory information in the name somehow, but this may be a bit unwieldy.

So you'll end up with a bunch of text files with odd markup stored in a repository separate from your project. Surely there must be a way to take these text files, written with special markup, and turn them into something (dare I say) pretty? Well of course there is - Github takes the text files and creates web pages doesn't it? So yes, it can be done and it will be done. There's an open source program called Pandoc that describes itself as a swiss army knife for transforming markup formats. If you look at the list it can exchange between a lot of formats. Very neat, very useful. Now instead of text files you can get PDFs or... DocBook.

Now with the PDFs you get PDFs that look like nice, printable versions of web pages. Basic but serviceable. But engineering documents from real engineering companies don't just look serviceable - they look complicated. They're full of revision history blocks, referenced documents, government standards and the aforementioned 'UNCONTROLLED' backdrops. You can still do this in this approach but you need a lot more finesse. Enter DocBook. DocBook is used to create... books. You know all of those programming books with different animals on the front? Like this one? If my history is correct, they're all written in DocBook and in fact O'Reilly invented DocBook so they could write their books easier. That's why the all pretty much look the same. That and I guess those folks are boring.

The great thing about DocBook is that it's customizable. The input files are just XML, but the output is usually PDF - just print it off, bind it, draw a fish on the front and you've got a book. Or, if you want an engineering document, you describe some table layouts for revision history, title page, etc, fill out that information in your XML file, transform it and then you've got an engineering document. True, that will be a LOT of work, but so is trying to use Word to do the same thing. Best of all, Git is version control, so your revision history is built-in: you can parse Git commit logs to fill out the revision history section. If your referenced documents are in version control (which would be a good idea) then you can link right to them. And DocBook has all sorts of other neat features built in: automatic table of contents creation, automatic figure referencing with hotlinks, you name it. It's worth looking in to.

Text is great, yes, but we all scream for graphics. The Github wikis can reference documents from your project repository on the Github wiki, so including graphics in the online wiki is not really a problem, but what about on locally-produced PDFs? This might bet hairy. Pandoc has a different format for specifying image links than the Github wiki has. Luckily Pandoc is an open-source project so you can modify it to your heart's content if you so like. I might just figure out something else. So the workflow looks like this:

Draw your tables, graphics, etc in whatever program you use locally.
Use command-line tools (as part of a makefile) to export the local graphics to a GIF or JPG format so they can be included in your documentation.
Save the newly-exported graphics in a common area of your project repository.
Commit your changes to Github.
Write your documentation in a Github wiki and reference the graphics you just committed. This will produce easily-accessible online documentation.
Retrieve the wiki changes from Github to your local wiki repository.
Modify the local copies of the wikis to allow Pandoc to run on them seamlessly.
Run Pandoc on the wiki text files to create either PDF output or DocBook output and copy it to the correct place in your project repository directory structure
If you just want PDFs, you're done. If you created DocBook output then there will be another step to distill the DocBook to PDF after running it through all of your custom stylesheets.
Commit your changes and you're done

Tada! You have professional-looking PDF documentation derived from a wiki and various graphics. And what's great is that most of these steps are automated once you set up the makefiles. The only non-automated steps are actually writing the documentation, making the graphics and creating the stylesheets. Aren't you happy?

Tool Vs. Patterns

2011-05-18T04:52:00.001-07:00

It's hiring time where I work - mostly new grads. That means lots of confusion. The disparity between what new grads expect and what actually happens in industry is sometimes wide.

For instance, my company might say 'We're looking for people with knowledge of VHDL to program FPGAs'. Plenty of students take a VHDL class or two - so you'd think that'd be a good fit, yes?

Yes... and no. The tool is not the end product. Whatever is made will be made with VHDL, true, but the end product will not materially depend on VHDL being the tool used to create it. In fact you could use Verilog to achieve the same, or even schematic capture. Or even step back a bit: digital logic is fundamentally NAND gates or NOR gates. You can make any digital device from NAND gates - everything from a half adder to an iPod. But you won't get a job anywhere just because you memorized the pinout of an SN7400 IC - the important part is what you're creating moreso than how. Fundamentally, being familiar with common patterns used in your field of choice is the mark of a good developer moreso than a strict adherence to syntax.

I'll illustrate with an anti-example that's more in my line. Let's talk C code. 'But it compiles!' has been long the defensive cry raised from many a young coder to defend their petty messes. Their unspoken assumption is that adherence to syntax is the sole qualification of a good C coder. This is patently untrue and a prime example is the anti-pattern of placing all of your code into main(). We've all done it - in fact it's the first thing you learn in class because they haven't taught you function calls yet. But is it a suggested practice? Not in the slightest. It produces code that's difficult to read, difficult to control (requiring the use of many gotos (warning: considered harmful) and just in general a pain. Of course the proper way to write code is the use of function calls to compartmentalize functionality and efficiently reuse it. The only people who deny this are defensive greenhorns more concerned about compiler errors than writing maintainable, readable and efficient code. Knowing syntax is a necessary condition for being a good programmer/engineer/developer, but certainly not a sufficient condition.

What about a straight example of why you should pay more attention to patterns than syntax? Something that looks awful but is brilliant, effective and clean. I present to you Duff's Device:

send(to, from, count)
register short *to, *from;
register count;
{
register n=(count+7)/8;
switch(count%8){
case 0: do{ *to = *from++;
case 7: *to = *from++;
case 6: *to = *from++;
case 5: *to = *from++;
case 4: *to = *from++;
case 3: *to = *from++;
case 2: *to = *from++;
case 1: *to = *from++;
}while(--n>0);
}
}

(Code formatting is always iffy on this blog so you'll have to forgive me.)

What manner of insanity is this? It's truly mind-breaking when you first look at it. So much so that when I show it to new grads/students the first words out of their mouths are more often than not 'Well that won't compile'.

And at this point I get an evil, evil smile on my face. They've shown their hand - more concerned with syntax than patterns. I assure them that it does and the response follows 'But it's not right!' Oh but it compiles! So it must be right, no? Oh the joy I have at this point - I've defeated a mere child in a battle of wits (Ok, I'm not a great person). None of them attempt to figure out what's going on - which is sad. Because once you understand the purpose and function of this code you recognize at least two patterns right off:

*to = *from++

This is your basic memory copy with a twist - the destination address isn't incremented. If you're really smart and had a focus on embedded systems you might realize that's because it's meant for a memory-mapped I/O register so the address won't change between writes.

switch(count%8){
case 0: do{ *to = *from++;
case 7: *to = *from++;
case 6: *to = *from++;
case 5: *to = *from++;
case 4: *to = *from++;
case 3: *to = *from++;
case 2: *to = *from++;
case 1: *to = *from++;

You might recognize this as loop unrolling. Loop unrolling is used to minimize the overhead incurred from jumping around in a loop: instead of doing one thing n times you do four things n/4 times. Useful.

Go ahead and read the full explanation of Duff's Device. If you can follow, you'll see why it's an amazing piece of code and why it vindicates the idea of patterns. Tom Duff understood assembly programming and knew how to implement loop unrolling in it - he was familiar with the pattern. He wanted to do it in C but couldn't think of a straightforward way so he abused the hell out of C syntax to make it work. It makes you want to cry for joy and in pain because it looks like that poor, efficient program is being tortured to death.

This is why patterns are more important than syntax. Patterns are like tools: if you're working too hard, chances are there's tool to help you be lazy. And new engineers usually work way too hard - most often at reinventing the wheel. They'll figure out some complicated method of ensuring single-access to a variable because no one taught them about semaphores. Oops. Patterns help us write efficient and correct code because they are distilled knowledge - the lessons of programmers before us given form. It pays great dividends to know about them.

That's what 'x years of y experience' on a resume means: I know a lot of patterns, I have a lot of tools in my toolbox, I won't make stupid mistakes because I'm past that. If knowing syntax is intelligence then knowing patterns is wisdom (go D&D!). If you don't have both you will never be as awesome a programmer as Tom Duff.

Offering Advice

2011-05-02T18:27:00.000-07:00

I spend an inordinate amount of my free time on a site called Chiphacker. It's a Stack Overflow for electronics, embedded software and general EE nerdiness. Sometimes you get wonderful questions like this one:

Hey guys, I'm trying to properly bias the LM34476U chip (datasheet here) for operation in the saturated MODE. However, I want to be able to dynamically change the bias to on mode when my signal source changes. I've sketched out a design using the MAX4640AB 4:1 multiplexer that will switch out bias resistors but I'm getting a split second where both resistors are conducting and the feedback causes ringing in my output that I can't get rid of. Can anyone recommend methods to reduce the bandwidth of the LM3447 to eliminate the ringing? I've checked the app notes but the method they suggest there doesn't apply directly to my circuit topology and I don't know where to connect the feedback capacitor with my setup.

Wow, just wow. It's questions like that that I open up my circuits text book for.
Literally - check out this question here. I was even nice!

But sometimes there are questions asked that... spawn blog posts. That spawn stand up comedy routines ('Hey guys did you hear about the guy that asked a stupid question on the internet?'). Questions asked by people with obviously little knowledge of engineering, analog design, programming, digital logic, etc. Out of respect for the guilty I will not post one of the questions here but instead show you a creation of my own twisted mind that makes me approximately as angry as the real thing:

Hey guys, I've just had a great idea that I'd like to follow up on but I don't know where to start so I was hoping you could give me some pointers.
It seems to me that a lot of disabled people have problems with picking things up, so I want to build a robot that will roll around and pick up the things that they need picked up and you can control it with voice commands.
I've got a lot of software experience with VB6 and I've been reading up on the PIC so I think I can write the code for it when I find an implementation of the .NET framework for the PIC.
What I don't know about is the motors and stuff and the voice control maybe. I need the robot to roll around with wheels and its hand has to be just like a persons hand so it can pick everything up.
It also needs to know what the words for things are and it should probably understand if you point at something too.

My cousin is disabled and he has such a hard time... I really want to help him and think this is a good idea. Do any of you guys know a website or book that can help me figure this stuff out?

Thanks guys, I know together we can work this out!

This just makes me angry. Doesn't it make you angry? The first thing that pops up in my head is that this isn't really a question. If you had to distill it it wouldn't end up as 'Is this possible?' or 'What will be involved in doing this?' but instead 'Tell me how to do this'. It's like saying 'I want to get into Heaven - tell me how to do this.' Man, you'd be in for quite a discussion let me tell you....

But what are the specific ways I hate this question: Oh, let me count them:

Hey guys, I've just had a great idea that I'd like to follow up on but I don't know where to start so I was hoping you could give me some pointers.

First off, I hate it when you use any honorifics ('Dear sirs, I am many problems with this software having! For much to help please!') or really, any form of address. Don't call me 'guys', don't pretend like you know me unless you post regularly. Yes, I do check. Secondly the phrases 'I don't know where to start' and 'give me some pointers' indicate this person has no idea how to do what he wants to do. At all. Not even close. When the last time you had a great idea didn't you at least have some clue how it might be done? You think 'I want to know when I should stop pulling my car into the garage' and then immediately assume that this is a question you shouldn't even consider - that you should go consult a professional? Even if you start thinking about it and conclude you don't have enough specific knowledge to formulate a final solution that at least leaves you with some specific questions to ask. If for some reason you decided it needed to be electronic you'll quickly reach the question 'How can I measure distance electronically?' Then you ask THAT question to the professionals. Saying 'I don't know where to start' means you didn't bother to think through it even a little or you honestly don't understand what you want.

It seems to me that a lot of disabled people have problems with picking things up, so I want to build a robot that will roll around and pick up the things that they need picked up and you can control it with voice commands.

This boggles my mind. Do I want to help disabled people? Yes. Do I understand some of the issues that may be affecting them? Yes - apparently they 'have problems with picking things up'. Could I design an apparatus that might help disabled people pick things up - via voice commands? Perhaps - but it would take years. You see, I'm a simple embedded software engineer (before this I was a controls engineer - before that an electrical engineer and before that? Farmer) and the request you just made involves at least four different fields of study (Mechanical engineering, embedded software development, computer science with a specialty in artificial intelligence, digital signal processing, analog design - all of the top of my head) and would require someone with a college education in each. I'd probably need one PhD as well - just for good measure. It's not as if I could do wonderful things to help the disabled but I just get drunk instead. This is HARD.

I've got a lot of software experience with VB6 and I've been reading up on the PIC so I think I can write the code for it when I find an implementation of the .NET framework for the PIC.

Okay... this means you don't understand any of the potential tools you might be working with. Do you guys know what the PIC is? An 8-bit microcontroller that can't find its ass with both hands. Actually, scratch that - the PIC only implements one hand in hardware and requires you to emulate the other hand in software if you want to access it. This thing is dumb as sin. It most certainly doesn't have an implementation of .NET and even if it did the first thing he'd probably try to do is make 'Hello World' using console.writeline or whatever prissy function they use because they're not man enough to call it printf. You have to work with bits when you use this thing. Software people don't even know what bits are anymore - this guy is way out of his league.

What I don't know about is the motors and stuff and the voice control maybe. I need the robot to roll around with wheels and its hand has to be just like a persons hand so it can pick everything up.
It also needs to know what the words for things are and it should probably understand if you point at something too.

Oh okay, what you'll admit you don't know about is maybe 3/4's of the entire thing? So at the very least it's only ten years of school? Do people seriously think that these matters are trivial? Many legitimate college graduates are simply lost as far as practical skills in their first job and even then a mechanical engineer (for instance) isn't a machinist - he may be able to design the mechanical parts but he can't make them. You need someone who works with his hands and hopefully still has all of his fingers - that means he's good! And I'm not even going to get into how difficult it is to 'make it understand words'. Suffice it to say the most advanced processing hardware in the world takes years to learn a language and usually doesn't learn it correctly anyhow.

My cousin is disabled and he has such a hard time... I really want to help him and think this is a good idea. Do any of you guys know a website or book that can help me figure this stuff out?

Thanks guys, I know together we can work this out!

Oh of course - a book! Or website! All he needs is a website - one that tells him EXACTLY WHAT TO DO. Every step from A to Z to make this magical device that barely exists in his mind because he hasn't thought it through very much. Yes, I wrote that webpage - it's on Geocities. I dare you to find it. Face it - even if such a thing exists someone's selling it and they're not going to tell you how to make it. They're not going to document it completely and put it on the web for you to reproduce so you can not pay them anything. People like their effort to be rewarded - and this is effort

People will try to defend this person. 'But Angry', they'll start, 'he just wants to know if it's possible!' Screw that - everything is possible. Period. It all depends how much you want to pay. And I can't tell you how much it will cost. Do you realize that real companies have people who are paid - full time, real money - to estimate how much jobs will cost? And (call me cynical) when you tell him that the price for development of this miracle is in the millions he'll say 'I don't understand why it needs to cost so much!!! You can't be for real!' You see, he has already demonstrated that he lacks the ability and knowledge necessary to judge the difficulty of the ideas he presents. Remember that as far as he's concerned he simply hasn't found the right website to tell him how to do it. After that it's easy.

'He just wants to know what it will take!'

Seriously? Remember how he can't tell whether he's asking for something reasonable or something insane? When I tell him it's something insane guess what? That's not the answer he's looking for. He has every reason to ignore me and perhaps ridicule me. I can't say for sure whether he has a rigid mindset that won't listen to reason but I am saying for certain he has no facility to judge whether something is reasonable or not. So why even ask? Why not ask 'Am I way off base here?' instead of 'Show me how to do this'.

'You should at least be nice to him'

Well... I could. It's true. I don't have to do anything - so many questions simply remain unanswered, bereft and eventually die in obscurity. It feels so fulfilling to see no answers, no comments, no nothing on a question months after it's been asked. But I have my limits. I am the Angry EE for a reason. And that reason is that some things I just can't let go. On the internet anyhow - where all I have to do is type. Otherwise? Waaaay too lazy. But I will not refrain from essentially answering his question in the only way one paragraph can. I am not on retainer for more than one paragraph - no one on stackexchange sites should be. It will be pithy, it will be a little acerbic, but it will be right. If you choose not to believe it, I will follow up with comments that make fun of you more and more.

The moral of the story is to do work for yourself. Do some, any! Then you can ask more specific questions - questions which I will be less hesitant to answer than blanket questions about topics you obviously don't understand and haven't thought about. I answer lots of questions on this site and there's plenty of good ones which means that plenty of people don't expect me to do their homework for them. I like those people. Yeah, don't expect me to do your homework. Homey don't play dat.

(Does anyone get that reference?)

THE FUTURE

2011-04-23T19:05:00.001-07:00

For me, this weekend I realized that I was living in the future.

I've always been bullish on the future. I read a lot of science fiction and science fiction loves the future Time travelers from our time go into the future for some reason and they see all of this cool technology: warp drives, transporters, ray guns, nanoconstructors - you name it. Science fiction writers love science but most of the time they skimp on the social aspects of the future. Do we have racism? Is there free love? Are drugs legal? Is religion abolished?

Sci-fi writers are often less eloquent on these matters. Roddenberry wasn't. He showed us a world in stark contrast to the one that was developing around him. Racism: gone. Freedom: for everyone in every way. No one is sick because we all decided that wasn't going to happen in our world. No one is hungry because not a single person on Earth could bear to see a person starve. There's no war because we're way too grown up for that. There's no money because we're all so rich we don't feel the need to measure it anymore.

But the uniforms he put his characters in were absurd.

Seriously, go-go boots? Miniskirts? Captain Kirk is pretty much wearing pajamas. I mean take a look at military uniforms in our time. They've changed only somewhat in the past 300 years. They're full of creases, corners, polish, buckles and complication. Uniform maintenance is the primary method of making grown men cry in the military. I don't foresee a general relaxing of military culture to the point that you can chill in pajamas on the bridge of the flagship within Star Trek's timeline.

Specific predictions about the future can get tricky. Very few people predicted even really obvious stuff like computers and cell phones. And clothes are worse. Fashion is unrecognizable year to year. The best rule I've figured out is the 20-year rule: whatever you're wearing now will be stylish again in 20 years (assuming it is currently stylish). It works well enough in a general sense, but I'm also quickly losing sense of what is fashionable right now anyhow.

Which is why I was blindsided this weekend. We went to the beach (note to everyone in the frozen north: it is simply gorgeous out here and the water is plenty warm). I saw a bunch of surfers - many of them much much younger than I. And they were ripping up the waves and...

Wearing war paint.

Yeah, streaks of red, blue and yellow all over their faces, arms, chests, you name it. They looked like they belonged in Braveheart. I don't normally attempt to fathom the culture of kids these days but this one left me curious. The only explanation I could think of turned out to be the right one: it's sunblock.

And of course it's sunblock. Let's do the math: you need to wear if you're going to be out in this sun and it pays to be generous with it. When I was bailing hay in summers on the farm I used pure zinc oxide - creamy, greasy and great at picking up dust and dirt. But you had to use it - normal stuff wouldn't get the job done and even with the zinc oxide you had to slather so much on that it was opaque. I looked like one of those nerds you see in movies who never goes outside - pure white nose peeping out from behind a wide-brimmed hat and long sleeves in the middle of summer.

Oh if only I could have looked cool AND gotten sun protection....

Oh, and now in THE FUTURE you can. You can cover yourself in sunblock and look like a warrior instead of a nerd. This is the sort of great idea that I never had or even considered, but seems so foreign when you suggest it:

"In the future," the traveler started, "the kids wear war paint!" The pitch of his voice raised on the last half of his sentence as if he had made a particularly incisive statement. "They go to their sports games all painted up with their team colors, fancy designs on their faces to intimidate the other team - who of course have their own paint. Some wear it for every day pursuits too - go to the store and you might see some of them there."

How odd! How interesting! How different! THE FUTURE! Of course, in this case the future isn't foreign or odd. If you start from certain suppositions (you need sun block, sometimes you need it really thick, but it looks dumb) then it's not a far leap to the conclusion (non-dumb-looking sun block should be invented).

Yes, THE FUTURE has arrived for me and it's a good thing: someone had an idea that seemed foreign and senseless to me but is in actuality insanely great (and it's making the world a better place). Kids are wearing more sunblock, protecting their skin and stopping cancer - but they don't look like idiots. Yes, in THE FUTURE the right thing to do is also the cool thing. I want more future please.

Overflow Overflow Overflow

2011-03-16T17:35:00.001-07:00

For the second time in the past few weeks it turns out I'm my own worst enemy. I work for days to track down a bug in my latest embedded application only to find out that (surprise) the jerk who put the bug there is none other than me.

Some background: the ARM Cortex M3 has this nice interrupt vector called FAULT. When something bad happens (divide by zero, memory protection error, etc) the program packs up the current processor state and vectors directly to the fault interrupt handler. This is a very good thing - you can put a fault handler in that interrupt, resolve the problem and continue on your merry way. Perhaps. The thing about faults is that they tend to be somewhat hard to recover from. For instance, let's look at my two most recent faults I've had to deal with.

The first fault was a stack overflow. I know that some of you have probably heard of a stack, and some of you have heard of a stack overflow but if you're anything like me you had no idea what this actually meant and simply assumed it would never be a problem for you. Well, it most likely will be for you at some point. It's like the fraternity hazing of the embedded world: you can't consider yourself an embedded programmer until you've had your first stack overflow. Let's have some background: A stack is a Last-in-First-Out memory structure that is essentially scratch space for your program. Whenever you call a function the return address is pushed on the stack as well as function arguments. Inside the function the arguments are popped off and used, and when the function is done the return address is popped off and jumped to. You can use the stack directly but it's not typical to do so if you're programming in a high-level language like C - just let the compiler do the dirty work. Now the stack is stored in memory just like everything else (you'll see in a second why this is at the root of the problem) and it has a specific length. So it may start at memory address 0x20001E0 and if it's 16 bytes long then something else starts at location 0x20001F0. So you've only got 16 bytes to play with for all of your function arguments and return addresses and such. In a VERY simple system maybe this would be enough but it's very doubtful. For instance, if you call a function within a function you have to push a lot more stuff onto the stack. If you call a function within that it gets worse.

Now here's the tricky part - even though the stack is special and unique and essential to calling functions and making your program work correctly your processor doesn't care. It can't really tell what memory is stack and what isn't. So if your stack is 16 bytes long and you push 17 bytes on to it your processor will happily obliterate whatever data was residing just after your stack and replace it with what you just pushed onto the stack. This is stack overflow.

What happened to me the first time is that I didn't allocate enough stack space and it wiped out some important information residing just beyond the stack. Now if what is just beyond the stack is data, then overwriting it is bad, but not fatal for the program. After all, data is data. If I'm adding two numbers and one of them is overwritten by the stack then I can still add it, but the result is unimportant and nonsensical. What's worse is when pointers get overwritten. If you don't know pointers... then you probably have lots of company nowadays. But it's simple: pointers are memory locations. You interpret the value in a pointer as pointing to a memory location. So to load data from a pointer is a two step process: read the memory address stored in the pointer, then look at that memory address and grab the data there. However, if your stack has overwritten this pointer then it might point to the wrong memory address or (more likely) not point to memory at all. You see, RAM on the Cortex M3 (the one I'm working with) starts at 0x20000000. So if a pointer tells you to look at 0x1FFFFFFF then it's not just telling you to look at the wrong memory address - it's telling you to look at something that's not even memory. If it tries to do that it triggers a hard fault and you get to figure out why. If it's your first stack overflow then this process lasts days. Enjoy!

So that's one type of overflow that can hurt you. The second is buffer overflow. There's no magic here either - a buffer is basically an array. Arrays are bounded - they have a definite size. But even if you declare your array to have a size of 512 your compiler won't stop you from requesting the 513th element - it picks out the memory just beyond the end of your array and reads it. And you can write to it just as easily. And of course, there's usually something important there that you really shouldn't overwrite.

Arrays can be difficult to work with, so people create circular buffers. Normal arrays would be linear buffers: start at zero and go to the end. Circular buffers wrap around so that if you try to go outside the array you loop back around to the beginning instead of trailing off the end. That is, they loop back around to the beginning if you code them correctly. Tell me what you think the result of this code is:

head_pointer = (head_pointer + increment_value % buffer_size)

If you said the modulus (this thing: % ) operation would be evaluated first - you're right! And you're smarter than me! That code doesn't do what I wanted it to do: increment the value then wrap it around if it was greater than the buffer size. Instead it wrapped increment_value around if it was bigger than buffer_size and then added it to head_pointer. The net effect of this is that the pointer never loops around to the beginning of the buffer - it just keeps growing and growing and growing and gobbling up memory as you write to it. If you let it go on long enough it will overwrite a pointer with gibberish and trigger a hard fault.

Yes, I did this. At least this sort of stupidity on my part gains me valuable experience. Lesson learned: I am very foolish sometimes.

C Gotchas

2011-02-17T16:48:00.000-08:00

Holy crap, two updates in one day? Yeah... I decided that if I find myself sitting at my computer thinking 'I wonder what's on Slashdot' or 'Do I have any more street cred over at Chiphacker?' I decided I should do something potentially useful and update my blog. Doesn't have to be long, doesn't have to be good, dosen't ahve to ahve correct spellign - just do it. After all, the first step to making money with a blog is to update it every day. Second step? Have people actually read it. Step three profit baby!

But I'm sure you're all here for the real meat of this post and that is going to be the answer to the question 'What stupid thing did Steve do today that cost him hours of time?' It didn't take me hours, but here's a snippet of code that caused me trouble. By the way - you win $10,000 if you spot the bug and submit it before I hit 'Publish' for this post.

#define VALUE 0xF01FUL

short i = 0xF01F;

if( VALUE == i)
{
print("They are equal\r\n");
}
else
{
printf("They are not equal fool\r\n");
}

Ok hotshot, what prints?

If you said 'They are equal' you are in fact wrong. I hope that feels good. But do you know why you are wrong? Here comes the science.

We have two things being compared here: i is a short int. On most processors/architectures that is a 16-bit signed integer. The #defined value is in hex (obviously) and supposedly the same value as the variable but has a little 'UL' on the end of it. That signifies that it is to be treated as an Unsigned Long variable. This corresponds to the unsigned int (32 bit) type on most processors/architectures.

That might already give you the first inkling of why these two aren't equal: one is 32 bit but the other is 16. But you veterans out there (if you consider having taken an introductory C class in college being a veteran) will think 'Ah, but those bottom 16 bits are the same, so it shouldn't matter!'. You would be right, but C doesn't follow your rules. In C all integer comparisons are done on a 32-bit basis. Basically, C expands every integer used in a comparison to 32 bits to determine if they're the same.

'Aha!' you say with a sly smile, 'I was right then! Even if you expand them both to 32 bits they're padded with 0's in the upper unused portions (obviously!) so they come out to the same thing!'

But once again you're incorrect in your assumptions. Why oh why do you assume that they're padded with 0's? Because the only other option is to pad them with one's and that would change the value? You obviously forgot how signed data is represented on a computer! In signed integer types in C the most significant bit is the sign bit - if it's 1 then it's negative. Seems simple enough. So let's follow your line of thinking and expand our (signed) variable to 32 bits:

16-bit value: 0xF01F
Expanded to 32 bits: 0x0000F01F

Wait a darn tootin' second! This was a negative number (the most significant byte was F which means all 1's which means the most significant bit was 1 - which means negative). Now that we've expanded it it's suddenly.... not negative. Well that can't be. It's not the same value then - positive vs. negative. Kind of a big change. So to preserve the value we'd have to expand it and pad with 1's - like this:

16 bit value: 0xF01F
32 bit value: 0xFFFFF01F

Let's check my math with a signed integer calculator that you can find online (via the Google: http://planetcalc.com/747/):

0xF01F: -4,065
0xFFFF01F: -4065
0x0000F01F: 61471

Yep... padding with 0's doesn't produce the same result if the integer is defined as signed. So where you see

if(0xF01F == 0xF01F)

C sees:

if(0x0000F01F == 0xFFFFF01F)

And then it looks at you funny for thinking they're the same.

But I don't look at you funny. I'm only so dismissive and rude because I just made this mistake today and the pain is still fresh. Someday we'll laugh about this.

But for now if you mention it I will end you.

Quick Debug Tip!

2011-02-17T04:37:00.000-08:00

You've probably been told you should always read returned error code from functions - especially if you're working with a new API. It's too easy to assume you have everything working because it compiles and then it all falls flat when you try to run it. But the question is what do you DO with the status? It's not always clear. There usually a lot of them and often you won't be handling most of them in a release configuration (hopefully you will have learned how to avoid most of them by the time you release). Some are potentially ill-defined (how many times have you seen a code like ERR_UNKNOWN returned three or four different places in one function?). But what you should always do is read the code and check it - like this:

status = api_function(args);
if(status != ERR_OK)
{
//Apocalypse?
ERROR();
}

This is good practice - always do this. Even if the if statement is blank, still do it. Just get your hands into the habit of reading returned error codes and checking them.

But you may have noticed I put something called ERROR() in there. That's a placeholder for a real error handling strategy. Just start by defining it as a macro (Note, this may not be correct, it's early and I'm not in my right mind):

#define ERROR()

Now it exists but it doesn't DO anything. This is a fancy way of putting nothing inside the brackets but still reminding yourself that you have to do something later. If you do this for every returned error code then you will have a hook in place to do something if a proper status code isn't returned.

Now depending on what point in development you're at and what kind of system you're running you have several options. While still debugging I find it easiest to just define the macro to be something like this:

#define ERROR() disable_ints();for(;;){wdt_pet();}

This will entirely block the program (including interrupts) while simultaneously petting the watchdog timer so it doesn't restart the processor (just in case the watchdog interrupt isn't maskable on your processor). If you don't have a watchdog timer you can ignore that part. This approach works best when you have some sort of debugger. You start the program, wait a second and then pause execution to see if it's stuck in any of these loops. This approach also works in a multi-threaded system assuming that your scheduler runs in a maskable interrupt. When you release this code you should remove that macro so that your system doesn't hang out in the field for an ignorable error.

For a more advanced approach that you might actually want to use in a release environment you can potentially define different levels of error such as ERROR and FAULT. FAULTS would obviously be more important and warrant more attention while ERRORs might simply be counted and then ignored. Most of the time dire errors can't be handled locally, so your only option is to report it to the operator (if there is one) and usually his/her only option is to hit reset and hope everything goes back to normal. But at least there's a process!

There are other interesting wrinkles in this error handling game. The ARM Cortex M3 for instance has a fault interrupt that is called whenever something bad happens (try to access memory outside of RAM, divide by 0, wear white after labor day, etc). It pushes the state of its registers, stack pointer, program counter, etc on to the stack and then visits the interrupt. You can use the information it saves to create a report (because sadly most faults that force a visit to the interrupt cannot be recovered from without a reset). The processor you're using may have similar error-handling features. Take a look.

To summarize - always check returned error codes. Even if most of the time you can't DO anything with them you can at least hang the program so you know you have a problem to fix. You might be able to get fancier later but as my favorite super-national paramilitary group used to say - Knowing is half the battle!

(Actually that's crap - they were all-American when I was growing up and I'm too old to change now! Get off my lawn! GO JOE!)

What are you doing?

2011-02-15T17:20:00.000-08:00

Stop! Right now. What. Are. You. Doing?

Wow, let's stop that, I felt like a telegraph there for a minute (STOP). But the question stands: just what do you think you're doing? I intend this post mainly for people who have stopped developing software/hardware and have taken a break to absorb my acerbic wit. If I stopped you in the middle of enjoying a bowl of ice cream please believe me when I say I did not intend that you should question why you're putting it in your mouth. There's a good reason for that: it's ice cream. Duh.

But to all those coming for some witty banter - fresh from a break from developing the latest microcontroller-inspired widget - let me ask the question again. Just what do you think you're doing?

Probably your answer is going to be 'coding' or something similar. Good. Microcontrollers need code - that's obvious. But let me ask you this: how certain are you that the code you're writing right now is the code that's going to end up inside of that microcontroller when all is said and done? Uh, analysis? Maybe 15% certain I'd say - depending on what point you're at in the design. If you're early on in the design your chances are closer to 1%.

This is not your fault - well, that's a lie. It probably is your fault - but I'm trying not to scare you off. After all, this happens literally to everyone. Everyone. No one goes through a project and doesn't have one of those moments where they realize they have seriously miscalculated the scope of their project, or the simplicity or realized that they forgot about some other major hurdle. Then they end up decimating their code - and not in the literal sense where 90% of it is left after. No, more like 1% is left - and that's probably a header.

I typically see this problem because people don't consider whether their code actually works. They read specs, requirements and other documentation and then write a lot of C code. Or Java, or Python or whatever. It's all the same - almost literally because as I said 99% of it will typically be gone by the end of the project (so it makes little difference what language it's in anyway). Sure, it probably compiles - with only a few dozen warnings (It's all small stuff - it doesn't affect how the program works. Some casting will fix it, it's fine!) but there's no telling whether it actually does what is expected of it because you didn't set up benchmarks, tests or sanity checks on anything. Your development process goes something like this:

Read
Code
Code
Code
Code
Code
Delete 'old' code
Wish that you had used source control because that wasn't old code at all
Code
Code
Cod (not a misprint - you're enjoying fish at this point)
Ode (poetry break)
Code
Code
Code
Integration
Blank stare
Blank stare
Disbelief
Delete 99% of code
Recode
Recode
Etc

And let's be honest - if at any point in the design process you stop to think about it you're going to think 'Man, integration is going to be a bitch.' There's no project for which that isn't the case - integration is always difficult. But there's a way to make it easier - don't save integration for last.

Why is it that when you work on something for some reason project management always seems to think that it's their job to keep developers apart for as long as possible? It's probably because the Waterfall Model says that development and integration are two separate phases and one (development) is a prerequisite for the other (integration). So no skipping ahead to integration! What will project management do if not enforce the flawed and ultimately unhelpful vanilla implementation of the Waterfall Model on the poor helpless engineers under its command?

Of course not everyone blindly follows the Waterfall Model, or eXtreme Programming (seriously, the initialism is XP, not EP, so I capitalized the right letter there) - it certainly isn't the case where I work. No, what you need to do is integrate as soon as possible.

A project typically consists of several independent parts which can be integrated and tested without bothering the other parts. Projects usually also consist of several pieces of hardware, code or technology you've never worked with before. Let's be honest - when was the last time the new chip you used followed its own datasheet exactly? And for that matter, when was the last time two engineers working on opposite sides of a communication channel, attending the same meetings, reading the same documentation and potentially being in the same love triangle decided to implement their portions of a project in a compatible fashion? These aren't signs of immature engineers or bad project management or difficult documentation - it's just life. These things happen. The difference between an inexperienced engineer and an experienced one is basically how jaded he/she is. Optimism is not a useful trait when everything is contractually obligated to go wrong.

Given these problems it makes no sense to write literally everything and then go back and make sure it actually works. Here are several suggestions:

Use unit test for complex algorithms to catch rookie mistakes such as off by one errors (we all make them).

Walk over to the other engineers office/cubicle and ask copious amounts of questions. Nine times out of ten on a project the arbiters of what actually gets made are the engineers themselves. Just to get everyone on the same page it helps to ask of your fellows 'So what CRC method are we actually using and how does it work?' Then document it.

If two chips have to talk to each other chances are they will hate each other and refuse to speak. Start early with actual chips and force them to get along as soon as possible.

Abstract away interfaces so you don't have to worry about specifics in unrelated parts of code. You don't have to know whether your serial interface actually works if your application's only method of accessing it is a ring buffer.

Verify all assumptions as soon as possible. Chances are you're wrong (life just hates you like that).

This is hardly an exhaustive list but I think you get the idea. In case you don't get the idea I'll state it as plainly as possible:

Code isn't useful unless it works! Don't sit there with questions, concerns and unverified assumptions but pop out 1K lines of code a day. In a month you'll be left with 500 lines of good code and a looming deadline. Be pragmatic, be wary and be prepared.

Note to anyone who is reading this who may actually know me and/or work with me: I am not vindictive, frustrated or lacking empathy for this situation. Believe me I have been stuck in it plenty of times and it was all my fault. But I will be the last person to berate you and the first person to stick up for you in a meeting or directly to your/our boss. Sure, this may be a rookie mistake but we're certainly all allowed them. If we're not then we have no opportunity to become better engineers.

Terminology Galore

2011-02-03T17:41:00.000-08:00

If you're anything like me you hate terminology. You know, those special, magical technical words that people use that you don't know the definition of. Terminology. It'd be great if it weren't so imprecise. You'd think (well, hope) that a word has one definition. This is not even close to the case with regular English (and even worse with British English) but can't one hope for a more direct mapping from technically-minded people. A word should just mean one thing, right?

Take the word 'driver'. On your desktop PC you have drivers - all kinds of them. On an embedded system you have drivers - all kinds of them! But they're not the same kind of drivers - not exactly the same anyway. If you wanted to fill a job writing Windows drivers you might not want to fill it with someone who write embedded systems drivers or even Linux drivers. So if you saw such a job advertised you'd want to make sure what kind of job you were getting into.

So some recruiter calls you and asks 'Do you have driver writing experience?' And you really want to ask what he means but you know he doesn't know. The best answer you're going to get is "What kind of experience do you have?" I would respond with something like "I've written multiple device drivers for bare-metal microcontrollers and real-time operating systems, is that what you're looking for?" And if you're lucky the notes he writes about your experience will be something like "bear-metal.. big iron? Iron Man? multiple operati.. operation systems. Operation - I loved that game...." And what he tells you is "Absolutely absolutely, I'll get in touch with them and let them know. That's great that's great!"

And that's only the answer I would give now - because now I have some idea what a driver is and how it isn't a board-support package or hardware abstraction layer (I think). But if you're anything like me a month ago you're a bit lost. You see I had developed drivers before (I think), and just didn't know it. So let's define some terms!

I consider a driver (in an embedded system) something that hides registers for you. For instance, here's some code that configures a timer on on MSP430 for creating a servo control pulse:

//Clear timer A config

TACTL = 0x04; //TACLR = 1

TACTL = (0x02 << tassel =" 10">
(0x00 << id0 =" 00">

TACCTL0 = 0x0000;

//Configure compare and capture unit 1 for output compare mode 3
TACCTL1 = 0x0000;

TACCTL1 = (0x00 << cap =" 0">

(0x03 << outmod0 =" 011">

//Set CC1 to generate 1.5ms pulse - neutral

//MSP430 user's manual page 11-14

//?? what's going on here?

TACCR1 = 0x0000; //SET output line HI at 0x0000

TACCR0 = PULSE_1MS; //RESET output line (LO) at 1.5ms

//Set timer A period to 2ms

TAR = TIMERA_PERIOD;

//Start timer

TACTL |= (0x02 << mc0 =" 0x01">

(0x01 << taie =" 0x01">

Thats... a lot. A lot of bits. A lot of hex, a lot of OR'ing. A lot of bad formatting. Oh my, I can't handle this.

I'd rather do something like this:

timera_conf(SRC_SMCLK,DIV_1);
timera_cc_conf(FUNC_COMP,COMP_MODE_3);
timera_cc_setpw(1500 /*us*/);
timera_interrupt_enable();
timera_run();

See? No bits. That's a driver.

Now, this is an internal peripheral. Those are easy. Well, easier. We know that drivers basically set registers. When it's an internal peripheral then accessing those registers is just as easy as saying 'register = value'. But it's harder if you have (for instance) a peripheral connected over SPI. You still have to set registers but that requires you writing data over SPI - usually commands like 'I WANT TO WRITE TO THIS MEMORY LOCATION. IT'S A CONFIGURATION REGISTER YO' and then the peripheral responds 'YO DAWG THAT'S COOL WHERE THE DATA AT?' and then with another SPI transfer you say 'HERE DA DATA AT!'. So basically you'll have a peripheral driver utilizing the SPI driver. It's a whole bunch of driver on driver goodiness.

So what about all the other crap? Like a Board Support Package. Well a BSP... supports a board. For instance the ez430-USB development kit has one LED on it (this is the extent of its on-board peripherals). It's located on P1.0 which is on physical pin 3 which can be accessed on port (yadayadayadayada). You don't want to know all of that - you just want a heartbeat LED. You want it to flash. So you have a smart guy write a function for you - a BOARD SUPPORT FUNCTION!

void bsp_led_toggle( void )
{
P1 ^=0X01;
}

This is great - I don't have to know where the LED is, I can just say 'toggle that please!' and it gets done. That's board support - it supports the board. Whatever's on the board needs functions so I don't have to know all about it.

And what about the dreaded HARDWARE ABSTRACTION LAYER?!?
The HAL just makes sure you don't actually need to know what your hardware looks like to use it. For example, you can turn a general-purpose I/O port into a TTL serial interface - you just have to be careful with timing and such but it's certainly possible. Now imagine on your Arduino you have the regular UART and a software-based UART. You want the same interface to both: get a byte, put a byte, turn it on, turn it off. So you write up a bunch of functions and then you just say something like:

byte = uart_get(fake_uart);
byte = uart_get(real_uart);

Same interface, same bytes, different underlying hardware. That's what a hardware abstraction layer does.

Hopefully with some of these definitions you'll be a little more educated about what all these weird definitions are. Good luck!

Desk Update

2011-01-16T10:45:00.001-08:00

I mostly finished my desk/workspace. I think it looks good. You can see pictures of it here: http://picasaweb.google.com/sfriederichs/DeskPics?feat=directlink

I totaled up the cost for the materials. Not tools or time, just materials. The new things I bought came out to about $340, but I also used some wood from my old desk. I'll figure about $60 for that and put the materials cost about around $400, plus about a full week of time spent on it. Don't even ask about the tools and miscellaneous consumables (staining pads, etc). Tools are worth it, but consumables are just something you have to live with.

A Proper Workspace

2011-01-08T10:14:00.000-08:00

When it comes to work surfaces I am no fan of store-bought desks - I'm very angry at them. I need to do real work and to do that I need a solid dependable work surface. Sadly, you won't find that in most stores. Or, if you do, you'll pay $1000 for an 'Executive' desk that does its job well (and usually that job is much different than the one you want it to do). If you buy from a store, you're likely to find something like this in your price range: http://www.officemax.com:80/catalog/sku.jsp?productId=prod1811868

This monstrosity is a disaster. I mean, even the picture alone signifies that they just didn't care. Look at the CRT and circa 1996 inkjet printer. They didn't even update the picture for the new millennium. I'll start picking this thing apart in no particular order:

3) Teak Laminate Top. Teak Laminate Top. When you're just not important enough for real wood, we'll mix sawdust and glue together to give you particle board and plaster it with just enough real wood to make you think we spent money on you. This is low low low LOW quality. And sadly, if you're anything like me you've seen plenty of laminate (and hated it).

6) Penumatic work surface adjustment - how likely is it that the pneumatic parts are similar to the ones you see in office chairs? You know, the office chairs that fail working particularly well after a few years? Yes, for now it's an adjustable work surface, but come three years along you'll have to drill holes through the supports and stick bolts through them to make sure that it doesn't keep dropping on you overnight.

2) Keyboard trays - these are universally awful. For me anyhow. Yes, I'm sure they work for some people but how likely is it that this keyboard tray will work for you? You'll either end up disconnecting the thing (painfully) or developing carpal tunnel. I have not had a single work desk (that I didn't make myself) be 'ergonomic' in any fashion and the keyboard trays were some of the worst offenders. Just look at the wrist rest - bunched up right against the keyboard. That looks painful. Look at the odd angle. Yes, I'm sure it works for some people, and I'm sure it can be altered, but let's face it - they use cheap hardware on these things and they never stay altered. They wear down and flop back to the worst possible position.

7) CD storage on the desk - since when did this ever work at all? Do you own 12 CDs? Then this is the desk for you! Pathetic. You need a real storage solution for CDs and DVDs - not the pathetic attempt seen on this desk.

9) No monitor stand. Really, this is a necessity. Your neck is just as important as your wrists.

4) No cable management. It will look ugly.

1) No front legs. Just try to lean on it and see what happens. And I don't hold out much hope for those metal support parts. They're probably hollow and will bend/break eventually. This is not how you make stable work surfaces.

5) Leg room - nil. Seriously, where do my legs go? Can I stretch them? No I'm going to hit the wall or one of those supports.

8) Too small - it's 29" deep which is good, but only three and a half foot wide. Not nearly enough room to spread out several documents.

But hey, what can you expect for a desk that's only $80? Oh, wait, what's this:

Top component of the Balt Ergo Sit/Stand Workstation - ORDER BOTH TOP AND BASE

Oh, this only half of it, so it's $170 or so right? No? Nearly $400? Oh, well that's a great price....

This will not do. I cannot work on those things. I don't just need a computer desk - I need a real workstation - computer desk, writing desk, solder station, filing system, parts storage, tools storage and sever closet all in one. I need it to be solid. I need it to not hurt me. I need it to look nice. And I don't want to spend $1000 on it.

So I make them myself.

Now, this may surprise some of you, but I did not start out as an electron pusher. I grew up on a farm and learned about many many things. I can spot weld, cut metal with a torch, help pour concrete, drive a tractor, be a human gate, give a pig a shot, stack hay and shoot guns. I am also not to shabby with woodworking. Wood is my preferred material. It's light, strong and pleasing to the eye. It's also phenomenally cheap and the tools to work it are similarly inexpensive. And when you mess up it's easy to hide (as my dad always said 'A little putty and a little paint makes a carpenter what he ain't').

I started making my own desks with very simple construction. Here's an example of something I hacked together with 2x4's and wood screws that is very similar to my first desks:

Such a table is not difficult to put together. Most of the pieces you'll buy pre-made from places like Lowes. The top is a single 2 ft x 4 ft sheet of 3/4" plywood screwed into the frame. The frame and legs are 2x4's all around - butted up against each other and screwed with cheap wood screws. Everything was cut to length right in the store then brought home for assembly. The only touches of flair are counter-sunk holes for the screws (keeps the screw heads from sticking out) and paint, along with a coat of spar varnish to protect it from the elements.

It's not pretty, but it has its benefits. It is exactly the table I wanted - no more no less. It is solid, very solid. I can stand on it no problem. It won't tip or jostle and the legs stay in place. And it is cheap. Materials were at most $50 and my labor is free (I'm a slave driver).

That's all well and good, but it's still not really a piece of furniture - I wouldn't hand that down to my grandchildren. And it's not very shall we say... featureful. It's four legs and a top. No drawers, no shelves, nada. For a workstation I need something a lot more complicated. I need this:

This is my workstation. I just made it over Christmas with the help of my dad. It's just shy of 8 foot long, 30" deep with a 12" deep monitor shelf along the back - all pine. It has four stylish legs all the way at the corners of the desk for maximum stability, and the part in the middle is also weight-bearing. I sanded the hell out of it and applied two coats of a stain/polyurethane combo. Half of it (on the left) is meant for electronics and the other half is for the computer. There are cable holes on the computer end for easy cable routing. My laptop there is sitting on an Ergotron laptop arm. There are spaces for drawers on both the electronics and computer end. The part in the middle is slide out storage for printers (both my inkjet and laser). Take a look at one of them in action:

Underneath, there's power strips running all along the back of the desk, as well as a piece of pegboard that holds my wireless router, NAS, print server, power adapters and other IT-related paraphernalia. This is temporary at this point and everything is going to go in the drawer when it gets put in later:

There is a second piece that is still waiting to get put in that will support rails for plenty of shelving storage and a bookshelf on top as well as extra lighting. Here's what it looks like right now:

(Oh and don't complain that it's covered in papers. This is what a desk is supposed to look like. If your desk is clean it means you're not doing any work.)

And what did this wonder cost me? Less than $500 at this point, and three solid days of work. Well, the tools cost a fair bit by themselves, but I think it's worth it when you consider what you can make with them.

I'll post pictures of the completed project when it's done.

Something I'm Proud Of

2010-09-25T14:11:00.000-07:00

Engineers are supposed to make things, right? You'll cruise the hobby blogs and read about all these cool projects that light up and/or tweet things - everyone seems to be making something nowadays. I'm not so great in that respect. I really haven't made anything apart from my job. It's a little embarrassing, but I don't really get off my butt unless someone orders me to do something. But then I'm great - I make scripts. PCBs, code, you name it.

Well it turns out that lately I've actually accomplished something - and it all started with a problem. While working on a microcontroller design at work I was readily infuriated by breadboards. You just can't get away from them and you shouldn't try. Everything really out to be breadboarded before it's finalized - otherwise you'll have loads of incorrect PCBs laying around and lots of wasted money. But breadboards are hard to work with: wires pop out, you can't put surface mount components on them, they don't have nice modular connectors and you can't always stick a probe in the holes to see what's going on. So I made something to fix that problem for me.

This little doohicky is a breadboard test point/connector adapter. It has a row of 8 pins that plugs into your breadboard and then connectors for the power bus (there's two different configurations of power busses that i've seen - .1" pitch and .15". It has holes for both). First, it has 8 surface mount LEDs for each of the pins making it very useful for status displays. Second, it has a 2x5 connector that will fit a standard IDC cable plug. I have plans to make an IDC to male pin cable that plugs into it. Third, it has two sets of test points - there are male headers for logic analyzer female plugs to fit onto (along with the obligatory 9th connection - ground) and then there are high profile test points that work great with scope probes. And also for the scope probes there are four high-profile test point ground connections. No more silly jumper wires to the ground bus on the breadboard. All of the test points are isolated from the breadboard by 10K resistors. That way, if you accidentally short two test points together (perhaps one that is connected to ground and another to +5V) you won't break whatever is connected to those pins. If, for instance, you do have the worst case scenario and +5V shorts to ground, you have an equivalent 20K of resistance between them and nothing short circuits - just 2.5mA flows through the connection. Lastly, there is a 1.5A resettable fuse between the breadboard VCC and the VCC on the IDC connector, so if something you accidentally connect the +5V from the IDC connector to ground the fuse will trip and not hurt your power supply on the breadboard.

I plan to sell these at... well, wherever will take them. Sparkfun, Ladyada, etc But not yet. There are several design problems that need to be ironed out. A bit more parts selection, maybe a few more components and this board will be ready to sell. I hope everyone else is as excited as me.

Anyone?

Anyone? Bueller?

Oh well.

Don't Save Bad Code

2010-09-16T07:35:00.000-07:00

As coders we're often asked to do things we don't know how to. Use a new library, use a new microcontroller, try a new compiler. So we start to experiment - make something 'just to see' if you can program your microcontroller or figure out how a library works or do simple socket communication. Then you think 'Wow! That worked! Let's try more things! I'll just start coding everything here in main()....' And then you get eaten by raptors.

You've made a mistake. You weren't thinking about organizing this mess or even making sure it was robust - you were just seeing what you could do. Your main() is 1000 lines and you're not sure what all of this code is supposed to. You didn't comment because it was all so obvious earlier, but now all you know about this line is that if you comment it out the program crashes. Whoa boy.

But still you feel the compulsion to try to work with this mess. You want to massage it, start breaking things off into functions, add a few defines, rename the variables and make it work again. But step back - is this what you're supposed to be doing? After all, you started this to learn the library or a framework and, well, haven't you? Isn't that information tucked in your head by now? Is this code even remotely close to your final product? Could it be? Does it work? Is it well-organized and robust? Didn't you have a real development plan somewhere here? Complete with projections and unit tests? And haven't you learned enough to start on the real work, rather than just trying to save this one .c file out of sentimentality?

You shouldn't try to save old code just because it's already written. Old code has no particular value over newly-written code for several reasons.

First, there is no cost to writing new code in place of old code. When working with physical objects if you make a mistake with a material you have to pay to replace the material. If I were whittling wood and I shaved off a little too much I would try to work with it. But with software we have source control, so if I make a mistake I can revert it or even start over without losing anything. I still have access to the important things: my old code and the knowledge I've gained from writing that code.

Second, new code presents an opportunity for a more correct solution. So you've sat down and written a lot of bad code. You feel like deleting it would mean deleting the progress you've made towards a correct solution. But surely you haven't been asleep this whole time? Your mind has been watching the code you're writing and doing its own work - pondering the patterns behind the madness, taking note of vulnerabilities, seeing patterns that could become functions, etc. You've gained a lot of experience just by doing it wrong and at the very least you have a better idea of what NOT to do once you start over. A fresh start allows for a more pure expression of the solution that can't be shoehorned into your ill-conceived beginning.

Third, you will improve your tests by trying things different ways. If you use a FOR loop then you won't encounter errors that you might if you used a WHILE loop. If you're using dynamic memory allocation instead of static you have more things to worry about. Each new method of solving the problem before you opens up new areas for failure. Will this sub-function I just started using return 0x00 for any reason? Because I return its value without checking it and zero would not work well for a return value. So write a unit test to ensure that the return value is never 0. That's one more level of certainty that you're writing correct code, not just code.

Fourth, writing new code allows you to follow best practices more closely. So after a few hours of hacking together your test program-turned release candidate you decide to see what happens when you run splint on it. Turns out it produces so many warnings that your ego crawls into a hole and hides. What's the right course of action? Fix the warnings one by one until the splint output is clean. What do most people do? Turn off splint and wait for it all to be caught in integration. It's much easier to run splint on a 5-line function than 1000. Starting over gives you the chance to comment, run analysis tools, develop unit tests and format your masterpiece when it's a much more manageable size. This will only help you in the long run by front-loading much of the work that leads to clean, well-running and tested code at integration time.

Whenever you are writing - be it code, a short story, a joke, a play, a manual on annoying your cat remember this: the measure of how well you have written isn't the number of words - it's whether you accomplished what you wanted with those words. Eliminating incorrect code instead of massaging it back to life costs you nothing, lets you write more correct code, expands the range of your unit tests and helps you follow process more closely. So don't save that crufty old code, replace it and move on.

Just How Awesome is Batman?

2010-06-25T08:29:00.001-07:00

Unlike my usual practice of the title of the blog post having nothing to do with the content I have posted a relevant title today. And also, there's nothing about electrical engineering or computers or code or controls or math in this post (except insofar as Batman can do any/all of those things awesomely) - just Batman.

Truly, Batman is amazing - he's crazy prepared at all times. (Note: I reference the Justice League cartoon when I'm talking here - haven't done much comics reading). When AMAZO stole Superman's powers, Batman nonchalantly whips out kryptonite from his belt and fends him off - prompting Wonder Woman to ask "You carry kryptonite around with you all the time?" Of course he does - he's the Goddamn Batman. If Supes suddenly went rogue (as opposed to going rouge which involves cross-dressing) then Batman would be on it. His utility belt is like Schroedinger's cat. While the cat is both alive and dead at the same time, his belt does and does not contain every single item in the universe - particularly the one that would be most useful. He's just lucky enough that when he opens a specific compartment the probability waves collapse into the exact item he needs. Or not - he'll just MacGuyver up something from whatever he finds in there. I mean, it might be pocket lint and somehow he'll defuse a nuclear bomb and give everyone ice cream cones (with sprinkles).

It's a well-known fact that Batman has a way to defeat every single Justice League member if they turn evil. But this brings us into the barber paradox of the day: Can Batman defeat himself? The logic of the situation is similar to the barber paradox but instead of shaving it involves death served cold. We start from several assumptions:

1) Batman is always prepared. Always.
2) Batman doesn't want to die
3) Batman has plans to kill every member of the Justice League himself
4) Batman is in the Justice League
5) Batman is the Goddamn Batman

So, some conclusions. From 1 and 2 we surmise that Batman will always have a plan ready to prevent his demise. This only makes sense - who wants to die? And by adding in 5 we can assume that this plan will always work. It always has in the past. In the 80+ years of his comics I don't think he's ever died. I mean, Bane broke his back but he didn't die - he healed up again and went on being Batman. That's about as close as it gets. Maybe in some alternate timeline or something he dies. I hear the Joker got reality-warping powers once somehow and might have killed him but that didn't work out for the Joker because the whole mess was cleaned up by the end of the comic and Batman was alive again. Yes you heard that right - when Batman and reality itself are in conflict Batman wins. That's tough.

From 2 and 3 we surmise that Batman must have a plan to kill himself if he goes rogue and by adding in 1 we can surmise that he's ready to follow it through. It makes sense. I'm sure he's been under mind control or something often enough to want to have a backup plan in case he starts acting all evil. Most likely if he really needed to die he'd just have Supes or WW pummel him to death or slice him in half with heat vision (or lasso him to death somehow - not effective, but kinky and I think he'd like to go like that. Plus he can't lie when the lasso is around him so he'd probably admit it turned him on).

So this gives us two contradictory conclusions: Batman can't be killed and Batman has a plan to kill himself that will succeed. Both of these can't be true. I'll throw out a few examples:

Batman knows that he needs to go down if he turns evil, so he tells Supes to beat him to death under a very specific set of circumstances - maybe some kind of mind control that can only be broken by blending his brain into a milkshake or something. But! We know that Batman really doesn't want to die AND he has a plan to kill Supes. What stops him from just going through with that plan? Not much - he's Batman. So Supes is dead, and then Wonder Woman, Green Lantern - pretty much anyone who gets in his way. That's not a very effective plan. Maybe the ENTIRE Justice League could jump him at once but hell I just thought of that in ten seconds - Batman probably considered that one when he was still in the womb. I'm sure he has a plan for the whole Justice League turning evil at once. In fact I'm sure there's a precedent for that somewhere in comics history. And I'm sure Batman resolved it with ice cream again. So scratch that.

Okay, so maybe Batman can just kill himself. If he was evil he wouldn't consciously want to do it but maybe he implanted some post-hypnotic suggestion triggered by a code word that would force his heart to stop or something. And then he makes himself forget about it somehow and only lets certain people know. People like... Superman again? No, because if he really fears Supes going evil (enough such that he keeps kryptonite on his person) then he's not going to give him a kill switch. And even if he did do that whole hypnotic kill switch thing and forced himself to forget about it what stops him from one day discovering that someone implanted a hypnotic suggestion in his mind somehow and rooting it out? He doesn't know that he put it there - it could have been Grodd or someone else's insurance policy. Just yell 'Steeplejack!' in their next battle and Batman makes tea for him instead of delivering a beatdown. Can't take that chance.

I wouldn't even put it past Batman to install a tiny computer with a tiny AI in his suit somehow prevents him from stabbing himself in the chest, or replacing his heart with a nuclear bomb (the Goddamn Batman needs no capacity to love, or circulate blood. The blood moves on its own because it fears what will happen if it doesn't) that goes off as a last resort and his consciousness transferring through the ether to a clone tank on the moon that he put there and made himself forget about so no one could sabotage it. HE'S FREAKING BATMAN PEOPLE! He put a space station with a fusion laser in orbit a week after Martians attacked Earth. The buses weren't even running but apparently Wayne Enterprises had orbital capacity. Fusion technology didn't even exist! He invented it and then did it and surprised the hell out of the rest of the league and was all like 'Oh you like this space station? Yeah of course I can afford it. I'm the Goddamn Batman. Go to your quarters - the mattresses are stuffed with money and I can flood your rooms with poison gas if you get out of line. I mean, strike that last part. I'm Batman.'

Does anyone have a logical resolution to this mess?

Quick question: In a fight between Batman and Chuck Norris who wins? Answer: No one wins - there are only losers. Specifically, you and everyone you love.

New Unoriginal Idea

2010-03-09T08:48:00.001-08:00

I don't think that I learn much at my job, but I was surprised the other day to learn something new about microcontrollers. Well, coding in general anyhow. You see, when working with microcontrollers I use a finite state machine to control program execution. No, wait, what does that mean? That's technical gobbledy gook. It means that I have a variable and I define a bunch of different values it can take. When my program starts it goes into INITIALIZE mode and it initializes stuff. Then when its done I usually put it into IDLE where it doesn't do much of anything and usually waits for user input like a button press or something. Then depending on what happens I put it into other modes like DO_STUFF_1 or DO_STUFF_2. Different events can change the mode it's in and that produces different behaviors. The idea is that it will be a nice and orderly flow from one mode to the next and that depending on what mode you're in you won't handle certain inputs, or you'll handle them differently. The goal is predictable behavior.

But microcontrollers are anything but predictable. You see, there's these awful things called interrupts. They are what they sound like - they interrupt your program and do something else. They happen usually when hardware has done something - say a button has been pressed. When that happens, whatever you were doing previously stops and this new code executes. That's not usually so bad, but you almost always want to change the mode you're in based on what happens in that interrupt. Okay, so change the mode. Fine, whatever. I don't care.

Except I obviously do. This might happen:

CODE ALERT! CODE ALERT!

do_special_stuff_for_different(modes)
{
mode1:
easy_command();
easy_command();
---OOOOPS INTERRUPT HAPPENED HERE!----
change_mode_to(mode2)
---INTERRUPT OVER BYE BYE NOW!--------
command_that_checks_mode(); //Oops, doesn't run, wrong mode!
command_that_depends_on_the_above(); //fails, above didnt' run!
...
} //Always close your braces!

You see what happens there. Ideally, you want all the commands for a mode to run regardless of whether the mode changes halfway through.

This can be done. Just use a queue data structure to change the mode. A queue is a list of things. I can put things into the back end and take things out of the front. So when I want the mode to change, I can just put the new value for mode into the queue and then handle it in one and only one place in the program:

is(empty(mode_queue))
{
no:
mode =new_mode_from_the(mode_queue);
yes:
//Do nothing!
}
do_special_stuff_for_different(modes)
{
mode1:
easy_command();
...
}

So you only change the mode before you handle events for that mode, and never while you're handling them. This eliminates confusion for the poor code.

But, my unoriginal idea improves this - I think. Say that you're in MODE1 and you have an interrupt that wants to change the mode to MODE2. Ok, it puts the new mode into the queue and we'll handle it when the time comes. But now, before it can be handled you get another interrupt that tries to change the mode to MODE3. Well, put it in the queue and we'll handle it. When everything is said and done, you have the mode change from MODE1 to MODE2 to MODE3 right after one another. It seems OK, but wait! When we designed the system, we never intended to move from MODE2 to MODE3 because in MODE2, we set things up to get ready for MODE4 which we intend to follow MODE2 directly. But that doesn't happen. When you're in MODE2 you try to put MODE4 on the queue, but it gets in line behind MODE3. And meanwhile, if MODE3 attempts to put MODE5 next, it also fails. So you've got two different things trying to happen in sequence at the same time.

It might work, but it might not. We intended in our design to move right from MODE2 to MODE4 but we didn't do anything to ensure that it actually would. That's where my unoriginal idea comes in. These modes are stored on a microcontroller. That means they're stored with bytes, and bytes are made of bits - eight of them to be precise. Eight bits can store 256 different modes if you encode it that way, or it can store eight if you use something called one-hot encoding. One-hot encoding means that only one bit is ever a '1' at a time - the rest are zeros. Wherever the '1' is determines what the value is.

So, when we want to change modes we don't put the next mode we want to move to into the queue, we XOR the current mode with the next mode and put that into the queue. The XOR operation works on bits and it says if two bits are the same, the outcome is zero. If they're different, the outcome is 1. Since all of our modes are encoded with one-hot encoding, there shouldn't be any same bits. Wherever there was a one in either of the modes' bytes, there will be a one in the resulted XOR'd byte. Using this method we don't only record the next mode we want to go to, but we record the mode the program was in when it wanted to go there. It says 'I want to go from MODE2 to MODE4' instead of 'I want to go to MODE4'. Then, when it comes time to look at the queue and change the mode, we can make a better decision. If the request is 'I want to go from MODE2 to MODE4' but we're in MODE3 right now, it doesn't happen. It goes on to the next request which was 'I want to go from MODE3 to MODE5' and that one happens.

Now, there's an obvious flaw in this: the request to go to MODE4 is lost totally. The upshot is that if I press two buttons at the same time only one thing happens. For buttons that's good enough, but for other inputs it's not. For some inputs what you really mean to say is 'No matter what mode I'm in now, I want to go to MODE8'. How do you do that? Simple, it's.. No, wait, how DO you do that?

Oh, wait, I know how. Assuming you followed all of the above steps, then when you XOR the information in the queue with the current mode you get out a valid new mode and just switch to that. If you don't, then you can assume that you want to perform one of these unequivocal mode changes. You just have to figure out what mode you want to change to. So, in order to handle that, you can do this: instead of putting the XOR'd mode into the queue, just put the next mode you want to go to. When the program attempts to XOR it again with the current mode it WON'T get a valid mode out of it. So, it will XOR the data again with the current mode and if you meant to do one of those unequivocal jumps, it will produce a valid mode from THAT XOR operation. If you didn't mean to do an unequivocal jump, then it will produce an invalid mode. Tada!

Debugging

2010-03-08T06:19:00.000-08:00

This post isn't for me. I never have to debug. I follow a strict development process in which all of the documentation I read is perfectly accurate and describes all edge case behavior. I also follow code writing procedures so well that every exception is handled, every input is sanitized completely (even to the point of predicting accurately what the user meant to enter instead of what the user did) and every return value is checked. All reported errors (which are entirely due to... well, not me anyhow) have expository messages bordering on small novels that not only tell you what caused the error but actually fix it themselves in all known copies of the program, including those in the past. Yeah, that's right - my code is so awesome that it goes back in time and fixes itself before you even see an error message.

Well, I doubt you believe that. I'm not sure what to believe anymore - the full truth of the situation is caught somewhere in an unstable time loop that will someday tear reality apart. So in the meantime, I actually debug things. Yes, things that I created. This is because you can never get away from debugging. Because documentation is Just. Plain. Wrong. Because the user put international unicode characters that you didn't even know existed into your text box. Because every library you rely on has bugs and undefined behavior. And it's only worse with circuits: the EM spectrum is effectively infinite. Wavelengths you only see in your nightmares can ruin your life in an instant. It happened with the earliest digital computers - freaking cosmic rays would cause a single bit in memory to be changed and trash your machine state. Yes, the very fabric of nature hates you and wants to give you blue screens.

The best, the very best that you can hope for is that your code or circuit does what you designed it to do. Put all your effort into meeting the spec first - later, after you're almost done debugging and trying to figure out why the slave device won't respond to a perfectly valid message, well, that's when they'll tell you that the specs they gave you were wrong and you have to wipe out some code and start over.

In any case, when all you have is a spec, all you can do is implement it. Let's imagine a scenario that involves both hardware and software. I'll imagine a small embedded sensor that has to relay data to its gateway - probably an AVR based 8-bit microcontroller with firmware that I've made, communicating with a host system over TTL-level serial (8-N-1, 115200 bps) and it collects 0-5V data with the ADC. It uses a packet-based serial communication scheme (START, MSG ID, LENGTH, DATA, END sort of deal) to transmit data every second to the host.

Ideally, you code everything, solder everything, hook up power and communication and it works. Then you go home to your supermodel wife/husband/plaything and sports car and sip scotch while listening to death metal. But the reality is that 99.9999% of the time it won't work. The questions you have to keep asking yourself:

'What might be wrong if it doesn't work?'
'What's the quickest way to test that?'

Before I begin, let me note that I said 'quickest' way to test things. When you think you're done and you're testing your code or hardware the actual system it's meant to interface and work with, then chances are your time is limited. You'll probably be close to delivery of the final product, or you'll have another group that needs to use the actual system to test. In either case, quickest is best - you want to front-load as much of this integration phase as possible into design as possible. Don't ever assume it's going to work and then attempt to throw debug capability into your design after you amazingly find that it doesn't 'just work'.

So, look at the host computer and see what it says. No communication with slave?

'What might be wrong if that doesn't work?'

You should know why specifically it shows that message - in this case let's say that it shows it because it didn't receive an answer to an initialization message. Maybe it's just not getting the message

'What is the quickest way to test that?'

I think the easiest way is to put an LED on my slave device and light it if it finds the message. So we'll create the light and lo and behold it doesn't light.

'What might be wrong if it doesn't light?'

If it doesn't light, then the serial port may not be working correctly.

'What's the quickest way to test that?'

I would create a wrap-around self-test mode. Just loop the TX lines of your slave to RX and send a test message, then light an LED if it receives what it sent out.

'What might be wrong if THAT light doesn't light up?'

At this point, if it can't communicate with itself then it's just not sending anything out (for one reason or another) or what it's sending out is gibberish and can't be read back.

'What's the quickest way to test that?'

Monitor the signal to see if it looks like a legitimate serial transmission. To do that, you'll need test points to hook up your logic analyzer or oscilloscope. So you'll either see a valid waveform, an invalid waveform, or no waveform.

So ask yourself again: 'What might be wrong if I see valid waveform or no waveform?'

If you see a valid or no waveform waveform then it's probably software: the receive or transmit buffer isn't being filled perhaps.

And then: 'What's the quickest way of testing that?'

What *I* would do is use a secondary serial port as a debug terminal so you can view the contents of the RX and TX serial buffers. Then you can see if they're being filled or not, and go on from there.

So look back at all that. Just by considering what might not work we added several things to our system: test points, a wrap-around self-test functionality and a debug terminal. If you have all of those things ready and you've tackled some of the other 'Why might this not work and what can I do to quickly test it?' scenarios then you're poised to quickly find the root of the problems you're seeing in minutes and hours instead of days and weeks. Integration (that is, combining what YOU make with what OTHER people make) is the most difficult part of any project. Specs aren't going to help you because no spec is ever complete enough to define a system. Everyone makes assumptions and doesn't tell anyone and that makes testing a pain. So save yourself some pain and plan ahead. You'll thank yourself later.

Wherefore Art Thou Sparkfun?

2010-03-02T09:39:00.000-08:00

Did you know there's an angyee.blogspot.com? I think it's in Portuguese and run by a female. It's all pink anyhow, but pink can be manly, so I don't know. It might help if I read Portuguese.

I just read an interesting flame war over on Hackaday about whether Sparkfun is overpriced and, well, dumb. Overpriced is for certain, but that's the name of the game. Everyone who sells you something they bought from someone else will be 'overpriced' in that you don't have the money or storage to buy 1000 of anything and wait for others to buy them from you. You're buying the convenience of getting one and only one of something.

In fact, I just bought some ATMega328 DIP chips from Sparkfun. After the flame war I was feeling guilty about it so I checked DigiKey for their price. It's the same. The exact same. The only difference is that Sparkfun offers less of a volume discount. Their stock counter says they have over a thousand of them, so they got the nice $3.04/unit price and are selling them at $4.30 - just like DigiKey. Yes, they're making money, but why not just buy from DigiKey?

That brings me to the second thing that you get from Sparkfun - an explanation. When you search for ATMega328 on DigiKey you get five different chips in varying packages and a comparison table that's about a half mile long. If you don't know that you want the DIP version you're screwed. Lucky me, I do know.

But this totally avoids a more important point: How do you know you want an ATMega328 at all? Sparkfun spells it out for you: it's the Arduino chip but with more program memory. More is better right? Even if you don't know how much program memory you need, why not have more? Plus they tell you that it will work in place of the chip already in your Arduino but wait, don't change that chip yet! You need the bootloader programmed on there for everything to work properly. DigiKey would not tell you that. Heck, DigiKey will barely make it easy for you to give them money.

That's another thing Sparkfun has going for it: their web design is not straight out of 1995. It has pictures and a shopping cart and colors! Oh, and navigation. DigiKey has an awful Flash version of their print catalog. Yes, they thought it would be good to simulate turning pages on the internet. That hasn't been a good idea since Geocities. Of course, DigiKey started with its paper catalog and must really feel attached to it. I can't fault them for sticking with success, but the idea of even using a 1000 page catalog is... so, so unappealing. And a waste of paper.

So the second point is whether Sparkfun is dumb. And by dumb I mean facile. Childish. Dorky but in a bad way. The issue at hand was creating projects like '5 foot Nintendo controllers'. I must admit that so much of the hobby electronics seems like a big dick waving contest. Who can create the coolest steampunk doo-dad? I don't care. Or this gem recently featured on Hacknmod: 'Twitter Controlled LED IKEA Table using Bluetooth'

Let me just pause a minute to let that sink in.

...

...

Okay. Bluetooth is neat. My Wiimotes are Bluetooth and I think it's great I can hook them up to my computer and read their data. It makes a lot of sense as a Personal Area Network protocol. I like headsets. So I don't dislike Bluetooth. I also think RGB leds are interesting - with Red, Green and Blue we can make all colors! It'd be great for use as a status indicator - red is bad, green is good, orange is warning, etc. I like... coffee tables? I guess. I don't dislike them anyhow.

But I just picture this geek sitting at this table with his friends, itching to show off his new toy that he made. So in the middle of the conversation the whips out his phone and says 'Watch, I'm going to Twitter to my table and change its color! I made this! Hold on. I have to type out the message... almost there... Oh I don't have 3G, this might take a bit... Ok sent! Now it should turn green.. uh. any second... uh.. Maybe I have to reboot the computer... hold on...'

Or you have worse projects like TV-B-Gone. The sole purpose of that invention is to make yourself feel good at others' expense. In a bar? Football game that everyone else is watching too loud? Can't hear you and your friends talk about how counter-culture you are? Whip out your TV-B-Gone and turn off everyone else's entertainment! That TV wasn't doing anything for you!

So a lot of this hobby electronics community consists of everyone trying to outdo this month's steampunk widget or Twitter-using Ardunio-controlled status symbol. You know where it goes from here: What started as an honest effort to educate people, have a good time and share things turns into a contest. I saw it happen with LAN parties. Instead of having fun and laughs people started the biggest hard drive contest, or the most porn contest or the biggest monitor/best system specs contest. My friends and I held LAN parties in high school. Some we charged for. Despite lugging their system across town and PAYING to get in some people just wouldn't play computer games! They'd hang around, look over people's shoulders and slow down peoples computers by transferring porn while they were trying to play Unreal Tournament. It just wasn't worth it after a while and I fear the same thing will happen with this 'community'.

In conclusion: Overpriced - probably for some things, not for others. Do your homework. Facile? Childish? Lacking in true content? A bit. There's still plenty of good stuff over at Sparkfun and their top-selling items are components - not kits. But I worry about everyone else. When blogs just echo each others' postings of Twitter-controlled steampunk widgets instead of producing something... substantive, well, I get concerned.

On Requirements

2009-09-15T08:15:00.001-07:00

I like looking at job postings. It's kinda like shopping for a new TV. 'Oooh, this one is for designing analog circuits!' instead of 'Ooooh, this one has three HDMI ports!'

But they're still funny, funny things. There's so many problems with how people are hired nowadays. For instance, I rarely see jobs that ask for less than 5 years of experience in... something. Whatever the job is about. Want a job making circuits? Five years experience in making circuits, mandatory. Your resume is not even considered if you don't have five years of valid work experience. Go work for $random_big_corporation with a well-known name developing circuits for five years and we'll consider you.

What exactly is the value of five years of 'professional' work experience? Surely you have a number of designs under your belt, a list of components you use, probably a few contacts for samples, support, etc? But if you've been hacking together circuits since you were five and have a web page full of stuff you've done, but only two years of work experience then you're not supposed to apply. What does the five years working for a company guarantee them? Certainly not quality. Anyone can remain employed with a certain title and be bad at their job. I see plenty of people like that, but when they leave they'll put right on their resume - '5 years of bad designs, rework, lost profit and profanity. But my title was analog design engineer the whole time.' That guy wins.

And what's with the specific skills? 'Must use Eagle for schematics and layout.' Ok, I'll learn Eagle. Not my preferred solution but I'll do it if it gets me a job. Oh wait, it doesn't say that. It says 'Proficiency in Eagle schematic capture and layout required'. What? When you're looking for a mechanic do you put 'Must have proficiency with Craftsman brand crescent wrenches'? I've used plenty of schematic capture programs and none of them are black magic (except maybe PSPICE. Grrrrr....). I'm sure I can use Eagle very well if you give me a few hours to play with it. Heck, I can DOWNLOAD it and play with it - I'll get it done before I come to work for you! The tool is not supposed to define the job. If anything, arbitrary requirements only diminish the talent pool of people who can help you. (Note I said 'arbitrary' requirements. Some requirements are sadly justified.)

So your resume has to have a bullet point - 'Created 12346463113 circuit desgins for a toaster using Eagle'. Perfect. That gets past the filter. Why do the tools matter so much? True, if I'm going to work for a company that's doing a large program in C#, then I had better have worked with C# before. But that's what coders do. Code monkeys. People who get specifications in and produce code for output. Then a freaking computer checks their work with a unit test and tells them if they did it wrong. These people are automatons. Their job requires almost no original thinking. Engineering is supposed to be about creativity - finding solutions to problems. I always thought I would be sat down and they'd say something like 'This hyperdrive keeps overloading when we turn it on. We need to reach the Tok'Ra in three days before Apophis comes in his mothership and beats the crap out of us.' To which I'd say 'I'll put my engineering mind to work on that problem right away. And I need to work with Major Carter. Alone. Naked.' What's NOT supposed to happen is for them to then say 'Oh by the way, it all has to be done in Lisp. You have 5 years of Lisp experience, right?' Whatever happened to the right tool for the job? You know - how it's supposed to make things easier/cheaper/faster?

And don't you love how every job has its little 'niche'? This position is for an automation engineer, not an electrical engineer. This one is for an embedded application developer, not a software engineer with a specialty in embedded systems. The proposed scope is so small it's like it's not even worth it. Even if you have superior experience it doesn't matter.

'This position requires experience with PLCs'.
'Oh, those are microcontrollers with a whole bunch of electronics. I could design one of those. Can I have the job?'
'No, you just said you don't have any experience with PLCs.'
*facepalm*

Or better yet, your similar experience is worthless:

'I studied to be a controls engineer but I think I fit for this DSP position. DSP and controls stem from the same theoretical background and I've also had extensive training in numerical methods for computer systems. Also, I created an embedded sensor platform that used DSP, so I have practical experience in embedded programming and DSP. Can I have the job?'

'I'm sorry, we're not hiring controls engineers.'
*facepalm*

If I had to boil it down to one question it would be:

Why are businesses afraid to hire anyone but the person who exactly fits?

There are so many modifiers on these job listings that you'd think they had millions of people to choose from and they could afford to be picky. Ideally on the internet they do, but then they do stupid things like say 'Local candidates only' or refuse to pay for people to come out and be interviewed. Or not pay for relocation. Or not help a spouse get a job. So you're limited to whatever people can be found at hand. And they're not going to find their perfect match. I have friends who look for mates like this. Yeah, they're single. And they're missing out on a great part of life.

While it's true that companies cannot afford turnover, they also cannot afford to delay hiring for too long. If a job sits, the work doesn't get done. Or you overwork the people you do have which leads to turnover. All out of an.. ideal? Is it a value nowadays to not train people? To not give people a chance? Not to invest in them? To be picky? To not try new approaches, hire people with new skillsets?

While I can't fathom the reasons that businesses follow these hiring practices I can tell you that it is costing them people. Good people. Specifically, young people. Comparatively untalented, inexperienced good young people. These hiring practices are biased against those who haven't had the chance to develop whatever random skill that is in vogue nowadays.

But that's ok. Soon the baby boomers will retire. And then they'll have to be less picky.

On Being 'Well Rounded'

2009-09-08T11:15:00.001-07:00

Following up my previous post on how well-rounded I believe I am, I am realizing that being well-rounded is not good. At least, not in my position. Let's consider my position and see why being well-rounded but not deep in any particular skill is not good.

I work for a large company and it employs many people. MANY people. We have all kinds of engineers, technicians, secretaries, project managers, etc. If it can happen on a project chances are we've seen it and hired for it. Need someone to create a schedule and manage a budget? We've got project managers and they've got secretaries. Need someone to design an antenna feed? We have seasoned RF engineers. Need someone to program an 8-bit microcontroller, create interfacing electronics and build an enclosure for everything? Well, no. We don't need one person to do that when we could have three separate people do it. People who excel in their individual areas and can get the job done quicker and better than one person.

Let's be serious - you wouldn't combine all of those tasks into one person unless it was absolutely necessary or desirable. When might those confluence of events happen that make it necessary or desirable? Let's tackle necessary first. It might be necessary to do it with one person because that's all you have. You either can't find people who are skilled in those areas or you can't use them if you did find them. The skillset might not exist in your company if it's small and might not exist in your area if you're in a small town or other backwater type place (like most of where I grew up...). So you use what's available to you and hope that it can get you by. And it might be desirable. You could contract out the work but then you have communication issues and overhead costs and such. If the work isn't particularly difficult then it might not be worth it to bring new people on even if they could get it done in half the time. So a wide array of basic knowledge could be useful if you couldn't hire more people and didn't care too much about time lost and/or wasted.

But at a large company those don't generally hold true. You usually have a ready supply of many differently-skilled people. You probably have at least two people to choose from who have your skill, so you'll always choose the person who is better at what you want to do. Then this person gets more experience and hones his/her skill and eventually becomes an expert. Then, unless this person is unavailable, you will always go to this person if you need his/her skill. In the situation where two people know the skill but have unequal levels of experience, the more experienced person always wins.

And time is usually very important at a large company. There's never enough of it. Competition is fierce and to maintain market position you have to be timely. Contrast with a smaller business: there is little/no market position to maintain so releases of products can be more easily delayed without losing too much money over it. Not that deadlines aren't important to a smaller company, but if they let a deadline slip they are likely to lose less money than a large company, even as a percentage of income.

Both of these factors contribute to drive people to specialization in large companies. With so many people being available, you have to set yourself apart to make an impact. In some circumstances intimate knowledge of a project may make it more worthwhile that you perform multiple tasks with different skillsets yourself. Or they could just put the person with the right skillset underneath you and have you lead him/her.

That's the major difference: A large company is more able and more likely to trade labor for time, where a smaller company doesn't have this luxury. If you aim to be a jack of all trades, shoot for a small company that can't support many people.

Update: I've done a little thinking and I believe there are a couple more things to be said for this line of thinking. 1) The 'time first' mentality tends to push people towards specialization in skills and it also pushes entire organizations towards specialization in skills. If you know Labview and you can do the job in Labview, chances are you'll do it in Labview. So a C# programmer is going to have a hard time. Over time the company becomes a monoculture and tries to apply its one major skillset to all problems (for good or ill). 2) The 'time first' thinking need not be limited to skill sets. If you can buy an off-the-shelf product that does what you need then it makes no sense to build it in-house. You're essentially buying the skillset you need ready-made. This also can lead to a monoculture of suppliers - with certain businesses being preferred over others because of past experience. In and of itself, buying this experience rather than 'growing' it in-house is an acceptable tradeoff for certain decisions but definitely not all decisions. 3) The monoculture induced by either of the above methods can kill a business. If this trend moves forward to its logical conclusion (which is admittedly far-fetched) then you'll have a handful of experts skilled at using a small number of skills/products, or you'll just buy everything from someone else. You run into two problems with this: what happens when your extra-skilled engineers retire/get hit by a bus/find a better offer? Brain drain. What happens when all you do is buy other people's products and put them together? Brain drain again - you now have people whose sole skill is figuring out what other products to buy and re-sell. You're no longer an engineering firm - you're at best a project management firm and at worst a re-seller.

Neither of the above outcomes are guaranteed to happen. There are many other pressures that will come to bear on an organization before this brain drain becomes absolute. However, it may still become terminal before it is absolute and the organization may not survive. There are two ways to combat brain drain: First, never outsource (directly or by means of buying COTS products) your core competency. If you write e-commerce software then outsource your server management, not your programming. If you build Arduinos then outsource your board fabrication, not your board layout. If you do that then you are paying people to develop their skills at the expense of your own. Then, when they are the experts they have no reason to keep you around, or at least, no reason to let you have a big slice of the profit. Second, you must, MUST invest in pursuing different/new technologies and you MUST invest in training less-skilled people. If you have a choice between two people of different skill levels, consider choosing both. Split the work up along skill-level lines and let the less-skilled person cut his/her teeth on the easy stuff and let the more-skilled person handle the hard stuff. But keep them together - working together, seated together and give them the same access to resources/people. Chances are the less-experienced person is younger than the more experienced one and will be around longer than him/her. You'll need the same level of experience if not more in 20 years time, so plan for it!

It is less efficient to make these investments in lower-skill areas and people, but unless you plan for the future you could find your company on the chopping block.

Complications...

2009-08-26T06:13:00.000-07:00

I consider myself a fairly well-rounded engineer. My specialty is control systems but that's in name only. It doesn't say embedded systems on my MS diploma because I had already taken all of the embedded classes for non-grad credit. It says communications on there because I sat through all of the comm classes (and got A's in them) but even though I know a lot I wouldn't consider that my bag. It doesn't say 'math minor' on my BS degree because I was two classes short (I decided to get a head start on my MS instead of the math minor). I've been programming since I was 6 in BASIC, C, Perl, Python, etc. That doesn't make me a CS major, but the years after that of learning to use source control, taking OOP classes, using unit tests and documenting with Doxygen makes me at least a friend (you guys will be my friend, right?). I built robots when I was a teenager and learned the basics of electromechanical systems as a controls engineer, so I'm claiming some mechanical experience as well. And I can speak German.

But I find as I grow older that some areas I just don't know about. Worse, I think I've built them up so that they seem so difficult that it's not worth it to even try to learn about them. TCP/IP, networking, ethernet - all that is something I've been interested in but thought the barrier to entry was too high. I've fussed about around the edges - I can tell you that the ethernet physical layer uses differential signal and (I believe) a 350MHz signal. I've done web programming and that introduced me to ports and such, but I don't take the time to go and just learn the stuff. I would start and I'd see the 7-layer OSI model and my eyes would just glaze over. Who the hell needs 7 layers (unless it's lasagna)? What are they cramming in there that it NEEDS 7 layers. Is it really this complicated to send data across the internet? I deal with microcontrollers and RS-232. I can send data that way, but for some reason people see the need to make twittering power meters out of the same microcontrollers. Other than being just a foolish idea it seems way too complicated if you have to interpret every layer of this 7-layer monstrosity just to do something as asinine as say 'Your refrigerator is now using 78 watts of power!'

So I figured I wouldn't deal with it. Who needs internet connectivity? I'll get along just fine. But still, it vexes me. It's something I don't know. I have a friend who does all this sort of stuff, makes routers in his free time. So I ask him 'What do you DO? What do you know that I don't know?' And he replies 'I can tell you exactly how a machine responds to a ping request. Where the message goes, where it comes from, what happens. It's complicated but I know it.' Basically, he knows how the OSI model works, what each level does, what happens. And the trick is it's mostly programming after a point. But not straightforward programming: complicated CS-type programming. Layers of abstraction built upon layers of abstraction so in the end you don't know that you're dealing with bits and voltages. You see, software people are TERRIFIED of bits and voltages and they will do anything they can to abstract those away. They have sacrificial programmers that write drivers to interface the bits and voltages to sane, reasonable things like floats and strings. They'd have a minor episode if they saw a character instead of a string. They NEED the 7-layers of the OSI model to be able to effectively ignore the basic truth of what's happening: bits are being represented by voltages. By the time you get to the 5th layer you're dealing with all kinds of structs, connection IDs, sessions and voltagephobia so that you don't know what's going on. If you're me that is.

If he tried to explain it all I think I wouldn't have understood. He could have explained it in Elvish and it would have made as much sense. So I ignore it again BUT IT STILL BOTHERS ME. Especially since I designed what I thought was a very complicated, nice and worthwhile messaging system for microcontrollers built on top of RS-485. It had circular buffers, message queues, priority, timeliness, guaranteed message delivery and all that jazz. It was complicated but I just started at the bottom and worked up. Pretty soon *I* had a few layers just to implement serial communication between a couple of devices. You had to have addresses and ACKs and collision sensing and much more to handle all this. It gets complicated quickly. Even when you start simple you always have the edge case. "Oops, what about this? Better add an address field. Oops, better add parity. Gotta ACK that for sure.."

It adds up. So I thought I might take another crack at TCP/IP and you know what? It turns out it's a bunch of things I already know. Just more queues and buffers and ACKs. When a ping happens (I think this is how it goes...) it's sent to the right computer (as determined by its IP address) where the network layer reads the message, finds it's an ICMP ping packet and puts the information in a special struct then pushes it into a queue for the ICMP handling process to take care of. When it gets around to it, it creates a response packet, tells the network layer where to send it (what IP address) and puts all that data into another struct and pushes it onto a queue for the network layer. That's it. Not too complicated - just data structures, processes and lots of teamwork. I learned something about TCP/IP.

TCP/IP is a complicated system, there's no question. But it does a LOT. It can reliably send data halfway around the world to a computer you didn't know existed in a few seconds. That sort of success is built up on layers and layers of success - each layer being tested and battle-hardened until it has so many special cases that it looks like an insane asylum. But once again - IT WORKS. Simpler systems would not. I don't begrudge the creators of the magical internet their seven layers anymore. It's not simple but it's not incomprehensible. It uses tools and methods that I was aware of - I just wasnt' aware of how they pieced them together. Now I am. The moral of the story is that if there's something you want to learn or need to learn don't think that it's impossible just because it's complicated, and don't condemn the creators of the system just because it's complicated. Just take it slow, re-read a lot and eventually it will make sense. Eventually.

Content or Not Content?

2009-08-20T09:30:00.000-07:00

That title up there is actually a clever double-meaning. I'll get to those meanings in a second. Suffice it to say I'm smart and it makes me feel smarter to withhold information from you for a few minutes longer. It won't be terribly long, so just bear with me.

I've often heard that writers can't write if they're happy. After all, how boring is happy? Have you ever imagined yourself perfectly content with not a care in the world? How long did you keep imaging that before it you moved on to something more interesting? It takes me about five seconds. I imagine myself sitting in a chair with a smile on my face and... then I repeat that. Forever. That's dull. Writers know this either consciously or subconsciously and strive to keep themselves aggravated. After all, writing is about conflict. Protagonists, antagonists, twists, man vs. man, man vs. nature, man vs. God, etc. So it doesn't do to live a life of contentment because the more conflict you're familiar with, the more you have to write about. Those writers make good books (in case you didn't catch it, the first meaning of the title was 'content' as in 'the opposite of conflicted').

So perhaps I can take it as an indication of my level of contentedness that I don't write here much. That's actually incorrect or at least a confusing argument. For instance, what kind of content could I put here? I started this blog to detail cool ideas I had and hope that other people would say 'wow, that's neat! I didn't know you could do that!'. I have some of those. From time to time. But my day-to-day work isn't exactly to work with super-awesome ideas all the time. It's somewhat hard to breed super-awesome ideas ("Hey guys, let's make robots with neural networks that train themselves to efficiently compete for limited resources against other robots!") when the only ideas you work with are super-pedestrian ("Hey guys, let's use Labview for test software again! All right! Just like the last 50 times! High-Five!"). Now the total contents of this blog are by no means all of the super-awesome ideas I've come up with, but it's enough of them. What I DIDN'T want to do with this blog was to 1) whine about my life and how horrible it is, 2) just post links to cool things OTHER people have done and 3) update about how I haven't had time to update.

I hate whining. And my life isn't awful or even excessively boring. What's the worst thing that happened to me lately? I had a fight with my wife this morning. Sure there was yelling. Sure there was screaming. Sure I kicked a robot (poor Roomba!) Sure we were angry. So what? That's not bad, that's a fight. That's a normal fight. I'll tell you what we didn't do. We didn't threaten to leave each other. We didn't say things like 'My ex-girlfriend never did this to me! That's why I like her much more than you!' We pushed each others' buttons and eventually apologized to each other. Wow. I'll tell you the number one thing I fear is getting in fights with my wife, but that's not the worst thing in the world. And how boring is my life? Well today I had to go through my Labview sub-VIs and re-arrange the connections one-by-one. I have about 20 of them. Yeah it's boring but I grew up on a farm. My dad used to have me stand in front of open gates and keep the pigs in just so he wouldn't have to close the gate and reopen it when he wanted to get out with the tractor. I did that for hours. In winter. I think I'll survive my office.

I also hate when people just link other things on blogs. 'Hey look at this cool kegerator robot!' Ok. That's good. But most of the EE/hacking blogs I read linked to that site on that particular day. It was a little excessive to read about it five different times, and I think we'd live in a worse world if I made it six. Just a link, no commentary, no analysis, no further ideas. Just aggregation. Go blagosphere. If anyone is actually reading this be assured I will NOT do that to you. If you, for some reason, subscribe to this blog you will not get that sort of 'content' (second meaning of the title!). You will get something original even if it's just meaningless ranting like right now (which, honestly probably isn't that original. Read Ecclesiastes).

And posting about how you have no time to post? No. Just Say No. Let me say one thing: it is essential to actively maintain something you want to build a following around. If you want people to read it then update it every day even if it's not particularly useful information or even tangentially related to your subject. People will either read it for the sake of reading or stop reading it altogether. But the people who read it for the sake of reading it will continue, daily, religiously. Then you can put ads in front of their face and get a shiny nickel. But it has to be NEW or at least something they don't ALREADY KNOW. If you dont' update your blog for a while then guess what? People know that. They can see it. So if you update and say 'I haven't updated for a while' they will respond 'We know' and leave. People aren't interested in things they already know. Better to update less often and with actual information than to tell people something they already know.

So that's my rant. Can you expect high-quality updates from me from now on? Probably not. I would actually have to do interesting things, or in fact, things at all. I'd have to have something to write about and it turns out I only rarely have that. And, if I start writing about non-engineering things I risk becoming like everyone else. And I HATE everyone else. They make me angry, hence the name of the blog.

When the robot is smarter than you...

2009-07-06T12:14:00.000-07:00

...then you had best let it drive!

Teleoperation is one of the largest growth areas for robotics. All kinds of robots are being used in situations that are dangerous to people: battlefields, burning buildings, defusing bombs, etc. I don't know about you, but when I defuse a bomb or do any other precision work I like to have all of my senses about me. Heck, I like to have all of my senses about me when I'm walking down the street. You couldn't pay me enough to remove one of my senses or any of the well-tuned reflexes that my body has developed over time.

So why is it when we ARE paying people to build robots we ignore most of those sense? In many of the tele-operated robots I've seen you one sense: a teeny camera. And you have two little sticks that move your tracks or wheels. That's it. And what happens? You drive your robot around half blind into walls. Somehow when we design robots for autonomous operation we give them all kinds of sensors and instructions for how to deal with their input. If an autonomous robot saw it was about to run into a wall it would correct itself and move on. But somehow when we put a human in the loop we forget how difficult it is to control these things so we give it a camera and call it done. There's a human in the loop so why bother?

Why not give a teleoperated robot the same senses and reflexes we give the autonomous ones? The same senses and reflexes we ourselves couldn't function without. If a human operator is about to ram the robot into a wall then DON'T LET HIM! Put ultrasound transducers on there and when the wall gets too close stop. Chances are you're not trying to run into the wall but instead run parallel to it. Then this behavior works perfectly. Get more in-depth with it. There are cars now that will auto-parallel park. This is amazing and probably safer than letting me do it myself. Robots should do the same thing. For any complicated tele-operation there should be a way to do it automatically. Most people can't back up a truck to a loading dock without someone watching for them. Is this any different?

One obvious application of these ideas is an area that technically isn't tele-operation: powered wheelchairs. Have you seen them? They are beyond dumb as far as controls and intelligence are concerned. They consist largely of a battery, motors and a way to steer. Of course, at this stage of the design life for powered wheelchairs the main problems they are dealing with is not enough power and not enough battery life. But when these are dealt with you'll still have a powered wheelchair that is more than happy to run into anything faster than it could before, and for a longer duration!

For the sake of pedestrians in the vicinity and the paint jobs of objects nearby wheelchairs should be semi-autonomous. I hate to make generalizations but for the most part older people's reflexes and fine motor control are on a downward trend. For the people confined to wheelchairs it's likely very well gone. And there are many diseases that impair fine motor control. It's not right to expect our senior citizens to be able to control their wheelchairs perfectly with a joystick (in fact I wouldn't count on anybody's ability to control such a device with a joystick - the input method has to be somewhat tailored to the system being controlled and a joystick doesn't move the same way a wheelchair does).

The most basic of features you could put into a semi-autonomous wheelchair would be obstacle avoidance. All the walls would be safe as attempting to run directly into them would cause the chair to stop a few feet in front. You could not worry about crowding out others if the wheelchair by default hugged a wall as it moved forward and if it deftly maneuvered around a door frame without significant interaction then so much the better. If you wanted to sit at a table without knocking your feet on the table legs just have the automatic systems do it. Perfect fit every time. You could even define more advanced algorithms. If you wanted to be able to open a door from the wheelchair you'd need to move within arms' range of the handle, turn it, then prop the door open with the wheelchair and move through. It might require some extra hardware and a lot of testing to be able to identify the door frame, back up, prop it open, etc. But the benefit to someone who couldn't open that door before would be significant.

Let's face it - no one wants to be stuck in a chair. People weren't designed to move around in chairs we were designed to walk. Trying to adapt our thinking and motor skills to a physical system totally different than our natural method of locomotion is difficult for anyone. Automatic systems can and should make this easier on those who have no other choice. It's an improvement for quality of life for everyone involved just to let the robot do some of the driving for you.

Power Power Everywhere...

2009-05-13T18:17:00.000-07:00

But specifically from the sun. We will never run out of power (never being equivalent to several billion years) because of the sun. Consider that most of our power comes from consuming things. Basically we burn things - we turn complex hydrogen-carbon chains into simpler ones and live off of the resulting energy output. Coal, wood, sugar - it's all the same. Combustion. The sun is better. The sun is powered by gravity itself. By the very shape of the universe! There's a lot more energy out there than sitting in every oil field, forest or coal mine. But how do we access it? The simplest method is by laying in the sun and getting warm. As much as I like a tan, I LOVE electrical power! So I use solar panels.

Let's talk about them for a second. Solar panels are a lot like batteries in some ways. You start out with solar cells which are maybe an inch square of real estate. They're made of silicon in an extra-special arrangement which causes them to produce voltage and current when light strikes them. But how much of each? There's essentially two main parameters that matter. The first is open-circuit voltage, the second is short circuit current. They're fairly self-explanatory: if you shine sufficient light on the cell and don't put an electrical load on it then you'll measure the open-circuit voltage across its output; if you shine a light on it and put a short circuit on the output then you'll measure the short-circuit current through the output. In between those points is not a straight line, but more of a graph with a knee. Take a look at the graph I show below and you'll see what I mean.

You'll notice that the cells are almost constant voltage sources except when you attempt to draw currents close to the short-circuit current. You can either use a simplistic model of a constant voltage source or you can use a more complicated model that accounts for the actual behavior more:

If my calculations are correct, Il is the short-circuit current and the open-circuit voltage is equivalent to Il*RSh. Props to Wikipedia for having such awesome images that I can steal for free. I like free.

So that's your single solar cell. They're a lot like the cells of a battery as I mentioned last time: to get a useful voltage out of them you have to put a lot in series. The difference between these and batteries is that you also typically have to put several sets of cells in parallel to get a decent current. I have an 18V solar panel made out of perhaps 32 solar cells - 8 in series and four sets of these in parallel. Its open-circuit voltage is 18V and short-circuit current is 300mA. This actually makes it rather useful for charging batteries!

When charging batteries you need two things at the same time: voltage and current. Well lucky for us voltage and current together are POWER! And we can get power out of our solar panel! Success! Just connect wires to things and make it go! Well, not that easy. You need specific voltages and specific currents to charge our batteries. I have some Ni-Cad batteries that need to be charged at a maximum rate of (off the top of my head - don't hold me to this) their total capacity divided by 10. So I have 5 batteries that have a total capacity of 6A-Hours, so the max rate I charge them at is 6/10 = .6 Amps. And for charging these batteries current is the most important part: as you push more current into them their voltage goes up and you stop putting current into them when they reach about 13.8V. So you need your voltage to be at least above whatever the battery voltage is at the moment. Just figure you'll need at least 13.8V to get this to work.

So we have a specific power IN from the solar panel and we need different power OUT. My first guess is to use an LM317 in constant current mode. However this has problems. The main one is that as you try to draw constant current out of the solar panel its voltage will drop. And the LM317 is a linear device. For linear devices the rule of thumb is that the current into the device is the same as the current out, and the voltage out is less than the voltage in. Thus, you will need to make sure that your solar cell voltage stays above the battery voltage. Good luck, because even if you do make sure of that you will be dissipating the 'extra' voltage from the panel IN the LM317 as heat - thus losing it. Solar panels aren't amazing power sources, so I'd rather not waste any of the power that it does generate.

You can also do power transformation with a switched-mode device. The rule of thumb for switched-mode devices is 'power-in equals power-out'. The only loss is due to efficiency. Another great thing about switched-mode devices is that they can produce nearly any output voltage off of its input voltage. So you can always keep your output voltage above the battery voltage even if the solar panel voltage goes below it. The trick is that switched-mode power ICs are usually set up as constant voltage supplies. They employ a feedback voltage to set the output to the right voltage. They're a little control system! I love control systems! SOOOOO CUTE! It's contained in a single IC! Adorable!

Anyway, I know about control systems and I reckon that I can modify the feedback signal so that it's based off of the current going to the batteries instead of the voltage at the output. I would current sense resistor to monitor that current and then use a differential amplifier to amplify and scale the voltage from the current sense resistor. This would be the new feedback signal to the switched-mode power supply. Thus it would regulate its voltage to make sure that the proper current goes into the batteries.

My fingers are yet again getting tired so I'll mention that this is just the battery charging circuit. You still need an output stage connected to the batteries to power things off of the batteries. Also, there's an improvement on the simple battery charging circuit I described here. It's called a Maximum Power Point Tracker that gets even more power out of the panel than before.

Wherefore Art Thou, Camping?

2009-05-07T11:57:00.000-07:00

Exactly how many doohickeys can you take with you camping and still be considered to be 'roughing it'? I know many people for who camping means 'in a camper', i.e. they need their flat panel TV and fridge full of beer or they're not going. I will sleep in a tent, so I consider myself somewhat superior to these people. But being an electrical engineer I just LOVE gadgets and I can barely stop myself from bringing them along. I know not to even THINK about bringing a laptop, but I'm still bringing our battery-powered fan. Of course, instead of batteries I'm using an emergency jumpstart car battery thingamajig with a cigarette lighter power adapter. If you want to power AC devices you'll need an inverter - just make it has sufficient maximum power (I've experienced the letdown of not having enough power firsthand when my friends and I figured out we couldn't run a smoke machine off of an inverter plugged into a car. Memories!) I haven't graduated to thermoelectric coolers yet but soon I will.

The question is how to you power all of your gadgets in the wild? A car battery is a good start but for a perfectionist like myself it just won't do. Car batteries are heavy. If you have to carry anything any distance you'll with you hadn't brought it. You can find better batteries than the lead-acid car batteries. Nickel-Cadmium, Lithium-Ion and Lithium Polymer are all better options (i.e. greater power density per kilogram). Technology is only one thing to look for when choosing a battery. There are four main things to look for in a battery: Voltage, Capacity, Current Capacity and Technology/Battery Type (Ni-Cad or Li-on or whatever).

Voltage is easy for the most part. Car batteries are 12V nominally and for the most part that is the magic voltage. Since your car runs on 12V everything that plugs into your car runs on 12V so there's lots of devices out there that will happily run on this voltage. Other popular voltages are 9V and 5V, but if you have a 12V source you're in luck because you can use dead-simple linear DC-DC conversion to get down to those voltages. Look up the LM7805, or in fact, everything in the LM78xx line (the last two digits are the nominal output voltage). They're really simple to use - for an electrical engineer anyhow. Fun fact: a battery is not a single unit - it's made up of cells. Depending on what the battery is made of, each cell has a different voltage. It's intrinsic - entirely dependent on the technology used in the battery. You get other voltages by stacking these cells up in series. Lead-acid is 1.5V per cell I believe, so a 12V battery has 8 cells in series. Another fun fact: it's not called a battery unless there is more than one cell used in it.

Capacity of the battery is also straightforward. It's measured in Amp-Hours. If my battery is rated at 7A-Hs then it can in theory deliver 1A for 7 hours before it's empty. Of course this is simplistic. Voltage degrades as the battery empties, so if you need 1A at 12V for seven hours you may not be in luck. After hour three the voltage may go down to 11.5V, then down to 11V next hour, etc. If you're using linear step-down conversion then you're out of luck for a couple of reasons:

You need a certain minimum voltage difference between the input to the voltage converter. If your battery voltage goes too low then you won't meet this requirement and you won't get any power at all. This can be alleviated with a switched-mode voltage converter

Assuming the power output is constant, lower volts means higher amps. As your voltage goes down your converter will draw more amps just to have the same amount of power and this will cause your battery to drain faster. Switched-mode voltage converters are not immune to this

The degradation in voltage is a function of the battery technology, so your mileage may vary.

Current capacity is distinct from the other capacity. Current capacity is the ability of the battery to source large amounts of current. Not all batteries are equal in this respect. As with any non-ideal voltage source a battery has a certain amount of internal resistance. This means as current flows out of the battery, power is dissipated in the internal resistance. This creates heat. Heat causes explosions, especially when it's applied to exotic chemicals. Remember when Dell's batteries were exploding? They got too hot. They got too hot because despite the fact that Dell spec'd the batteries with a peak current of say 5A for one minute, the manufacturer ignored that and went cheap on the batteries. This caused them to heat up when the laptop pulled the amount of power from them it thought safe. They got so hot that the chemicals got angry, and *POOF*! Exploding laptop. In general batteries are high-performers for current sourcing - that's the reason your emergency jumpstart kit works better than the alternator on your Yugo - current capacity.

Technology is a good one. I'm not into chemicals and ions and things but it's somewhat exciting to see the effect that all of that sciencey stuff has on actual performance and use. As I said before, technology affects all sorts of things on a battery: nominal battery voltage, voltage degradation over time, capacity to weight ratio, discharge capability, charging method, discharge behavior, etc. In general your newer battery technology has better capacity to weight ratio, less voltage degradation, higher capacity and greater current capacity. What it doesn't have going for it is simplicity and reliability. Simplicity in that they're more difficult to charge. Lead-acid is dead simple: apply 13.8V to a 12V battery and let it go for a while. It will charge completely. Lead-acid is surprisingly hard to damage as long as you don't exceed its recommended voltage. Newer technology is harder - it's rather finicky about how much current you push into it and what voltage and you must stop trying to charge it when it's done or else you'll break it. As for reliability consider that you can discharge a lead-acid battery almost down to 0V and it will still live - just charge it up again (deep cycle maritime batteries are the best for this). Most other battery types can't handle this. If you let them get too low then they're dead.

My fingers are tired so I'll leave until a different time the discussion of the solar-powered power unit I'm making. For my camping gadgets of course!

Firmware in all things

2009-05-04T10:30:00.001-07:00

So today I'm home from work for a bit to wait for the repairman (or repairwoman) to come and take a look at my dishwasher. It has lights flashing on the front and won't do anything no matter how many buttons I press. You may be wondering why I don't just fix it myself. Despite all of the experience I have with fixing household appliances (none) and my familiarity with similar devices (huh?) I've decided that it's just not worth my time to try to fix the thing myself (although having to stay home from work to wait for the repairman somewhat negates that position). I tried some things. The manual says that it may be a bad heating element and to check the wiring. I checked the wiring (well, looked at it anyhow) and nothing. Some advice online said to try pressing a sequence of buttons in rapid succession - nothing. I gave up after that.

Why? Because it's obvious this isn't a hardware problem. And if it's not a hardware problem it's a just plain HARD problem. Mechanical systems are easy - they either work or they don't. No ill-defined states, no invalid inputs, no built-in tests. If something can't happen it physically can't happen. If one gear is moving then the gear in direct contact with it also has to be moving. If something sounds wrong it's probably directly linked to the problem - just follow the connections.

But somewhere in this dishwasher is firmware - code. Code breaks all of the rules. Impossible things happen in code all of the time. Jump to the wrong memory address? You could end up executing impossible code. 'This shouldn't happen! I never called this function! It's impossible!' Mess up with pointers or indices? You could start reading impossible values. 'Umm, an unsigned 8-bit integer CAN'T be 1024....' Use a case fall-through instead of explicit checks? 'That's IMPOSSIBLE! That value isn't handled! Why isn't it going to default...'

Code is arbitrary. I have a flashing light. That's not a symptom of the problem - it's an indication. I need a manual to tell me what it means, and if that doesn't suffice I need the REAL manual - the one they only give to the service technicians (well, sell anyhow). It means whatever they tell me it means. A mechanical device is not arbitrary. If my engine is overheating it doesn't do something illogical like lock the drive shaft automatically - it just starts heating up. All according to rules Mr. Newton figured out (differential equations baby!). You can track it and explain it with rules you find in your high school physics book. The code only follows the rules that Mr. Programmer set forth and he's not bound to adhere to any standards and even if he was he wouldn't tell you. Furthermore, the state of a machine is obvious and can be ascertained by observation (perhaps complicated observations, but still, observations). Code need not give any indication of state and typically doesn't, or it's not very useful (green light - good, red light - bad!)

And if you aren't rigorous in your code you can have undocumented behavior. It's entirely possible for you not to be able to reach one of your states or not be able to leave it if you make simple mistakes. And if your testing doesn't catch it then you'll have to reboot your dishwasher or some other such silliness.

Computers are very powerful, but simple. They execute instructions at memory addresses. Everything else is defined by the designer. If he or she is incomplete in the definition of the system then irrational behavior follows. Mechanical systems are bound by the laws of physics: they cannot perform physically impossible functions, they must move smoothly between states that are defined by their physical characteristics alone, and failures are characterized by smooth transition to a new state(one which is obviously broken). Software doesn't follow these rules and that makes it much more difficult to troubleshoot.

PS: It was bugs Living on the electronics. Dirtying them up. I'm not a dirty person I swear. It wasn't a vague error code or random impossible to get to state, it was just flat out broke. At least even computers have flat-out broke modes...

Laziness Continued

2009-04-30T19:36:00.000-07:00

I am a fan of taking steps out of processes. Before tonight in my TinyCAD library creation process I had to do a few things:

Update my CSV file with new attributes and/or data

Update the schematic symbols in TinyCAD

Export the symbol library as XML to a certain folder

Run a Python script to turn each symbol in that XML file into its own XML file

Run my main library creation Python script

Open TinyCAD to check for errors. If any symbols are wrong, repeat from the second step

I hadn't really used it earnestly yet but it seemed like too many steps. So I decided to add functionality to my library creation script to pull the symbol data from the symbol library file directly instead of XML files created from that library. Tonight I got it to work, so now the process is:

Update CSV file

Update symbols in TinyCAD

Run library generation script

Check for errors

This is a great improvement. I may actually get work done now!

Exceptional Programming

2009-04-28T18:48:00.000-07:00

I grew up programming. My first computer was a Laser 128 - an Apple IIc clone. I learned AppleSoft BASIC. Those were the days when the command line was also the BASIC interpreter. I programmed everything on that computer. Well, if 'everything' is some science fair projects and cute programs with blocky graphics. I grew up and got an 8086 and cut my teeth on QuickBASIC, then eventually C. In college I was taught C++ and I also picked up Perl and PHP. But despite all of this I never learned about exceptions until I earnestly got into Python.

We see exceptions all the time on our computers. "Firefox caused an exception in blah blah blah blah." I didn't really know what it meant past 'the program screwed up'. I had done most of my programming in C which doesn't have exceptions (or if it does I've never used them). Sometimes I screwed up in my programs and all sorts of crazy things happened. Gibberish printing out on the screen, random lockups, the computer starts beeping and won't stop, etc. I can't be sure, but once I think I caused the gibberish to come out on the printer. I'm probably imagining that though.

But after working with Python I found out those screwups were actually called exceptions. And what's more, you could handle them. Just put everything inside a try statement and if something bad happens you can catch it below and go on your merry way. It's great to just be able to deal with it and continue. But I noticed something. Exceptions weren't always exceptional. Try to access element number -1 of an array? C would happily let you do it and you could go crazy trying to figure out why. But in an exception-driven language it would catch it for you and stop you. But some functions throw exceptions instead of just telling you that you did something wrong. Well, I suppose that's HOW they tell you you did something wrong. But if you have three statements inside a try block and you just get a generic exception back, you're going to need to do more work to figure out what went wrong. And imagine my horror when I learned that some people will use exceptions for something as mundane as input validation! Maybe I'm just old fashioned, but in C you did your own input validation. You didn't just accept whatever the user gave you and then throw a fit when it turned out to be wrong. But what's worse is that this wasn't on your run-of-the-mill PC, but on an embedded device!

Exceptions? On an embedded device?

I've always been told that embedded devices needed to be ultra-reliable. All failure modes had to be accounted for and handled. True, exceptions are one way of handling failure, but for a lazy programmer it's entirely too easy to wrap the entire program in one big try block and just restart in case something bad happens. It's too easy to not even plan for the failure modes because you have exceptions. That may fly for your DVD player but certainly not for your airplane.

So what's the alternative? For one, strict input validation for everything - not just user data. Don't even assume that your own code will always pass you valid values in functions - always check! But then how do you signal an error? In C many functions would return a non-valid value if there was an error. For instance, abs() might return -1 if there was an error (and you had better check!). But this scheme doesn't always work. For some functions there may be no possible invalid values. You can't take a chance on using a valid value as your error signal. Sure, you'll bury the true meaning of the return value somewhere in the documentation but honestly who's going to look until there's a problem? And by then they've already cursed you for being so tricky.

No, the solution is to return a status along with your return value. In some serial systems the receiving device will respond to any message with a status message to tell you that it successfully received your information. Typically zero is 'OK' and everything else has a specific value - either the byte as a whole means something or each bit has significance. This can be done with functions as well. You can either pass all of the parameters as reference and the status as the return variable, or return the status and parameters in a struct if you don't like pointers. Of course it may be more work to work with a struct (and it's dangerously close to object-oriented programmng!). In this way a function can tell you that it failed and why it failed. You can use an enum to define all of the different error messages and then handle each of them. Of course some errors cannot be handled by your code alone. If data shows up late in a control system there's not much you can do about it except not use it in calculations. And your status returned will tell you if it was late (assuming you check it).

Exceptions are very good form for PC applications but it's easy to be lazy with them. Embedded programming sets a higher bar for the programmer, so bring your A game when developing for embedded devices. Always check inputs, parameters and status returned to make sure that you are working with valid data. The passengers on your airplane will thank you.

Update: I had a conversation with one of my colleagues on this issue and we're of the same opinion - in a sense. In essence he agrees with me except but he thinks exceptions are a valid method of achieving the same goals. BUT - we both agree that in embedded development you shouldn't be catching general exceptions or just wrap your entire main() in a try block. You have to know what failure you expect to happen and exactly how to correct it in EVERY case - exceptions or not. The way exceptions work on PCs (Oops, Firefox caused an exception - it's quitting) is unacceptable for embedded development. I shudder to think that the same approach would be applied to an embedded device by an unaware programmer, but that doesn't indict exceptions in general. Just their misuse.

Laziness (and Python)

2009-04-25T18:51:00.000-07:00

I wrote enough last time to scare some of you about serial communication. I was going to write about the design of a simple serial messaging protocol to make all of you readers feel better, but my laziness got to me. I mean, it's all designed in my head, but I figured I'd need tables and diagrams and figures and such to really explain it well. That riled up my innate laziness, so I decided to write about that instead.

To put this in some perspective, I am supposed to be the library maintainer for TinyCAD. It's an open-source schematic capture program, and development on it just started again after a few years of languishing. My duties are to put together libraries of basic schematic symbols (resistors, capacitors, diodes, some ICs, etc) for other people to use.

I'm not doing a good job. I have released no libraries yet.

I do admit - I'm lazy. And this job requires a fair bit of tedious work. The libraries are stored in Microsoft JET database format and the symbols can only be edited with TinyCAD's built-in drawing tools. They're OK, but not if you want to make two dozen symbols at once. Or worse yet, make slight changes to two dozen symbols at once. Or just add a bunch of meta-data (part name, manufacturer, part number, etc). You have to create all the fields for every part, edit them manually, save, etc etc. The entire process was not made for batch creation and editing. I was in danger of getting absolutely no work done at all. Or worse yet, I was in danger of doing lots of tedious work and then re-doing it when I needed to make slight changes to every part I had already made. Something that realistically I should only have to do once and then forget about it.

I am no fan of manual operations, especially when they are tedious and repetitive. Humans are not geared towards that sort of work - we make more mistakes and are much less efficient than a small shell script. When I was making wirelists my brain would just shut off after a while and I would do all sorts of really wrong things. Or I would cut and paste things that shouldn't be cut and pasted because although I thought two things were the same they weren't. My wirelist had tons of errors that luckily weren't too expensive to fix. But the lesson is clear - don't use people for repetitive operations. They're bad at it.

So when I was faced with the prospect of doing just that for these libraries I said 'no' and started immediately with the laziness. Laziness is bad of course because it keeps you from getting work done. And since I wasn't required by force of law or paycheck to make these libraries I wasn't too insistent on starting a process I knew would be frustrating and error-prone. I did nothing until I learned about Python.

Python is a scripting language that is designed to be easy and do everything. It is very close to succeeding. As I said before these symbol libraries were made with Microsoft Jet Database. It's the back end of Microsoft Access. I am not entirely impressed with Microsoft products but I knew SQL queries and it supports those so I had some baseline. And I had Python. A little searching and I figured out which module I had to import to allow access to the database. A little more searching and I figured out how I could insert the BLOB (Binary Long OBject, or maybe Binary Large OBject? - it's how raw data is stored in fields in databases) for the symbol drawings. Then I said to myself, why don't I store the symbol text data in a CSV file so I can just type things out once? Python has an import for that too. TinyCAD can export the parts data as XML? Great, I can import DOM to access the XML and create a CSV out of the current libraries.

If you've ever used Matlab scripting you'll feel right at home. Heck, there's even SciPy - it mimics many of the functions of Matlab (graphic, matrix operations, basic math, etc) in case you're too poor to buy Matlab. You can use wxWidgets or any other GUI library to create GUI apps. You can access almost any database. You can draw with TKInter. You could lose your voice from listing all the things you can do. It's easy, comprehensive and powerful.

Python gave me a reason for my laziness. I can do more work, more accurately and faster than if I had tried to do everything by hand. Bottom line: if you're avoiding doing some tedious work, pick up Python or any scripting language and do yourself a favor by making a tool to do the work for you. You'll thank me.

The scary world of serial protocols...

2009-04-21T09:50:00.000-07:00

Serial is one of those things that may seem easy at first glance but in reality is so complicated that you want to hang yourself. I used to see plenty of job descriptions that required something along the lines of 'knowledge of serial protocols'. What a joke! 8-N-1, 115200bps, nine pin serial connector DONE! And you get BITS out the back end when you're done. Are people afraid of BITS?

Some people are afraid of BITS (I'm looking at you CS majors), but that's not the trouble with serial protocols. The trouble is that 'serial protocols' is really vague. The first protocol you might think of is good old RS-232. It's a lovely standby, 8 bits per message, no parity, one stop bit and your choice of baud rate. Why would anyone want to deviate from that? But what do all of these settings actually mean? It's swell if you have a GUI to enter these values in, but it gets harder when you have to configure a microcontroller with nothing but assembly. And even then what actually HAPPENS when you send a message? What does the waveform even look like? These may be idle questions for you until the first time that something doesn't work and you have to dig deeper than the GUI to fix it.

For instance what are the voltage levels of RS-232? The answer - heh. There is no answer. According to the standard it's supposed to be +/- 15V. That'd be easy except that no one follows the standard!. You might get +/-12V, or 0 and 5V (TTL levels). The scary thing is that most of these work because transceivers are often not too picky about voltage levels. And then when you figure that out you'll be surprised to know that the voltage levels are the inverse of what you'd expect - a '1' is -15V, a 0 is 15V. If you don't know that you'll have at least 20 minutes of confusion.

Add to the fact that even if you've figured out your RS-232, there are MANY more serial interfaces out there - all different. I2C and SPI are synchronous - there's a clock signal transmitted unlike RS-232. You don't have to worry about addresses with RS-232 since you've only got two devices communicating, but not so with the others. And have you even thought about hardware handshaking? Unnecessary with RS-232 but crucial to the others. I hope you can figure out why you need an open collector output with I2C...

And don't even get me started about data representation. Do you think bits are just bits? That 0x35 is always equivalent to 53 decimal? Not so fast. That could be ASCII-encoded which would mean that it represents the digit '5', not a decimal value of 53. If you look at your serial data stream in HyperTerminal the output you are seeing is decoded from ASCII. That means that 0x35 will display '5', not 53 decimal. ASCII encoding is useful when data is being displayed on a terminal, but it's also used in other circumstances where you'll tear your hair out because of it.

For instance many serial buses have packets with headers, footers, checksums, etc. Packets are usually started with 0x02 hex and ended with 0x04. You can transmit the length of the data packet when you send it so that your device will know when to start looking for another one, or you could choose not to. In that case, what if the data you're sending has an 0x04 in it? That would tell the device to stop listening and would ignore the rest of the data. So, you encode all of your data with ASCII. If you need to send the value 0x04 (decimal value 4, obviously) then you encode it to ASCII and send it as two bytes - 0x30 and 0x04. When you receive it, chop off the 3's and push the nibbles together and get 0x04 again, there's your data.

And that brings up another point - nibble order. Do I send the least significant nibble first (the '4') or the most significant ('0')? What does the standard say? There isn't one. You have to be told how to interpret the data by whoever put it together, and God help you if you didn't do it yourself.

Suddenly 8-N-1 doesn't sound as simple as it used to. Yes, it's all just BITS, but they're scary bits. They're bits that mean whatever the person on the other end of the conversation wants them to mean. You have to parse them, reorder them, combine them, split them and take a magnifying glass to them to make any sense of it. Watch out kids - it's a jungle out there.

Better pathfinding

2009-03-04T10:03:00.000-08:00

I think Computer Scientists are so cute when they pretend that they can build robots. Now I'm not knocking all of them, but there's a select group of shall we say unworldly computer scientists who are too far up in the clouds for robotics. Software is of course very necessary for robots but so are electronics and mechanics. Just TRY to build a robot on your own or even work with a pre-made robotic base if you don't understand the first thing about torque vs angular velocity and how it relates to power. Or beat your head against a wall when your robot takes a heading of 3 degrees vs the 0 degrees you told it to. Why DOES it do that? Surprise! Your sensors aren't that accurate even though your numerical precision is.

Anything having to do with kinematics is going to confuse the bejeezus out of someone who isn't familiar with it. True, it's all stuff you learned in physics but the devil is in the details. It took me a good month to really internalize the roles of current and voltage when dealing with motors (in short - current is torque, voltage is speed but it's only this simple at steady state!). And while it's very straightforward to say the reason your car can't turn on a dime at 50mph is momentum, it's a much harder thing to tell me exactly how quickly your car CAN turn at that speed. This is the realm of physicists, mechanical engineers and me - the controls engineer. It's part of my job to do the calculations that tell you how much effort you will expend to turn your car at that speed (and whether it's safe).
So it's no surprise to me that I hear more about what data structures to use for pathfinding, how to make it computationally less complex and whether it always produces the shortest-cost path (all GREAT fodder for computer scientists) than I hear about whether the path is actually something useful. Most discussions I've heard ignore what will actually happen when you attempt to follow that path. For instance - if your GPS has you making 90 degree turns every 5 seconds I don't care how 'least-cost' your path is in terms of the criteria you assigned because it requires too much effort (starting, stopping, slowing, turning, etc). It also certainly won't be the least time intensive path.

The level of 'smoothness' of motion is a concern in controls. Jerky motion requires much more power than nice smooth motion. It's hard to get things to stop and change direction all the time. I once worked with a digital servoamplifier which had several different 'motion profile' modes. I needed to feed it position commands every .1s. So the first mode I tried had it move as fast as possible to the commanded position and then STOP. Then start up again when it got the next command, move as fast as possible and then STOP when it reached the position. That's the first thing you'd try if you had never designed anything like this before and it turns out to be an awful approach. A much better approach was the second motion profile - attempt to arrive at the commanded position but keep moving in the expectation of the next position command.

Now apply this to pathfinding: Are you going to go from Point1 to Point2 then STOP, change direction and go to Point3 (then STOP)? Not ideally. You want a smooth path. You need to take into effect several things: current speed, heading, difficulty of turning, etc. You make a model of your vehicle and compute how difficult it would be to actually execute all of those crazy moves the algorithm things up. So instead of just finding the shortest (distance) path and you weigh the two factors against each other. Just make sure that your weights prevent impossible actions ('Go straight ahead at 50MPH then stop after 50.18 yards and make an instant right turn and immediately resume 50MPH. It'll get ya there in 23 seconds flat. LEAST COST BABY YEAH!'). This will lead to more realistic paths whose costs more accurately affect realistic situations. That's better for everyone.

A quick implementation of this would be to add it to the movement cost in A* (I've heard this called the G score of a square). In the simplest A* algorithm I implemented the movement cost for a 90 degree movment (forward, back, left, right) was always 10, and for a diagonal movement was 14 (sqrt(2)*10 because of the added distance to reach a diagonal square). I would keep track of my current heading in terms of degrees. Assume that the top of the screen is zero and right is 90 degrees, bottom is 180 degrees left is 270 degrees. Heading in this case will be either 0, 45, 90, 135, 180, 225, 270 or 325 degrees. Just add a cost proportional to the change in heading to the G score - this will keep your path from having too many jerky motions. You'll have to fiddle with the weight to balance least cost vs. least effort (you don't want a lazy robot do you?). You can then get more advanced by incorporating speed and momentum into the model. To incorporate speed, allow the pathfinding algorithm to search more than just the squares around it but with a caveat: it's difficult to speed up and slow down. Thus, my normal speed is 1 square/turn - that gets me to the forward, back, left and right squares for a cost of 10 usually. When I want to go to the diagonal squares I have to increase my speed to 14. If I want to go to the forward square two squares ahead of me my speed has to be 20 - I have to speed up. Speeding up takes effort so add to the G score a cost proportional to the absolute value (absolute value because any change in speed requires an effort - you don't gain effort for going from hi speed to low speed) of the difference between my current speed and next speed AND use the speed in the effort cost calculation for turning (higher speeds make turning harder).

Once you start down this path there are many physical parameters you can add to the system. I have yet to implement this but I look forward to some excellent pathfinding in the future!

Process Process Process!

2009-02-19T07:12:00.001-08:00

I know I probably won't make any friends with some engineers when I champion process. Many (younger) engineers are free spirits that love to do things their own way. To them each problem is a unique flower that requires a highly-specialized engineering ninja to swoop in, dismember the flower's head, toss it in the air, slice it up and carefully place a decorative plate underneath the pieces as they fall to form an intricate and subtle artistic pattern which he will present to the customer. At least that's his intent. Conversely, a more seasoned engineer will look at the problem and say 'Oh God not THIS again....' and go back to his back of tricks to pull out the Excel spreadsheet or Python script or piece of hardware that solved this problem for him in minutes last time (after taking hours if not days to create in the first place!). The seasoned engineer isn't trying to impress anyone - he's trying to get the job done. Last time he had this problem he DID try to impress someone and ended up hurting himself by taking way too much time to finish - he probably missed a deadline and maybe a large chunk of money by trying to be clever and 'unique'. So he figured out a process to get the job done quicker with fewer missteps. He wants to help the younger engineer by teaching him how the process will help him save time and effort, but what the younger engineer hears is 'Don't be creative! Don't be clever! Fit into this box!'

The truth is we all want to be creative to some degree. It's part of our hierarchy of needs - right at the top. Many people see process as drudgery that inhibits creativity. Who wants to file a bunch of forms to design something? Who wants to document when they could invent? And I agree. I would question the truthfulness or sanity of someone who said they truly enjoyed documenting their latest Lego creation rather than spreading the pieces all over the floor and putting them together. But we have to have a sense of perspective about this. Documentation and process save us from drudgery and boring work. It's not FUN to have code that just won't compile, or a bug you can't track down for the life of you. It's not FUN to build a Simulink model that does nothing other than vibrate itself to pieces. It's frustrating and sometime humiliating. It certainly doesn't help my self-esteem to fail so much.

So when you DO figure out how to make the program compile or the model work, you write it down. You share it with everyone - partly to prove you're smart, but partly so they don't have to go through it like you did. And then when you get a few more good tips you compile them together, format them, codify them, and put them somewhere that everyone can see. You keep track of the resources you used, the rules you followed, the steps you took - all so no one has to go through the same experience you went through: just follow this process and go right from A to F, skipping B-E! Yep, you just created a process, and now some young blood is telling you that you're cramping his style and he's going to do things his own way. Oy.

Folks, process may not be glamorous but it's there to help you! It's there to make sure you don't have to do extra, unnecessary work. It's there to help you get to the fun part more quickly. Trust me - someone who was there before you figured all this out and wants to save you the trouble. Just go with it.

The Dangers of Object-Oriented Programming

2009-02-12T14:55:00.001-08:00

In theory I love OOP (Object Oriented Programming) - up to a point. I love having properties. If I want to set one of my discrete I/O ports active then I like to be able to say something like DIO[0].State = Hi. Perfect right? It's at least better than, say: DIO_State[0] = Hi. I think that's a little sloppy.

I'm not so hot on methods. Someone who is might try this: DIO[0].Assert(). Of course you'll also need DIO[0].Decline() (? I'm not fully sure on the opposite of assert at the moment. It's been a long day.) But then you also need something like DIO[0].Tristate() and DIO[0].UnTristate() and DIO[0].Toggle() which start to get a little hairy.

But why do you need those? Oh, that's because you made the State property 'protected' so that people can't set it directly. That's another tenant of OOP - don't let people touch things that they shouldn't. Instead provide 'safe' methods to change properties instead of letting people do it themselves (because they're EVIL! EEEEEVIL! They'd set your properties to invalid values and THEN what would you do?)

And then you go farther! Obviously since you save time when you use objects (another tenant of OOP) you decide to use them everywhere. So you've got relays too, and relays are kinda like DIO right? They go on and off - it's perfect! We'll just extend the DIO type to encompass relays too. So I can Assert() and Decline() relays to close and open them, and I can toggle them and I can Tristate() them... Ummm... wait... Relays don't tristate, do they? But it's a method of the relay object type so maybe they do? I forget...

Ok! Wait! I've got the answer! OOP also has this thing called inheritance where you can define an object type and then create more specific types from it (and from reading the tenants of OOP it seems this is good design practice). So I'll create an object with basic on/off functions and then extend that into DIO which will have tristate functions!

So now we have a parent type that I'll call a, ummm, 'Boolean' object. And Boolean will have a 'Open_Switch' method for the relays and a 'Close_Switch' method.. Or, hmmm. It won't make sense to 'Open_Switch' a DIO port, so we'll just have to call it something more general like 'Frob'. Yeah, 'Frob' will set the DIO high or close a relay and 'UnFrob' will do the opposite. And then 'XFrob' will toggle.

This is really coming together!

Of course that was all in my imagination and I made it worse than it had to be (a little anyhow). But it shows the problem with using OOP which is OVERusing it. Writing methods for toggling and setting a DIO port - sure go ahead. Extending a DIO type to relays? No, too far. Don't get sucked in to using things like inheritance just because it exists. A relay is not a specialized DIO port even if they have a couple of the same functions and properties!

PS - It seems Jeff Atwood agrees with me somewhat.
PPS - And again

Now I know he doesn't perfectly champion every single word in my post but I think we agree on general principles here.

Killer Tip

2009-01-01T20:23:00.000-08:00

Well, it's not killer per se. If you've had a bit of experience you probably already do this. I've just started to document what I'm doing with each of my projects so I can start right off the next time I want to work on it (instead of spending a half hour figuring out what the next step is). I just create a file called TODO.txt in the main directory for each of my projects and I update it every time I work on the project saying what I did and what's probably next. Hopefully that will keep me up to date on what I've done.

Now here's the REAL killer tip: automatic timestamp. If you use Notepad++ hit Ctrl+F5 to automatically enter the date/time where the cursor is. And this, THIS is the real killer. It comes from good old Notepad. That's right, Notepad.exe takes the cake for awesome features with this beauty: create a new TXT document and type '.LOG' (no quotes) on the first line. Save it, close it, then reopen it. The date and time is automatically entered right above your cursor. Just start typing.

I can't believe Notepad of all things has such a useful feature. It doesn't even have proper 80 character line wrapping.

(Edit) I've done a little searching and found out that you can also add the timestamp in Notepad by hitting F5. Not nearly as cool though.

More Fuzzy Logic

2008-12-31T13:28:00.000-08:00

Ok, so it may have been hard to follow my previous post, especially with no graphics. Well fear not, I just dug up some old materials from my senior project and there is a nice graphical explanation of fuzzy logic. It's a little big because it was meant as my poster for the final competition, but it should be followable.

Fuzzy Logic

2008-12-29T07:58:00.001-08:00

We all have our little pet methods that we like to use. I for instance will try to use microprocessors (specifically Atmel or PIC) to solve a problem I encounter. You may use FPGAs or CPLDs. Some crazy people will use .NET and embedded WinCE or somesuch. Everyone has different tools and methods that they are familiar and comfortable with.

One of the tools that I use to solve problems is fuzzy logic. I want to say it's not very well known, at least, I don't get many appreciative nods when I mention it to colleagues. I plan to talk about it as a solution to various problems so it will be useful for you listening (Hello? Anyone out there yet?) if I were to explain it and its use.

Where to start? It might be best to start mathematically. In a mathy sense, fuzzy logic is a method to map m-dimensional input space to n-dimensional output space in a continuous nonlinear fashion. The output may or may not be 'smooth' (ie, differentiable at all valid points in both directions), but my gut feeling is that it is. Don't ask me to prove it, I won't. I'm sure it's proved in some paper somewhere. If math is really your thing then I'm sure this will matter to you, so by all means, look it up.

So what does that mean? Fuzzy logic is basically a function. It can take any number of inputs and produce any number of outputs. The output can depend on one or many of the input variables. So if we have a fuzzy logic function that takes two inputs and produce two outputs: [a,b]=fuzzy(c,d) The inputs and outputs are usually limited to a certain range (If you're mathy, it's called input space and output space. Non-mathy types: you too can feel important and smart by saying things like My fuzzy logic function maps a three-dimensional input space to a two-dimensional output space. And I always drink my tea with my pinky sticking out - I'm distinguished that way.). An input may be limited to values from 0 to 20, and for values outside that range you'd either cut it off to the limit value (ie, 0 or 20 depending on where you go outside the range) or extrapolate somehow. I generally choose to cut the values off. Outputs are also limited as a result of the limitations to the input - since the input space is limited so is the output space.

Now there are all kinds of multi-dimensional nonlinear functions. Usually they're defined something like this:

z=f(x,y)

x varies from 0 to infinity
y varies from 0 to infinity

for x=0 and y=0, z=0
for x>0 and y=0, z=0
for x>0 and y>0, z=x*y

This scheme covers all values, is nonlinear, and if you're careful enough in the definition produces a continuous function. The beauty of fuzzy logic isn't the fact that it IS a nonlinear mapping of multi-dimensional input space to multi-dimensional output space (say THAT five times fast!), but the HOW of how it accomplishes that mapping.

This is where the 'fuzzy' part of fuzzy logic comes in. In the above example we have some logic and it's very precise. Inputs have precise values, EXACT values even. And we have logic: If x IS 1 AND y IS 1 THEN z IS 1. What if I don't want to be so exact? What if I want to say IF x is BIG and y is SMALL THEN z IS MEDIUM?

You might be asking yourself 'Why would I want to create a function to express such a concept?' If you're starting off from a mathy 'multi-dimensional input space' type of position then you probably don't see the need for such a statement, or a means to evaluate it. If you're mathy then you like numbers and variables and ranges - oh my! But if you're an actual person (the jury is still out on whether mathy 'people' are actually human) then fuzzy logic is for you. Take the example of tipping at a restaurant.

At a restaurant you usually have to give a tip which is some percentage of your total bill. I evaluate this percentage based on the level of service I receive and the quality of the food. We'll score them both based on a ten-point scale with 0 being the worst, 10 the best. I will generally tip between 0 and 33% of a bill. Let's functionize that:

0 < Tip% < .33
0 < Food < 10
0 < Service < 10

Tip% = f(Service,Food)

Ok, so how much should I tip? Fuzzy logic has the answer.

The first step in fuzzy logic is to 'fuzzify' the inputs. We do this with what are called 'input membership functions'. That's just a mathy way of expressing a concept like: '0 is definitely (100%) awful service while 2.5 is definitely not (0%). All values in between that vary linearly'. If you graph it, that would be a triangle with your peak at the 'definitely' point and then a straight line drawn to your 'definitely not point/points' Since 'awful service' was at the low end of our range for service it only has one 'definitely not' point. For 'OK' service we might place the peak at 5 and the low points at 10 and 0. It will be a triangle that spans the whole range with its peak at 5.

For both Service and Food we'll define 'Awful','OK', and 'Great' levels. Awful peaks at 0 and bottoms out at 5, OK peaks at 5 and bottoms out at 0 and 10, and Great peaks at 10 and bottoms out at 5. Now for any numeric input value from 0 to 10 we can describe how accurately the service is 'Awful', 'OK', and 'Great'. For instance:

Service = 5
0 = Awful(Service) %The service is not at all Awful
1 = OK(Service) %The service is definitely OK
0 = Great(Service) %The service is not at all Great

Service = 7.5
0 = Awful(Service) %The service is not at all awful
.5 = OK(Service) %The service is somewhat OK
.5 = Great(Service)%The service is somewhat Great

It is worth noting that these membership functions don't have to look like triangles - they can look like squares or gaussian functions or trapezoids. The complexity of the calculations to determine value of the membership function increases, but not the general idea.

Now that we know to what extent the level of Service or Food matches our descriptions (Awful,OK,Great), we can create the output with rules. This requires output membership functions. These work pretty much like the input membership functions but in reverse. Each output membership function has an output number associated with it as well as the degree to which it is asserted. For tipping I'll say that the tip is 'Small' if it's 0%, it's 'Normal' if it's 16.5% and it's 'Big' if it's 33%. These membership functions also have shapes associated with them but for an expedited example it only matters where their nominal output values (or center points if you're thinking with shapes) are on the output axis. The output number is determined by doing a weighted sum of each nominal output value times the degree to which it is asserted. Thus we can say that the tip should be 33% Low, 50% Normal and 75% Big, and if we do a weighted sum we get:

0*.33+16.5*.5+.75*33 = ~8.5% tip (roughly)

So once we know how much each output membership function is 'asserted' we can tell how much we should tip. How do we tell how much each output membership function is asserted?

That's where rules come in. To determine the level of assertion we make rules that read like:

1) IF SERVICE IS GOOD AND FOOD IS GOOD THEN TIP IS NORMAL
2) IF SERVICE IS BAD AND FOOD IS GOOD THEN TIP IS NORMAL
3) IF SERVICE IS GREAT AND FOOD IS BAD THEN TIP IS LOW
etc...

You'd probably make a lot of these rules for a tipping system, and depending on how much you value food versus service your rules would be different than everyone elses. So in the above example if the degree to which the Service is Good is .5, and the degree to which Food is Good, how do we evaluate the AND? There are several ways. You can multiply the two .5's and get .25 for the level of the tip being Normal or we could choose the maximum of the two values. There are other ways, but it has to make some sense. You can also have OR's as logical operations (the OR operation is usually a minimum of the two) and NOT (usually 1-value). How the operations are evaluated changes based on the type of system you are putting together, so don't be too shy about the 'right' way. My AND's are usually the product and my OR's are usually minimum.

So using these operations you assign some weight to each output (Normal, Big, Low). As seen above, multiple rules produce weight for Normal. When this happens, you either add all of the weights together or select the highest. When you've gone through each rule you'll have a weight for each output:

Low = 0
Normal = .5
High = .5

Then you go back two steps above and perform the weighted average of the output values. In this case you'd get:

0*0+.5*16.5+.5*33 = 24.75% tip

That must have been some good service (or maybe food)!

So, if you've followed me this far, you can see that fuzzy logic excels at creating nonlinear multidimensional functions in a way that humans can relate to (ie, in a way that matches our decision-making process). It produces real number outputs from real-number inputs, and it has a firm mathematical basis (that you can look up if you want, I'm satisfied that it works). I hope you can see all of the potential uses for this, because my hands are getting tired from typing so I'm signing off. Tune in later for some practical controls-related uses of fuzzy logic.

Timekeeping Cont'd

2008-12-28T06:49:00.000-08:00

As the idea has been rolling around in my head I've thought up some logical extensions. Both timekeeping units - master and slave - could be implemented in programmable logic, probably a CPLD. The slaves could have an internal delay oscillator (several inverters in a row) which would be the basis of measurement of the incoming clock signal. The slaves could also do clock doubling or tripling of the incoming clock signal to provide a fast clock to whatever programmable logic or microcontroller is their client. That would be accomplished by use of the internal timer/counter in the timekeeping slave unit - determine how many ticks the external period is in your internal clock, then divide by 2,4,8,16, whatever and get a faster clock that OUGHT to be fairly correct compared to the incoming clock signal. The unit could also provide the bare external clock signal (suitably buffered) for use with an RTC timer/counter on the client device.

I was also thinking that due to all of that transmission line stuff that I try to ignore it might be better to transmit the clock signal as a sine wave. Doing that with programmable logic would be harder, but it might be required in certain circumstances. Certainly a differential signal is probably necessary in most environments. I would say that repeaters might be necessary, but repeaters will introduce delay. It might not be too much but it wouldn't be consistent. You may hook one slave up after a single repeater, and hook another up after 10 repeaters, and the signal will be more delay for the second slave.

I'm beginning to wonder if this is patentable. In my miniscule experience (<2 years in the controls field) I haven't seen anyone use this system, nor have I seen anyone attempt to synchronize different subsystems to this degree. Usually if you want everything to be on the same clock you put it in the same unit - programmable logic, microcontroller, whatever. For hard real time applications that's pretty necessary. The question is whether people have thought of this but ignored it, not thought of it, or not thought it practical?

Timekeeping

2008-12-20T16:16:00.000-08:00

Recently I've been designing an embedded hard real-time environment for controls purposes in my head. I've gotten somewhat detailed with it and you can see some of the plans I've made here: http://steves-wiki.wikispaces.com/ Look at the microcontroller architecture headings (and look at everything else while you're there). It's not finished yet but I think I have some good ideas down there. One of the major issues in a hard real-time system is timekeeping. If you have two or more chips of any type (microcontrollers, programmable logic, RTC, etc) that need to keep or be aware of time then you have a problem.

Ideally there is a 'time' right now. December 20, 2008 7:22:33.44 PM. We humans need to coordinate times on the level of minutes usually. Our clocks are always minutes off of each other and we still function well enough, but control systems need to keep accurate timing on the order of hundredths of a second if not less. As my mentor likes to say "In a real-time system if it's late, it's wrong." We have accurate timekeeping hardware for embedded chips. Crystals are very accurate if used correctly, so just attach it to your controller chip and go. There's only a problem when your system consists of multiple chips with multiple clocks. Even if they all have accurate crystals still all have to be accurate to each other. One chip is doing the controlling and if it needs data from different chips or systems then it MUST be there before it has to send out its next control update or at the very least you'll have degraded performance. Not only must each part have an accurate clock, they all must have the SAME clock.

How do you do this? Off the top of my head I envision a timekeeper, maybe a small microcontroller with an RTC. It will keep accurate time and transmit the current time to each subsystem that needs it. Great! Except not. Serial messages take time to create and send, that's delay. That means when you send your message with the 'time' in it, it's not the real time anymore. If you can characterize this delay and you know that it's insignificant compared to your timekeeping rate then you might be OK. But then you need to have a dedicated serial port for the timekeeper, or use a bus (but then you have to deal with bus contention, possibly delayed time messages, etc). Further, what happens if your messages don't get through? You'd need some kind of ACK response from each sub system, then your timekeeper has to process those. Even then if you don't get an ACK what do you do? Resend and hope for the best. Throw a fault to your superior if you consistently don't get a response.

Alright, let's go a little deeper into this serial idea - I like it. Let's assume that the time is represented with a 16-bit value that is the count of milliseconds since the system started keeping track of time. Our control system's sample time is 10ms, so we'll send time updates every .5ms. With a message structure of 8 bits, no parity and one stop bit, that will be 10 bits total for every data byte, and two bytes of data equals 20 bits. 20 bits times 2 is 40 bits every millisecond, and then multiply that by 1000 for the number of bits per second, and it's 40kbits. That's doable, but we need some overhead and some time in between messages to respond and process, so at least double and probably triple it, find the nearest baud rate and you've probably got 115.2Kbits. Still doable.

Of course every subsystem still has to have a crystal to support serial communication, you use up a serial port, and generally I just don't like this solution too much. Too kludgy, too imprecise, too wasteful. So naturally I'm about to propose a more wasteful but not kludgy and imprecise method (and it will be somewhat more robust).

My method involves a unit that is the timekeeper unit. It has a very accurate crystal and RTC. It will be especially protected from temperature variations. It keeps accurate time and generates a clock signal of say 100KHz. It will be a square wave, probably made differential or somesuch to improve transmission characteristics. Each subsystem will use this signal as the clock into a timer which it will use to keep track of the current timestamp. The timekeeper will have a control line with a pullup that the timekeeper holds low. Each subsystem monitors this line and keep count with the clock signal as long as it's low. Each subsystem is also tied to an open-collector ACK line. Whenever they lose the clock signal they pull it low and the timekeeper signals a fault or does something to correct the situation. If the timekeeper goes away then the control line will go high and let all of the subsystems know. At this point the subsystems go independent: they've been measuring the clock signal with respect to their own internal clocks and can keep fairly accurate time now without the timekeeper (for a while anyhow). When the timekeeper comes back it sends a special signal on the control line and the subsystems know to reset their timestamps to 0 and pick the clock signal up again.

Is this a great system? No. It only works in specific situations - if you have multiple subsystems in a control system that are all actively involved in the process of creating a control signal, you need to know absolute time and not just absolute time differences, and all of the subsystems are far removed enough from each other to make this worthwhile (probably not on the same PCB). Otherwise this could be a cute four-wire timekeeping system.

Finally some anger

2008-12-11T10:04:00.000-08:00

I'd like to share with you something that makes me angry. I'm sure all of you who are engineers have some specific talent or area that you excel in. My specialty is control systems. For those of you who don't know what control systems are let me explain. You know what cruise control is right? Set a speed and your car presses the gas or depresses it to make the car go the right speed. That's a control system: desired output, error, correction. Designing a control system can be easy or hard. For cruise control it's probably not too hard. Heck you can probably get away with something like 'IF CAR IS GOING TOO FAST DON'T PRESS DOWN ON THE GAS. IF IT'S GOING TOO SLOW PRESS DOWN ON THE GAS'. That's a fairly dumb control system, and any engineer could conceive of it and implement it fairly easily.

But if it gets any more complex, or things don't work out like you planned (ie, your car wildly revs and slows your car) you have to retreat and use MATH. That's control theory: how to use math to figure out how to make systems (anything that can be described as a differential equation, this includes physical, electrical - in fact almost anything even if you have to dumb it down) behave exactly how you want them to. The key in that last sentence is math. If something goes wrong with your cruise control system you could 'use math' to change your system to be something like 'IF CAR ISN'T GOING THE RIGHT SPEED PRESS DOWN ON THE GAS BY A*ERROR'. This WILL work better. But you don't know why. The reason is control theory, and until you understand what it is and how it works your cruise control just won't work right.
That is what controls engineers do - they use some tough math to make things behave correctly. It's not easy, it takes a long time to learn and a longer time to perfect. I have a master's degree in controls engineering, so trust me. Trust me also, that job postings like this make me angry:

An aluminum manufacturing company is seeking an ELECTRONIC CONTROLS ENGINEER. Candidates must have 2 - 5 years experience with Allen Bradley PLC's in a manufacturing environment. Job responsibilities include programming process control systems, calibrating, repair and troubleshooting plant instrument, electrical and control systems. A Bachelor's Degree in Electrical Engineering is required. Excellent benefits package available including matched 401k, company-shared health/dental, and paid life insurance. Salary is commensurate with experience.

What's a PLC? I never heard of that in my controls classes. Processes? Plant instrumentation? What are the inputs to my system? What am I controlling? Temperature? Shaft position?

Oh that's right. This job posting is abusing terms. They want an electronic controls engineer, but what they mean is a process controls engineer or manufacturing engineer. Their goal is process control: if I have a conveyor belt I need to know how fast it should go, where things on it are, what actions I have to perform now, etc. That's automation, and it's cool. PLC's are microcontroller systems with some nifty serial interfaces and analog components such as relays, TTL drivers, etc all built-in. Sometimes they use C for programming, sometimes they use BASIC. I know, ew, but it gets the job done. And it's a job that needs doing. But it's not controls engineering!

You couldn't take the person that automates that factory and put him in my chair. I've designed antenna control systems that can point to within an accuracy of .7 degrees using an embedded processor. I've modeled an entire dish antenna system (including mechanical, electrical and software components) in Simulink. These two things are NOT the same job. I'm not blaming anyone or saying anyone's at fault, but it's very frustrating. Ya'll feel me?

Wire Lists, Revisited

2008-12-08T08:41:00.000-08:00

More wirelist fun at work today. I just got introduced to an even more complex and unwieldy and hard to understand wirelist format. It's very daunting - especially concerning the sheer amount of information I have to absorb to create such a list effectively. To give you an idea, I have at least 15 connectors on my testing unit with probably an average of 25 contacts each. these are brought straight through to eight larger more populated connectors that range from high-current connectors wil 20 contacts to low-current low-voltage connectors with a max of 192 contacts. Most of these are utilized. Then, the doohickey that plugs into THAT eight-connector monstrosity has several complications: wiring on its inside to connect signals, and multiple cables and connectors coming off of one plug doohickey. Then it's finally the unit under test with its 15+ connectors. And in between the doohickey and the UUT I have to keep track of which wires are twisted, gauges, colors, etc. Furthermore, this process embodies the problem I stated in the original wire list post: The same information is stored in different methods in different files. Very much ripe for errors.

Of course this is a good thing. I need to know that my XML scheme can entirely define all the information I need to reproduce all of the wirelist files I have at my disposal. This is a very complicated system and if I could represent it I'd be happy. Already I've made several changes based on what I've seen so this should be good experience.

Printers

2008-12-07T18:01:00.001-08:00

My wife and I just got done writing, printing folding, stuffing, posting and addressing about 60 holiday letters to various friends and family, some of them rather distant ('Janet? Ok, so if Arnie is my great-uncle, that's my grandfather's brother, Janet is the wife of one of his kids. Yeah they get a letter). I am continually amazed by my printer's (HP Photosmart C4280) ability to stop working reliably when I have a large task for it to perform. Sometimes it doesn't pick up paper, sometimes it doesn't print straight. Sometimes there's roller marks on it. But ONLY when I ask it to do something significant.

It's a good printer. I got it to replace my old Deskjet 932C which broke horribly when I asked it to do my wedding programs. Maybe it's just that I put odd paper into it at these times. Or maybe it's a conspiracy. I'm voting for conspiracy - after all printer ink is a conspiracy isn't it? (Although this printer's ink is actually cheaper than the last printer's) I just spent $75 in printer ink to prepare for this Christmas letter, and sure enough, it needed both cartridges changed.

What do you do to combat this evil? Buy in bulk? No dice, ecause then your printer will break and you'll have to get print cartridge #80 instead of print cartridge #75 (both manufactured by HP of course, why is there a whole WALL of different types of cartridges?). Then you'll be let with a lot of useless cartridges. Could you perhaps buy online to find a lower price? It's probably lower online, sure, but can you wait a week to get it? I usually buy ink when I run out and I need it for a big job, like today. So waiting isn't an option. Get a laser printer? Cost, and B&W only. Color laser printers are awfully expensive, and let's face it - you don't print that much at home. If you do, like you have a business, yeah, buy a laser printer and write it off. But for a home user? Naw, I rarely print until I print a lot. Then I spend a lot of money.

So what should I do? Nothing. HP and the other print companies have our number. They've planned this ink-debacle perfectly. And quite frankly I don't care. They've structured their whole business around the way I buy and use ink. So they made it expensive. Money isn't the only thing I have to optimize. The printer by and large works, and it has a scanner/copier in it! Score! I have simple needs. When I print large jobs I waste a bit of paper and have to buy new ink. It happens so infrequently that I honestly dont' care. I could more profitably care about beer. What if I'm out of beer? That matters more to me than ink, and HP knows it. They ought to include a coupon for a 24 pack of Miller with every ink cartridge. Hey, two birds with one stone.

Oh wire lists!

2008-12-04T10:43:00.000-08:00

As an electrical engineer, wire lists are the bane of my existence. Let me give you an example:

Suppose that you have a DB9 connector (that's an old serial port connector for those of you not in the know) and connected to it are several LEDs: LED1, LED2, LED3, etc. We seek to organize and list the various pins, connectors, and wires that are associated with this scheme. So, the first list we make is a pinout of the DB9:
Pin,Signal

LED2

LED3

LED6

LED1

LED4

LED5

Just like in real life, this list is not simple. Wouldn't it be great if, for instance, LED1 was on pin 1, LED2 was on pin 2, etc? Not only is that not the case, I didn't even put the LEDs in numerical order! Woe is me! Why am I being so mean to myself? Because that's the way real life is sadly, plus it will all turn out in the end if I have my way anyhow.
So there's my pinout, I'll stick it in an Excel pinout file. But now I want to view this information by LED number, so I make another file that looks like this:
LED #,pin

Great! Now I'm organized. So if I want to make a diagram of the connector I'll just head to the file that's ordered by pin number and draw that out. Then when I'm going to write my code to turn the LEDs on I'll refer to the list ordered by LED number and use that to determine what value I have to shift to an output register to turn that LED on. Then everything will work perfectly and my documentation will match reality AND my code.

Except, oops, it doesn't. Look at LED6. In the list ordered by pin number I put it on pin 4, but in the list ordered by LED number I put it on pin 9. Maybe it was my handwriting - after all 4's look like 9's if you're sloppy. Now I have a diagram that shows LED6 on pin 4, but in my code I assume it's on 9. That LED will not light and I get to spend hours trying to figure out why.

What's the fundamental problem here? We have what should be the same information represented two different ways. But instead we have two separate sets of information each represented different ways. When you split the two lists up with nothing to keep them synchronized and nothing to provide a sanity check between them you're setting yourself up for headaches.

People do this every day. It doesn't always lead to tragedy but it sure does lead to aggravation. What can be done about it? You could use a database. I'm not such a huge fan of databases. I'm sure that that all of my reasons are susceptible to 'yes but's and 'not really's but I've always found them overcomplicated, slow, too confining and difficult to use. Plus if I told people to use databases they might accidentally use Access. And let me tell you: in my experience the only thing worse than misusing Excel as a database is using Access as a database instead.

My preferred method is XML. However XML is so big and vague and useless that it's just not worth it. There are no schemas that define how to write out wire lists and there are no programs to view the resulting XML. That is why I'm writing them.

What I've figured out so far is to use the following organization:

Signal list - this details the names of all the signals present in the wire list (like LED1 above). Each signal will be name and you will be able to attach voltages, currents and other electrical properties to them

Connector - this will define the type of connector, part number, pin type, gender, refdes, etc. It will have a pinlist under which each pin is assigned a signal from the signals list, or N/C

Cable - this will define the wires and connectors. It will reference two or more connectors from the connector list and will have a wirelist section. Inside the wirelist there will be wires, each of a specific type (twisted shielded pair, solid conductor, etc), color, part number, etc. Inside the wires are conductors which are linked to signals from the signal list

Got it? Connectors->pins->signals. Cables->wires->conductors->signals. Signals->electrical characteristics. Everything links together. Let me give an example of a signal list:


<signallist>
 <signal>
  <name>Odd_AC_Signal</name>
   <voltage type="AC" waveform="sine">
    <name>AC Portion</name>
    <amplitude value="115" tolerance="1" unit="V" measurement="RMS" reference="GND"/>
    <frequency value="60" tolerance=".1" unit="Hz"/>
    <phase value="0" tolerance="0" unit="degrees"/>
   <voltage type="DC">   
    <name>Offset</name>
    <magnitude value=" 200=" tolerance="10" units="V" reference="GND">
   </voltage>
  </signal>
</signallist>


And here's the connector:


<connector>
 <refdes>J4</refdes>
 <name>LED Connector</name>
 <type>DB9</type>
 <partnum>M24-09</partnum>
 <gender>M</gender>
 <pinlist>
  <pin pos="5" signal="LED1">
  <pin pos="1" signal="LED2">
  ...
 </pinlist>
</connector>

And then the cable:

<cable>
    <name>LED Cable</name>
    <refdes>K4</refdes>
    <partnum>32345</partnum>
    <connector>J4</connector>
  <wirelist>
   <wire type="SC">
    <conductor signal="LED3" color="brown"/>
   </wire>
   <wire type="TP">
    <conductor signal="LED1" color="blue/white"/>
    <conductor signal="LED2" color="blue"/>
   </wire>
 </wirelist>
</cable>

I think the cable definition is especially nice because you can see which conductors are twisted, what color each conductor is, etc.

Naturally this isn't all fleshed out yet, but you can see the possibilities. From this information you can make almost any document you'll find necessary: tables can be made with HTML or FO/PDF, wiring diagrams can be made with SVG. And you only write it out once, then the XSL transformation will handle the rest - putting signal names to pins or wires, creating a connector pinout or a cable pinout, etc. That means fewer places to mess up. I think it's a good idea, but it will take a lot of work. Right now I'm defining the XML schemas for everything above. When that is done I'll be able to write an XML file that has all of the information in it. Then from that point I just have to write XSL transformations that turn it into output documents.

It is a lot of work, but I think worth it.

2008-12-03T13:57:00.000-08:00

I just got back from a trip to California for Thanksgiving. I live in Florida. In case you're not familiar with geography, that's a long way. Let me outline what we went through to get there:

To California:

2AM - Melbourne to Orlando Airport, arrive by 3:45AM

3:45AM - Park in the satellite parking and take a bus to the terminal

5:00AM - Flight to Atlanta

7:30AM - Two hour layover in Atlanta

9:30AM - Atlanta to Orange County Airport (5 hours!)

11:15AM - Arrive at Orange County Airport, pick up luggage and take a taxi to the train station

11:30AM - Arrive at train station, eat something and wait until 1:45PM for the train

1:45PM - Train is late, wait until 2:15PM

2:15PM - Train to Guadalupe station (5 hours again!)

8:00PM - Arrival at sister's house, very tired

On the way back we got luckier:

2:30PM - Board train

2:45PM - Buy new tickets on the train because the ticket machine at the station doesn't work and we're in a cellular dead zone so they can't call HQ to verify our ticket (at least they cheerily refunded the old tickets)

7:30PM - Arrival at station, take bus to LAX airport (nice service)

8:15PM - Arrival at LAX, security, etc

9:00PM - $11 margarita, $11 entree at an airport restaurant (the margarita was good)

10:15PM - Flight to Orlando (direct!)

6:00AM - arrive at Orlando, pick up luggage, take bus to car

7:30AM - arrive at home in Melbourne, tired, take a sick day because California is too dry for my sinuses and they're angry

For those of you keeping score, it's 19 hours out there, 14 hours back. Moral of the story: when at all possible get direct flights.

Added bonus: trains are pretty nice. They have electrical outlets in every seat. I learned how to play RA3 with a trackpad. I highly recommend the train if you're headed in that direction and its' cheaper than a flight. Again, HIGHLY recommend the train, even if it is filled with highschoolers.

I can haz irony?

2008-12-02T07:10:00.000-08:00

Well I started a blog. I always hated blogs but I suppose I can't withstand the Web 2.0 any longer. Also I hear you can make money with these, somehow. I seek to monetize my anger, thus, I have a blog. I think that's the definition of the internet.

One of my goals in this blog (other than time-wasting) is to write about things that interest me. I interest myself mainly for technical thingies and my preferred topic of the day is DocBook.

In theory DocBook is a good idea. The idea is to take all of the content you put in your document and enclose it in XML tags which determine layout and formatting for you. Take Word for example. When you write something in Word the content and the medium are directly intertwined: you can't (as far as I know) get to a mode in Word where you don't have to worry about page breaks and tab spacing. Everything that you see in the Word window will be there on the print page. If you've been playing the document-creation game for some time then you know that Word is a WYSIWYG (What You See Is What You Get) editor. A WYSIWYG editor is a very good thing. I may not be an old-timer but I've worked with computers long enough to remember when there were WYSINAAWYG editors (What You See Is Not At All What You Get). Trust me, they were a pain. With so many surprises every time you tried to print something. Without WYSIWYG editors desktop publishing would not exist.

DocBook takes a different approach. Rather than start off with the end product (ie, how does the page look? What are the margins? What point font do I use? etc) it chooses to focus on the content and meta-information first and formatting second. If you want a book you start off with a tag and then some tags and all the text goes in a tag and so on. There's special tags for cross-references, figures, acronyms, empahsis, etc. Trust me, if you've seen it done in a book somewhere there's probably a tag for it in the DocBook standard. After you've created all of the content you verify it against the standard, then run it through a converter and get some kind of print output - maybe a PDF, HTML, Help file, etc. Since it's XML you could also theoretically create a transformation to OpenOffice Document or Word XML Document (I've seen some rudimentary Word XML transformations but nothing great). THAT file is your WYSIWYG copy, tada! You're done!

Well, you're really not. The chapter titles look like 'Chapter 1 - The Phantom Meanace' where you wanted them to look like '1- The Phantom Meanace', and you wanted a bigger font for the body text, and when you want to emphasize text you'd like bold AND italics.... In short it doesn't look like you want it. Now you've met the meat of DocBook - customization. By default DocBook outputs don't look so... custom. They're bland and boring, and I'm sure the spacing is off by a bit. The good news is that there are options for some of the changes you want like numbering the chapters. They're easy to set - just put the options you want in an XSL file, include it in your conversion, and it'll work! The bad news is that for a lot of the more minor issues you get to write all-new XSL code and figure out just what the property or attribute you're looking to change is called, then maybe find it in the DocBook XSL files, copy and paste to a custom XSL file, change what you want, try again, change, try, change... It is not easy. It is not straightforward. Get your Google and your patience ready if you want to significantly change DocBook's default output.

So why use DocBook? It seems to be a step backwards towards YHNIWYWG (You Have No Idea What You Will Get) and YHNCOWYWG (You Have No Control Over What You Will Get, also known as the Outer Limits editor). In some respects it is undesirable. However, consider that once you have the formatting correct, it will ALWAYS be correct. One knowledgeable person can create the formatting rules and then any number of people can use them flawlessly thereafter. In a large organization with a need for a standard formatting this can be a real timesaver.

DocBook files are text. Plain ol' text. They're also XML. Use your favorite text editor or programming GUI. I like Notepad++ - it will do the tabbing and tag completion for me! If I need, I can create a python script to read the DocBook XML files and parse them for information, or edit them, or do a simple search and replace. I can create a python script to retrieve information from a database and format it into DocBook XML. Text is also version control friendly.

Consider also that creating DocBook is free. Free as in beer, maybe free as in freedom, I haven't done my homework. With a budget of $0 I can create a PDF from raw text in under an hour. The tools I prefer are free, there are books you can buy but there are also many online resources, free. Word, in contrast, is not free. OpenOffice is, but am I the only one who hates its interface? I don't like using it honestly.

DocBook is write once, publish many: once you've written a DocBook XML file you an convert it many different formats without any additional work.

DocBook encourages reuse and portability: you can include other DocBook files to reuse common sections such as titlepages. You can create an XSL transform to transform any XML data into a DocBook format. Got an HTML table you want in DocBook format? You can make a transform to do it in a jiffy. Got a CSV file? Write a python script to generate a table from it, somewhat jiffily. Got data in a database? Python again FTW. How much data do you automatically want to include in your DocBook XML file? Create a batch file or script to retrieve the most recent data from published sources, transform it into DocBook XML, make sure it is included in your document, then transform your DocBook into your final published format.

Ok, so most of that is a pipe dream that will take LOTS of effort to set up, but remember, once you have a system in place and documented, that system stays there. Put the knowledge in the world, put the process in the world and it won't get lost, and you wont' have to remember it yourself.

For more information on how you (and by you I mean a Windows-using engineer) can get started with DocBook visit my DocBook wiki at Steve's DocBook Reference