I own an Arduino. And why not? I've always liked the AVR - mostly because Atmel has always had a good focus on hobbyist-level tools. WinAVR is an excellent free toolchain and can interoperate with a wide number of IDEs due to the way it mimics the functionality of the GCC toolchain (THE standard C compiler). The Dragon is an extremely cheap ISP with debug functionality and HV/Parallel programming interfaces for when you screw up your fuses and it can program nearly all chips in the AVR line. The AVR architecture is similarly impressive due to its high MIPS/MHz ratio and hardware multiply functionality.
Wow, that was in-depth, I hope I didn't lose anyone. Hey, vast majority of Arduino users: did you get any of that? I'm sorry, I'm sorry, I don't mean to be a jerk. No, the vast majority of Arduino users don't understand that. But that's okay. I'm an electrical engineer - trained and all! I've been programming microcontrollers for, oh.... Oh God - eight years now. I think I might be... old.
So I'm old, and I'm an electrical engineer. That's why I know these things - to me it matters. If an AVR is a bad choice for a project I have to know why so I will be able to tell whether to replace it with a PIC, a 68K or ARM Cortex-M3. Most people who use the Arduino are not electrical engineers, develop embedded software for a living or necessarily have ever programmed before. Now, despite the fact that this causes some pretty hair-pulling questions on Chiphacker (I'll bet that question is gone by the time I post) it's a good thing. Embedded systems have the potential to make our world a much smarter and better place in the hands of those with the right knowledge and world-view. I mean, the most basic application I can think of would be intelligent power sockets for everything - track usage of power, schedule things, auto-turn off of dangerous items, etc. You could make your house more efficient (power-wise) and safer (auto turn-off on a curling iron? Yes please!). Someday, the halting, uncertain steps taken by people who have never soldered before will lead to a better world through practical software.
Which, to me, makes it all the more important that if you're going to have a good idea and succeed at making it ubiquitous you ought to know a thing or two about writing safe, deterministic software that will last forever and gracefully handle all errors.
This is just as hard as it sounds. But worth every bit.
You've seen it just as much as I have. I mean, look at the gas station - the electronic pumps that take your credit card. First off, they beep as if they have something to say and then it takes them three seconds to start to update the display. And then it updates a little bit faster than molasses flows. So it asks you for your zip code. You punch it in the numbers and every time you hit a key you hear the beep beeps but don't see the numbers show up. No, for some reason that takes about 10 seconds per digit. So if you mess up you need thirty seconds or so to fix it. But that's only a little annoying - try this on for size: has your dishwasher ever crashed? Would it refuse to clean anything and spit out an error code you can't look up because you don't have the service manual? Did unplugging it and plugging it back in fix it? Do you know how many lines of code are in your car? Did you know I have a microprocessor in my PEN?
While it's true that anything remotely dangerous with software in it is (generally) tested to some degree to make sure it doesn't kill you we have to ask ourselves - is the shoddy software we see in something so simple as a fuel pump with a display and a keypad something we want to put up with in (eventually) every single device we own? What is so frustrating is that we know how to write software. We know how to read keypads, we know how to write numbers and letters on a display. We know how to dial a phone and ask a remote system whether that credit card number is valid. And if people would just write software correctly we could avoid creating a million devices that need to have their batteries removed every day due to blue screens.
When I say 'we' know how to write this software I mean... I don't quite know who I mean. Dudes at NASA who couldn't rely on taking out batteries because the batteries were IN SPACE! Those dudes knew how to write software that worked - period. The stuff in cars, airplanes, medical devices is not perfect but as a friend of mine would say it's not a cell phone either. The quality is simply higher because otherwise people die. Right now, that sort of expertise costs money - you need the right people to design AND test your software so that it performs to the standards of whatever certifying body you need to satisfy. If there aren't any standards you need to satisfy then you don't bother - you hire someone who grew up on VBScript and PHP instead of someone who grew up on assembly language and eating rusty nails for breakfast.
But the point is that the expertise is out there; the information is out there. Our collective knowledge knows how to write good software, so the goal should be to get the information out there and make it easy to use.
So, back to the original point of this post. What does this have to do with Arduino? Well I'm sad to say that while Arduino is great it's pretty basic. It is somewhat brain-dead: it encourages blocking calls (ie, idling in a do-nothing loop until something happens) rather than use asynchronous interrupts. It skips over more advanced topics that form the basis of some of the techniques which ensure safe software such as callbacks/function pointers. It shuns even basic lessons such as parts selection (newsflash: you're going to use an Arduino. You will buy a shield - you don't need to know what's on it). You're certainly not going to learn anything about design for manufacturability (unless you're making a shield). Ever heard of a state machine? Circular buffer? Process stack pointer vs. machine stack pointer? All very important!
The Arduino is a good place to start, but if you're serious about making something worthwhile that has software in it, your attitude can't be 'I don't care about good correctness, suitability or efficiency. I just care if I can get it to run'. This attitude leads to terrible software and terrible products to boot. It can be avoided. Over the course of... the rest of my life I intend to write a series of articles/guides that instruct people on what the essentials of writing good software/good overall design are and how to apply them. This will involve every aspect of the process I can think of from microcontroller selection to getting a PCB made. I'm going to write the articles, publish the code, open-source the development kits and take a lot of pictures. Yes, there may be selling of things involved, but you only have to buy them if you're lazy. All of the tools (as far as possible) will be free to use and open source and all of the design files will be made available with an open-source license.
If you want your software to be faster, more responsive, more efficient and just overall better, listen for a while. Even if I don't teach you something I may point you in the right direction.
Sunday, February 26, 2012
Wednesday, December 14, 2011
Labview is Actually Not All That Bad
In stark contrast to the flurry of epithets I hurl at Labview on a daily basis at work I will now say good things about it - because it deserves it!
The blocks are actually pretty old school. If you get stuck trying to decode a string in Labview your tools look surprisingly like what you'd expect from the C standard library. Granted - you don't need to worry about NULL termination or anything, but most of the basic building blocks (like string subset, search and replace, etc) promote code that acts (in form) like old-style C code. It's not like writing the routines yourself (iterating through each character, comparing, discarding, buffering, etc.) but the solution you come up with to manipulate strings will probably feel like an old-style C solution does. I also like the way the control structures work - you get nice features like shift registers (that teach you about instantiating a variable before the loop) or straight-through tunnels that just take the last value from the loop and pass it out. You can make complicated while or for loops with conditional terminations or conditional continuances. These are very welcome forms to have present. There's a lot of nice tools and abilities in Labview that don't feel new and gimmicky, but instead old and tried.
Second, while most Labview VIs are just a mess plain and simple what I've found is that a messy VI usually means messy badly-organized code. A clean, well-organized VI means you've created sub-VIs in the right places, organized the right data into clusters, used the right type of loop, etc. If your VI is clean and organized, chances are your code is too. Thus, by seeking to visually clean up your block diagram you can actually write good code. Of course, the opposite is not true: you can have great code that still looks like a mess. Because you just can't get around that in Labview sometimes. But chances are if the block diagram looks good the reason is because of a highly-organized coding mind behind it. It helps people learn how to organize code by presenting it as a visual problem instead of an esoteric abstract one - and many people are simply visual learners.
Third, I'm ecstatic as hell that whenever I Google anything related to Labview I come up with an answer - period. Someone has tried to do what I'm doing or for some insane reason National Instruments make a guide on how to manipulate TestStand sequences programmatically from Labview. Quite simply put I spent hours trying to figure out a problem on my own and when I get the bright idea to Google it the answers is invariably THERE. Done in five minutes. At this point in my career I'm a little more focused on results than banging my head against a wall to 'learn' so that feels really good. I have no idea why all of this support is out there but it is and it makes me happy.
Fourth, Labview is actually free of a lot of the object-oriented crap that plagues many trendy languages today. Yes, Labview is actually a bit trendy by itself, but (and this impresses me because the more I think about it the more I realize it's true) Labview is actually pretty old-school. Some old-school hard core stuff you can't do (like function pointers - but then only kinda) but I'm pleased that I haven't seen the term 'inheritance' once when dealing with Labview. True, all the things that make me hate C++ might be in there but I haven't been forced to deal with it and I haven't seen anyone else's work that deals with it either.
Don't get me wrong - I still find plenty to hate about it (and I may get to that later) but the more I consider it Labview feels like C but graphical. And that's not terrible.
The blocks are actually pretty old school. If you get stuck trying to decode a string in Labview your tools look surprisingly like what you'd expect from the C standard library. Granted - you don't need to worry about NULL termination or anything, but most of the basic building blocks (like string subset, search and replace, etc) promote code that acts (in form) like old-style C code. It's not like writing the routines yourself (iterating through each character, comparing, discarding, buffering, etc.) but the solution you come up with to manipulate strings will probably feel like an old-style C solution does. I also like the way the control structures work - you get nice features like shift registers (that teach you about instantiating a variable before the loop) or straight-through tunnels that just take the last value from the loop and pass it out. You can make complicated while or for loops with conditional terminations or conditional continuances. These are very welcome forms to have present. There's a lot of nice tools and abilities in Labview that don't feel new and gimmicky, but instead old and tried.
Second, while most Labview VIs are just a mess plain and simple what I've found is that a messy VI usually means messy badly-organized code. A clean, well-organized VI means you've created sub-VIs in the right places, organized the right data into clusters, used the right type of loop, etc. If your VI is clean and organized, chances are your code is too. Thus, by seeking to visually clean up your block diagram you can actually write good code. Of course, the opposite is not true: you can have great code that still looks like a mess. Because you just can't get around that in Labview sometimes. But chances are if the block diagram looks good the reason is because of a highly-organized coding mind behind it. It helps people learn how to organize code by presenting it as a visual problem instead of an esoteric abstract one - and many people are simply visual learners.
Third, I'm ecstatic as hell that whenever I Google anything related to Labview I come up with an answer - period. Someone has tried to do what I'm doing or for some insane reason National Instruments make a guide on how to manipulate TestStand sequences programmatically from Labview. Quite simply put I spent hours trying to figure out a problem on my own and when I get the bright idea to Google it the answers is invariably THERE. Done in five minutes. At this point in my career I'm a little more focused on results than banging my head against a wall to 'learn' so that feels really good. I have no idea why all of this support is out there but it is and it makes me happy.
Fourth, Labview is actually free of a lot of the object-oriented crap that plagues many trendy languages today. Yes, Labview is actually a bit trendy by itself, but (and this impresses me because the more I think about it the more I realize it's true) Labview is actually pretty old-school. Some old-school hard core stuff you can't do (like function pointers - but then only kinda) but I'm pleased that I haven't seen the term 'inheritance' once when dealing with Labview. True, all the things that make me hate C++ might be in there but I haven't been forced to deal with it and I haven't seen anyone else's work that deals with it either.
Don't get me wrong - I still find plenty to hate about it (and I may get to that later) but the more I consider it Labview feels like C but graphical. And that's not terrible.
Wednesday, October 5, 2011
Good Coding Practices #...?
I have a semi-ongoing series in good coding practices... to the extent that I ever update my blog anyhow. Lately I've taken up work on an iTunes Android Sync tool. Is anyone else amazed that there are very few good programs that will sync iTunes playlists to an Android device? There are some out there, certainly but for some reason each of them tends to have one or two major flaws: way too slow, randomly renames your songs to the titles of different songs, costs money - the usual complaints. So I looked into it and it turns out that iTunes maintains an XML version of its library information file. But then I discovered that the Android playlists are stored in a highly technical format called ASCII-encoded text files. Let me tell you, it took forever to crack that puppy.
Well when all you have is a hammer every problem looks like a nail. When you have Python every problem looks... easy. So I decided to make a Python program to:
1) Read the XML library file.
2) Figure out what playlists are in there
3) Figure out what songs are in those playlists
4) Figure out where those songs are
5) Make Android playlists from the iTunes Playlists
6) Copy the playlists and music files to the Android device
7) DANCE!
So at first I tried using my favorite parser - SGML parser. But it turned out that SGML parser doesn't handle truncated tags. You know - the ones with nothing in them? With only a start tag that has a / in it and then it's done? Yeah, those. So I had to switch to expat which isn't so bad either.
But enough of that! I'm going to show you what I did that's a good coding practice. The iTunes XML file has several parts in it: a general section that describes the library, a tracks section that describes each track and assigns it a unique ID, a playlists section that describes the playlist and lists the unique track IDs in the playlist.
I wanted to start off by parsing all the goodness of the general library section and ignore the rest while at the same time planning ahead so I would... be able to figure out where to put the code to parse the rest of it as well. To that end I present a random code snippet:
Some explanation: this function handles data inside of tags. It handles key tags specially, but handles tags that contain data (integer, string, date, etc) differently still depending on which section they reside in. So you can see I've written the code that handles the data in the library section but left out handling data in all the other sections. But this is by design: if I wasn't planning ahead I wouldn't have put the if statement that checks what the parent is in that function. I would just have put the code that handles data for the library section without verifying that I was still in the library section - and then it would have handled a whole lot of data in the rest of the file.
By putting the parent key check in there and explicitly listing the different situations that I want to code for I'm doing two things. First, I'm specifying the exact situation I expect this code to run in - putting my assumptions right out there in the code. Second I specify all of the other situations that I haven't yet coded for but want to in the future. I'm using the code to inform myself (in the future) that I need to put code there that does something different. That's the good coding standard.
It can be used in a variety of languages. In Python use the above form but make sure that you put the pass statement in an empty case - otherwise it gets angry. In C you can use #warning directives to produce a warning when you know you'll have to write some code but just haven't yet. Like '#warning Will Robinson, you didn't handle the default case!'
Well when all you have is a hammer every problem looks like a nail. When you have Python every problem looks... easy. So I decided to make a Python program to:
1) Read the XML library file.
2) Figure out what playlists are in there
3) Figure out what songs are in those playlists
4) Figure out where those songs are
5) Make Android playlists from the iTunes Playlists
6) Copy the playlists and music files to the Android device
7) DANCE!
So at first I tried using my favorite parser - SGML parser. But it turned out that SGML parser doesn't handle truncated tags. You know - the ones with nothing in them? With only a start tag that has a / in it and then it's done? Yeah, those. So I had to switch to expat which isn't so bad either.
But enough of that! I'm going to show you what I did that's a good coding practice. The iTunes XML file has several parts in it: a general section that describes the library, a tracks section that describes each track and assigns it a unique ID, a playlists section that describes the playlist and lists the unique track IDs in the playlist.
I wanted to start off by parsing all the goodness of the general library section and ignore the rest while at the same time planning ahead so I would... be able to figure out where to put the code to parse the rest of it as well. To that end I present a random code snippet:
def handle_data(self,text):
if self.current_tag == KEY_TAG:
self.current_key = str(text)
print "Key: " + text
elif self.current_tag == INTEGER_TAG or self.current_tag == STRING_TAG or self.current_tag == DATE_TAG or self.current_tag==TRUE_TAG:
if self.current_parent == LIBRARY_KEY:
if self.current_key in libraryKeyList:
print self.current_tag + "=" + text
self.tempDict[self.current_key] = text
elif self.current_parent == TRACKS_KEY:
pass
elif self.current_parent == PLAYLISTS_KEY:
pass
elif self.current_parent == TRACK_KEY:
pass
elif self.current_parent == PLAYLIST_KEY:
pass
else:
pass
self.current_key = ""
Some explanation: this function handles data inside of tags. It handles key tags specially, but handles tags that contain data (integer, string, date, etc) differently still depending on which section they reside in. So you can see I've written the code that handles the data in the library section but left out handling data in all the other sections. But this is by design: if I wasn't planning ahead I wouldn't have put the if statement that checks what the parent is in that function. I would just have put the code that handles data for the library section without verifying that I was still in the library section - and then it would have handled a whole lot of data in the rest of the file.
By putting the parent key check in there and explicitly listing the different situations that I want to code for I'm doing two things. First, I'm specifying the exact situation I expect this code to run in - putting my assumptions right out there in the code. Second I specify all of the other situations that I haven't yet coded for but want to in the future. I'm using the code to inform myself (in the future) that I need to put code there that does something different. That's the good coding standard.
It can be used in a variety of languages. In Python use the above form but make sure that you put the pass statement in an empty case - otherwise it gets angry. In C you can use #warning directives to produce a warning when you know you'll have to write some code but just haven't yet. Like '#warning Will Robinson, you didn't handle the default case!'
Sunday, July 3, 2011
On Excellence
I fancy myself pretty good at C. Not great, but pretty good. I can find my way around source code, I can write from scratch, I can debug with the most average of them. I'm handy in a variety of ways and I eat source code for breakfast.
This didn't happen in college. This didn't happen in high school. This didn't happen in freaking grade school. I have been programming since I was 6. I started off with AppleSoft BASIC on an Apple IIc knockoff (Laser 128c was the correct answer for those of you playing the home game). After AppleSoft BASIC it was GW-BASIC on the 8088 and QuickBASIC on the 286 and up. But roundabouts high school I decided I had to learn C - because that was the language that grownups used. So I bought a C book (the right one as it turns out - if you want to learn C get this book first), downloaded DJGPP and got to work!
I'm having a computer weekend (putting a computer whose hard drive failed back into working order) so I'm going through old files looking for utilities and just plain reminiscing. I decided to see what my old C code looked like.
Oh God, it's awful.
It's terrible. Here's an example (with some helpful comments from future Steve):
In case you were wondering, I used malloc() and, no, there is no corresponding free() call. I relied on the fact that the OS would free the memory once the program exited. There were variables defined in .h files (no, not extern defines, plain old defines). There were what should have been arrays of constant strings were 'initialized' using sprintf (copy constant to string) rather than just initializing them when the array was defined (as any normal person would do).
And the best part is that the whole program I made basically amounted to a regular expressions parser. All I needed to do was remove images and other formatting from HTML files so they'd be easier to print off and use less ink. That could have been done a lot easier.
This was 1999. So, becoming average takes at least 12 years of constant use of a skill - and I still screw up. Looking at this I can see a lot of myself in new grads coming out of college - the same mistakes, the same assumptions, the same basic design assumptions that end up making bad code (even if it runs). It falls in line with Malcom Gladwell as he writes in Outliers: if you do something for 10,000 hours you'll be great at it. It's not necessarily innate skill, it's practice, practice, practice. It's why I'm a professional programmer and not a professional trombone player - I just code a whole lot more than I play trombone.
So knowing this I can see where new grads come from - hey, they haven't had 10,000 hours of programming, they probably haven't had 10,000 hours of anything engineering related from their college experience. 10,000 hours is 3.5 years at 8 hours a day and that's just for one skill. Engineering is a whole plethora and if you don't know where your career is taking you, why bother practicing one skill over another?
That's my strawman argument - I don't agree with it. My question is - if you know you won't have you 10,000 hours in whatever you want to be good at by the time you graduate college (and by extension, be able to show some really awesome work to a prospective employer) why didn't you start earlier? Did you plan to be average? To be right in the middle of the pack, to not stand out? To be, essentially, replaceable by any other member of your graduating class? Did you plan to go out into the job market and have a big corporation tell you what you should be good at instead of deciding it yourself? Didn't you get into engineering for a reason?
I see people on both sides of this question and you can tell them apart right away in an interview. The people who don't know why they're engineers - the ones who didn't start early excelling at something they loved and wanted to do - come to a job interview and basically want you to tell them what kind of career they should have. They hit the middle of the road for all of their classes - probably picked whatever electives were the most popular because they didn't really care about the difference, didn't have an opinion on what they wanted, didn't find anything particularly exciting and just followed their friends into a class. Their senior projects were whatever they were assigned and they just sort of did them. They don't speak about them with passion, they just wanted to graduate and they needed a project, so they did it. And they're not dumb - a lot of them have 4.0 GPAs for what its worth. But since they didn't know what they wanted to do they never got in-depth on anything. They never really put together the pieces that every single topic in electrical engineering is inexorably linked to every other. Analog circuits mean differential equations, considerations of bandwidth, frequency response, frequency content of waveforms, Fourier Series, linear algebra, matrix equations and any other number of fields of study. The coursework isn't a checklist, it's a symphony of learning. But if you don't have passion it is just a checklist. Mom and dad want you to be an engineer. You're smart, so you do well in classes and you graduate with a high GPA. You go for a job at a big corporation and they grind you into whatever kind of employee/engineer they want. Yay for you - you're average.
But the ones who have passion and drive and love what they've done - they stand out immediately as well. They had a definite plan when they went to college. They'll tell you how they took apart TVs when they were a kid (good Lord it's dangerous - let your kids do it but make sure those capacitors are discharged) or how they wrote dumb little computer games in Visual Basic to entertain themselves. They just won't shut up about their senior project or whatever personal projects they have if they haven't started their senior project. Their eyes light up when you suggest ways they could improve their project ('Ooooh my God... I wish we could go back and work on that more - I'd make it so much better!') or they kept working on it themselves after they graduated. They've learned weird programming languages for fun. In essence, they love what they do and they just won't stop doing it. They don't ask you for direction, they tell you - I'm this kind of engineer and I love doing it, do you need me? And the answer is usually yes, we need you.
So essentially the choice is yours: You can be average or excellent. There is certainly a long road between the two, but you have the choice to take the journey and practice practice practice. And if you find out that whatever you had your eye on doesn't really interest you then fine - move your target, pick something else. Excel at something. Hell, maybe you'll have 10,000 hours of random junk you've practiced. That's okay - it makes you an excellent generalist. Don't just sit around and play video games - ply your craft. I guarantee there's a payoff even though it's a long way down the road. Yes, a very long way. But it's worth it.
This didn't happen in college. This didn't happen in high school. This didn't happen in freaking grade school. I have been programming since I was 6. I started off with AppleSoft BASIC on an Apple IIc knockoff (Laser 128c was the correct answer for those of you playing the home game). After AppleSoft BASIC it was GW-BASIC on the 8088 and QuickBASIC on the 286 and up. But roundabouts high school I decided I had to learn C - because that was the language that grownups used. So I bought a C book (the right one as it turns out - if you want to learn C get this book first), downloaded DJGPP and got to work!
I'm having a computer weekend (putting a computer whose hard drive failed back into working order) so I'm going through old files looking for utilities and just plain reminiscing. I decided to see what my old C code looked like.
Oh God, it's awful.
It's terrible. Here's an example (with some helpful comments from future Steve):
i=-1;
fseek (readfile,0,SEEK_END); //Set starting point to end
size = ftell(readfile); //Find file size
fseek (readfile,0,SEEK_SET); //Set starting point to start
//FS - Seriously? Is there no better way of getting the size of a file?
//FS - Oh god, who gave me malloc?
readfiledata = malloc(size); //Allocate memory for char
printf("Filename is %s\n", argv[1]);
printf("Size is %d\n", size);
printf("Copied %s to steve.tmp\n", argv[1]);
//FS - WHAT?! WHAT?! Index of -1?
i=-1;
do
{
//FS - SERIOUSLY!? Is there no easier way to get all the data in an array? Did you not look?
readfiledata[i] = fgetc(readfile); //FS - OH GOD YOU ACCESSED INDEX -1 OF AN ARRAY!
i++;
}
while(!feof(readfile));
In case you were wondering, I used malloc() and, no, there is no corresponding free() call. I relied on the fact that the OS would free the memory once the program exited. There were variables defined in .h files (no, not extern defines, plain old defines). There were what should have been arrays of constant strings were 'initialized' using sprintf (copy constant to string) rather than just initializing them when the array was defined (as any normal person would do).
And the best part is that the whole program I made basically amounted to a regular expressions parser. All I needed to do was remove images and other formatting from HTML files so they'd be easier to print off and use less ink. That could have been done a lot easier.
This was 1999. So, becoming average takes at least 12 years of constant use of a skill - and I still screw up. Looking at this I can see a lot of myself in new grads coming out of college - the same mistakes, the same assumptions, the same basic design assumptions that end up making bad code (even if it runs). It falls in line with Malcom Gladwell as he writes in Outliers: if you do something for 10,000 hours you'll be great at it. It's not necessarily innate skill, it's practice, practice, practice. It's why I'm a professional programmer and not a professional trombone player - I just code a whole lot more than I play trombone.
So knowing this I can see where new grads come from - hey, they haven't had 10,000 hours of programming, they probably haven't had 10,000 hours of anything engineering related from their college experience. 10,000 hours is 3.5 years at 8 hours a day and that's just for one skill. Engineering is a whole plethora and if you don't know where your career is taking you, why bother practicing one skill over another?
That's my strawman argument - I don't agree with it. My question is - if you know you won't have you 10,000 hours in whatever you want to be good at by the time you graduate college (and by extension, be able to show some really awesome work to a prospective employer) why didn't you start earlier? Did you plan to be average? To be right in the middle of the pack, to not stand out? To be, essentially, replaceable by any other member of your graduating class? Did you plan to go out into the job market and have a big corporation tell you what you should be good at instead of deciding it yourself? Didn't you get into engineering for a reason?
I see people on both sides of this question and you can tell them apart right away in an interview. The people who don't know why they're engineers - the ones who didn't start early excelling at something they loved and wanted to do - come to a job interview and basically want you to tell them what kind of career they should have. They hit the middle of the road for all of their classes - probably picked whatever electives were the most popular because they didn't really care about the difference, didn't have an opinion on what they wanted, didn't find anything particularly exciting and just followed their friends into a class. Their senior projects were whatever they were assigned and they just sort of did them. They don't speak about them with passion, they just wanted to graduate and they needed a project, so they did it. And they're not dumb - a lot of them have 4.0 GPAs for what its worth. But since they didn't know what they wanted to do they never got in-depth on anything. They never really put together the pieces that every single topic in electrical engineering is inexorably linked to every other. Analog circuits mean differential equations, considerations of bandwidth, frequency response, frequency content of waveforms, Fourier Series, linear algebra, matrix equations and any other number of fields of study. The coursework isn't a checklist, it's a symphony of learning. But if you don't have passion it is just a checklist. Mom and dad want you to be an engineer. You're smart, so you do well in classes and you graduate with a high GPA. You go for a job at a big corporation and they grind you into whatever kind of employee/engineer they want. Yay for you - you're average.
But the ones who have passion and drive and love what they've done - they stand out immediately as well. They had a definite plan when they went to college. They'll tell you how they took apart TVs when they were a kid (good Lord it's dangerous - let your kids do it but make sure those capacitors are discharged) or how they wrote dumb little computer games in Visual Basic to entertain themselves. They just won't shut up about their senior project or whatever personal projects they have if they haven't started their senior project. Their eyes light up when you suggest ways they could improve their project ('Ooooh my God... I wish we could go back and work on that more - I'd make it so much better!') or they kept working on it themselves after they graduated. They've learned weird programming languages for fun. In essence, they love what they do and they just won't stop doing it. They don't ask you for direction, they tell you - I'm this kind of engineer and I love doing it, do you need me? And the answer is usually yes, we need you.
So essentially the choice is yours: You can be average or excellent. There is certainly a long road between the two, but you have the choice to take the journey and practice practice practice. And if you find out that whatever you had your eye on doesn't really interest you then fine - move your target, pick something else. Excel at something. Hell, maybe you'll have 10,000 hours of random junk you've practiced. That's okay - it makes you an excellent generalist. Don't just sit around and play video games - ply your craft. I guarantee there's a payoff even though it's a long way down the road. Yes, a very long way. But it's worth it.
Monday, June 20, 2011
Fathers Day
I just read a nice article about someone whose father taught him how to build, use tools, make things. Reading that, it occurs to me that I've never heard a 'My father taught my how to code and I'm grateful for it.' story. It seems like everyone who codes well has been mostly self-taught - a loner. I wouldn't have wanted it then, but looking back now I would have liked to have been taught something so wonderful by my father. I hope we have these stories someday.
That is all.
That is all.
Wednesday, May 25, 2011
Documentation
People say they hate writing documentation, but what they really hate is Word. And even Word would be okay if no one cared about formatting. Once you have to conform to these corporate styles things get so awkward - oh, you used 11 point font instead of 10, your margins are .05" off, you can't use a table here because it doesn't justify correctly. I've been in peer reviews where the only comments people have are formatting (and spelling errors). It's such an anti-pattern.
Wouldn't it be much better if documentation were like wikis? Where anyone can find the document they want to edit? Where all you need is a web browser to edit it and it's just text? Sure you have to use *'s instead of bullets maybe, or -'s or something, but have you looked lately how many different options you have for bullets in Word? It's insane. I'd rather have one ugly bullet.
So sure wikis are simplistic, but they're straightforward and you get to focus on writing instead of margins. But they don't work for real engineering, right? Real engineering documents are version-controlled, have complicated title pages, fancy diagrams and backgrounds that say things like 'UNCONTROLLED'. Wikis couldn't ever.... or could they?
Step one: version control. Github now has wikis. But Github doesn't just do wikis - anyone can do wikis. Github does version-controlled wikis. Wikis are written in text-based markup: typically Markdown, MediaWiki, etc. But they're all text - just text. Github saves each page you create in the wiki as a text file in a repository separate from the project you're working on. The only non-ideal thing about this whole setup is that all of the wiki pages are stored in one directory - no structure at all. So if you want to create a block diagram for a sub-assembly in a sub-directory you'll have to figure out how to store that information somewhere. I'm considering storing the directory information in the name somehow, but this may be a bit unwieldy.
So you'll end up with a bunch of text files with odd markup stored in a repository separate from your project. Surely there must be a way to take these text files, written with special markup, and turn them into something (dare I say) pretty? Well of course there is - Github takes the text files and creates web pages doesn't it? So yes, it can be done and it will be done. There's an open source program called Pandoc that describes itself as a swiss army knife for transforming markup formats. If you look at the list it can exchange between a lot of formats. Very neat, very useful. Now instead of text files you can get PDFs or... DocBook.
Now with the PDFs you get PDFs that look like nice, printable versions of web pages. Basic but serviceable. But engineering documents from real engineering companies don't just look serviceable - they look complicated. They're full of revision history blocks, referenced documents, government standards and the aforementioned 'UNCONTROLLED' backdrops. You can still do this in this approach but you need a lot more finesse. Enter DocBook. DocBook is used to create... books. You know all of those programming books with different animals on the front? Like this one? If my history is correct, they're all written in DocBook and in fact O'Reilly invented DocBook so they could write their books easier. That's why the all pretty much look the same. That and I guess those folks are boring.
The great thing about DocBook is that it's customizable. The input files are just XML, but the output is usually PDF - just print it off, bind it, draw a fish on the front and you've got a book. Or, if you want an engineering document, you describe some table layouts for revision history, title page, etc, fill out that information in your XML file, transform it and then you've got an engineering document. True, that will be a LOT of work, but so is trying to use Word to do the same thing. Best of all, Git is version control, so your revision history is built-in: you can parse Git commit logs to fill out the revision history section. If your referenced documents are in version control (which would be a good idea) then you can link right to them. And DocBook has all sorts of other neat features built in: automatic table of contents creation, automatic figure referencing with hotlinks, you name it. It's worth looking in to.
Text is great, yes, but we all scream for graphics. The Github wikis can reference documents from your project repository on the Github wiki, so including graphics in the online wiki is not really a problem, but what about on locally-produced PDFs? This might bet hairy. Pandoc has a different format for specifying image links than the Github wiki has. Luckily Pandoc is an open-source project so you can modify it to your heart's content if you so like. I might just figure out something else. So the workflow looks like this:
Wouldn't it be much better if documentation were like wikis? Where anyone can find the document they want to edit? Where all you need is a web browser to edit it and it's just text? Sure you have to use *'s instead of bullets maybe, or -'s or something, but have you looked lately how many different options you have for bullets in Word? It's insane. I'd rather have one ugly bullet.
So sure wikis are simplistic, but they're straightforward and you get to focus on writing instead of margins. But they don't work for real engineering, right? Real engineering documents are version-controlled, have complicated title pages, fancy diagrams and backgrounds that say things like 'UNCONTROLLED'. Wikis couldn't ever.... or could they?
Step one: version control. Github now has wikis. But Github doesn't just do wikis - anyone can do wikis. Github does version-controlled wikis. Wikis are written in text-based markup: typically Markdown, MediaWiki, etc. But they're all text - just text. Github saves each page you create in the wiki as a text file in a repository separate from the project you're working on. The only non-ideal thing about this whole setup is that all of the wiki pages are stored in one directory - no structure at all. So if you want to create a block diagram for a sub-assembly in a sub-directory you'll have to figure out how to store that information somewhere. I'm considering storing the directory information in the name somehow, but this may be a bit unwieldy.
So you'll end up with a bunch of text files with odd markup stored in a repository separate from your project. Surely there must be a way to take these text files, written with special markup, and turn them into something (dare I say) pretty? Well of course there is - Github takes the text files and creates web pages doesn't it? So yes, it can be done and it will be done. There's an open source program called Pandoc that describes itself as a swiss army knife for transforming markup formats. If you look at the list it can exchange between a lot of formats. Very neat, very useful. Now instead of text files you can get PDFs or... DocBook.
Now with the PDFs you get PDFs that look like nice, printable versions of web pages. Basic but serviceable. But engineering documents from real engineering companies don't just look serviceable - they look complicated. They're full of revision history blocks, referenced documents, government standards and the aforementioned 'UNCONTROLLED' backdrops. You can still do this in this approach but you need a lot more finesse. Enter DocBook. DocBook is used to create... books. You know all of those programming books with different animals on the front? Like this one? If my history is correct, they're all written in DocBook and in fact O'Reilly invented DocBook so they could write their books easier. That's why the all pretty much look the same. That and I guess those folks are boring.
The great thing about DocBook is that it's customizable. The input files are just XML, but the output is usually PDF - just print it off, bind it, draw a fish on the front and you've got a book. Or, if you want an engineering document, you describe some table layouts for revision history, title page, etc, fill out that information in your XML file, transform it and then you've got an engineering document. True, that will be a LOT of work, but so is trying to use Word to do the same thing. Best of all, Git is version control, so your revision history is built-in: you can parse Git commit logs to fill out the revision history section. If your referenced documents are in version control (which would be a good idea) then you can link right to them. And DocBook has all sorts of other neat features built in: automatic table of contents creation, automatic figure referencing with hotlinks, you name it. It's worth looking in to.
Text is great, yes, but we all scream for graphics. The Github wikis can reference documents from your project repository on the Github wiki, so including graphics in the online wiki is not really a problem, but what about on locally-produced PDFs? This might bet hairy. Pandoc has a different format for specifying image links than the Github wiki has. Luckily Pandoc is an open-source project so you can modify it to your heart's content if you so like. I might just figure out something else. So the workflow looks like this:
- Draw your tables, graphics, etc in whatever program you use locally.
- Use command-line tools (as part of a makefile) to export the local graphics to a GIF or JPG format so they can be included in your documentation.
- Save the newly-exported graphics in a common area of your project repository.
- Commit your changes to Github.
- Write your documentation in a Github wiki and reference the graphics you just committed. This will produce easily-accessible online documentation.
- Retrieve the wiki changes from Github to your local wiki repository.
- Modify the local copies of the wikis to allow Pandoc to run on them seamlessly.
- Run Pandoc on the wiki text files to create either PDF output or DocBook output and copy it to the correct place in your project repository directory structure
- If you just want PDFs, you're done. If you created DocBook output then there will be another step to distill the DocBook to PDF after running it through all of your custom stylesheets.
- Commit your changes and you're done
Wednesday, May 18, 2011
Tool Vs. Patterns
It's hiring time where I work - mostly new grads. That means lots of confusion. The disparity between what new grads expect and what actually happens in industry is sometimes wide.
For instance, my company might say 'We're looking for people with knowledge of VHDL to program FPGAs'. Plenty of students take a VHDL class or two - so you'd think that'd be a good fit, yes?
Yes... and no. The tool is not the end product. Whatever is made will be made with VHDL, true, but the end product will not materially depend on VHDL being the tool used to create it. In fact you could use Verilog to achieve the same, or even schematic capture. Or even step back a bit: digital logic is fundamentally NAND gates or NOR gates. You can make any digital device from NAND gates - everything from a half adder to an iPod. But you won't get a job anywhere just because you memorized the pinout of an SN7400 IC - the important part is what you're creating moreso than how. Fundamentally, being familiar with common patterns used in your field of choice is the mark of a good developer moreso than a strict adherence to syntax.
I'll illustrate with an anti-example that's more in my line. Let's talk C code. 'But it compiles!' has been long the defensive cry raised from many a young coder to defend their petty messes. Their unspoken assumption is that adherence to syntax is the sole qualification of a good C coder. This is patently untrue and a prime example is the anti-pattern of placing all of your code into main(). We've all done it - in fact it's the first thing you learn in class because they haven't taught you function calls yet. But is it a suggested practice? Not in the slightest. It produces code that's difficult to read, difficult to control (requiring the use of many gotos (warning: considered harmful) and just in general a pain. Of course the proper way to write code is the use of function calls to compartmentalize functionality and efficiently reuse it. The only people who deny this are defensive greenhorns more concerned about compiler errors than writing maintainable, readable and efficient code. Knowing syntax is a necessary condition for being a good programmer/engineer/developer, but certainly not a sufficient condition.
What about a straight example of why you should pay more attention to patterns than syntax? Something that looks awful but is brilliant, effective and clean. I present to you Duff's Device:
(Code formatting is always iffy on this blog so you'll have to forgive me.)
What manner of insanity is this? It's truly mind-breaking when you first look at it. So much so that when I show it to new grads/students the first words out of their mouths are more often than not 'Well that won't compile'.
And at this point I get an evil, evil smile on my face. They've shown their hand - more concerned with syntax than patterns. I assure them that it does and the response follows 'But it's not right!' Oh but it compiles! So it must be right, no? Oh the joy I have at this point - I've defeated a mere child in a battle of wits (Ok, I'm not a great person). None of them attempt to figure out what's going on - which is sad. Because once you understand the purpose and function of this code you recognize at least two patterns right off:
This is your basic memory copy with a twist - the destination address isn't incremented. If you're really smart and had a focus on embedded systems you might realize that's because it's meant for a memory-mapped I/O register so the address won't change between writes.
You might recognize this as loop unrolling. Loop unrolling is used to minimize the overhead incurred from jumping around in a loop: instead of doing one thing n times you do four things n/4 times. Useful.
Go ahead and read the full explanation of Duff's Device. If you can follow, you'll see why it's an amazing piece of code and why it vindicates the idea of patterns. Tom Duff understood assembly programming and knew how to implement loop unrolling in it - he was familiar with the pattern. He wanted to do it in C but couldn't think of a straightforward way so he abused the hell out of C syntax to make it work. It makes you want to cry for joy and in pain because it looks like that poor, efficient program is being tortured to death.
This is why patterns are more important than syntax. Patterns are like tools: if you're working too hard, chances are there's tool to help you be lazy. And new engineers usually work way too hard - most often at reinventing the wheel. They'll figure out some complicated method of ensuring single-access to a variable because no one taught them about semaphores. Oops. Patterns help us write efficient and correct code because they are distilled knowledge - the lessons of programmers before us given form. It pays great dividends to know about them.
That's what 'x years of y experience' on a resume means: I know a lot of patterns, I have a lot of tools in my toolbox, I won't make stupid mistakes because I'm past that. If knowing syntax is intelligence then knowing patterns is wisdom (go D&D!). If you don't have both you will never be as awesome a programmer as Tom Duff.
For instance, my company might say 'We're looking for people with knowledge of VHDL to program FPGAs'. Plenty of students take a VHDL class or two - so you'd think that'd be a good fit, yes?
Yes... and no. The tool is not the end product. Whatever is made will be made with VHDL, true, but the end product will not materially depend on VHDL being the tool used to create it. In fact you could use Verilog to achieve the same, or even schematic capture. Or even step back a bit: digital logic is fundamentally NAND gates or NOR gates. You can make any digital device from NAND gates - everything from a half adder to an iPod. But you won't get a job anywhere just because you memorized the pinout of an SN7400 IC - the important part is what you're creating moreso than how. Fundamentally, being familiar with common patterns used in your field of choice is the mark of a good developer moreso than a strict adherence to syntax.
I'll illustrate with an anti-example that's more in my line. Let's talk C code. 'But it compiles!' has been long the defensive cry raised from many a young coder to defend their petty messes. Their unspoken assumption is that adherence to syntax is the sole qualification of a good C coder. This is patently untrue and a prime example is the anti-pattern of placing all of your code into main(). We've all done it - in fact it's the first thing you learn in class because they haven't taught you function calls yet. But is it a suggested practice? Not in the slightest. It produces code that's difficult to read, difficult to control (requiring the use of many gotos (warning: considered harmful) and just in general a pain. Of course the proper way to write code is the use of function calls to compartmentalize functionality and efficiently reuse it. The only people who deny this are defensive greenhorns more concerned about compiler errors than writing maintainable, readable and efficient code. Knowing syntax is a necessary condition for being a good programmer/engineer/developer, but certainly not a sufficient condition.
What about a straight example of why you should pay more attention to patterns than syntax? Something that looks awful but is brilliant, effective and clean. I present to you Duff's Device:
send(to, from, count)
register short *to, *from;
register count;
{
register n=(count+7)/8;
switch(count%8){
case 0: do{ *to = *from++;
case 7: *to = *from++;
case 6: *to = *from++;
case 5: *to = *from++;
case 4: *to = *from++;
case 3: *to = *from++;
case 2: *to = *from++;
case 1: *to = *from++;
}while(--n>0);
}
}
(Code formatting is always iffy on this blog so you'll have to forgive me.)
What manner of insanity is this? It's truly mind-breaking when you first look at it. So much so that when I show it to new grads/students the first words out of their mouths are more often than not 'Well that won't compile'.
And at this point I get an evil, evil smile on my face. They've shown their hand - more concerned with syntax than patterns. I assure them that it does and the response follows 'But it's not right!' Oh but it compiles! So it must be right, no? Oh the joy I have at this point - I've defeated a mere child in a battle of wits (Ok, I'm not a great person). None of them attempt to figure out what's going on - which is sad. Because once you understand the purpose and function of this code you recognize at least two patterns right off:
*to = *from++
This is your basic memory copy with a twist - the destination address isn't incremented. If you're really smart and had a focus on embedded systems you might realize that's because it's meant for a memory-mapped I/O register so the address won't change between writes.
switch(count%8){
case 0: do{ *to = *from++;
case 7: *to = *from++;
case 6: *to = *from++;
case 5: *to = *from++;
case 4: *to = *from++;
case 3: *to = *from++;
case 2: *to = *from++;
case 1: *to = *from++;
You might recognize this as loop unrolling. Loop unrolling is used to minimize the overhead incurred from jumping around in a loop: instead of doing one thing n times you do four things n/4 times. Useful.
Go ahead and read the full explanation of Duff's Device. If you can follow, you'll see why it's an amazing piece of code and why it vindicates the idea of patterns. Tom Duff understood assembly programming and knew how to implement loop unrolling in it - he was familiar with the pattern. He wanted to do it in C but couldn't think of a straightforward way so he abused the hell out of C syntax to make it work. It makes you want to cry for joy and in pain because it looks like that poor, efficient program is being tortured to death.
This is why patterns are more important than syntax. Patterns are like tools: if you're working too hard, chances are there's tool to help you be lazy. And new engineers usually work way too hard - most often at reinventing the wheel. They'll figure out some complicated method of ensuring single-access to a variable because no one taught them about semaphores. Oops. Patterns help us write efficient and correct code because they are distilled knowledge - the lessons of programmers before us given form. It pays great dividends to know about them.
That's what 'x years of y experience' on a resume means: I know a lot of patterns, I have a lot of tools in my toolbox, I won't make stupid mistakes because I'm past that. If knowing syntax is intelligence then knowing patterns is wisdom (go D&D!). If you don't have both you will never be as awesome a programmer as Tom Duff.
Subscribe to:
Posts (Atom)