The Skeptical Methodologist

Software, Rants and Management

The Most Important List Your Manager Will Never Read

There has been no silver bullet in software development, but there have been a few dozen good lead ones that I’m finding it more and more hilarious that go completely unheeded.  Whether it’s skepticism from management about new development practices, or skepticism from developers themselves because some new technique violates a dogma that they held in particularly high esteem.  This second type of hold up is particularly dangerous since it usually results in a developer shooting themselves in the foot just because they refuse to learn something new.

I’ve decided to put together a list of five of these lead bullets that if you aren’t familiar with them, you really ought to make yourself so.  These are things that really can aid in  your productivity, but not in any sort of slave driving way.  They’ll help you ENJOY programming again, and as such, should be important to be perused by managers as well as developers who want to institute practices to get the most out of their developer’s potential.

I would also like to hear from you as to what YOU believe has really helped in your productivity, since this list won’t be in any way exhaustive and will of course suffer from huge biases on my part😉

And so, to begin…

1. The REPL Loop

Most languages people still use today have a familiar code-compile-run cycle born out of the roots of punch cards, but it need not be so.  REPL, standing for ‘read-evaluate-print-loop’, institutes a type of interactive programming that facilitates incredibly fast turn around compile times and ‘live’ coding and testing sessions.  REPL was instituted first by LISP, however, many other languages have some sort of REPL today.

Adding a REPL window to your task bar that normally has your IDE, text editors, etc., running will give you a quick place to build those ‘dirty’ programs to test out a new idea or concept, or to refresh exactly how some API works.  The best REPL capable languages assume this amount of interactivity in the built-in API and provide functions to interactively explore packages, like a Help() function that can take in a method, package or object and tell you what you can do with it, what it does, what traps you should avoid in it’s use, etc.

2. Dynamic Typing / Type Inference

These are two related technologies that really yield completely different type systems, but in the cases of productivity, tend to work in similar ways to the developer and provide the same development boost.  Compare this:

vector<int, myAlloc> x = make_vector(objects);

to

x = make_vector(objects);

It seems like a small difference, but allowing the developer to ‘forget’ type information drastically cleans up the code.  In dynamic typing systems, this can open the door to certain types of bugs that static typing prevents, so usually dynamically typed languages rely more on unit testing to catch these bugs.  But in a language with type inference, even light weight type inference that will be added to C++0x via the new use of the auto keyword, this takes a huge cognitive load off of the developer that the compiler should have been doing anyway.  Type inferred languages allow all the benefits of static typing but none of the insane verbosity, of which Java’s probably the worst offender.

Make no mistake in the dynamic versus static debate, dynamic typing is more productive, while static typing, on average is safer.  The point being, when you can, you should go dynamic where you want and static where you have to.  Few languages without extensions or some mixture can use this combination easily, however, it’s a maxim that’s pretty well understood.  Type slows down productivity as a trade off for finding more bugs.  Type inference is a happy medium.

3. Object Oriented Programming

This is no major surprise.  But let’s go ahead and call this a tie for third with something that’s less well known among mainstream developers and managers.

3. Functional Programming

Object orientation is useful because it breaks up the cognitive load on the developer.  Information hiding allows the author of an object to tell the user “this is what’s important to you”.  It drastically cuts down on the amount of mental modeling the user has to do to have an object.  It descends from modular programming, both of which are useful not only for program modeling and understandability of a program, but also for managing a program.  Having one team implement one module, meeting a specific interface, while another team implements another module, is a very nice way to break down tasks and features without the two teams stepping on each others toes.  Object orientation just takes this a bit further with some more bells and whistles.

What is not acknowledged though is that many of the more verbose ‘patterns’ that arise in Object Oriented languages, again Java’s a big contributor here, should actually be concerned more as work-arounds than true solutions.  Functional languages like Lisp and Scheme allow the passing around of functions as first class things, as opposed to the way C and C++ allows you to pass around function pointers.  You can achieve some level of functional programming in C and C++ via the use of the functor idiom, however, like I said above, this is just a workaround for a missing feature.

In many cases, what may take 10,000 LOC in a pure OO language can be put in a mere 1000 lines in a functional one.  Functional languages do better to capture certain features and patterns that are implemented over and over again, like different things one does to a collection, as well as clever ways to capture state without resorting to objects.  Then again, there are functional programs that are put much more clearly and succinctly in object oriented style.

The two are not competitors, but supplements, and having developers that are familiar with both methods will allow them to take advantage of times when ideas can be expressed more clearly in either of the two ‘paradigms’ styles.  This results in less work on the developers part as they are not fighting against the language, say by introducing about a dozen different objects to implement the ‘strategy pattern’ when really, all they need is a passable function.   Or alternatively, implementing a huge object hierarchy that holds certain bits of state when all that was really needed was currying and closures.

A third competitor for this tie might be considered metaprogramming techniques which allow for easily constructed domain specific languages, which I would believe to be the next ‘evolution’ in paradigms.  But for now this should probably be considered ‘too hard’ for average developers since the language support just isn’t there in most cases.  Yes, LISP has had this sort of thing since the beginnings, decades ago.  But let’s face it, for some reason or another, LISP has never caught on as a mainstream language that most managers and developers who cling to their Visual Studio will trust.  I don’t know why.  And perhaps that will change in the future, but until then, or until a metaprogramming is easier in the current batch of popular languages, just having a java developer think in terms of map, filter and reduce should make us happy enough.

4. IDE’s / Text Editors

Another obvious one… but wait, both?  No, obviously IDE’s are the modern approach to proper tool integration some might say.  But others still cling to their ‘old’ VI and Emacs like the dinosaurs they are.

Basically, the lesson here is do whatever you’re LESS familiar with.  Text editor aficionados laugh at IDE users because they require so much ‘framework’ to get going, while IDE users fear the spartan interfaces of the anciently inspired range of text editors.  Both of these are pretty useless stereotypes.  Text editors can easily be extended to support every feature even the most modern IDE has, from debugger integration to build cycles and more.  Likewise, IDE’s should be thought of by Emacs and VI users as a pre-configured text editor with tabbing, code folding, auto-completion, etc, already set up.  Admittedly, a heavy-duty text editor user won’t find as much going to a mixed environment as a heavy duty IDE user – from a developer perspective, text editors give you all the power of IDE’s with a ton more extensibility.  A good .vimrc fits a developer like a glove.  But, from a management perspective, providing a clean, intuitive and easy to use IDE is obviously cheaper than training everyone on the intricacies of maintaining their own text editor profiles, and as such, technical managers that themselves rely on text editors probably need to experience the world of Visual Studio and Eclipse from a cost performance perspective.

Managers and developers who do all their coding in VS or Eclipse would do better to become familiar with one of the more popular text editors.  A heavy IDE user probably won’t get much of a thrill out of setting up IDE conveniences like auto-completion and what not in VI, but will become more productive by not having to rely on an IDE that takes up half of their memory for every minor programming problem.  Like the REPL, you don’t always need a solution file and a 104 step wizard just to build a program that plays with some API to learn more about it.

Likewise, while text editors are nearly infinitely personally extensible, they are not targeted by tool manufacturers like IDE’s are.  There are dozens of high quality, very useful integrations and plugins for the major IDE’s that are hard if not impossible to fully replicate in a text editor.  On-line static analysis and integration with bug and issue trackers name a few.  Certainly, these things can be done in text editors if the tool providers have provided a scriptable command line interface, or, instead of figuring out your own Emacs Lisp scripts to make these things work in a one-off fashion, you can just bite the bullet and use their offered integrations to get these features up and running for the majority of your developers in about five minutes.

5. Frameworks

Again, this one should be an obvious productivity enhancer, but time and time again developers end up forgoing a library based approach and would rather roll their own.  This happens for a few reasons.  Some developers are simply ignorant that a standard library already provides what they are trying to do.  Others arrogantly believe that what they can accomplish in a few hours of hacking will naturally be more efficient or safer than what it’s taken 30 years of mature Fortran back end to do.  A good example of the second is why every single site seems to want to roll it’s own linear algebra code, even though there is freely available insanely fast libraries already available.

From a developer perspective, it is our job to educate ourselves on the libraries availed to us by our languages and environments.  This means whenever we run into a problem we don’t know how to solve, we should FIRST research the problem to make sure no one else has not already solved the problem either using built-in or liberally open source code that we can use.  Remember the maxim, “Good programmers write, Great programmers steal.”  It does you no service to reinvent the wheel – and even though libraries have been around since almost the first languages with punch cards in the form of common subroutines, we STILL manage to try and reinvent solutions to common problems almost every day.  Even if an API isn’t ideal, it doesn’t do things in exactly the way you’d like, you still need to weigh whether you can be more productive with a slight learning curve + a good library, or just writing the whole damn thing from scratch.  In most cases, learning a new library is better.

From a management perspective, managers need to stay on top of all the tools they can provide their developers.  I know some sites are very skeptical of even liberal, non-viral open source licenses.  While developers’ anti-pattern with regard to libraries is “reinventing the wheel”, managers play their role with the classic “not invented here” syndrome.  If you honestly don’t trust outside code, if it is open source then you can still use it within your current review and testing processes.  Simply work with your developers to figure out which parts they are using, and treat that code as if you have produced it – review the source for errors, test heavily used parts, etc., treat it like legacy code.  Only most open source code is far better written and with far fewer bugs than legacy code.

The first lead bullet ever developed to improve software productivity was the idea of reusability, and it’s a shame we still haven’t gotten the picture.  Generally the most elegant and efficient solutions to some of our most commonly run into problems are already captured in well tested, highly used libraries.  What we end up producing in their stead is ugly, slow, unreadable code that every time it’s reviewed, has another boundary case exposed as not dealt with.  In other words, whatever you’re working on, it will never have as low of a bug count as the C++ STL or Java API’s, so if you’re rolling your own based on skepticism of others code, you’re doing it completely ass backwards.

Like I said, this list is hardly exhaustive.  And most of these things listed here we’re all already familiar with, at least a few of them.  The point is that these are things we need to be intimately familiar with, and more importantly, those making high level technical decisions need to be intimately familiar with, if they hope to compete in producing code fast and as bug free as possible.  Our abstract factory builder strategy patterns use ten thousand lines to do what could be said in ten, while the code-compile-run cycle turns what ought to be a five second endeavor into a five minute one, which is a growth of over an order of magnitude slower.

There is no one language or environment to rule them all, so mixing and matching should not be discouraged.  Some might balk at dynamic typing, for example, as far slower than statically typed and compiled languages.  Ok, then dynamically type most of your code, then go back in and rewrite the slow stuff in C.  It’s a tried and true approach to getting 99% of the developer speed and processor speed of both worlds.  Similarly, a REPL in a language that’s specifically not being used can still be used to prototype approaches and algorithms for another language.  Prototyping how a system ought to work in Python or Lisp will catch most of the high level design flaws much earlier than waiting to have a full C++ system rolled out, and it will be more malleable too.

The bottom line is that while there is no silver bullet, that does not mean that we don’t have a responsibility to use all the lead ones we’ve accumulated over the years.  Reusable code is decades old, yet we are still rewriting matrix algebra packages.  Why write a set of scripts to integrate Emacs with your latest unit test tool when there’s a plugin for eclipse?  Likewise, never, EVER rely on cut-and-paste for jobs an Emacs macro can do more safely and extensibly (luckily, there are in fact some integrations of Emacs and VI key bindings with some popular IDE’s.  Combining the two approaches in this way might be interesting, although I personally have not tried it.)  It’s a simple principle that the less we have to type to implement a feature, the faster we’ll implement that feature, but it seems to be ignored, even today.

October 12, 2008 - Posted by | Software Culture, Software Methodologies | ,

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: