Software tools

I’ve been thinking about tools recently.  It started when I compared myself to my father who, like me, had an office job for pretty much all his career.  Every day he took a briefcase to work, which contained (paper) documents, pens and a calculator.  I have never used a briefcase, and instead take a small rucksack containing a laptop and associated paraphernalia – charger, mouse etc.

Then it occurred to me that my laptop was analogous to Dad’s garage full of tools to help him sort the house – there’s a garage inside my rucksack.  Dad’s garage had things like a workbench, drill, screwdrivers, plane etc. to help him make things out of wood, like coffee tables and compost bins.  The tools installed on my laptop are a text editor / compiler / debugger (together as an IDE), a web server, web browser with developer tools, Git client etc. that help me make software.  Then it occurred to me that just as Dad would sometimes use his tools to make other tools, the tools I use to make software are also software, i.e. the product of software-making tools.

I recently had an online conversation with someone who wanted to learn about test automation, which meant learning a specific language (C# in this case) but also programming in general.  I pointed out some tools that would be necessary or useful as part of this.  It’s not obvious to a novice how much tools are part of programming, and yet now I take them so much for granted that I often fail to notice them.  C#, Visual Studio and the CLR are a really good example of the inter-dependence between language and tools.

Starting from scratch on Mars

In the book The Martian, the astronaut Mark Watney is stranded on Mars and must solve a daunting set of problems to stay alive and then get home.  I imagined how it would be if one of the problems he had to solve was getting a specific computer to work, for instance the only computer that could control the communications link with Earth.  The problem is, while the computer’s hardware is fine, for some reason it has no software.  What should he do?

There’s a picky question for us as co-authors with Andy Weir to get out of the way first – what does “no software” mean?  No BIOS?  No operating system?  I’m going to hand wave that one away and ignore it.

Next there’s an interesting question for us co-authors – does Mark Watney have a working laptop with developer tools on it?  If he does, then the problem turns into something like developing a mobile app (on Earth).  When you develop a mobile app, the computer that the software runs on (your mobile) can’t develop the software.  Instead, you use another computer (your laptop) to develop the software, and then copy it from your laptop to your phone to run there.

The compiler on your laptop in this case is acting as a cross-compiler – it runs on platform A to create code for platform B.  The output of the compiler can’t be run where it’s made (on the laptop), unless you use another program on the laptop that will emulate the phone.

Mark Watney could use his laptop to create replacement programs for the crucial computer, starting with the lowest missing layer of abstraction e.g. the operating system, and then working his way up to the program he actually needs – the control software for the communications link.

Writing all this from scratch isn’t for the faint of heart, but it’s easy compared to the alternative – when he doesn’t have a laptop.

Coding your way up by your bootstraps

If he doesn’t have a laptop with compiler etc. then life gets interesting, potentially fatally so.  If there is no way, even involving hardware like magnets, switches or wires, to get a new program into the crucial computer then it’s probably Game Over.  This is tough for him and boring for us, so I’m going to assume he has something.

Maybe there’s a computerised coffee maker, and he works out how to rig a wire from the coffee maker to an input port of the crucial computer, and realises that using the buttons to select different kinds of coffee will send different bits over the wire.  These bits somehow accumulate in the crucial computer and it treats those bits as executable code.

You might think that the coffee maker has saved the day, but not really.  It would be such a slow, error prone and just generally crazy way to program anything that you want to move on from it as soon as possible.  This is where the concept of software-making tools themselves being software comes into play.

If you jump forward to the end of this new version of the novel, Mark Watney writes code in a text editor running on the crucial computer, compiles it using a compiler that’s there too, and that code successfully and correctly controls the communications link to Earth and all is well.  However, we’re at an earlier chapter of the novel where the computer has no text editor, no compiler and maybe no operating system either.

How do you create a text editor without the compiler that creates it?  How do you create a compiler without the text editor that lets you write its source code?  How do you create either without an operating system to run on?  (An operating system which is, in turn, another program and so needs a text editor and compiler to already exist before you create it.)

This is the classic chicken and egg problem, or to be more computer science-y: there’s a circular dependency.

The answer is to break the circle and change it into a spiral.  That’s an accurate but rather cryptic description of the approach, so I’ll explain.  Instead of trying to use the coffee machine to create the equivalent of Visual Studio and Windows 10, he will use it to hand-craft the compiled version of the absolutely simplest version of the missing programs (text editor, compiler and/or operating system). That is, he won’t be sending bits to spell out C# source code.

Instead he will have to write the source code physically with paper and pen, do the compilation by hand, and then send the bits that make up the compiled code.  Note that there won’t be any nice operating system support such as virtual memory, task scheduling etc.  He really will be programming on bare metal.  He will have returned to the very early days of computing – the era BF (Before Fortran, the first compiler).

This will still be amazingly tedious, error prone and painful, even though it’s to make such a cut-down version of the tools.  It’s important to know that the object of creating these tiny versions of the tools isn’t to let him create the communications control software.  It is just to let him make a slightly better version of the tools.  You can think of the versions created via the coffee maker as sitting at the bottom of a vertical spiral.  They mean that he can now use the crucial computer’s keyboard and monitor rather than the coffee maker.

Because he now has version 1 of some tools to help him, it will be ever so slightly less painful to write version 2 of the tools, which will be only a tiny improvement on version 1 and sit on the next level up of the spiral.  He can then use version 2 to write version 3 and so on up the spiral.  Eventually he will get to the last chapter of the book, where the operating system, text editor and compiler are up to the job of letting him write the communication control software.

Starting from scratch in the comfort of your own home

You can experience some of this process, without the constant danger of imminent death, in the comfort of your own home.  There’s an open source course called NAND 2 Tetris.  It starts with just NAND gates and you use them to build the other types of logic gate – NOT, AND, OR, XOR.  Once you have done that, you can pretend that your building blocks have moved up from the base abstraction layer (NAND) to the full set of logic gates.

With these building blocks you need to make a simple counter and other similar things.  These then become your building blocks and you repeat this process several times to move up the abstraction levels from hardware into operating systems, then compilers and finally writing Tetris.  It gives you a top-to-bottom overview of a computer.

Summary

It is easy to not realise or forget how important tools are when programming.  It’s well worth investing time in yourself where it comes to tools – learn what they’re capable of and how you can use them to achieve your goals most easily.  Learn the keyboard shortcuts!