Tools every CFD

developer needs

A guided tour through the zoo of

software tools
Copyright © 2023 Tom-Robin Teschner

Version 1.0.0

Programming is no longer just you and your code editor (and it hasn’t been this way for quite
some time). Productivity enhancement and organisational tools are abundantly available
and a prerequisite for many job positions these days. This guide will provide you with a
quick overview of the most important tools you should be using.

Cover page image: Link

Table of Contents image: Link
Section image: Link
 Contents

1 Tools to save your life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Integrated Development Environment (IDE) 1
1.2 Build System 4
1.3 Debugger 6
1.4 Version Control 8
1.5 Your terminal 15
1.6 Dependency Manager 18
1.7 Static Code Analyser 28
1.8 Memory sanitiser 29
1.9 Automatic Code formatting 33
 1. Tools to save your life

The saying goes, “if you only have a hammer, everything looks like a nail". Theoretically
all you need is a compiler or interpreter and a text editor to write software, though we have
left the dark ages some time ago and if you are still using notepad or gedit to write your
code you probably ought to rethink your programming practices. But there are so many
more tools available that help us write good and clean code, and in this guide we will look
at the most important tools to get you started.

1.1 Integrated Development Environment (IDE)

Let’s start this section with the most important tool, your Integrated Development Envi-
ronment, or IDE. An IDE is nothing else than a glorified text editor. It usually comes
with goodies such as language-specific syntax highlighting, code-specific formatting rules
(i.e. automatically indenting code on new lines) and to some degree static code analysis
(highlighting issues before even running or compiling your code). A good IDE should make
you more productive, and you should really feel a pain when you open one of your source
files in a text editor such as gedit or notepad to make a few changes because all of these
things an IDE is exposing us to (and you hopefully got used to) are missing. If you are still
comfortable editing files in a plain text editor, it’s time to use an IDE or learn to use the one
you are currently using.

I have tried a fair bunch of IDEs myself, I started with just a text editor like notepad,
2 Chapter 1. Tools to save your life

but then started to use all different kinds of IDEs. To name but a few, Matlab, Kate, Code-
blocks, Vim, Jet Brains (PyCharm, IntelliJ IDEA, CLion), Visual Studio, sublime text 2 and
3 and eclipse. You may find Atom missing on this list, it is a good editor / IDE, but I always
had something else to use and never got around to try it properly. It is now discontinued,
and I would not recommend it any more.

Though I have saved the best for last, the IDE I am currently using and which is un-
likely to change is Visual Studio Code (or VS Code). It is cross-platform, open-source(-ish),
pretty lightweight but features just the right number of tools out of the box. The tools which
are shipped with Visual Studio Code follow the philosophy, “do only one thing, but do it
right". Each tool has evolved over the years and in particular the Git integration is pretty
amazing and for that reason alone I would recommend it (read more on Git in Section 1.4).

The most important thing VS Code is getting right, though, is its ability to do text editing.
The default shortcuts are pretty standard (in-fact, after switching from sublime text to
VS Code I was right at home and did not have to work out new muscle memory for new
shortcuts) but make your work that much more productive. A quick overview of navigating
around in VS Code can be found here:

Resource 1.1 VS Code: Often used shortcuts

The next resource is getting a lot deeper into the basic shortcuts available in VS Code
(spoiler alert: there are much, much more than those shown).

Resource 1.2 VS Code: Cheat sheet

You can customise all of these shortcuts as well (from the File →
− Preferences →
− Keyboard
Shortcuts menu). Though if you only take 5 shortcuts away from this section, these are the
top 5 ones I use and these will enhance your productivity exponentially:
List 1.1 — Most often used shortcuts. 1. Ctrl + Shift + K: Delete current line
2. Shift + Alt + ↓ / ↑: Copy current line up or down
3. Alt + ↑ / ↓: Move line up or down
4. Ctrl + D: Select current word and highlight all occurrences of the same word in the
document. Repeated pressing of Ctrl + D will select the next word.
5. Ctrl + /: Make selection (or current line if nothing is selected) a comment
1.1 Integrated Development Environment (IDE) 3

Another feature for why VS Code is so popular is its extension marketplace. If there is
something you want your IDE to do, chances are someone else has thought of it already and
developed that feature. If you go to the extension toolbox within VS Code, you can search
for extensions and find the ones you’ll need (or, develop your own ones, but that is probably
something beyond the scope of the current document). To get you started, here is a list of
extensions I use pretty much on a daily basis:
List 1.2 — Most often used extensions. 1. All the C/C++ extensions from Mi-
crosoft: Adds syntactic sugar to your IDE (syntax highlighting, intelisense, all
the good stuff you need to edit C/C++ files. Similar extensions do exist for other
languages such as Python, Fortran, etc.
2. CMake, CMake Integration and CMake Tools: If you are using CMake (and
you should! See Section 1.2), this will not just add syntax highlighting, but also a
toolbar for building your project.
3. Codeium: One of the best extensions out there, and, completely free (yes, com-
pletely free!). Think ChatGPT but integrated into your IDE, it comes with a chat
box like ChatGPT to ask questions, but it also analyses your code as you type and
will provide auto-completion as you type, and it is pretty much spot on. It’s like
intellisense on steroids and has not let me down so far, a pretty neat tool to enhance
your writing speed.
4. Test Explorer: If you are using software testing in your project (and you should!
That warrents an entire own discussion though and won’t be covered in here), the
test explorer extension will discover tests automatically in your project, add them
to the list, and you can run them conveniently from within your IDE.
5. VS Sequential Number: Such a small yet effective extension. If you have multiple
lines selected in your document, it allows you to insert numbers at those selected
lines, and it will increment (or decrement) each number by 1 (or a specified value).
Pretty useful for unrolling lists or quickly adding print statements to your code
(more on “should you add really print statements to your code?" in Section 1.3).
6. Todo+: A glorified to-do list with nice syntax highlighting and annotation capabili-
ties. It’s best to see this in action, search for it on the extension marketplace within
VS Code and look at the demo to get an overview, but suffice to say that it can be a
pretty useful thing to keep your to-dos within the same project and just crossing off
items as you go along.

I do actually use many more extensions than the one listed above, but I think it is fair to
 4 Chapter 1. Tools to save your life

say that these are the most commonly used ones. And if you have a look around, you’ll
realise there are extensions you may not even have thought about being part of your IDE.
For example, if you search for Kanban, you’ll get a few good integration of Kanban boards
into VS Code. No idea what Kanban boards are? There is a good quick introduction over at
Atlassian. As you use VS Code more and more, you’ll notice the need for specific tasks, at
which point it is worthwhile to check if there is not already an extension available for you.

Before I close this section, I just wanted to comment on some negative sides of VS Code
for full disclosure. It is developed by Microsoft, and Microsoft is not in the business of
developing free software for developers like you and me. We don’t pay them money to
use their software, but we pay them with our data. Your code is likely analysed and send
back to Microsoft, most likely to train their AI coding assistance (GitHub Co-pilot, similar
to Codeium) but there are likely many more use cases for your data that I am not aware
of (that part of VS Code is not open-source and added by Microsoft with each release of
VS Code). Microsoft has been at the center of a controversy (Detailed in this lawsuit and
this rather strange legal responsibility commitment by Microsoft) over essentially software
piracy (stealing closed-source codes and use that to train their GitHub Copilot). If you don’t
want to participate in this exercise, there is a true open-source version of VS Code available
called vscodium (not to be confused with codeium mentioned in the extensions above),
though not all extensions will work with vscodium (essentially all Microsoft-led extensions
are disabled).

1.2 Build System

Once you are comfortable with your editor, you need a way to build or execute your code.
If you are working exclusively with an interpreted language (you shouldn’t for performance,
but there are ways to get close to compiled languages, so you may be one of those unicorns
able to get away with interpreted languages), then this section is not as important for you.
However, build systems these days are so powerful that they don’t just build and compile
your code but also help you with executing different parts of it, so this section may still
contain some useful information for you.

Similar to IDEs, I have gone through a good number of build systems such as Make,
CMake, Meson, Ninja, Autotools and Waf. There are a plethora of build systems out there,
this list just scratches the surfaces, but luckily there is not really a choice here, and in this
1.2 Build System 5

day and age it is fair to say that everyone really ought to use CMake as their preferred build
system.

To understand why we should use CMake, and why there are so many build systems
out there, let me give you a quick run down. First, we had Make. Make uses Makefiles
to provide compiler specific instructions on how to build your project. The problem with
Make is that it is not portable, i.e. you write it for say a UNIX distribution such as Ubuntu,
and it won’t work on Windows out of the box. So you have to provide a lot of logic to make
it cross-platform compatible. The other issue is that there is a lot of boilerplate code you
have to write which really doesn’t change between different projects, so why should you?
Make is not exclusive to building source code (although this is its primary target area), but
rather Make provides an abstract language to Make stuff (i.e. automate tasks).

Along came CMake and in theory it was a better deal, all the boilerplate stuff was hidden
behind some logical syntax, but the problem with CMake was that it grew faster than it
could be maintained, meaning that leading up to version 2.8 it became a complete mess.
Back in these days, documentation was also pretty bad and so even though it was widely
used, it wasn’t getting much love. Thankfully, we have transitioned and now live in the
version 3.x age. Since version 3, CMake is also referred to as Modern CMake, indicating
that it has gone through a process of completely rewriting itself. The syntax is largely the
same, with additional features compared to pre 3.x versions, but the logic in CMake has
shifted (for the better). These days, documentation is still pretty bad, but more external
material is becoming available helping with that.

Then you have build systems like Meson, whose only goal was to overcome the chal-
lenges of the old CMake. Since we have modern CMake, I unfortunately don’t see much
use for build systems like Meson, though I am in love with the excentric main developer
(Jussi Pakkanen) of Meson and have contributed to the project. I love Meson, but I’ve
stopped using it. The reason: Every C/C++ project usually supports CMake, and it makes it
much easier to integrate other projects into your own (and this becomes important if you are
building for speed).

Waf is a build system probably not many have on their radar, I used it on a commer-
cial CFD solver I worked on and well, the way I would describe it to someone who has
never heard about or used it is, you need a degree in computer science to understand the
 6 Chapter 1. Tools to save your life

documentation and how to use it. It is probably pretty neat, but when I was working with
it I had a feeling it is like writing your own build system in python, but you do get some
useful classes that help you with boilerplate stuff.

Unfortunately for us, some (older) projects still haven’t transitioned to CMake, those
project may still use Autotools as their main build system, especially CFD libraries. See for
example PETSc. I don’t really want to get into Autotools here, but let me just summarise it
with one word: Radioactive. Don’t use it (it is old, outdated, and doesn’t fit with modern
software development).

When it comes to CMake, the best resource I have found by far on this matter is the
book "Professional CMake: A Practical Guide" by Craig Scott. If you are serious about
CMake, you need to have this book in your library.

Resource 1.3 Professional CMake: A Practical Guide

1.3 Debugger
Now we are getting into the controversial stuff. I love debuggers and think they are one of
the best ways to, well, debug your code. If you look around, though, most debugging is still
done with a technique referred to as caveman debugging. Caveman debugging refers to a
practice where you add numerous print statements into your code and then run the code,
inspect the output and try to figure out if the output makes sense (and ultimately why your
code is not working). This way of debugging is one-way, though, and doesn’t allow you
to interact with the code and/or react dynamically to the code as it changes variables and
memory. Print statements may be one way of finding bugs, but you are limited in what you
can do, and it is a rather outdated practice, hence the term caveman debugging.

Like build tools, there are a few debuggers available for you to choose from, some of
them working better than others on certain platforms, but in the end, we made a conscious
choice to use an IDE which supports debugging, and so we should make full use of that. VS
Code has a great integration for debugging and since I started to use VS Code, I noticed
that I used debuggers a lot more and slowly moved away from the print statement type
debugging referred to above. This is especially true if you are using CMake and have the
CMake Tools extension installed, which gives you an easy way of launching the debugger
1.3 Debugger 7

with one click while steering the behaviour within the IDE.

Stepping back for a second, a debugger is essentially just a command line application,
which you can instruct to run your code. You can instruct the debugger to stop at a particular
line of code, or when a certain memory location is changing. You can inspect what values
your variables hold at any given point in time (that is, once execution of your application is
paused, e.g. when hitting a breakpoint) and then reason about their correctness. This is a
very high-level description, and you are able to do so much more with debuggers that would
fill entire books.

If you wanted to really get to know your debugger, then by all means, feel free to learn
the debugger from the ground up. There is always a value in knowing these tools not just
from within your IDE but as standalone applications (similar to the argument made above
about build systems, if you don’t know how to manually build your code, you won’t be
able to debug errors in your build script, or at least not very easily). A good lightweight
introduction to this can be found here:

Resource 1.4 Quick start guide to debugging

The above quick start guide is for the popular GDB debugger, available on all UNIX plat-
forms. It is a good tool to learn if you want to get into debugging and are not sure which
one to go for. The following cheat sheet may then become handy to remind you of the most
useful and commonly used GDB commands:

Resource 1.5 GDB cheat sheet

However, debugging is one of those things which primarily happens in your development
environment, i.e. seldom on a remotely deployed cloud or server, so it makes sense to have
the debugger living close to your code base within your IDE. As alluded to above, VS Code
makes debugging rather enjoyable, and you can get a comprehensive overview of how to
get started with debugging and VS Code here:

Resource 1.6 Using a debugger within VS Code

Once you have gone through the above guide, you should have a very comprehensive
overview of how to do debugging within VS Code. It will feel strange and slow at first, sit
through the pain barrier, once you come out on the other side your programming life will
 8 Chapter 1. Tools to save your life

change (for the better) and your productivity will go up.

1.4 Version Control

OK, so I have one question for you, how do you back-up your code? Are you backing up
your code at all? I used to think putting everything into my cloud storage is a good idea
for backing up code, and to some degree that is true. If you are just working on a project
by yourself, fair enough, you don’t need a full-blown version control system, but even
then, what if you want to share your code with other people? Sure, you can provide zipped
versions and send them by email on-demand, but let’s push this further, how do you ensure
the correctness of your code whenever you make a change? Or, how do you make sure
code documentations stays up to date whenever you make a change? At this point, your
cloud storage solution becomes rather impractical and the version control systems we have
available these days offer much more functionality than just simply storing your code.

What is version control, then? At its simplest form, it is a system that allows you to
keep a snapshot or record of your code for each and every version of your code. Say you
write your CFD solver, and you just implemented a new turbulence model, well, that is a
new version of your code, so you take a snapshot and commit that to your version control’s
memory. You may later add a few more models and implement a new time integration
scheme, each change is a new version of your software and each version will be stored
in memory (actually, the version control system is clever enough not to store the entire
code base but rather only changes made between versions to keep storage requirements low).

Let’s talk then about the meat of the matter, which software is available to help with
version control? Well, if we have a look at what Wikipedia has to say on the matter, we
realise that there is, again, a plethora of tools available (you notice a trend here? We are
never short of software solutions for our problems ...). I would say, though, there are only
really 3 candidates which are important for coding, which are: Git, Mercurial and SVN. Of
these 3, I have only ever used Git and seen the other two in the wild, but it is fair to say
that Git is the gold standard of version control for coding and this is the tool you should be
learning (every serious programmer will know git and so should you).

Along with Git, we have a few online cloud storage providers that integrate with git very well.
You can use git entirely offline and just keep a local copy of all your versions, though, this is
1.4 Version Control 9

not very secure and may not provide you with a long-term storage solution of your code, or
even just a backup. Thus, you really want to pick an online storage solution and stick with
it. By far, the most popular platform is GitHub. The only sensible rivals to GitHub I can
think of are GitLab and BitBucket. I have used all of them but gravitated towards GitHub,
these days, though, they all provide a similar level of service, and it is really down to you to
decide which one you prefer, so read up on them and then make your decision. Though, it is
probably fair to say, most people will be familiar with GitHub and may search for code over
there so if you want to write open-source software, this is the right place to be in my opinion.

To get started with git (which is again just a command line tool), you’ll need to first
install it. This should be the easy bit. Then, you can really start to dig in. Create a simple
software project (just a single text file) and start editing and committing your changes with
git. A really lightweight, yet effective introduction can be found here:

Resource 1.7 Quick start guide to Git

Once you feel comfortable with the basic workflow in git, the next step is to understand its
branching model. A nice visual representation is shown in Figure 1.1 which is taken from
this post, which in itself is a very good read as well.
Let’s go through this in more detail to understand what is going on. On the left, you see a
timeline. Going from the top to the bottom, each coloured dot represents a new version, or
commit, of your code to the version control memory. Commits further down on the page
have been added at a later time during the code development cycle. With git, you can work
on different branches at the same time, which is a real useful feature.

Think about a case where you are working with multiple people on the same project,
in this case everyone is working on their own branch and then only later merge all their
work into the, well, aptly named master branch. While you would ideally not want to merge
anything yourself into the master branch (for that, you want to have some form of code
review in place and only the code review tool should be able to merge your code with the
master once it has passed review and some checks were done to ensure that no conflicts will
occur upon merging), though if you are only working as a single person on your project, it
is fine to directly merge into the master.

If we wanted to follow the procedure outlined in Figure 1.1, though, we would work
10 Chapter 1. Tools to save your life

Figure 1.1: Git branching model, adopted from this post.

1.4 Version Control 11

on the develop branch most of the time. We may work on different features at the same
time (borrowing from the example above, we may have one RANS model and one time
integration scheme we want to work on at the same time), so each of these features would
get their own feature branch. Once these features are completed, they get merged into the
develop branch. Once the develop branch has a logical set of features implemented (say you
finished with all RANS models that you wanted to implement, and they all sit on the develop
branch now), you can merge your develop branch into the release branch. At this point, you
would engage in code review, a practice where another programmer looks over your code
and ensures it makes sense, is easy to read and follows standard coding conventions. Once
the code review stage is passed, you can merge that finally into the master branch, and it
becomes available to the public.

Every merge into the master branch should be followed by a unique tag or version to
identify the change (hence the name version control). Not every version needs to be a
new release, but you may choose to periodically make releases available. Your version
control system (or rather, online cloud storage provider) should make this a one-button-click
exercise, and GitHub does.

Finally, there is a hotfix branch, which is reserved to fixing bugs quickly. Again, if you are
working on your own project and, especially if you are writing a CFD solver, chances are
you are not exposing any system critical bugs which needs to be patches quickly. You may
well get away without a hotfix branch. In-fact, when I was working on a commercial CFD
software solver, we did not have a hotfix branch but would just patch the code as needed
whenever we discovered a bug, but that was treated the same as a (smaller) feature branch.

We did talk a lot about version control thus far, but have actually not yet talked about
versioning itself. This is another topic with a bottomless pit, so I try to make this short,
concise, and to the point. In most cases, we will be content using semantic versioning.

Resource 1.8 Quick start guide to Semantic Versioning

You may not have come across the term semantic versioning, but you have seen it countless
time in practice. Every time you see a version of the form MAJOR.MINOR.PATCH, you
are dealing with a semantic versioning. For example, version 1.4.3 or 2.5.11 are examples
of semantic versioning. Here both 1 and 2 are the MAJOR part, 4 and 5 are the MINOR
12 Chapter 1. Tools to save your life

part and 3 and 11 are the PATCH part. Each of these parts is determined by your code
automatically (well, you have to set the version manually, but there are rules describing how
to change either the MAJOR, the MINOR, or the PATCH part). What are the rules then?
Well, here is a quick summary:

List 1.3 — Semantic versioning quick guide. • You can only ever increase the
MAJOR, MINOR, and PATCH number, it can never decrease.
• You increase the PATCH number only if you have made changes to your code but
not actually implemented a new feature (e.g. fixed a bug, cleaned up code, added a
test).
• You increase the MINOR number whenever you implement a new feature (e.g. new
turbulence model, new discretisation scheme).
• You increase the MAJOR number only when you introduce new code that breaks
backward compatibility.
• Special rules apply to MAJOR version 0. As long as you are in version 0.X.X, you
do not need to preserve backwards compatibility.

Let’s expand on this a bit. A bump in the PATCH number indicates one of the two following
things; either you have provided new code that fixes a bug (this is probably the scenario
most often encountered) or you have made other changes to the code without providing a
new feature, for example, rewriting parts of your code that were not following some style
guidelines, or you just improved readability in parts, but the core functionality of the code
has not changed.

An increase in the MINOR number indicates that you have actually provided a new feature.
A new RANS model, a new time integration or spatial interpolation schemes are all typical
features you may implement which warrants an increase in the MINOR part. And, we
follow this religiously. You can be forever on MAJOR version 1, i.e. 1.X.X, and have a
MINOR version of, say, 873, it doesn’t really matter, adding a new feature does not warrant
an increase in the MAJOR number, even if it seems tempting to indicate progress. Don’t
do it, increasing the MAJOR part is associated with a lot of time and cost, so you really
only want to do that when it is absolutely necessary. When is that the case? When we break
backwards compatibility.

Backwards compatibility means that if I am using your software, say a linear algebra
1.4 Version Control 13

solver library that I can use to solve my implicit system of equations, and I started to
use it from version 1.4.0, I should not need to worry if later I decide I want to use a new
feature from your library that you have implemented in version 1.7.0. I should be able to
upgrade the version without my code falling apart, i.e. my code should still compile without
issues after I replaced version 1.4.0 of your linear algebra solver library with 1.7.0. But, as
software projects grow, you will realise design decision you took at the beginning are not
fit any more at a later stage, or you have decided to go a different direction than originally
planned, and you may have to introduce changes that mean that if I start to use your latest
release 2.0.0, I am no longer guaranteed to be able to compile my code after I upgrade from
version 1.7.0 to your latest 2.0.0 version. The compiler may complain, the code may not
compile, and I have to look at your documentation and see what changes I have to make
in order to use version 2.0.0 from now on. And, these changes can be rather small and
insignificant at first glance. Let’s say we have three functions in our linear algebra library
called:
• construct_matrix()
• precondition_matrix()
• solve_system()
Let’s say we later decide that we don’t like snake_case as a naming convention and want to
use camelCase instead (incidentally, if you have no idea what I am talking about, you may
want to check the following resource):

Resource 1.9 Quick start guide on naming conventions

Suffice to say, if you have a variable name construct matrix, you can name it either con-
struct_matrix (snake_case), constructMatrix (camelCase), ConstructMatrix (PascalCase),
or CONSTRUCT_MATRIX (SCREAMING_SNAKE_CASE). Different programming lan-
guages have different conventions, the above post talks about this in more depth.
Back to our example, changing from snake_case to camelCase, our function names now
become
• constructMatrix()
• preconditionMatrix()
• solveSystem()
If I used your 1.7.0 version in my software, I would have code somewhere in which I make
a call to solve_system(). However, with your change to naming convention, which now
requires a call to solveSystem(), my code does not work any more, and the compiler
14 Chapter 1. Tools to save your life

won’t produce an executable. I have to figure out why the call to your library is not working
any more, costing me time and effort to sort out one part of my code which was working
perfectly fine before the update (remember, this is a simplified example, in real life the
changes are much more subtle and thus cost time and effort to figure out). If you do not
have a lot of people using your software, then this does not really matter that much, but
once people start to rely on your software, you don’t want to alienate your users with
frequent changes that break their codebase. Be mindful of how often you want to intro-
duce backwards incompatible changes, but if you have to, bump up the MAJOR version
to communicate to everyone clearly that this will break their code if they use the new version.

There are two exceptions to the above, if you bump up the version from say version
1.X.X to 2.X.X, you may not yet want to completely commit to the new MAJOR version,
i.e. you may want to release a preview of version 2.X.X to your users, gather feedback,
and then make changes before officially releasing your updated code. You can do that by
adding a hyphen and then a release tag along with it. For example, you may have want to
label your pre-release of version 2 as 2.0.0-alpha.1 and then, every time you make changes
to this version, you bump up the version after alpha, i.e. 2.0.0-alpha.2, 2.0.0-alpha.3 and
so on. While you are in this alpha version, you indicate to users that this is potentially an
unstable version and that version 2.0.0 will potentially contain backwards incompatibilities
to versions 2.0.0-alpha.X. Users who are happy with that may then use your new features
introduced in version 2.X.X while knowing they may have to change some of their codebase
once the release is being made official. In most cases, though, most developers will aim for
backward compatibility even in pre-releases, but it is not guaranteed.

The second exception is version 0.X.X. As long as you are in MAJOR version 0, you
are free to break your code as many times as you want. The build system Meson I men-
tioned above is one example, which stayed comically long in version 0.X.X (for about 8
years and 6 months), so don’t be afraid of staying in version 0.X.X and only ever go out of
it once you feel ready and think that your software will likely only add features, rather than
fundamental changes that require changes in using it and thus a change in MAJOR version
number.
 1.5 Your terminal 15

1.5 Your terminal

As a CFD practitioner, there is no way around it; you need to be comfortable using a
terminal / command prompt / console. It doesn’t matter which operating system you use,
although UNIX-based distributions such as Ubuntu really educate their users early on about
the importance of the terminal. macOS is not much different, although you can enjoy that
operating system without ever needing to use the terminal, though it is very similar to a
UNIX-based distribution. Windows is a bit different, it always tried to position itself as
a user-friendly operating system, and this is typically in contrast to what we as developer
need. In my view, though, Microsoft has really pushed for providing the right tools and envi-
ronment for developers outside their flagship development environment Visual Studio. One
such development is the introduction of the Windows Subsystem for Linux, or WSL. This
provides a native UNIX kernel on Windows and the user can choose between some popular
distributions, though for most parts, Ubuntu should be the default choice here. WSL is giv-
ing you access to a proper UNIX-based terminal on Windows. This means that software that
previously was only running on UNIX, such as OpenFOAM, can now be executed with ease
on Windows directly (I know, there are ways to make it work on Windows, but all installation
options use some form of UNIX emulation). WSL has largely removed my requirement for a
dual boot system, it is that good, and one of the first things I install when installing Windows.

Windows’ native terminal, though, has also matured to a point where I find it to be similar in
terms of power and functionality to what I am used from my Ubuntu terminal. To be clear,
I am not talking here about Windows’ arcane command prompt, but PowerShell instead.
Either way, if you are completely new to the terminal and just need some pointers, here are
some good resources for getting started with either terminal:

Resource 1.10 Quick start guide to Ubuntu’s terminal

Resource 1.11 Quick start guide to Windows’ PowerShell

Resource 1.12 Quick start guide to macOS’ terminal

Which ever operating system you are using, make sure you are familiar with its terminal,
you will need it. You should be comfortable moving around folders, creating and deleting
files, executing commands and, eventually, be comfortable writing your own scripts that you
can execute in your terminal. On Ubuntu and macOS, you would generate bash files ending
in *.sh while on Windows, you would generate batch files ending in *.bat. Within these
16 Chapter 1. Tools to save your life

files, you can write code that automates parts of your operating system. For example, I have
both an install.sh and install.bat file which I run after each fresh installation of Ubuntu and
Windows, respectively. Within that script, I instruct both Ubuntu and Windows to download
all software I typically need and set up my PC for me. I don’t want to be sitting in-front of
the PC, clicking on different executables and installing software manually unless absolutely
necessary (some software still does require this), but most of the smaller utility tools you
can be automatically installed in this way. Once you feel comfortable navigating around
your terminal, you should consult one of the below references to get comfortable writing
scripts in either operating system:

Resource 1.13 Quick start guide to writing bash scripts (Ubuntu, macOS)

Resource 1.14 Quick start guide to writing PowerShell scripts (Windows)

The next thing you need to be comfortable with is each operating system’s package manager.
Luckily, we do have one available on each platform, though not everyone is necessarily
aware of it. We use apt on Ubuntu, brew on macOS and winget on Windows. winget is
not necessarily a well known package manager, but if you are on Windows, give it a try, it
makes your life really a lot easier and the same goes for the other package managers.

To get started, you need to install the App Installer from the Microsoft store (and make sure
that your user account has admin privileges, otherwise it won’t work). For macOS, you’ll
also need to install Homebrew first and on Ubuntu you can simply use the default package
manager apt, which is tightly integrated into the operating system. On that note, you will
likely find some references where people use apt-get instead of apt; apt has superseded
apt-get both in terms of support and functionality, and you really should be using and favour
apt over apt-get. Keep this in mind when you find resources using apt-get (whenever you
see apt-get, simply replace it with apt). Using your package manager should be easy, it
typically breaks down to one of the following three tasks:

List 1.4 — search for packages with name <app>. • Ubuntu: apt search <app>
• Windows: winget search <app>
• macOS: brew search <app>
1.5 Your terminal 17

List 1.5 — install packages with name <app>. • Ubuntu: apt install <app>
• Windows: winget install <app>
• macOS: brew install <app>

List 1.6 — uninstall packages with name <app>. • Ubuntu: apt remove <app>
• Windows: winget uninstall <app>
• macOS: brew uninstall <app>

There is one downside on Windows; the package manager only seems to be concerned with
installing applications and not libraries. This is not a huge issue, as there are applications
which you can use to manage libraries / dependencies efficiently, which are discussed in the
next section.

Before moving on, though, there is one last item I want to touch upon, and that is how you
edit text files in the terminal. You man reasonably say that we have talked about choosing
an IDE and then sticking with it for all of your text editing. This is true as along as you are
on your own system and edit files there, but what if you write or use a CFD solver which
needs to run on a remote high-performance cluster? Downloading text files all the time just
to make small changes and then re-uploading them is not a good idea, and so you need to
be comfortable making changes to text files directly in the terminal. Thankfully, virtually
all high-performance clusters operate on a UNIX distribution and so long you are able to
use one of the terminal-based text editors on Ubuntu, you should be fine for all of your text
editing requirements within the terminal.

There are really two text editors which come to mind; nano and vim (and its derivatives).
Typically, you will find both on a cluster, and so learning either one of them should be fine.
I say learn, but both of these really sit on completely opposite sites of the spectrum. First,
we have nano, which really doesn’t need any explaining, and you should just be able to
open a file using the command nano filename, which will open the filename you specified
within nano. All commands you need to survive in nano are written at the bottom of the
screen, it’s Ctrl+O for saving a file and Ctrl+X to exit the application. If you know that, you
are already a nano power user, really, it is that straight forward.

Vim, on the other hand, is a beast with a steep learning curve. It comes with its own
interactive tutor, which makes learning the basic a lot easier. You can type vim filename
 18 Chapter 1. Tools to save your life

to open a specific filename or, vimtutor, if you just want to learn the basics. I would
definitely recommend giving Vim a try, it is indeed so powerful that some people use it as
their go-to IDE (yes, you can heavily extend Vim through plugins, akin to extension in VS
Code, and give it that IDE look and feel). It is probably wise to stick to nano initially but
try Vim every now and so often. It helps to have the most common commands available as
well, and you may want to print off the following cheat sheet:

Resource 1.15 Vim cheat sheet

Looking at the above resource, you’ll notice that there are quite a few commands available,
and committing these to muscle memory will take time. To help you ease the transition, here
are my top 3 text editing commands, which are the same as for VS Code given previously
(you’ll notice this list has reduced to 3 from 5, this is because Vim does not support multiple
cursors out of the box and isn’t a concept much used in Vim anyway, equally, toggling
comments is not as straight forward, but there are ways to achieve this).

List 1.7 — Most often used shortcuts. 1. dd: Delete current line
2. yyP or yyp: Copy current line up or down
3. :m -2 and :m +1: Move line up or down (plus or minus sign is required)

This is it, if you can master the concept presented in this section, you are well on your way
to become a power developer. You will have to interact with the terminal sooner or later,
even if you have set up your IDE in a way that you never have to look at it. If you aren’t
comfortable using your terminal, then you will have a hard time troubleshooting issues
when they get wrong. Take some time to familiarise yourself with the concepts explaining
in this section and the resources linked herein. This will be a foundation from which you
can expand into other areas, and you will see in the next sections that we really need to be
comfortable with our terminal to use the tools we’ll introduce next.

1.6 Dependency Manager

Sooner or later, you’ll find yourself in the need to use a package or library someone else
developed, and you want to use it in your code. Really, you should always try to find a
library to do the work for you instead of writing it yourself. Why? Well, someone else has
already put together a package for you to use, it is probably tested (you’d hope so) and is
maintained, so if there are bugs within the software, someone else is fixing them for you, for
1.6 Dependency Manager 19

free, no time required from your side. Some larger libraries out there have serious funding
behind them, meaning that people are getting paid to work on the library and make it the
best possible solution for your specific application need, you can’t compete with them, so
use their work and benefit from the hours they put into making their library as good as it
can be.

When it comes to dependency management, there are a lot of things to consider, all of which
warrant their own discussion. First you have to chose what type of library you are going to
write or use, there are static, dynamic, and headers-only libraries. Static libraries are like
zip files, all functions are included in one archive and then bundled into your executable
when you compile your code. Dynamic libraries, on the other hand, are still like archives,
but do not get bundled with your executable. This has two advantages; first, your executable
file size is reduced and second, should you ever change the dynamic library version, you do
not need to recompile your code, both of them work independently. Headers-only libraries
are not compiled and contain all their code in the headers. This means that they will become
part of your codebase once you compile the code, and so you don’t have to worry about
compiling them if you want to use them. Small utility libraries are typically headers-only,
which can be quickly integrated into your code.

In this section, I don’t want to go into the depth of how to compile libraries on your
own, but instead I want to provide a high-level introduction and some resources, and look at
software solutions to help you manage dependencies.

The first thing that we should discuss are ways to get libraries into your project. If you are
working on Ubuntu, chances are that the library you need is already available through the
package manager, i.e. apt. So you can perform an apt search <lib> to see if the library
you need is already available. This approach, while simple, is not the best one, though.
Imagine that you are working on your own CFD solver which requires the latest features
of a certain library, so you install that through apt and are happy about how easy it was to
get that library to work in your project. Then, later, you install some other CFD code, say
OpenFOAM, which requires its own dependencies (dependencies and libraries are really
just different words for the same thing). In-fact, to me, it seems that OpenFOAM requires
each and every library under the sun and so whichever library you are using, OpenFOAM
does probably use the same, though a lower version. Now you run into a version conflict,
where OpenFOAM can’t work with the library you need for your project because it may not
20 Chapter 1. Tools to save your life

be backwards compatible and likewise, your project can’t use the version that OpenFOAM
needs because it doesn’t include the features you need in your project. This is not an
artificially constructed example, this happens all the time with my students who are happy
enough to work with apt only to later realise that things do not work any more and have no
idea why (to be fair, this is a difficult thing to spot if you are new to developing CFD codes).

In-fact, what I have described above is such an important problem, that there are so many
solutions out there to address exactly that problem. You may have heard about some of
them, but you may have been unaware that these solutions exist just to address the above
issue (some of them also address other issues, but the problem described above is common
to all following listed solutions).

Docker: Docker is essentially a virtual machine, but instead of making each instance
of docker a virtual machine instance (which would require its own associated memory and
hard disk space), it runs so-called containers which are managed by the main docker applica-
tion (removing the need to associate resources to each container). What are the advantages?
You can create a container (typically based on a UNIX distribution) and define exactly
what dependencies you want to be included for each container. You can have different
containers for different CFD solver applications, so one for your own project and one for
OpenFOAM, for example. Each container installs their own dependencies and the correct
versions required, and so you just spin up the container you need for the software you want
to use. You can easily share your containers through Docker Hub, and so chances are if you
are looking for something specific, someone may have already created the container you
need, and you can simply download that one and run it. Docker is a tool which deserves
its entire own section, there is quite a lot to unpack, but I’ll leave it at this. If you like the
idea of containerising your applications, then give the documentation a go and see if this is
something you want to use:

Resource 1.16 Quick start guide to Docker

Kubernetes: Kubernetes is very similar to Docker. If you have used both, you will point out
that actually they are very different, in the sense that Kubernetes is a container orchestration
tool while Docker simply runs container (there is also Docker Swarm which manages
several containers at the same time and is more alike to Kubernetes). To be honest, I have
always been put off by the seemingly steeper learning curve for Kubernetes, so I have never
1.6 Dependency Manager 21

found a reason to ditch Docker for it. But I wanted to include in this list for completeness.
If you are, though, only interested in managing your libraries and their version, then Ku-
bernetes is overkill. It is like installing a full installation of Windows on your smartwatch,
sure, we have the technology to do that, but you probably don’t need a full operating
system on your watch when a bespoke one will do just fine. Though, there are some training
material available if you wanted to give it a try and really need the tools Kubernetes provides:

Resource 1.17 Kubernetes training courses

update-alternatives: This is a small command line tool for UNIX-based distributions which
allows you to install several packages or libraries side by side and then switch between
version if you need. This makes your life easy in the sense that you can use apt as your
package manager to install all libraries you need, and then you switch between the versions
as you use different software. The downside is you have to manually change between
versions every time you need a specific one, though this is something you can automate
by writing a small bash script that will do this for you. Keep in mind, this will only work
on UNIX-based systems (which includes WSL on Windows). A good introduction can be
found here:

Resource 1.18 Quick start guide to update-alternatives

EasyBuild and Lmod: These two tools are, unfortunately, UNIX specific and if they were
cross-platform, i.e. available natively on Windows, I would give these probably my highest
recommendation. If you know that you are exclusively working on a UNIX system, then
you owe it to yourself to learn both tools. Really!

The idea behind EasyBuild is simple, it provides a setup script which provides instructions
on how to install dependencies for you. You can have a setup script for different versions
of an application, and EasyBuild will then build that software for you. If the software you
want to build requires additional libraries, EasyBuild will install them automatically, and
this means that each library will be compiled for you on your system. This process can be
slow for larger applications, you may find that you need hours to install certain applications,
however, compiling all of your dependencies from source means that your compiler can
optimise each library and software it installs for your operating system, something which
you won’t get from your package manager (which is sort of a compromise between the
best possible performance on most platforms). And, EasyBuild takes this serious, even
22 Chapter 1. Tools to save your life

the compilers you’ll use are compiled from source so that your compilers are optimised
as well. Once you have all software and libraries installed, you’ll end up naturally with
different versions for different libraries, as each software will likely specify their own
preferred version (and each dependency of the software you are installing will have their
own sub-dependencies with their own specific library version needs).

At this point, you need to have a good way to switch between different software ver-
sions, and this is where Lmod comes in. Lmod allows you to dynamically load and unload
different versions of software and/or libraries you want to use. The nice thing about Lmod
is that you load only the package you want, say a specific library, and Lmod will load all
required dependencies for that library in the background for you. Once you want to load
a newer version of that library, Lmod will look for any sub-dependencies which need to
change and then unload versions you no longer need and silently load newer versions of
these sub-dependencies. This system works really well and chances are, if you are working
on a high-performance cluster, you will interact with a system similar to this. I say similar,
as there are more implementation that achieve exactly the same thing.

One of these solutions is Environment Modules, it looks exactly the same to the end-
user in terms of the commands they have to type into the terminal, but the backend is based
on the somewhat arcane TCL scripting language. Lmod is based on Lua, which makes it a
bit easier to read and learn. Environment modules also doesn’t provide you with any means
to install software, it assumes you have that already ready and just need a mechanism to
switch between versions, similar to update-alternatives. Either way, both implementation
are so simple to pick up, that I can provide a list with 5 most commonly used commands,
and you see how easy it is to use them. Once you know these, you will be already a Lmod
or Environment modules power user:

List 1.8 — 5 most important Lmod / Environment Modules commands. • module avail:
List all available modules
• module spider <package>: Search for all available versions of <pacakge>
• module load <package>: Load a specific package. If no version is specified, the
default version will be loaded. The name <package> is case-sensitive.
• module list: Show all currently loaded modules
• module purge: Unload all modules
1.6 Dependency Manager 23

Before finishing off this section, I just wanted to drive home the ease EasyBuild provides
us to build software, while there is somewhat of a learning curve associated with it, once
you get the hang of it you will realise how simple it is to build libraries and other packages
from source. EasyBuild provides us with a lot of scripts for already available libraries,
and this list is constantly growing. You can even make your own installation script and
submit it to EasyBuild, and it will make its way into the next release. And even if a
setup script is available for a different version of the software you want to install, or a
different compiler toolchain, you can always ask EasyBuild to try a different version or
compiler toolchain, among other things, through command line arguments, so even if you
don’t find exactly what you need, chances are you can achieve exactly what you want
without having to write any code. Both EasyBuild and Lmod are really fantastic ways
to build software, unfortunately just not cross-platform (and since I moved to Windows
for my primary development, I no longer use it as in for my own local setup, but I still
interact with it regularly on our high-performance cluster). Either way, if I have whetted
your appetite, and you want to give it a spin, the following resources will help you get started:

Resource 1.19 EasyBuild Tutorial

Resource 1.20 EasyBuild Documentation

Resource 1.21 Lmod documentation

Resource 1.22 Quick start guide for Lua programming

Conan and vcpkg: Both Conan and vcpkg are package managers for C/C++, essentially.
This means you can use them to search and install libraries just as you are able to do with
your package managers on your operating system such as apt, brew and winget. The differ-
ence here is that both Conan and vcpkg have a much larger selection of libraries available
to choose from. When compared to EasyBuild, both solutions are very similar but differ
in some of their infrastructure choices; EasyBuild comes with only the build script and so
every package is build from source, all the time. If you decide tomorrow that you want to
install a new package, and you just removed the previously compiled package, EasyBuild
will get the source code again and compile it for you.

Both Conan and vcpkg build their software from source as well with their build scripts
(which you can provide yourself as well if you have a particular library you want to use
which is not yet supported), but offer additional support if you need to reuse the same library
24 Chapter 1. Tools to save your life

over and over again.

Conan provides an online centralised storage where previously build packages are available.
That means that if you need a specific version of a library for a specific target architecture
and operating system, say ARM 64 bit on a Debian-based UNIX distribution, Conan will
check if that has already been build before by someone else, and, if that is the case, download
the compiled version, so you don’t have to compile it yourself.

vcpkg provides what they call binary caching, which does not provide an online stor-
age but rather a local storage. So you do have to build every package from source once,
but then never again as long as your cache stays intact. If you need it in the future, vcpkg
will silently get the package you have already compiled and drop it into your project. If
you are familiar with other languages, then you will realise that both Conan and vcpkg are
very similar to pip (Python), npm (javascript), maven and gradle (java) and nuget (for .Net
projects, e.g. C#).

Both Conan and vcpkg provide a command line interface which is rather easy to navi-
gate and integrate well with CMake and other build systems. While on paper, both solutions
sound great, they both fail the PETSc-test. PETSc is an open-source library for handling
matrix vector multiplications efficiently for CFD applications, with support for parallelisa-
tion. It has tons of matrix solvers, storage formats, preconditioners, and is maintained by
professionals. Chances are, you can’t beat them, so you may as well use their library, it is
relatively straight forward to pick up and use.

The maintainers and developers are, though, from an older generation of software en-
gineering, if that term even applies to them. The way to get PETSc to build is outdated
and needs updating, but to this date this seems to be something they simply choose to
ignore. This means, PETSc can, essentially, only be installed on UNIX systems, i.e. Linux
distributions such as Ubuntu and macOS. No luck for Windows. Both Conan and vcpkg
pride themselves in supporting cross-platform development, so PETSc does not really fit
in with their model. However, a package manager which is unable to provide support for
one of the most widely used linear algebra package, at least in CFD circles, is not equipped
to handle day to day dependency management for CFD development, at least for me. You
could always bring PETSc into your project in other ways, but if I choose one route to
handle dependencies, then I want all my dependencies to be managed this way, otherwise I
1.6 Dependency Manager 25

just invite entropy and messy code into my project. That being said, if you think that PETSc
is not for you (and there are other linear algebra packages supported by both Conan and
vcpkg), then both of these options are worthwhile to be considered for your development.
To get started with either of them, use the resources listed below:

Resource 1.23 Conan tutorial

Resource 1.24 vcpkg getting started guide

CMake: Hang on, wait, what? CMake is a build system, at least, so you said a few pages
ago? Yes, it is, but part of the build system is to manage dependencies if you think about it.
And CMake does provide some provisions to manage these. I mention it here for complete-
ness, but it comes with the caveat that, in my opinion, it is a really promising concept and
similar to EasyBuild, but after weeks of trying to get even the simplest dependencies into
my project, I have given up on this. For some dependencies, it just doesn’t work (but when
it does, it is one of the best ways to manage dependencies). But I am getting ahead of myself.

The concept is that, as long as you bring in dependencies which are already built with
CMake, you can make these dependencies subprojects of your own CMake script (techni-
cally I should not say subprojects, as this is a reserved word in CMake that also exist and
which is doing something very similar to what I am about to describe but let’s brush over
this technicality). CMake will then compile these dependencies for you as you build your
own project. The nice thing about this approach is that if you change your compilation
type, e.g. from a debug to a release build, your dependencies will be compiled with the
same compiler flags, so you get a consistent build overall (meaning debugging your code
becomes very easy as libraries are also compiled in debug mode and this makes it much
easier to step through your code with a debugger).

This concept in CMake is known under the umbrella term Fetch Content, and using fetch
content you can bring in outside dependencies with ease. Just for completeness, the other
term I mentioned, subprojects, is very similar to fetch content, but this will build your out-
side dependencies without checking if you are building in debug or release mode, meaning
debugging just becomes a step more complicated. As I mentioned above, I have tried fetch
content extensively, and I remember spending about 2 weeks over my Christmas break to
get OpenGL working with it. I did manage to get it to work eventually, but then returning to
it after 6 months, I saw the same error messages popping up that I debugged for 2 weeks,
 26 Chapter 1. Tools to save your life

despite me not having changed the code at all. In that particular case, it was too brittle, but
I also had limited success getting other libraries installed, in particular HDF5, which is a
compression algorithm used by many storage formats (such as CGNS). It just didn’t want to
work.

I may have to revisit this in the future but lost so many days to it that I am a bit ap-
prehensive investing more time into it. But, it is, as I said, in theory a nice way of handling
dependencies as part of your build directly, for that I really like what fetch content promises.
If you want to read about it, avoid the official CMake documentation. The book on CMake
by Craig Scott I mentioned in the CMake section is a perfect starting point, but if you just
want to get a taste, have a look at the following:

Resource 1.25 CMake’s Fetch Content overview

Bespoke dependency manager: Finally, there is always the route of writing your own,
bespoke dependency manager for your project. This is probably not the best option (we
should not be reinventing the wheel but rather reuse existing code), but if you have paid
close attention to my issues with EasyBuild, Conan and CMake’s fetch content, you can
probably understand why this is sometimes the preferred route. You have full control over
how libraries are build, and you have fine-grained control, and this is sometimes just what
you need. Instead of dragging on this section longer than it needs to be, I thought it may
be more useful to just show you the code I used in one of my projects to overcome the
issues faced with CMake’s fetch content (it is in-fact from the same OpenGL project that I
mentioned above). I’m not showing here the full code, but just enough, so you get an idea
how simple it can be to write your own dependency manager. This is based on Python and
given in the following:

1 import os
2 import urllib . request
3 import zipfile
4 import subprocess
5 import multiprocessing
6 import platform
7 import shutil
8

9 install_dir = ’ thirdParty ’
10 download_dir = os . p at h . j o i n ( i n s t a l l _ d i r , ’ d e p e n d e n c i e s ’ )
11 b u i l d _ t y p e = ’ Debug ’
 1.6 Dependency Manager 27

12 nCPUs = s t r ( m u l t i p r o c e s s i n g . c p u _ c o u n t ( ) ) # g e t t o t a l number o f CPUs

available
13 p l a t f o r m = p l a t f o r m . s y s t e m ( ) # e i t h e r Linux , Darwin o r Windows
14

15 d e f main ( ) :
16 install_glew ( ’ 2.2.0 ’ )
17

18 def i n s t a l l _ g l e w ( version ) :
19 u r l = ’ h t t p s : / / g i t h u b . com / n i g e l s −com / glew / r e l e a s e s / download / glew − ’ +
v e r s i o n + ’ / glew − ’ + v e r s i o n + ’ . z i p ’
20 f i l e = ’ glew − ’ + s t r ( v e r s i o n ) + ’ . z i p ’
21 e x t r a c t _ d i r = ’ glew − ’ + v e r s i o n
22 s o u r c e _ d i r = o s . p a t h . j o i n ( d o w n l o a d _ d i r , e x t r a c t _ d i r , ’ b u i l d ’ , ’ cmake ’ )
23 b u i l d _ d i r = o s . p a t h . j o i n ( d o w n l o a d _ d i r , e x t r a c t _ d i r , ’ b u i l d ’ , ’ cmake ’ ,
’ build ’ )
24

25 download_dependency ( url , f i l e , download_dir )

26

27 i f not os . p a th . e x i s t s ( b u i l d _ d i r ) :
28 p r i n t ( ’ c o n f i g u r i n g GLEW . . . ’ )
29 s u b p r o c e s s . c a l l ( [ ’ cmake ’ , ’ −S ’ , s o u r c e _ d i r , ’ −B ’ , b u i l d _ d i r , ’ −
DCMAKE_INSTALL_PREFIX= ’ + i n s t a l l _ d i r ,
30 ’ −DCMAKE_BUILD_TYPE= ’ + b u i l d _ t y p e ] )
31

32 p r i n t ( ’ b u i l d i n g GLEW . . . ’ )
33 s u b p r o c e s s . c a l l ( [ ’ cmake ’ , ’ −− b u i l d ’ , b u i l d _ d i r , ’ −− c o n f i g ’ ,
b u i l d _ t y p e , ’ −− t a r g e t ’ , ’ i n s t a l l ’ , ’ − j ’ , nCPUs ] )
34

35 def download_dependency ( url , f i l e , i n s t a l l _ l o c a t i o n ) :

36 i f not os . p a th . e x i s t s ( download_dir ) :
37 os . makedirs ( download_dir )
38

39 f i l e _ l o c a t i o n = os . p a t h . j o i n ( download_dir , f i l e )
40

41 i f not os . p a th . i s f i l e ( f i l e _ l o c a t i o n ) :
42 p r i n t ( ’ downloading : ’ , f i l e )
43 u r l l i b . request . u r l r e t r i e v e ( url , f i l e _ l o c a t i o n )
44

45 i f not os . p a th . e x i s t s ( f i l e _ l o c a t i o n . r e p l a c e ( ’ . z i p ’ , ’ ’ ) ) :
46 print ( ’ extracting : ’ , file )
47 with z i p f i l e . ZipFile ( f i l e _ l o c a t i o n , ’ r ’ ) as z i p _ r e f :
48 zip_ref . extractall ( install_location )
 28 Chapter 1. Tools to save your life

49

50 i f __name__ == ’ __main__ ’ :
51 main ( )

1.7 Static Code Analyser

Static Code Analyser run over your code and identify any issues that your code may have.
You may say, isn’t my compiler already doing that when I am compiling? Yes, it is, but
it will only stop compiling when it hits syntax errors or something went wrong with your
types. A static code analyser goes beyond that and tries to look at your code and identify
any potential issues in your code that would compile, but potentially could introduce bugs
(i.e. semantic errors). It is also checking your code against standard coding guidelines,
e.g. those published by the C++ committee if you are using C++, and having a static code
analyser look over your shoulder while you are writing your code is just one more layer of
protection against software errors. Syntax errors are easy to identify, semantic errors are
not, but static code analyser do a good job at identifying and reducing potential issues.

There are a number of static code analyser out there, and it is in general a good idea
to use as many as you feel comfortable with. They may highlight the same issue in your
code, or they may highlight different issues and are complimentary. In either case, you
should be using at least one of them to get free code review and improvement suggestions.

One tool that stands out in particular is clang-tidy, part of the LLVM ecosystem, which
produces just fantastic tools overall. Clang-tidy is a command line tool that allows to specify
a list of checks and an input file, i.e. your *.cpp file. It will then check your *.cpp file
against all the checks that you have asked for. Theoretically, you could do this on a file
by file basis, but this is cumbersome, and you want to automate this. VS Code provides
an extension called Clang Tidy GUI, which allows to automatically check your project
for potential issues. You need to tweak the settings of the extension to make it work for
your project (in particular, specify the source and build directory within the settings), and
the nice thing is it integrates really well with CMake. There are limitations, though, it
requires a file called compiler_commands.json, which is not generated by all compilers.
Window’s cl compiler won’t generate this file, but using clang as a compiler (which likely is
installed if you have clang-tidy installed), this file will be generated. In general, clang is a
 1.8 Memory sanitiser 29

loved compiler in the community for its clear error messages, permissive licence, and speed.
LLVM has a bunch of really useful tools (and we have another section dedicated to one of
their other tools) and adding some of them to your arsenal will make you a more productive
and cleaner programmer.

To get a quick overview of what clang-tidy can do for you, I would recommend going
through the following resource:

Resource 1.26 Quick start guide to clang-tidy

You may also want to consult with the C++ core guidelines, which identify common code
issues. Clang-tidy as separate checks available that check against this document:

Resource 1.27 C++ core guidelines

1.8 Memory sanitiser

Somewhat related to static code analysers are memory sanitisers, at least for me. Mem-
ory sanitiser work similar to a debugger, in that they run your application through the
memory sanitiser, and it will log any changes you do to your memory. Why is that im-
portant? Well, memory issues are some of the most common bugs you’ll encounter, and
you want to have a way to reliably detect any memory issues to help you speed up debugging.

The go-to tool for memory issues is probably Valgrind, one of the tools which is used
by many but understood by very few. The good news is, if you only need to check if your
code does have some memory leaks, and you want to make sure your application runs
without memory issues, then this tool is rather straightforward to use. The easiest way to do
this is to look at an example, purposefully kept very simple.
1 i n t main ( )
2 {
3 a u t o b a d P o i n t e r = new i n t [ 4 2 ] ;
4 return 0;
5 }

In this example, we create a pointer badPointer for which we allocate memory, but then
we never free that memory. If you are not familiar with memory management in C++, think
 30 Chapter 1. Tools to save your life

about it this way: memory allocation is like using parenthesis; for every call to the new
operator (or every time we open a parenthesis), we need to make a matching call to the
delete operator (or close the parenthesis in this analogy). In this case, we don’t do that, so
when we run the program, we are leaking memory. In this case it is pretty harmless, but if
you write critical software, pointer memory management is crucial. For this reason, some
companies have policies in place which forbid the programmers to use the new operator.
This may seem harsh, but when the 2014 C++ standard was released, there was no need
to for calling explicitly the new and delete operator any more, thanks to smart pointers.
Technically they were introduced in the 2011 C++ standard, but some features were missing
and from 2014 smart pointers should be really the only pointers we use in our C++ program.

Back to our example, if we compile the above code and run it through Valgrind with
the following command:

valgrind --tool=memcheck --leak-check=yes ./<executable-name>

Then we will get an output similar to the following:

1 ==474== Memcheck , a memory e r r o r d e t e c t o r

2 ==474== C o p y r i g h t ( C ) 2002 −2017 , and GNU GPLd , by J u l i a n Seward e t a l .
3 ==474== U s i n g V a l g r i n d − 3 . 1 8 . 1 and LibVEX ; r e r u n w i t h −h f o r c o p y r i g h t
info
4 ==474== Command : . / a . o u t
5 ==474==
6 ==474== e r r o r c a l l i n g PR_SET_PTRACER , vgdb m i g h t b l o c k
7 ==474==
8 ==474== HEAP SUMMARY:
9 ==474== i n u s e a t e x i t : 168 b y t e s i n 1 b l o c k s
10 ==474== t o t a l heap usage : 2 a l l o c s , 1 f r e e s , 72 ,872 b y t e s a l l o c a t e d
11 ==474==
12 ==474== 168 b y t e s i n 1 b l o c k s a r e d e f i n i t e l y l o s t i n l o s s r e c o r d 1 o f 1
13 ==474== a t 0 x484A2F3 : o p e r a t o r new [ ] ( u n s i g n e d l o n g ) ( i n / u s r / l i b e x e c /
v a l g r i n d / vgpreload_memcheck −amd64− l i n u x . s o )
14 ==474== by 0 x109158 : main ( i n / home / tom / a . o u t )
15 ==474==
16 ==474== LEAK SUMMARY:
17 ==474== d e f i n i t e l y l o s t : 168 b y t e s i n 1 b l o c k s
18 ==474== i n d i r e c t l y l o s t : 0 bytes in 0 blocks
19 ==474== possibly l o s t : 0 bytes in 0 blocks
 1.8 Memory sanitiser 31

20 ==474== s t i l l reachable : 0 bytes in 0 blocks

21 ==474== suppressed : 0 bytes in 0 blocks
22 ==474==
23 ==474== F o r l i s t s o f d e t e c t e d and s u p p r e s s e d e r r o r s , r e r u n w i t h : − s
24 ==474== ERROR SUMMARY: 1 e r r o r s from 1 c o n t e x t s ( s u p p r e s s e d : 0 from 0 )

There are a couple of things we can notice here. On line 10, Valgrind is informing us that
we have had 2 allocations for memory, of which we performed one, but we only have 1 frees
(memory deallocation). On line 16 and below, we see a summary of how much memory was
leaked. On line 13 and 14 we get a stack trace, which helps us to identify exactly where the
issue came from. You need to read the stack trace from the bottom to the top, to make sense
of it. At the bottom, we are informed that the issue occurred in the main() function and on
line 13 we are told that the call to new was done, but this is missing a matching delete call
and so that is where we are leaking memory.

Another useful tool is coming from LLVM, their clang compiler has a memory sanitiser
build into it by default, so we can turn on memory checking during compilation. To do that,
compile your code with:

clang++ -fsanitize=address mem.cpp

After executing the executable that this compilation produces, you will see something
very similar to Valgrind, just a bit shorter:
1 ==499==ERROR : L e a k S a n i t i z e r : d e t e c t e d memory l e a k s
2

3 D i r e c t l e a k o f 168 b y t e ( s ) i n 1 o b j e c t ( s ) a l l o c a t e d from :
4 # 0 0 x 7 f 2 7 0 d b a 6 0 2 d i n o p e r a t o r new [ ] ( u n s i g n e d l o n g ) ( / home / tom / a . o u t +0
xdc02d ) ( B u i l d I d : d 6 7 0 9 c c 9 6 7 0 6 6 0 4 2 2 0 0 e 4 c 8 a b d c 5 5 9 8 3 c 2 3 b c 4 3 f )
5 # 1 0 x 7 f 2 7 0 d b a 8 5 2 8 i n main ( / home / tom / a . o u t +0 xde528 ) ( B u i l d I d :
d6709cc967066042200e4c8abdc55983c23bc43f )
6 #2 0 x7f270d539d8f i n _ _ l i b c _ s t a r t _ c a l l _ m a i n csu / . . / s y s d e p s / n p t l /
libc_start_call_main . h :58:16
7

8 SUMMARY: A d d r e s s S a n i t i z e r : 168 b y t e ( s ) l e a k e d i n 1 a l l o c a t i o n ( s ) .

On line 8, we get again a summary telling us that we leaked some memory for one of our
 32 Chapter 1. Tools to save your life

memory allocations, and we can see a corresponding stack trace again on line 4, 5, and
6. We can ignore line 6, as this is a call to a system library which provides us with the
c++ runtime environment. But on line 5 we see again that the leak occurred in the main()
function and on line 4 we see that we are calling the new operator which is never matched
with a call to delete as discussed above.

Just for completeness, if we modify our code so that we have no longer a memory leak, i.e.
if we change it to:
1 i n t main ( )
2 {
3 a u t o b a d P o i n t e r = new i n t [ 4 2 ] ;
4 delete [] badPointer ;
5 return 0;
6 }

Then we receive the following output with Valgrind:

1 ==34== Memcheck , a memory e r r o r d e t e c t o r
2 ==34== C o p y r i g h t ( C ) 2002 −2017 , and GNU GPLd , by J u l i a n Seward e t a l .
3 ==34== U s i n g V a l g r i n d − 3 . 1 8 . 1 and LibVEX ; r e r u n w i t h −h f o r c o p y r i g h t
info
4 ==34== Command : . / a . o u t
5 ==34==
6 ==34== HEAP SUMMARY:
7 ==34== in use a t e x i t : 0 bytes in 0 blocks
8 ==34== t o t a l heap usage : 2 a l l o c s , 2 f r e e s , 72 ,872 b y t e s a l l o c a t e d
9 ==34==
10 ==34== A l l h e a p b l o c k s were f r e e d −− no l e a k s a r e p o s s i b l e
11 ==34==
12 ==34== F o r l i s t s o f d e t e c t e d and s u p p r e s s e d e r r o r s , r e r u n w i t h : − s
13 ==34== ERROR SUMMARY: 0 e r r o r s from 0 c o n t e x t s ( s u p p r e s s e d : 0 from 0 )

The important part here is line 15, where the error summary state that there were 0 errors
and so no memory leak or other memory issues that were detected. We can also see from
the heap summary on line 10, that we had 2 memory allocations and 2 associated memory
deallocations (frees), so no memory was leaked. If you use clang’s address sanitiser tool
instead, you will see that the application will run and no output will be printed to the screen,
 1.9 Automatic Code formatting 33

indicating no memory issues were detected during runtime.

When should you be using memory sanitisers? I typically only resort to it if I have a
bug that is difficult to track down. If you get a strange behaviour with your code, and you
can’t fix it quickly, it is probably worthwhile running Valgrind or clang’s address sanitiser
over your code to see if memory issues could be causing your code to misbehave.

1.9 Automatic Code formatting

Last but not least, I want to talk about code formatting, or rather, coding style. Each code
should pick a coding style, either generate their own coding style or follow an existing,
established coding style. Why? You may be the only person working on your code for
now, but what if someone else is contributing to your code later? What if that person writes
code completely different to your own style? Well, you are going to have differences in
your code appearance which makes it ugly to look at but more importantly more difficult to
maintain. And this is true even if you are the only one working on the code. As your code
grows in size, you may start to mix up your own style and start naming things differently
or break statements inconsistently. Your code may still compile, but it will be harder and
harder to read and understand it. Think about it this way, imagine I would write this guide
with a different font family, size, line spacing and colour. You could still read the book, and
you could still get the same information out of it, but it would put unnecessary strain on
you. Keep it simple, keep it consistent, and you will find it easier to maintain your code as
it grows.

So, how can we ensure we use a consistent style? Well, we don’t want to put that on
our to-do list, but rather, we use tools to aid in this task. As far as I can tell, there is only one
serious tool here that competes for our attention and, yet again, it is coming from the fine
people over at LLVM. By now, we know about the clang compiler and its address sanitiser,
as well as clang-tidy, yet there is another tool we ought to add to our Swiss army knife of
software tools: clang-format.

If you have familiarised yourself with clang-tidy, then using clang-format will feel very
similar. It is a command line tool that allows you to specify a coding style and then a
filename, and then clang-format can be instructed to make changes to your file automatically.
This will ensure that each file is formatted in exactly the same way, and you don’t even have
 34 Chapter 1. Tools to save your life

to be a competent programmer to make your code look good (it may be an entire different
story whether your code is fit for purpose, though!).

The key ingredient here is the coding style. Clang-format comes with coding styles for
you to choose from, namely: LLVM, GNU, Google, Chromium, Microsoft, Mozilla, and
WebKit. Defining any of these styles will make your source code look exactly as defined in
these coding styles. It is a good idea to test all of them and see if you like one more than the
other, and then just adopt it for your project. This also means that your code will already
look more similar to someone else who has already worked on a project with the same
coding style, we don’t have to reinvent the wheel all the time (a rather important aspect that
software-engineering is teaching us is that we should reuse code, and this can be extended
to things like coding style as well).

You can also specify your own coding style, and set all parameters yourself. But you
will soon realise that clang-format is not a tool with just a few settings, you can customise
the way your code looks in ways you probably haven’t thought of. If you want to dive
into the details and see what each formatting option does to your code, have a look at the
following list of parameters:

Resource 1.28 Full list of clang-format parameters

I’d say the more common approach is to pick a style that you like the looks off, and then
provide a few additional styling guidelines to overwrite an existing style. The easiest way
to achieve this is by providing a so called .clang-format file in the root directory of your
project. This file will either contain all specific parameters from the resource listed above,
or, contain an instruction to reuse an existing style. For example, a .clang-format file
that I have been using in most of my projects looks like the following:
1 B a s e d O n S t y l e : Google
2 Language : Cpp
3

4 A c c e s s M o d i f i e r O f f s e t : −2
5 ColumnLimit : 120
6 AlignAfterOpenBracket : DontAlign
7 ContinuationIndentWidth : 2
8 A l l o w S h o r t L a m b d a s O n A S i n g l e L i n e : Empty
1.9 Automatic Code formatting 35

On line 1, I specify that I want to use Google’s style as a baseline, and then I override
5 parameters manually to make the code look more like what I want. The way that I
have set these parameters is essentially starting with just the first line and then, whenever
clang-format is changing my code in a way I find unpleasing, I look up the documentation
and figure out which setting I have to change to make the code look more like what I want.
Sometimes it is difficult to get it exactly right (unless you want to spend hours on fine-tuning
your style, I have come to accept a compromise between the coding style I like and what
clang-format is giving me.

How do you use clang-format then? Well, that part is relatively easy, a standalone call to
clang-format could look like this:

clang-format --style=Google path/to/file.cpp

Here, path/to/file.cpp is the file path to your C++ file, and we are using Google’s
style here to format the file. If you run this command, you’ll notice something strange; the
file you have specified as the command line argument will just be printed to the console
with the style applied to it, i.e. your original file will not have changed. If you wanted to
commit the changes and allow clang-format to make changes to your file, you have to use
the -i command line argument for in-place file editing, i.e.:

clang-format --style=Google -i path/to/file.cpp

If you have made the effort of curating your own .clang-format file, then there is no need
to use the --style argument, and clang-format will automatically apply the style specified
in this file, assuming you are invoking clang-format from the same directory in which the
.clang-format file can be found. For completeness, the call to clang-format would then
become:

clang-format -i path/to/file.cpp

Or, without the -i if you just wanted to preview the changes in the terminal. If you
are using VS Code, then you can use the extension Clang-Format that you can find on the
extension marketplace. This allows you to apply the style specified in your .clang-format
file automatically to each text file you are editing. Using this extension, clang-format will
36 Chapter 1. Tools to save your life

run over each text file every time you save the file. This has a nice side effect that as your
file grows in size, saving becomes a pain. Why is this a positive side effect? Well, your files
shouldn’t really grow in size, clean coding principles advocate small files with as few lines
of codes as possible.
If you want to turn the extension on for your current project, open the workspace settings (in
VS Code, you can do that by pressing Ctrl+Shift+p and then searching for open workspace
settings (json)) and then add the following line to your settings:

"editor.formatOnSave": true

This will ensure clang-format runs every time you save your files. That’s it, now you
have master yet another powerful tool from LLVM and this is hopefully another one you’ll
add to your project to make your code look that much more professional and, as a result,
make it easier to read and maintain. Though if you wanted to get deeper into the details,
perhaps learn more about the different parameters that are available, then I’d suggest looking
through this resource:

Resource 1.29 Clang-format quick start guide