Blogs - Andrea Radaelli

Keychron K3 Battery Mod

Tue, 24 Dec 2024 00:00:00 +0000

In the last year or so I became interested in 75% sized keyboards. I love this design because it strikes a good balance between compactness and functionality — as a programmer, I need my function keys!

For Christmas I finally decided to scratch my itch and bought a beautiful optical Keychron K3 off eBay. It’s been a joy to type on, but with a big downside: the battery life is very bad.

With the backlight off, I struggled to get three days between charges. I didn’t want to keep the keyboard to be constantly wired, so I thought I could replace the battery. I checked a few online reviews and found out new units are rated for about week of use, which isn’t much of an improvement. I should have read about this before impulse-buying the keyboard, but it was too late.

I had a few used phone batteries collected over the years, so I thought I might be able to increase the total battery capacity by putting them to good use.

The battery

The keyboard original battery

The K3 is powered by a 3.7 V 1600 mAh lithium battery, and I was surprised to find out that it was almost twice as large and had less capacity than my old phone batteries, which are rated 3.85 V 3300 mAh. I did a deep-dive into lithium batteries and discovered that those batteries are dubbed “high-voltage”. By using a slightly different chemistry than regular lithium polymer batteries, they tolerate a higher charging voltage (4.4 V vs 4.2 V) and can therefore squeeze a bit more capacity for our ever more power-hungry phones.

The mod

Luckily, replacing a normal lithium battery with a high-voltage one is totally safe: it will be undercharged, but given the larger overall capacity the result is still a win. Assuming a 30% loss of the rated capacity because of the lower charging voltage, plus wear from previous use, I would still get about ~ 2300mA/h — not bad!

I figured I could fit two phone batteries in the frame by cutting a bit of plastic. Connecting two batteries in parallel effectively doubles the capacity. Before doing so, I carefully discharged the one that had the highest voltage until both were roughly the same within ±0.1 V — this to avoid any funny business when connecting them.

Soldering battery leads is almost impossible because the tabs are made of aluminum. I found that a product called “aluminum flux” makes it very easy! It dissolves and prevents aluminum from forming its natural oxide layer.

The Final Result

Two batteries in the frame

After this mod, I now get a good month of daily use from my Keychron, and I feel so satisfied that I’ve saved it from the bin and gave it a second life.

Sports Stats

Fri, 22 Mar 2024 00:00:00 +0000

I recently took a “homework project” for a job interview. What began as a daunting task turned into a pleasurable learning experience of React and Next.js and I got a new tiny project to showcase here.

Homework Projects

For those who don’t know, job interviews usually begin with a coding interview which consists of about an hour of solving coding problems under a time constraint. Some people criticize these evaluation methods as not being very representative of what the day-to-day work of a software engineering looks like.

Alternatives to coding interviews are “homework projects”. These projects consists of somewhat vague requirements that needs to be translated into a working program. They aim to test your ability to make reasonable guesses at the missing requirements and build something decent from them. In my experience this approach better reflects what a typical day as a software engineer will look like. Compared to the “classic” coding interview, homework projects remove the stress of being questioned and evaluated under tight time constraints. However, they require you to dedicate a few hours (or even days) of your valuable time, with no guarantee of a getting anything back in return.

The Challenge

My homework was to build a fullstack web application to visualize some NFL statistics. The assignment also required using React and Node.js in the tech stack. I had no prior experience with React, but I was confident the learning curve wouldn’t be too steep since I already know Angular (a similar library). I had the time to dedicate myself fully to the homework, so I decided to take advantage of that and use the opportunity to learn React.

The Tech Stack

Since the landscape of JavaScript web development is constantly evolving, I started with researching the current “state of the art” and finally decided on Next.js and Typescript. For styling the frontend I settled on Tailwind CSS. TypeScript should come at no surprise as it is the de-facto standard for JavaScript development today. Thanks to its static typing, TypeScript allows the IDE to provide useful suggestion when coding and detect many errors at compile time.

Next.js is a web framework based on React that offers server-side rendering in addition to client-side. What truly impressed me about Next.js was its cohesive set of functionalities that cover both backend and frontend development out of the box and the App Router. The App router makes it easy to to create dynamic APIs and routes by using the files and directories with specific names. This approach not only saves time but also improves the project’s overall structure.

Tailwind is a CSS framework that provides a low-level set of utility classes that wraps basic CSS directives. Unlike other frameworks (ex. Bootstrap) it doesn’t try to provide classes for predefined elements like cards or modals. I was skeptical about it because it seemed an unnecessary layer on top of CSS but I decided to give it a go given its popularity.

Embracing the Learning Process

I find that I learn best when I am trying to build something. I started out by skimming through the official tutorials of React and then Next.js. Once I got the feeling on how to use the two I launched myself straight into the homework. At every step I would ask myself how to proceed, make a guess, and then check online the recommended way of solving it. This process starts slow, with me constantly searching and copy-pasting, but soon I start to internalize enough concepts to get going on my own.

Tailwind CSS

This project made me reconsider my skepticism against Tailwind. Sure, it is simply wrapping CSS in simple classes, but by doing so it reduces its verbosity making it more concise, it standardizes the use of sizes, and finally it forces all styling to be tightly coupled to your components. So really, instead of writing CSS classes for your components you are writing the components themselves directly, saving one step. Furthermore by not having additional CSS classes you do not have to manage ever growing CSS files in your projects. When you need to make modifications you only change the components that are affected.

E2E Testing

As I neared completion of the project, I decided to take it a step further by incorporating end-to-end (E2E) testing using Playwright. Playwright is a modern testing framework that allows you to automate browser interactions and simulate user behavior across different scenarios. The ease of writing and executing E2E tests with Playwright is astounding and makes me happy again about web development in 2024.

Conclusion: A Week of Growth and Achievement

As a developer I find that the best way to learn is by building something, and thanks to the interview project I now feel proficient in React, Next.js and Tailwind CSS.

As I submitted the homework project and prepared for the interview, I reflected on the rapid growth and newfound confidence gained through this accelerated learning experience. Whether you’re a seasoned developer or a beginner, embarking on intensive learning journeys like this not only sharpens your skills but also fuels your passion for innovation in the ever-evolving tech industry.

Live Demo

Screenshot

The live demo (desktop only) is hosted at: https://sports-stats-demo.vercel.app

Source code: https://github.com/skilion/sports-stats

Kiwi

Sun, 23 Jul 2023 00:00:00 +0000

Lately I wanted to increase my hands-on knowledge about Amazon Web Services (AWS).

For me, the best way to learn a new technology is trying to build something with it. So, I spent some time pondering and come up with a the idea of creating a simple “Twitter like” web application.

The requirements I set were:

To be able to see all the messages posted in the last 24h.
To be able to post new messages.
To automatically simulate multiple users posting messages every minute.
To be able to scale for thousands of visitors.

I started by sketching a rough design for the architecture and then I went “shopping” on the AWS product page to see which services I could use. This is the end result:

Architecture Diagram

Key decisions:

The frontend will be a static single-page application (SPA) which I could host directly on AWS S3.
The backend endpoint for the APIs will be hosted on AWS Api Gateway.
The backend services will be fully serverless using AWS Lambda.
The storage for the system will be managed through DynamoDB.
Queues will use AWS Simple Query Service (SQS).
The whole infrastructure will be committed in the repository as code by leveraging AWS Cloud Development Kit (CDK)

For the frontend, I chose Angular for no other reason than to keep it fresh in my mind. After all, this project is solely for learning purposes.

I made sure that I could host the whole system within the limits of the AWS Free Tier, so that I would not have to pay in order to host a simple demo web application.

Here are a couple of tricks that I have learned.

SQS Long Polling

Services monitoring SQS queues do so by periodically polling the queues to check for new messages. As SQS is priced on the number of request, it’s essential to minimize the number of polls.

The recommended approach to achieve this is by enabling “Long polling”. Long polling allows Amazon SQS to wait until a message is available in a queue before sending a response, significantly reducing the number of requests per second.

After setting the maximum wait time allowed for long polling, which is 20 seconds, I was expecting to see 3 requests per second in my queues. However, to my surprise, I noticed 15 requests per second in the monitoring dashboard. The mystery was solved by this Stackoverflow question. It explains that queues wired to Lambdas (as in my architecture) are polled by 5 instances of the same Lambda by default, and this cannot be changed. While this caused a higher request rate, it is not a major concern for me since I have only two queues, and I will still be well below the threshold of the Free Tier for SQS.

Backend endpoints in the Frontend

Integrating the URL of my backend endpoint into the frontend using AWS CDK proved to be a bit challenging.

Given that the frontend is static, it becomes necessary to hardcode the backend URL before building the Angular project. However, the nature of AWS CDK is to generate semi-random names for the declared services, making it impossible to guarantee that they won’t change with future edits. Ideally, the AWS CDK should trigger the frontend build as soon as it knows the backend URL, but unfortunately, that’s not possible.

The workaround I discovered is to split the architecture into two separate “stacks”: one for the backend and one for the frontend. The backend stack can then utilize the “output construct” to write the backend URL to a JSON file that can be directly referenced by the frontend.

Live Demo

Screenshot

The live demo is hosted at: http://kiwi.aradaelli.com

Source code: https://github.com/skilion/kiwi

Don't make plans you cannot change

Tue, 04 Apr 2023 00:00:00 +0000

I recently came back home from a 2 months trip in north and central America. During my travels, I had the pleasure of meeting two retired airline pilots who gave me a great piece of advice: “never make plans that you cannot change, because no plans survive contact with reality”.

Of course the bigger and distant in time the plan is, the more the quote proves true. It is something that experience teaches us, even if only intuitively.

The quote is not new and it has been associated to many famous people with slight variations. But the truth remains.

Hearing those words… made me think. Because I broke the recommendation completely when planning for my long trip. I had bought a lot of plane tickets, way in advance, to save money. However, during my travels, I kept meeting people, like the two gentlemen, who told me about new places I would have loved to visit. But changing my plans to include the new places was impossible at that time. It would have been too expensive to change my tight flight schedule… because I did not buy flexible flights!

In that moment I consciously realized what my unconscious mind already knew. In a moment, I realized how many recommendations that spawned from that quote: “never put all your eggs in one basket”, “have time buffers for unknowns”, “get insurance for XYZ”, “do not buy what you cannot afford twice”.

I think the quote is a precursor of all the previous recommendations because it acts at a more abstract level of reality. At the simplest level, every instant, we set a goal, act it through and reassess. We make assumptions, plans, and goals all the time. But we cannot possibly ever know that things will take place as we expect them to. So we keep our investments diversified, we make space for unexpected events in our plans, we get insurance for our travels, we don’t spend all our money to buy a car or a house.

I also want to mention it would be wrong to deduce that we should not make plans at all because of our inaccurate understanding of the world we live in. I know this because it can be a plausible conclusion in a moment of difficulties. We cannot dismiss planning exactly because we must live in this reality. If we do not make any plans, we will not get where we want. We are prey to our circumstances, and they will diverge from our desirable ones pretty quickly. If we just stop moving, we stop living. Our plans may be imperfect but if we expect them to fail, when it happens, we can adapt and move on.

Stay in the flow.

Unity on Linux

Thu, 30 Mar 2023 00:00:00 +0000

I have been dabbling with Unity lately. Linux is my main OS and I have encountered a few headaches setting up my development environment.

My Unity/Rider Setup

To be able to write scripts in Unity you need to use an external code editor. Of course if you don’t care about auto-completion and debugging you can use any editor, but that’s not a realistic use case.

The officially supported IDEs are:

Visual Studio
Visual Studio Code
JetBrains Rider

Visual Studio

If you are on Windows the no-brainier solution is Visual Studio. However on Linux it is not available.

Visual Studio Code

My first choice was VS Code since it is free to use. However, at the time of writing (Q1 2023), it is hard to setup and can’t be used for debugging.

To use VS Code you need to install the C# extension and a supported C# compiler to enable auto-completion. The current LTS version of Unity (LTS 2021) uses .NET Standard 2.0 and generates .csproj files targeting .NET Framework 4.7.1. That means you cannot use the official Microsoft .NET 5+ compiler for Linux because it doesn’t support 4.7.1. In theory you could manually rewrite the .csproj files, because .NET 5 is compatible with .NET Standard, but Unity will overwrite your .csproj files without notice. That leaves you with the only option of installing the old Mono .NET Framework which, unfortunately, is a dying project. I am using Debian 11 and there is no repository from Mono to install the latest version although Debian 11 has been around since August 2021… Fortunately the version of Mono for Debian 10 still works (note that you need to pin the repository in APT or else APT will install an older version of Mono from the Debian “stable” repository).

Even if you get auto-completion to work, you will have to use the UNofficial Debugger for Unity extension which I had no luck running at all. It would have been nice to find out before I spent hours installing Mono…

At this point we are left with only one option.

JetBrains Rider

Rider is not free at all, but if you are a student you can get the Educational version for free.

I tried to install Rider both manually (extracting the .tar.gz to /opt) and through Flatpak. However both attempts did not get recognized by Unity so I could not get Unity to launch Rider correctly from inside the game editor.

The only way that worked was by installing Rider with the JetBrains Toolbox App. Afterwards, Rider just worked. Both auto-completion and debugging. And I did not even need to install Mono! The official .NET distribution from Microsoft worked just fine.

Finally I got a working Unity development environment!

Music Reactive Led Matrix

Sun, 30 Jan 2022 00:00:00 +0000

Introduction

For New Year’s Eve 2021, I decided to build a LED matrix that would react to the music being played.

I got inspired for this project by visiting the Cyberdog store in Camden Town. I wanted to recreate the same feeling I had when seeing for the first time the awesome DJ booth they have in the basement.

I started planning and building it one month before. However the project was too ambitious for my experience with electronics, I made a few mistakes and I had to cut the features short to meet the deadline.

I am very proud of the final result, I managed to implement the main effect I saw in Cyberdog and make it work just in time for new year. It was far from perfect but enough to create the right atmosphere:

The Original Plan

My initial requirements were:

build a LED matrix by cutting up a few meters of LED strips
use an ESP32 microcontroller to drive the strips
have an app on my smartphone to create the animations in sync with music
make the app and the ESP32 communicate via Bluetooth LE (partially reusing code from my previous project)
be as portable as possible
budget of £100

The biggest cost would come from the LED strips so I checked EBay and decided to buy 20 meters of WS2812B Addressable RGB LED strips with a density of 30 LED/meter. With that I could create a 20x30 matrix and if I spaced each strip by 10cm I could make it 2x1 meters big. Big enough for a strong impact.

The WS2812B LED strips work with 5V. I estimated that 600 (20x30) LEDS would need a big 200W 5V power supply and thick wires to sustain the amount of current needed (about 36A assuming each LED consumes 0.06A at maximum brightness). To avoid buying the power supply and the thick cables I decided to just use my bench power supply instead, with a higher voltage (12V+), and then connect a few buck converters right before the LED strips to bring the voltage down to 5V.

The LED strips come with adhesive tape so I decided to stick them on a frosted vinyl film in order to be able to roll everything up when not in use. The frosted finish of the vinyl film helps hiding the cabling, leaving a cleaner look.

On the software side, I could build upon the code of my previous project and extend it to transmit an RGB matrix from my phone to the ESP32 which would in turn transmit it to the LED strip. In the app I could make some nice animations that would use the sound from the microphone as input.

With all this in mind, I ordered the material, waited for delivery, and started building.

Assembling the LED Matrix

Mistake 1: Bad Power Estimations

My biggest mistake was over estimating the power capacity of the buck converters.

I had experience with the super compact “Mini 360” buck converter and I remembered from its spec that it could output up to 3A.

I had read somewhere that a single WS2812B LED could consume a maximum of 0.06A, so two strips of 1m should consume a maximum of 2 * 30 * 0.06A = 3.6A. I thought 0.06A was probably an extreme case and that the real limit would be lower so I decided to connect each Mini 360 to 2 strips.

However, I overlooked the fact that 3A was the peak output of the Mini 360, only sustainable for a short period of time. In my practical tests the Mini 360 becomes dangerously hot over 1.5A and I realized that only when testing the LED matrix after all the soldering and cabling was done.

The deadline was close, un-soldering and re-soldering everything to add more Mini 360s would mean hours of work which I simply didn’t had. In order to avoid missing the deadline I added some code to the ESP32 that would automatically limit the maximum brightness of the LEDs thus reducing the maximum current to a safe limit.

Lessons learned:

double check the specifications of your components
thoroughly test a subset of your project before soldering everything
if you underestimate the power you may be able to scale down the usage your components and get away with it 😉

Mistake 2: Mixing 5V and 3.3V Logic

The WS2812B LED strips are driven by 5V data signals. The ESP32 uses 3.3V, so common sense dictates that you need a level shifter to interface the two. However this was not apparent to me because the LED strips work quite well most of the time even with a 3.3V input.

This is because 3.3V “high” signals are high enough to be recognized as “high” from 5V components, most of the time. Glitches and flickers start appearing when refreshing the LED strips at high frequency:

However, I disregarded this problem as either some weak soldering or wrong voltage levels on my buck converters and went onward finishing building the LED matrix. Only at the very end, when I took the time to properly debug the issue I found the actual cause of the flickering.

Since I did not want to wait for the delivery of a level shifter I started tinkering with the voltage levels of the buck converters and discovered that if I lowered the voltage of the buck converter that powered the first line of LEDs where the ESP32 was connected to, all the flickering would disappear.

This makes sense since I am actually making the LEDs work in a voltage range closer to the ESP32. This ‘hack’ works but it left my first two strips emitting a slightly different color than the rest. Luckily it is hardly noticeable when playing fast animations.

Lessons learned:

3.3V and 5V circuits require a level shifter to interface them
lowering the supply voltage of the 5V circuit (in this case) is enough to get away with it 😉

Mistake 3: Assuming Real-Time Programming Is Easy

Another mistake I made was to underestimate the complexity of building the software to drive the LED matrix.

I assumed that extending the code I already had from my previous project would not take much and left it for the last week before New Year. I did the change fairly quickly but then I hit a serious problem.

The communication between my phone and the ESP32 is done using Bluetooth Low Energy. The ESP32 supports Bluetooth 4.2 which, configured with a large Maximum Transmission Unit (MTU) and the Data Length Extension Enabled (DLE), should about 478 KBit/s as measured in this article (see Case 3).

For my use case I would need to transmit the color for 600 pixel at least 30 times per second.

600 * 3B (number of RGB colors) = 1800B
1800B * 30 = 54000B = 54KB
54KB * 8 = 432KBit

432KBit are about 10% lower than 478KBit, so, when planning, I thought I would not have any problem.

However, reality is not that straightforward. The Bluetooth API are not really made for maximum throughput out of the box and optimizing both an Android app and the ESP32 requires reading lots of documentation and lots of trial and error. In the end, I did not have enough time to properly do it.

My initial tests showed miserable performances, in the range of 1 to 5 frame per second, with noticeable change in speed from time to time. That was unacceptable to show animations that react in real-time with the music.

Animations that slow are very hard to watch so I decided to create the animations directly on the ESP32 instead of my phone. That is not ideal because it is much faster to develop and release for a smartphone than a microcontroller.

With the short time left I chose one animation, the histogram of the audio frequencies, and hardcoded it in the ESP32. My phone would then only calculate the height of the histogram bars and send it. Since the histogram itself is very little (20B, 1 byte for each column) I could get away with my limited bandwidth.

I had to make the histogram on the phone because it was the only device with a microphone. The ESP32 by itself cannot listen to sounds.

For curiosity, the histogram is calculated using the Fast Fourier transform.

Lessons learned:

real-time task are a long process of optimization, plan accordingly
if you can’t meet the deadline, cut the features and you may get away with it 😉

Mistake 4: Dangling Power Cables

I guess this is one of the things one has to learn the hard way.

Much like in software development you never do tests in Production, in hardware you do not leave your “delicate” components close to “dangerous” equipment like a power supply.

When testing the LED matrix I had my power supply at 12V connected directly with alligator clips. My ESP32 was also nearby supplying the data to the LED strips. Accidentally, the data line from the ESP32 entered in contact with the alligator clips of the power supply, immediately destroying my ESP32 development board and a few meters of LED strips.

Luckily this happened quite early so I could wait for the delivery of a replacement ESP32 and new LED strips. I now make sure to always isolate my dangling cables with some masking tape.

Lessons learned:

isolate parts of your projects that operate at different voltage

Final Result

Here is a video that shows the internal cabling of the LED matrix. Before hanging it on the wall I simply folded the rest of the vinyl film to cover the circuit.

Final Result

Conclusion

When I turned the LED matrix on at New Year my friends were ecstatic. It made all the hours spent soldering and testing worth it.

I have learned a lot from this project. It is something I had never attempted at this scale and I am now working on improving it to match my original plan. Stay tuned.

Comments

Why I Like D

Sun, 09 Jan 2022 00:00:00 +0000

My engineer friends are surprised when I mention I know and I even wrote code in the D programming language (aka Dlang).

D is not exactly a mainstream language. According to the Tiobe Index, which measures programming languages popularity, D doesn’t make the first 20th languages, as of today. Its peak was in 2009 when it reached the 12th position.

I discovered D in 2013. I can’t remember if it was from Hacker News or some random Google search. At the time I was looking for a new language to replace C/C++ because I had enough of the amount of obscure behaviors, crazy complicated template system, and long compilation times. When writing C++ I have limit myself to use a subset of the features of the language to avoid “shooting myself in the foot”. I wanted to switch to a more productive language where I could focus on writing code.

My requirements for a new language were: compiled, strongly typed, easy to learn and use (think Python), and binary compatible with existing C libraries such as OpenGL and the various operating system libraries.

My candidates at the time were D, Go, and Rust. They are all compiled languages, however Go did not have generic types at the time so I excluded it immediately. Being already familiar in C, D won over Rust easily, because the syntax of Rust is quite far from C compared to D. My only pet peeve against D was that it came with a built-in garbage collector and I was against it for performance reasons, but the GC can be disabled programmatically so I concluded that it was worth taking D for a test drive.

Later in 2016, I registered for Microsoft Onedrive to make use of a promotion and, being a Linux user, found the landscape lacking of a good client to sync my files. The best option was a client written in Python which I didn’t quite like so I decided to write my own client and to use D to become more familiar with the language.

In hindsight, that was a great choice. The need to write a good program for myself made me focus in learning D. I picked up one of the best books about it, written by Andrei Alexandrescu, and dived in. The book was great and it made me completely fall in love with the language.

Here are the reasons.

Better C

D main “reason d’etre” is removing the limitations that C and C++ have accumulated over the years, while keeping compatibility with their binary interface and similar syntax.

If you already know C/C++ that’s a great thing. Remove the pre-processor, the template system, the architecture dependent differences (ex. the size of an int), the cruft of legacy features that accumulated over the years (ex. different flavors of the standard library). Then start adding modern features like foreach, modules, delegates, dynamic and associative arrays (aka dictionaries or maps). The resulting syntax is so similar to C/C++ that you can will pick it up in a day. Just try out the interactive Dlang Tour if you don’t believe me.

A nice perk of the similar syntax when you are converting your code: you can assume that any expression that you wrote in C/C++ will lead the same result in D or it won’t compile (ex. because you may need an explicit cast).

D strives to keep binary compatibility with C and C++ programs. That is excellent if you want to start porting your codebase or simply have to interface with legacy libraries. Furthermore you can directly link static libraries compiled in C within your D executable—and vice versa—as long as you translate the header files for the library.

Consistency

D has been designed from the ground up by a small number of very experienced people, namely Walter Bright and Andrei Alexandrescu. Their unified vision is reflected both in the syntax and the standard library of the language.

The standard library is rich of features. It provides both basic things like containers, date handling, math functions and more advanced things such as concurrency, regexes, and cryptography. You even get interfaces to common libraries like SQLite and Curl.

The standard library is built from the ground up making extensive use of generic types and ranges (equivalent of C++ or Python iterators). A great example is std.algorithm. All common operations like iteration, sorting, search, and comparison work on any existing or custom type that you will create (assuming you implement the expected interface for a range).

Another thing I like are the syntax choices that make the language simple and predictable. For example you cannot have multiple inheritance of classes (only of interfaces) so you cannot fall into the diamond problem like in C++ or Java. Furthermore all that in C++ is achieved with the pre-processor and template metaprogramming can be achieved with the same syntax that you use for writing runtime code. More on this later.

Built-in dynamic and associative arrays

I like to define D as a wonderful marriage between C and Python. You have the advantages of compiled code with the powerful features of Python.

In my experience, when coding, I will use dynamic and associative arrays (aka maps or dictionaries) 80% of the time. Having them built in the language is a incredible productivity boost as you don’t have to import some library or build your own. Furthermore you have dedicated syntax that make using them much less verbose than it would as if they were and external library.

Some examples:

// Create a dynamic array of 10 ints, initialized to 0
auto array = new int[10];

// Take a slice of the second half
auto half = array[4..$].dup;

// Increment all elements by one (array-wise expression)
half[] += 1;

// Concatenate the two arrays
auto concat = array ~ half;

// Create an associative array
int[string] dict = [ "Hello": 10, "World": 23 ];

// Execute the code if "Hello" exists
if (int* value = "Hello" in dict) {
	// do something with value
}

// Element wise comparison
assert(dict == dict);

Generics Types

Generic types are a breeze in D. They are intuitive and the compiler does a lot of smart inference behind the scene to optimize the resulting code.

An example to give an idea:

// Generic types are pre-pended to the actual arguments
auto add(T)(T lhs, T rhs) {
    return lhs + rhs;
}

int a = 5; int b = 10;
// T is deduced to int
auto result1 = add(a, b); 

// Explicitly force T to be float
auto result2 = add!float(a, b);

float c = 5.0;
// T is deduced to float
auto result3 = add(a, c);

What is even better is that a safe subset of D code can be evaluated by the compiler at compile time thus allowing the possibility of checking complex conditions.

For example here is function that can only be used when the types satisfy the given condition:

T[] find(T, E)(T[] haystack, E needle)
// Check that T and E can be compared for inequality
if (is(typeof(haystack[0] != needle) == bool)) 
{
	// Actual code of find()
}

This example shows how to accept a variable number of arguments with different types:

// This function accepts variadic template parameters
void write(T...)(T args) {
	// This foreach loop is unrolled at compile time!
	foreach (arg; args) {
		// to() is a convenience function from the std library to convert types
		auto stringArg = to!string(arg);
		stdout.rawWrite(stringArg);
	}
}

write("There are ", 3, " cows");

Compile Time Code Generation (Metaprogramming)

I have hinted that D allows you to execute code at compile time, so why not execute code at compile time that generates other code? Let me explain that: you can generate strings, that contain D code, at compile-time, and have the compiler include them in your executable. Enter the magic world of the keyword mixin.

If you need to generate a large structure of pre-computed values, you can do it!.

If you need to generate boilerplate code and you want to avoid typing it, you can do it!

// Returns "int[name] = [0, 1, ..., count - 1]";
string createArray(string name, int count) {
	string result = "int[" ~ to!string(count) ~ "] ";
	result ~= name ~ " = [ ";
	foreach (n; 0 .. count) {
		result ~= to!string(n);
		result ~= ", ";
	}
	result ~= "];";
	return result;
}

// here the magic happens
mixin(createArray("myArray", 5));

assert(myArray[0] == 0);
assert(myArray[4] == 4);

Creating raw strings may be overkill in some situations. So there is also a way to create injectable scopes of code with mixin template:

mixin template Foo()
{
    void func() { writeln("Foo.func()"); }
}

class Bar
{
    mixin Foo;
}

void test()
{
    Bar b = new Bar();
    b.func(); // calls Foo.func()
}

If you need to generate super duper optimized code for custom use-case, you can do it! A great example of this are compile time regexes which can generate optimized native machine code to match any regex.

Garbage Collector

Some people may not consider the GC a feature, I certainly did not at the beginning. I came from a hard-core game developer mindset where you need to know the exact timing for every operation in your critical path. I lived by quotes like: “the programmer knows better how to manage memory” and “you cannot have unexpected pauses for GC collection”.

However it turns out that unless you are writing a computer game, a high frequency trading system, a web server, or anything that really cares about sub-second latency, chances are that a garbage collector is your best friend. It will remove the burden of having to think about memory management at all and at the same time guarantee that you won’t have any memory leaks in your code.

In case you are writing a performance critical piece of software, remember you can turn off the garbage collector! People on forums like to bash that in such case you cannot use many functions from standard library. So what? If performances are essential for your system you are likely already writing you own utility library with highly optimized algorithms and data structures for your use case, so you won’t really miss the standard library much.

Hacker Features

If you had ever worked in the most “esoteric” aspect of coding such as: writing shellcodes, assembly language, executable packing, obfuscation, process hacking, etc. Here are a couple of gems that you’ll love:

Hexadecimal strings:

// Same as "\xAB\xCD\x01\x23\x45"
auto s = x"ABCD012345";

Including binary blobs

// Inline the content of "resource.bin"
auto x = import("resource.bin");

Mature developer tools

Compared to Go or Rust that have the backing of big companies, D took many years to get a solid ecosystem of productivity tools. I blame this as the main reason for D not being more widely adopted today.

However nowadays there are:

3 compilers: the official reference compiler DMD, GCC, and LLVM which cover any architecture which you may want to develop for.
Autocompletion and debugging for all major IDEs: Visual Studio, VS Code, Vim, Emacs, Sublime Text.

Package Manager

In line with modern programming languages such as Python, Node Js and C#, D has it’s own official package manger, called DUB.

To be honest here, the package ecosystem is not nearly as developed as Python or Node Js. But this is to be expected given the lower adoption that D has. Nevertheless you have the most common use cases covered such as web development, protocols such as gRPC or REST, parsers, interfaces with commonly used libraries like SDL, OpenGL, Linux and Windows system libraries.

Conclusion

Hopefully this post has gave you a bit of curiosity to learn about D yourself. If that’s the case I suggest you to spend 30 minutes on this interactive tutorial which will teach you the 80% of the language feature and practice them from your browser.

“Ok, that’s nice…”, you may say, “…but what about real work, can I get paid to learn and write in D?”. Well sort of. There are a very few places in the world that hire people to write in D, but they are niches which you will love and they will love you back if you have the hacker mentality which has pushed you to learn this elegant programming language. You can find them on Google ;)

Comments

Designing Data Intensive Applications, Chapter 1

Wed, 22 Dec 2021 00:00:00 +0000

This post is a summary of chapter 1 from the book “Designing Data Intensive Applications”. Chapter 1 introduces the fundamental terminology and metrics to evaluate data-intensive applications.

At the time of writing, “DDIA” is the number 1 best seller on Amazon in the category “Data Modeling & Design”. In the book, the author, Martin Kleppmann covers the architecture of a broad range of databases and distributed data processing systems that are essential knowledge for engineers that want to to develop distributed, scalable modern applications that process huge amount of data.

Data Intensive applications

Data-intensive means that the application is bounded by the amount of data, its complexity, and the speed at which the data is changing. This is opposed to compute-intensive application which are bounded by the amount of raw CPU power available.

Data-intensive applications are usually built from standard building blocks. These blocks have been defined empirically over a long period of time. These blocks are:

Databases: for medium and long term data storage.
Caches: for speeding up reads and remembering the result of expensive operations for short term.
Search indexes: to search and filter data in various ways, efficiently.
Stream processing: for sending data to another process to be handles asynchronously.
Batch processing: for periodically processing a large amount of accumulated data.

However each of this blocks can be implemented in many different ways and most of the time an implementation can fall in more than one category. Therefore we need a more granular way to categorize data systems and we can use the non-functional requirements that we want from the system that ultimately we want to design:

Reliability
Scalability
Maintainability

Reliability

Reliable means that the system should work correctly (performing the correct function at the desired level of performance) even in faces of errors.

Errors can be caused by hardware (power outages, hard disk failures, network partitioning, etc.), by software (code bugs, exhaustion of resources, cascading faults), or by the humans that operate those systems.

Hardware errors can be reduced by adding redundancy, ex. backup generators, RAID systems, multiple network connections.

Software errors can be reduced with testing, behavior monitoring, process isolation.

Human errors can be reduced with sandboxes (i.e. non-production environments), gradual roll outs and fast roll backs, telemetry, training.

Scalability

As the system grows in data volume, traffic volume, or complexity there should be reasonable ways to handle that growth.

Growth can happen on many axis such as number of concurrent users, volume of information processed, number of read and writes, or something else. These factors depends on the architecture of the system and they ultimately affect its load.

Once these load factors have been identified we can define how their changes affect the performances of the system, again which performances we care about depends on the nature of the system. Examples of performance metrics are throughput, latency, or response time.

When reporting on performance metrics it is common to use the average. However it is usually better to use percentiles (such as the median) because they tells how many users actually experience that performance. Also by looking at the higher percentiles you can see how bad your outliers are.

Copying with load can be done in two ways: horizontal or vertical scaling. In reality you will most likely use a pragmatic mixture of the two approaches.

While distributing the stateless services across multiple machines is fairly straightforward, taking a stateful system from a single machine to a distributed model can introduce a lot of complexity. The architecture of systems that operate at large scale is usually highly specific to the application.

Maintainability

New people working on the system should be able to work on it productively either maintaining the current behavior or implementing new use cases.

Three design principles help when creating maintainable systems:

Operability: make it easy for the operations teams to keep the system running smoothly (health monitoring, debugging, capacity planning, establishing good practices for deployment and configuration, documentation, good defaults, predictable behavior).
Simplicity: make it easy for new engineers to understand the system, by removing complexity as much as possible (good abstractions, decoupling of components).
Evolvability: make it easy for engineers to make changes to the system in the future, adapting it for unanticipated use cases as requirements change (this is closely related to simplicity as easy to understands systems are usually easier to modify than complex ones).

Stock Market Screener Demo

Wed, 15 Dec 2021 00:00:00 +0000

In the last couple of years I have been actively investing my savings in the Stock Market. I have learned a lot on how finance works but long story short, I am not interested in investing in the stock market anymore since I came to realize that:

I don’t own enough capital to make substantial returns
Beating the market average is difficult (it requires a lot of time)

Therefore I’m better off by focusing on my job and investing in myself rather than than the stock market. At least for the next 5 years or so.

My surplus money can track the market average and I stay clear from crypto, on which I am not willing to risk any substantial amount of money, but that’s for another post.

Anyway, before my crush on the stock market faded-and feeding off my engineering spirit-I was fascinated by the idea of developing my own trading bots.

Developing trading strategies needs at minimum:

A strategy (duh!)
Market data to backtest the strategy

I decided to focus on market data first, and I quickly realized that there is no real good free source of it, unless you want to pay $$$. Then I found out about Alpha Vantage which is a market data provider with a generous free tier which gives unlimited access to global market data, but it comes with a strong rate limit on the speed at which you can read it.

So I went to work and developed a few Python scripts that would slowly crawl Alpha Vantage each day and collect price and company information in a database from which I could easily run query or make simulations via Jupiter Notebooks.

I managed to get a working crawler but then quickly lost my interest in developing strategies. Instead of throwing away my work I decided to turn it in a basic stock market screener to add to my portfolio by adding a web interface on top.

Architecture Diagram

The project is mainly composed of 3 components: the Data Ingestion service, the Database, and the Stock Screener web app. They are all hosted on Microsoft Azure.

The arrows in the diagram represent the flow of market data.

The Data Ingestion service is the crawler that I mentioned earlier. It is written in Python and uses httpx to download data from Alpha Vantage and pyodbc to talk with the database. I spent some time learning how to use async programming with Python to be able to parallelize the HTTP requests to Alpha Vantage, but then I hit the bottleneck of pyodbc which does not have async support at the moment.

Stock List

The Database is a classic relational SQL database which is good enough for my use case, without having to disturb the folks in the NoSQL world.

The Web App is built with ASP.NET Core. The frontend uses Bootstrap 5, htmx and Alpine.js.

This is the first time I have worked with ASP.NET. Overall it was a good experience, ASP.NET is a mature technology with plenty of documentation. I developed the web app entirely within Visual Studio 2022 to see what Microsoft is up these days and I found it very simple to use and quite snappy. I was surprised to find out that Visual Studio Code with the C# Extension is capable to replace Visual Studio entirely, debugging capabilities included! I could do the development entirely on Linux or Mac seamlessly.

Company Overview

The code is hosted on GitHub. A running demo can be accessed at http://stock-screener-web-app.azurewebsites.net at least until the billing starts to be higher than a few quids a month.

Clean Code, Chapter 5: Formatting

Fri, 10 Dec 2021 00:00:00 +0000

This post is a summary of chapter 5 of the book “Clean Code”, entitled “Formatting”. Chapter 5 contains guidelines to use when formatting your code.

Code formatting is important. The organization, layout and arrangement of your code is a form of communication. And the quality of communication is what distinguishes a professional developer from a beginner.

The Newspaper Metaphor

Write your code like a well-written newspaper article. At the top you expect a headline and a summary that allow you to decide if it something that you want to read. As you continue downward all the details and facts of the story appear giving you the entire picture.

Code should be written the same way. At the top put the high level concepts and algorithms and the bottom put the lowest level functions and details.

If the newspaper were just one long story containing disorganized information and facts, then it would not be usable, and we would not read it.

Vertical Formatting

As a rule of the thumb the shorter the file the better. The number of lines in a source file should rarely be more than 500 and if longer you should consider splitting the functionality. The sweet spot should be between 100-200 lines.

A typical source files has different concepts: package declaration, imports, class definition, properties, methods. We want to separate these concepts with blank lines to help the eye distinguish between the text blocks that make them up. Empty lines give visual cues to the eyes of the developer so that he can read the first line and decide whether to skip to the next block or keep reading.

Code that is tightly related should appear close together and vertically dense (without blank lines) to help the reader identify code that has similar functionality.

Local variables should be declared as close as possible to their usage.
Class properties should be declared at the top (or at the bottom depending on the convention) of the class but never in the middle. Properties are variables that should be used by most of the methods in the class.
Functions that call each other (dependent functions) should be vertically close.
Some functions may represents related concepts by having common naming or doing variations of the same basic task (conceptual affinity). They may not call each other but they fit nicely together. Examples: read and write, get and set, open and close.
Function call dependencies should point to the bottom of the file. It means that the most called functions like utility and shared functions should be toward the end of the file.

Horizontal Formatting

Similarly to file length, as a rule of the thumb the shorter the lines in the file the better.

You may have heard of the Hollerith limit of 80 characters per line, which is a bit arbitrary, and it does not really make sense anymore since we have wide monitors that can easily show up to 100-120 characters. However if the line starts to be longer than 200, it is a bit ridiculous and you must split it.

Use horizontal white space to associate things that are strongly related and disassociate things that are more weakly related.

Add space around operators to highlight precedence’s (ex. var x = -a + b*c*d - 4*a;)
Don’t add space between function names and parameters but add space between the comma separated parameters (ex. Console.WriteLine("Welcome ", username, "!");)

Don’t Use Horizontal Alignment

The alignment in code emphasize the wrong things. It is not useful to read code by column. If you have long lists that need to be aligned the problem is the length of the list and not the lack of alignment.

Example:

public class CompanyOverview {
    public string   Symbol                  { get; set; }
    public string   Description             { get; set; }
    public string   Country                 { get; set; }
    public float    Price                   { get; set; }
    public string   Currency                { get; set; }
    public long     MarketCapitalization    { get; set; }
    public long     OutstandingShares       { get; set; }
    public float    PeRatio                 { get; set; }
}

// Becomes:

public class CompanyOverview {
    public string Symbol { get; set; }
    public string Description { get; set; }
    public string Country { get; set; }
    public float Price { get; set; }
    public string Currency { get; set; }
    public long MarketCapitalization { get; set; }
    public long OutstandingShares { get; set; }
    public float PeRatio { get; set; }
}

Indentation

A source file is a hierarchy. There is information that belong to the file as a whole, to the individual classes, to the methods within the classes, and the code blocks within the methods.

Indentation adds a visual cue to perceive this hierarchy, it allows the developer to rapidly see the structure of the file. Without it code would be an unreadable mess.

Don’t break indentation for short statements. Maintain a consistent hierarchy even for one-liners.

public class Pizza {
    private readonly string name;
    private readonly List<string> ingredients = new List<string>();

    Pizza(string name) { this.name = name }

    public void AddIngredient(string ingredient) { ingredients.Add(ingredient); }
}

// Becomes:

public class Pizza
{
    private readonly string name;
    private readonly List<string> ingredients = new List<string>();

    Pizza(string name)
    {
        this.name = name
    }

    public void AddIngredient(string ingredient)
    {
        ingredients.Add(ingredient);
    }
}

Team Rules

Most of the time in your career you will be working with other people on an existing codebase. When working in a team, learn and follow the conventions that have been set by team mates. Everybody should agree on one convention and stick to it because consistency is more important than following best practices (which can sometime be arbitrary). As a programmer I expect the formatting I see in one file to be be the same in others. Having multiple styles adds unnecessary complexity.

Clean Code, Chapter 4: Comments

Sat, 27 Nov 2021 00:00:00 +0000

This post is a summary of chapter 4 of the book “Clean Code”, entitled “Comments”. Chapter 4 contains a few examples of what to do and not to do when commenting code.

Comments Do Not Make Up for Bad Code

Having to write comments is often a failure of writing expressive code. The code itself should be explaining what it is doing.

Don’t use a comment when you can introduce a function.

// Check to see if the pizza is cooked
if ((pizza.flags & COOKING_FLAG) && (pizza.cookingTime > 15))
{
    // ...
}

// Becomes:

if (pizza.isCooked())
{
    // ...
}

Don’t use a comment if a variable can be renamed to add more context.

// time when the user submitted the request
TimeStamp theTime;

// Becomes:

TimeStamp userRequestTime;

// length in kilometers
double routeLenght

// Becomes:

double routeLenghtInKilometers;

Good Comments

There are situations when comments are necessary and useful. Here are some cases.

Informative Comments and Clarifications

Informative comments give information when the code cannot be more expressive than it already is.

For example, when dealing with regular expressions it may be useful to add a description.

// match dates in the format dd.mm.yyyy only between years 1900-2099
var dateMatcher = new Regex(@"(3[01]|[12][0-9]|0?[1-9])\.(1[012]|0?[1-9])\.((?:19|20)\d{2})");

Explanation of Intent

Sometimes a piece of code may may look odd. It could make the reader think: “Why was it written this way?”.

When you are writing something unconventional do add a comment to help the future developers understand your intentions.

void updatePage(Content content) {
    body.set(content);
    view.invalidate();
    // Invalidate the view twice to avoid rendering errors
    view.invalidate();
}

switch (operationType) {
    case OperationType.READ:
    case OperationType.UPDATE:
        row.refresh();
        break;
    case OperationType.DELETE:
        row.markToDelete();
        break;
    case OperationType.CREATE:
        // do nothing
        break;
}

Warning of Consequences

Sometimes it is useful to warn about consequences that are not immediately apparent to the readers.

// Trigger the operating system limit of running threads
// It will make the OS unresponsive for a few seconds
for (int i = 0; i < 1_000_000; i++) {
    thread.start()
}

Amplification

A comment may be used to amplify the importance of some operation that may otherwise be seen as unimportant.

// Trigger the operating system limit of running threads
// It will make the OS unresponsive for a few seconds
for (int i = 0; i < 1_000_000; i++) {
    thread.start()
}

Public APIs

When using a third-party library I always look for good documentation. It makes using them much more easier.

If you are building a public interface that will be released to the world or used by other teams in your company, do add comments on it. It is preferred to use a standard format for this type of comments (such as Javadoc for Java or /// comments for C#) that allows you to generate documentation from your code.

TODO Comments

“To do” comments indicate that future work is needed but the developer just does not have the time to do it right now.

Nowadays most IDEs have functionality to keep track of them so you can go back and get rid of them.

They are is useful for example to explain why a function has an incomplete implementation and what should be done in the future.

void sendText(string message)
{
	// TODO: actually send the email
	return;
}

void bakePizza(Pizza pizza)
{
	// TODO: should use the new FireOven class
	var oven = new ElectricOven();

	oven.setTemperatureDegrees(250);
	oven.waitUntilTemperatureIsReached();
	oven.bake(pizza);
}

Bad Comments

Redundant and Noise Comments

Comment that does not add any useful information to the code is useless and should not be added. Like describing the content of a variable when its name already does a good job.

public class Pizza {
    // The ID of the pizza
    private int id;

    // The list of ingredients
    List<Ingredient> ingredients;

    // Default constructor
    Pizza()
    {
        // ...
    }

Journal Comments

Sometimes people add a comment to record what they have changed. This comments accumulate to create a journal or log.

However there is no need for such comments as your version control system does that for you already.

/*
  2021-01-25: Add support for HTTPS
  2021-02-03: Retry download three times before failing
  2021-02-05: Add getFileSize()
  2021-03-18: Implement IFileStream
*/

Position Markers

Position Markers ends up just adding noise. Its better to let the proper indentation and code formatting to give the visual structure to your code.

////////////////////////////////////////////////////////////////////////////////
// Actor class
////////////////////////////////////////////////////////////////////////////////
class Actor
{
	// ...
}

Commented Out Code

There is no need to leave commented code. Control version system are good to keep track of what was there before. If you have committed it you won’t lose it.

If you leave commented code the next developer will ask himself why. Are they important? Are they reminders? Or are they just leftovers no one bothered to clean up?

Clean Code, Chapter 3: Functions

Thu, 18 Nov 2021 09:50:39 +0000

This post is a summary of chapter 2 of the book “Clean Code”. Entitled “Functions”, Chapter 3 introduces a series of principles to be used when writing functions. Functions deserve a full chapter about them because they are the first level of abstraction in any program.

1. Single-responsibility Principle (SRP)

Your function should do one thing and that thing should clearly be stated in its name. At a glance, anybody can understand what it does without looking at the code.

What does this principle mean in practice? It means that your function should only perform a handful of operations that are one level of abstraction below its name. If the function tries to do more than that, then you should be able to extract more functions by refactoring it.

There are two signals that tells if a function is doing more than one thing:

The number of lines of code is more than 20 or 30.
There are more than 2 or 3 if, for, and while statements (nested or not).

void synchronizeRemoteItems()
{
	Server remoteServerInstance = getServerInstance();
	List<Item> items = getAvailableItems();
	foreach (Item item in items) {
		Identifier id = item.getId();
		if (remoteServerInstance.hasItem(id)) {
			RemoteItem remoteItem = remoteServerInstance.getItem(id);
			if (remoteItem.getHash() != item.getHash()) {
				remoteServerInstance.upload(item);
			}
		} else {
			remoteServerInstance.upload(item);
		}
	}
}

// Becomes:

void synchronizeRemoteItems()
{
	Server remoteServerInstance = getServerInstance();
	List<Item> items = getAvailableItems();
	foreach (Item item in items) {
		ensureItemIsSynced(item, remoteServerInstance);
	}
}

private void ensureItemIsSynced(Item item, Server remoteServerInstance)
{
	Identifier id = item.getId();
	if (remoteServerInstance.hasItem(id)) {
		RemoteItem remoteItem = remoteServerInstance.getItem(id);
		if (remoteItem.getHash() != item.getHash()) {
			remoteServerInstance.upload(item);
		}
	} else {
		remoteServerInstance.upload(item);
	}
}

Another relevant example is error handling. “Try/catch” blocks confuse the structure of the code and mix error processing with the business logic. So it is better to extract the bodies of the try and catch blocks out into functions of their own.

void synchronizeRemoteItems()
{
	try {
		Server remoteServerInstance = getServerInstance();
		List<Item> items = getAvailableItems();
		foreach (Item item in items) {
			ensureItemIsSynced(item, remoteServerInstance);
		}
	} catch (ConnectionException e) {
		logException(e)
		throw new SyncFailedException();
	}
}

// Becomes:

void synchronize()
{
	try {
		synchronizeRemoteItems();
	} catch (ConnectionException e) {
		logException(e)
		throw new SyncFailedException();
	}
}

private void synchronizeRemoteItems()
{
	Server remoteServerInstance = getServerInstance();
	List<Item> items = getAvailableItems();
	foreach (Item item in items) {
		ensureItemIsSynced(item, remoteServerInstance);
	}
}

2. Use the least number of arguments

The more arguments the more difficult is to understand what the function does. You should never use more than 3 arguments. If you find yourself in the situation it is likely that some of those arguments should be wrapped in a struct or class because they are part of a concept that deserves a name of its own.

Also, never use boolean flags. They imply that the function does two things, one when the flag is true and one when it’s false. What you really need is two distinct functions.

void drawRectangle(int x, int y, int width, int height);

// Becomes:

class Rectangle {
	public int x;
	public int y;
	public int width;
	public int height;
}

void drawRectangle(Rectangle rect);

void makePizza(bool vegan);

// Becomes:

void makePizza();
void makeVeganPizza();

3. Have No Side Effects

Side effects are unexpected changes to the state of the program. It may be in the form of global variables or unexpected variables of the class of the same method.

Side effects create temporal coupling. Temporal coupling is an implicit relationship requiring the client to invoke one function after another. Basically it means that a function can only be called at certain times and incurring in the risk of unexpected side effects at others.

If you must have side effects they should be clearly made explicit in the function name, ex. Initialize().

4. Avoid Output Arguments

Function that use output arguments are confusing and should be avoided in general. It’s not immediate to understand that an argument is an output and you certainly have to look the definition.

Output arguments were widely used before object oriented (OO) programming, however in an OO language the need for output arguments disappears because when you are a calling a method, the object owning it is the intended output argument (this).

void addCheese(Pizza pizza, Cheese cheese);

// Becomes:

pizza.add(cheese);

5. Command/Query Separation

A function should either do something or answer a query, but not both. Doing both leads to confusion and it likely indicates that the function is doing two things.

class Hashmap {
	public bool setIfKeyExists(string key, string value);
}

// Becomes:

class Hashmap {
	public bool exists(string key);
	public void set(string key, string value);
}

Returning error codes count as breaking this rule. It leads to creating deeply nested if statements because you need to handle the errors immediately. Using exceptions instead enables to keep the business logic and the error processing code separated resulting in simplified code.

6. Don’t Repeat Yourself (DRY)

“Avoid code duplication” is something you have been told to death if you had a formal education in computer science, and for a good reason. If there is an algorithm that gets duplicated N times, the source code will require to be modified N-times should that algorithm ever change.

Also the readability of your code base is improved if reduntant code is replaced with a single function.

Clean Code, Chapter 2: Meaningful Names

Fri, 12 Nov 2021 00:00:00 +0000

This post is a summary of chapter 2 of the book “Clean Code”. Entitled “Meaningful Names”, Chapter 2 introduces a series of principles to be used when naming variables, functions, arguments, classes, and packages.

At the time of writing, “Clean Code” is the number 3 best seller on Amazon in the category “Software Design, Testing & Engineering”. In the book, the author, Robert “Uncle Bob” Martin promotes a coding style that is simple to read and understand.

Before I started coding professionally, I understated the value of choosing good names for variables and functions. Today I believe that a codebase with well named variables, functions, classes, etc. can be almost free of comments because the names themselves are enough to give context to the reader about what’s going on.

Before diving in the rules, please note that the naming convention does not matter. A name could be camelCaseName, snake_case_name or even kebab-case-name. The only thing that matters is which convention you use.

1. Use Intention-revealing Names

The name of a declaration should make the reader understand exactly what it contains. As a rule of the thumb, if a declaration requires a comment, then the name does not reveal its intent.

Avoid one letter names. Disregard the fact that they are widely used in examples and snippets, in that context they are mainly used to save space. When you are writing code professionally you are writing code that will be read and re-read by many people after you. Space is not a concern, brain cycles are.

Don’t be afraid to make a name long. Long descriptive is better than short enigmatic.

Examples:

int d; // elapsed time in days

// Becomes:

int elapsedTimeInDays;

int len = myString.Length;
for(int i = 0; i < len; i++) { /* ... */ }

// Becomes:

int myStringLength = myString.Length;
for(int charNum = 0; charNum < myStringLength; charNum++) { /* ... */ }

private List<int> getContractIds(){
    var list = new List<int>();
    /* ... */
    return list;
}

// Becomes:

private List<int> getContractIds(){
    var contractIds = new List<int>();
    /* ... */
    return contractIds;
}

2. Make Meaningful Distinctions

When you have similar named declarations it becomes confusing.

A common case is number-series naming. They provide no clue of the author intention. For example a numbered series of parameters: contract1, contract2, contract3.

Another case is variables padded with noise words: contractInfo, contractData, contractObject. They should be distinguished in such a way that the reader knows what are the differences they offer.

public function copyMessage(Message msg1, Message msg2);

// Becomes:

public function copyMessage(Message destination, Message source);

3. Use Pronounceable Names

Names that you write will eventually be used in spoken language. This matters because programming is a social activity, people talk about and discuss code. If the names cannot be pronounced they will create additional burden when connecting sound to meaning.

var ddMmYyyy = DateTime.Now.ToString("dd/MM/yyyy");
var ddddMmmmYyyy = DateTime.Now.ToString("dddd, dd MMMM yyyy");

// Becomes:

var shortDateTime = DateTime.Now.ToString("dd/MM/yyyy");
var longDateTime = DateTime.Now.ToString("dddd, dd MMMM yyyy");

4. Use Searchable Names

Single-letter names and numeric constants are difficult to be searched. You may want to locate them for refactoring or simply reference but they end up being part of other definitions.

Numeric constants are bad in general and its a well known counterproductive behavior. Hopefully you have been told about this one in your early days of learning programming.

int d = 5;
int m = 0;
for (int i = 0; i < 365, i++)
{
    if ((i + d) % 7 == 0)
    {
        m++;
    }
}

// Becomes:

const int DaysInYear = 365;
const int DaysInWeek = 7;
const int Monday = 0;

int startWeekDay = 5
int mondaysCountInYear = 0;
for (int day = 0; day < DaysInYear, day++)
{
    if ((day  + startWeekDay) % DaysInWeek == Monday)
    {
        mondaysCountInYear++;
    }
}

5. Avoid Encodings

Prefixing the type to variable names, the (Hungarian Notation), is a reminiscence from a time when computer where several times less powerful and compilers far less developer friendly. Nowadays our IDEs can tell the type of our variables and the compiler can enforce static checking of types. Having the type encoded in the variable name makes harder to change the name and type of a variable plus it makes it harder to read!

Other obscure practices like prefixing m_ to class properties or prefixing _ for private members is unnecessary. Your classes should be small enough that you don’t need them. Moreover, your editing environment can highlight class members to make them distinct from other declarations.

6. Class Names Are Nouns, Methods Names Are Verbs

Classes and objects should have a noun or noun phrase names like: Contract, ContractParser, FutureContract. Again avoid ’noise’ words like Object, Data, or Info, that do not add anything useful to the name.

Methods should have a verb or a verb phrase like calculateValue, save, rollForward. Access, mutators, and predicates should be named for their value and prefixed with get, set, is.

class Contract {
    public void sign();
    public void send();

    public int getMoneyAmount();
    public bool isValid();
}

7. Pick One Word per Concept

Before writing new code to a repository it is essential to look at what is already there to imitate the naming style and the concept used.

Many concepts have synonyms ex. “Controller” have synonyms like “Manager” or “Driver”. Another example: “Adapter”, “Decorator”, and “Wrapper”. “Aggregate”, “Collection”, and “Container”. “Fetch”, “Retrieve”, and “Get”.

You should word between synonyms and stick to it across your codebase.

8. Use Solution and Problem Domain Names

Choose technical names for those thing which is appropriate. By technical terms I mean topics from Computer Science like algorithms, design patterns, mathematical terms, well-known acronyms.

When there are no technical names for what you are doing, use names from the problem domain. You should at least be aware of the lexicon used by the client of your software, or else in you can always ask a domain expert.

9. Add Meaningful Context

It’s rare that declarations are meaningful by themselves. Most are closely linked to others. It makes sense to enclose them together in well-named struct, class, or namespace.

int addressNumber;
string addressRoad;
string city;
string postCode;

// Becomes:

struct Address {
    public int number;
    public string road;
    public string city;
    public string postCode;
}

Setting up my website

Mon, 01 Nov 2021 00:00:00 +0000

Hello world!

Last week I got a back injury after a long streak of attending Brazilian Jiu-Jitsu sparring sessions. So I am now forced to take rest and I have decided to build my personal website to host my portfolio and blog.

In the past I would have built everything from scratch, which is fantastic to learn a lot. However it is quite stupid to waste 80% of my time reinventing the wheel and 20% writing actual content. So I have decided to give a go to Hugo a static website generator. So far I am quite impressed. It hits the sweet spot for me in terms of automation and flexibility. No doubt it is so popular.

Using a pre-made template is a breeze, however I want to customize it fully so I am writing my own theme from scratch.

Bluetooth RGB Light

Sat, 30 Oct 2021 00:00:00 +0000

I was scouting Amazon and eBay for a controllable light to bring to parties, but I was left disappointed. The low end market is full of low power lights which are not bright enough and are controlled through a lousy remote controller. To get decent devices you need to go in the high end space where you can easily spend a few hundreds quids.

So I thought I should come up with my own solution.

I started by coming up with a list of requirements:

Portable and battery powered
Multicolor
Powerful enough to light up a dark room
Controllable with a smartphone app
Programmable with custom effects

Given those requirements I started searching online for electric components that would fit the requirements.

Since the light would be battery powered, energy consumption is critical. The most efficient form of lighting is provided by LEDs. I wanted a multicolor light so it needed to be an RGB led. I settled on a 10 watts Chip-on-Board (COB) RGB LED.

For communication between the light and my smartphone I chose the ESP32 microcontroller which, for less than 5 quids, gives you: Wireless, Bluetooth, 32 KiB of RAM, and 34 programmable pins, Pulse-width modulation (PWM) signals, and much more which is not relevant for this project.

For driving the LED, I was initially thinking of buying 3 separate LED driver modules, but they are quite expensive (compared to buying the individual components separately) unless you order them from China through Aliexpress or similar. However that would mean waiting for at least one month for delivery, so I decided to build my own LED driver from discrete components. Comparing the chips that existing LED driver are using and the availability of those chip I settled for using the PT4115.

Only when the chips arrived I realized that I had ordered them in the Surface-mount technology (SMD) package. Those are intended to be soldered on a custom printed circuit board (PCB). However for my prototype I intended to use a simple soldering protoboard, so I had to solder “legs” to all the SMD components in order to be able to plug them in the protoboard.

Adding “legs” to the PT4115

The circuit to run the PT4115 is quite simple and a reference design can be found in the official specifications along with the rules to pick the side components.

For Reference:

DIM is the PWM control pin
CSN is the control sensing pin which helps the chip detect the amount of current going in the LED
SW is the switch pin which is turned on and off automatically to keep the right amount of current flowing through the LED.

PT4115 sample circuit

Initially I thought to build my own case for the light, but after a quick search on eBay I found many variations of outdoor flood lights which I could simply modify for my purposes.

Light Front

The light from eBay came with its own LED driver mounted on the back. It was quite bulky and unnecessary so it was the first thing I removed.

Original LED Driver

Then I proceeded to open the led compartment and finding out how much space I had to fit my own circuits. I was pleased to find out that the case was quite oversized and I had plenty of space to fit everything I needed.

Frame unmounted

Original LED

The next step was to build the LED driver. I used a small piece of protoboard and duplicated 3 times the circuit that I have showed earlier for the PT4115. In the next photo I have highlighted the small SMD resistors for “Rs”. It is not an elegant solution, but it works.

RGB LED Driver

I had to do the same trick of adding “legs” to the ESP32 module as well in order to solder it. For powering it I chose to use a premade buck converter module, the mini-360.

ESP32 WROOM

Then it was time to connect everything together and test the result. In the next video you can see that the the LED turns on only for a brief moment. That’s because as soon as the ESP32 finishes booting its pins enter a low state, effectively turning off the LED drivers and the LED itself.

Having verified that the circuit works I proceeded fitting everything nicely in the light case. I used the black tape to set in place some transparent plastic in order to isolate the circuit from the metal case and avoid shorts. The dangling cables that you can see taped at the top of the next photo are used to flash the firmware on the microcontroller.

Fitting everything in the light case

Next, it was time to write the firmware for the ESP32 so that it can control the individual colors and receive the commands from my smartphone. I read extensively the documentation for the ESP32 and found a couple of API for my use case: the LED Control (LEDC) API which allows me to output PWM signals and the NimBLE library for Bluetooth Low Energy communication.

After spending a couple of days understanding the examples from the documentation I managed to put together my own firmware that exposes one Bluetooth LE characteristic that directly maps to the 3 RGB channels of the LED. The code is available on GitHub.

The last remaining task was to build an Android App to connect to the ESP32 and send the light colors. I quickly hacked together a proof of concept that allows me to toggle the basic colors for testing, automatically switch between all colors randomly, flash the color at different speeds, smooth the color transition and control the light intensity (brightness). The code, again, is available on GitHub.

And here you can see the final result.

Overall it has been a fun project that allowed me to play with electronics and learn a lot. The prototype hits all my requirements but the brightness is much less than I expected. I think a proper flood light should be at least 50W to make a good impression. I’ll come back at this project and see if I can upsize the LED while still keeping everything battery powered.

Map My Mountains

Fri, 01 Feb 2019 00:00:00 +0000

Introduction

In Jan 2018, shortly after my graduation, I was offered a contract to work for a team of PhD students lead by Prof. Piero Fraternali. The team operated as part of the “Department of Electronics, Information and Bioengineering” inside the university of Politecnico di Milano.

I was hired to develop a Geographical Information System (GIS) from scratch to provide good quality data for an ongoing research project. The research project goal was to develop a Machine Learning (ML) algorithm to recognize mountain peaks from the Digital Elevation Model (DEM) of the earth. Essentially a fancy way to find mountains using the 3D model of the earth.

The outcome of any ML project is highly dependent on the quality of data used as input therefore it is worth to have a good pipeline in place that can clean and filter data.

Furthermore we planned to use the data produced by the new GIS for “Peaklens” (created by the same team) which is an app that recognizes mountains using the camera of your smartphone. It works great and it is completely free, you should try it out for yourself!

The Project

The project consisted of three main tasks:

Create a data model to store geographical data coming from different data sources with annotations and corrections.
Create multiple data pipelines to import publicly accessible geographical data sources into our data model.
Create a web application to allow human annotators to review the data and validate it through crowdsourcing.

Annotation View - 3D globe

1. Data Model

I kicked off the project with an iterative process of:

Collecting and defining the requirements as use cases.
Designing a data model to account for the use case.
Simulating the usage of the data model and finding new questions for the stakeholders.

The process required an interactive process with the people involved in the ongoing ML research project.

I sketched the data model as Entity-Relationship and iteratively improved until it satisfied all the requirements the system needed to support. You can see the final result here.

We had informal meetings to decide which database to use. The alternatives were either a traditional relational database (PostgreSQL) or a NoSQL document database (MongoDB).

In the end we decided to go with PostgreSQL because:

no need to scale (storage expectations <100GB, user base <100 people)
it supports geometrical types out of the box which are convenient for geographical data (points, polylines, polygons)
strong data consistency (the schema enforces types and relations)
we expected the need for multiple JOINs between different data sources and user annotations

Annotation View - 2D map

2. Import from Geographical Data Sources

Once the data model was defined, the next step was to import as much data as possible. For this reason I investigated publicly accessible geographical data sources, studied their format and APIs, wrote and tested scripts to download their data.

The data model is able to store reference to the unique original identifier of each downloaded element so we could have potentially contributed back to each source following the Open Source spirit of giving back to the community.

By the time my contract ended I had written scripts to import mountain peaks from:

View to toggle permissions per user

3. Crowdsourcing Web App

The last task was the development of a Web App to annotate and review the data. Here you can peek the requirements specification document.

For the tech stack we agreed on using Node Js for back-end and Vue Js for frontend, the main reason being keeping all the code base in Javascript. The web app was hosted on the private cloud of Politecnico di Milano which is based on Open Stack.

For visualizations I have used Cesium Js which provides interactive 3D globes and Leaflet Js for rendering 2D maps. For the UI I have used BootstrapVue which provides ready to use Vue components based on Bootstrap.

For the development process I have followed an Agile approach with 2 weeks sprints after which I demoed the new features and collected feedback for the next iteration.

I really enjoyed working on this project because it allowed me to get full-stack development experience and also some planning skills as I was fully entrusted in defining my tasks and delivering them. During the project I also mentored a student who then helped me in some development tasks.

View to create new crowdsourcing campaigns

Live Demo

Before writing this article I decided to host a live instance of the Web App on Amazon AWS so that you can could see it live. I contacted Piero who kindly gave me permission to share the source code of the Web App for my portfolio, since it is not in use anymore.

“Resuscitating” a project that is “only” 3/4 years old proved to be more difficult than I thought.

I expected it to be fully working as I left it. However simply running it locally, using the latest stable Node Js resulted in some weird connection issues with the database that kept making my unit tests fail. I traced it back to the NPM library I used to connect to the DB (pg-promise) and updating it fixed the problem promptly.

I have also fall victim of a strange bug where the Vue CLI Service did not exit after finishing building the production bundle. This didn’t cope well with my goal of having a CI pipeline because it makes the automated process hang and timeout. I fixed it again with more targeted package updates.

I have tried updating all the NPM packages at once but that resulted in many incompatibilities appearing between my code and the new packages. Since I did not want to spend time fixing code, I only did targeted updates for specific packages.

This proved me how unstable is the entire NPM/Node Js ecosystem. Today I rather use a self-contained web framework that provides everything out of the box such as ASP.NET, Django, Ruby on Rails, or Laravel, and stay away from the dependency hell that is the Javascript ecosystem with NPM.

For hosting the Web App I have used: Amazon RDS for the database (I created a PostgreSQL instance), Amazon Elastic Beanstalk for the back-end REST API, and Amazon S3 for the front-end (configured to serve a static website). Both the back-end and front-end are served on vanilla HTTP to avoid having to deal with SSL certificates.

Overall the setups works quite nicely, it fits well within the limit of the Amazon Free Tier. It also has continuous deployment thanks to Github Actions.

You can find the live demo at: http://mapmymountains.aradaelli.com

Source code: https://gitlab.com/wds-co/mapmymountains

Credits

I want to mention the whole team at Politecnico di Milano with whom I have worked from 2018 to 2019. In 1 year I have learned a lot and I still have a great relationship with them:

Piero Fraternali (Website-Linkedin-Google Scholar)
Chiara Pasini (Linkedin-Google Scholar)
Darian Frajberg (Linkedin-Google Scholar)
Carlo Bernaschina (Website-Linkedin-Google Scholar)
Nahime Torres (Linkedin-Google Scholar)
Sergio Herrera Gonzalez (Linkedin-Google Scholar)
Federico Milani (Website-Linkedin-Google Scholar)

Product Configurator

Fri, 01 Sep 2017 00:00:00 +0000

Intro

In parallel to my master studies I enrolled in an honor program called Alta Scuola Politecnica. The greatest thing about this program has been the graduation project which consisted in a collaboration with a real business.

I joined a team of brilliant colleagues, from different backgrounds in management engineering, mechanical engineering and physics. Together we acted as a consulting team for a business, understood their problem, performed a research about the state of the art and delivered a solution.

Screenshot of the website at the end of the configuration process

Background

The project I joined was sponsored by Inkmaker Srl.

Inkmaker is a global producer of ink dispensing machines. These machines mix ink to produce the exact color required for printing or painting products at industrial scale.

The ink dispensers are large and quite specialized machines. Clients may have strict constraints concerning the available plant area or uncommon requirements on the ink mixing capabilities. Therefore the development of each machine tends to be quite customized for each client.

However Inkmaker also have a portfolio of entry-level machines that address the demand of their smaller customer with just a few variations. The vision of Inkmaker was to enable their small customers to self design their equipment by providing a set of standard modules to be mixed and matched. Those modules could be combined producing different assembly combinations and layouts to satisfy most request with minimal involvement from Inkmaker, thus increasing the overall efficiency.

The solution identified by Inkmaker to realize this vision consisted in the development of a product configurator that would allow clients to design their own machines with a few clicks, resulting in reduced lead time and errors in order interpretations.

Screenshot of a machine assembly

My Contribution

My work was based on the results of my dear colleagues who took care of:

Analyzing the business processes of Inkmaker (sales, design, purchase).
Understanding the mechanical constraints of the product and define how to modularize it.
Developing a model for generating a machine starting from the client requirements using the available modules.

I applied my skills in two areas:

Performing a research of the state of the art of existing product configurators.
Implementing a prototype product configurator to showcase to Inkmaker.

State of the Art

My first task consisted in researching existing product configurators to identify the best ones suited for the requirements of Inkmaker.

I researched both:

Software available “off the shelf” that could be used to customize a generic product.
Established product configurators from major retailers to study their offering and capabilities.

I wrote a report that summarized the pros and cons of each product, their features, their user interface (UI) and their configuration process.

The outcome of the research lead to the discovery of a few products that were fit for Inkmaker use-cases. Here is a feature matrix comparing the products against our requirements:

Feature matrix comparing selected configurators

The best candidate “A” was Autodesk Configurator 360 (now discontinued).

Prototype

My second task consisted in learning the inner workings of Configurator 360, understanding how to integrate it in a customized experience for Inkmaker, and developing a prototype.

Configurator 360 was a cloud based software as a service (SaaS). CAD designs could be uploaded to it and then displayed through an interactive web widget.

I decided to build a web application that would first guide the customer through the configuration process, then calculate the best configuration for the ink dispenser, and finally provide the result to Configurator 360 which would then generate the engineering drawings, 3D visualizations, and bill of materials.

Screenshot of the website during the configuration process

The web application was built in C# ASP.NET with Javascript and JQuery for frontend. It was deployed on Amazon EC2 to provide a demo to show to Inkmaker. Customer details would be saved in a SQL database for reference and potentially integrate with the company ERP.

Architecture diagram

Credits

I want to credit the rest of the Modula team from which I learned a lot and together we delivered this fantastic project.

Alex Saja (Linkedin)
Alfredo Fantetti (Linkedin)
Arianna Rosa Brusin (Linkedin)
Giovanni Prencipe (Linkedin)
Sara Mottola (Linkedin)
Stefano Amato (Linkedin)

Santa Ski

Sun, 15 Nov 2015 00:00:00 +0000

This game was made in 48h, for a local game jam that I organised with friends when I still was in University (good times).

This game is a remake of a Microsoft classic: SkiFree. The protagonist is supposed to be Santa Claus, in case you were wondering.

I consider game jams great experiences for learning how to be pragmatical developers.

Given the short time available one needs to focus on implementing features that matter. You do not have time to waste on writing code that solve problems that are already solved by a mature media framework. Loading assets, audio and video access, are all problems that can be solved immediately and not something worth spending time on like I did for earlier game projects.

In 2 days, you barely have the time to build the core features of whatever idea you come up with.

For this project I decided to use C++ with SFML which provides access to graphics and input with very few lines of code.

I have drawn all the game art, and I am really happy how it has given the game a consistent look&feel, as opposed to downloading a bunch of random images from the internet and mixing them together.

Overall I am quite satisfied how productive this project has been compared to other ones where I decided to reinvent the wheel by building my own graphic framework from scratch. It is a testament to how building on top of well-established libraries saves time.

The source code is available on GitHub and it can be compiled on Linux.

We Are Made of Dreams

Sun, 12 Oct 2014 00:00:00 +0000

Intro

For me, Minecraft could easily be the greatest game of all time. The main reasons being that it is a “meta-game”, a game that allows you to build other games.

Minecraft

The first time I played it I was confused by the absence of a story to follow or objective to complete. But the more I played it, the more I started to be addicted because I was building my own world and creating my own rules! The utter simplicity and freedom that a world made of blocks could create really impressed me. Furthermore, people online created (and still are creating) staggering detailed worlds and custom games using official and custom servers.

After playing Minecraft for a long time, I could not resist to try to build my own 3D engine and replicate the technology behind it. I read many articles which explained the inner workings of the game and then get to code.

When I got a prototype working, I was puzzled on what to make out of it.

I didn’t want to embark in making a full video game, so I decided to build a self-contained experience, a demo; or in simpler terms, an “interactive video”.

Analysis

The code is written in D as at the time I decided it would become my go to language after a long search for a modern alternative to C++.

Reading through the code today, the most interesting things are: the world data structure and the renderer architecture.

The world is implemented as an octree. An octree can be visualized as a giant cube that encloses all the space available for your world. Each node of the octree divides the space inside the cube in 8 sub-cubes. Then again each cube is recursively split in 8 more sub-cubes, until you arrive to the leaf nodes. A leaf node is the smallest representable unit of the world and it is represented as a single block on the screen.

Octree representation, source: mshgrid.com

An octree is ideal for a “Minecraft like” world for two reasons.

First, it naturally fits a world made of cubes beacause of its direct mapping of leaf nodes to world blocks.

Second, it is very memory efficient when there are big empty spaces because the leaf nodes are not required to exist at all in memory. That is in contrast to a simple three-dimensional array which would still need to allocate the memeory but mark all the empty elements as “null”.

The renderer is built using OpenGL3 and therefore it uses the modern rendering pipeline using shaders. The render is also multithreaded by using multiple worker threads that build meshes in real time every time a world chunk is modified.

Another thing I love is the built in map editor that allowed me to create the world that you can see in the demo.

Downloads

The source code is available on GitHub and it can be readily compiled on Windows and Linux. Pre-compiled binaries are also available.

Swizzle

Sun, 27 Jan 2013 00:00:00 +0000

Intro

In 2013, all my friends were playing Ruzzle on their smartphones. It was the viral game of the year but I was the only one not playing it. I only had a dumbphone so I could not join in the fun.

I was fascinated by how addicting the game was, given its simplicity, and I started thinkering how I could replicate it.

Analysis

The game is written in D because at the time I was looking to replace C++ for making games. The more I used C++ the more I found myself spending time avoiding bad patterns and the growing complexity of the language with each major release.

For this project I did not want to use any off the shelf multimedia framework because I wanted to challenge myself and use as less dependencies as possible. So I ended up having to deal with a lot of interfaces in order to use C libraries such as OpenGL, FreeType, and libpng. It turned out to be a huge amount of work so I reduced the scope and only made it work for Windows and not Linux as initially planned. After spending much of my time making interfaces for the C libraries, the game logic to re-implement Ruzzle turned out to be minuscule in comparison.

The game logic is all contained in one class: WordTable. And the most interesting functions are two.

WordTable.reset(). This function prepares the game board by placing the letters. It starts by taking a random word and placing its letters close to each other in the board. Then this process is repeated until there is no more space left. This algorithm ensures that any game has valid words to be discovered. A simpler approach, where letters are placed randomly in the board could lead to corner cases where no words are available in a game.

Example of placing the words: HELLO, WORLD, MUSIC

WordTable.findPossibleWords(). This function finds all available words in the game board. It uses a recursive algorithm that tries to match each word in the word list by trying each neighbor letter at the current position.

Downloads

The source code is available on GitHub and it can be readily compiled on Windows using Visual D. Pre-compiled binaries are also available.

Jump And Run

Fri, 02 Sep 2011 00:00:00 +0000

Intro

“Jump And Run” is a game I developed towards the end of high school and the first project I am really proud of.

I was always attracted by the idea of building my own game but I had never tried it before. I had some basic experience with graphics from reading books and articles online but I only dabbled with some small graphical demos that I made during the long boring ours in our school lab.

One day I decided to build proper game that would be fun to play with levels and enemies to kill.

The end result suprised me, it turned out that I could turn my ideas into reality just by putting enough effort into it.

Here is a gameplay video of the first level:

Background

When I was a kid the first type of video games I played were 2D Side Scrollers.

I still remember the two games which influenced me the most: Super Mario Land and Captain Claw.

Super Mario was the first game I ever played. I played it on a Game Boy which my cousins lend to me because my parents were against buying game consoles (“You would waste all your time!”, so true in hindsight).

Super Mario Land gameplay

Later, when the first computer arrived in my home, my first PC game arrived shortly after. Captain Claw was so difficult for me and I never managed to beat the second level until a friend of mine (who had an internet connection! I still had not) discovered that cheat codes were a thing.

Captain Claw gameplay

“Jump And Run” is my tribute to those games and the endless hours I spent on them. I drew the art myself using Gimp, downloaded the music from the Free Music Archive and sounds from Freesound.

Analysis

Re-reading today the code today I see a lot of “code smells” like huge functions and bad names, however keep in mind that I just finished high school when I wrote it :).

The game runs on a custom 2D Game Engine which I wrote in C++.

I appreciate how neatly the generic parts of the engine are split in virtual classes that make sense. The framework of the engine with its initialization code is separate from the game logic. I remember being deeply influenced from the code of famous game engines of the time such as Unreal Tournament and Quake 3 Arena.

To draw on the screen I used OpenGL and for playing sounds I used OpenAL.

I have forgotten how much time I spent rewriting much functionality of the standard library such as string and memory functions, file access, and various data structures. I was motivated by the fact that many game development articles recommended to avoid the standard library as it is not optimized for speed in most cases.

Map Editor

I was very proud of the built-in map editor which allowed me to build maps with my basic drawing skills that would become alive with sound and enemies.

The game world is composed of small squares. Each square has a background and foreground texture taken from a sprite sheet. Furthermore each square has a “type” property that can make it interact with the player by being solid, lethal, or a coin for example.

Particle System

Another thing I remember fondly is the particle system. It makes killing enemies very satisfying by releasing a swarm of “blood” particles which collide with the game world.

Downloads

The source code is available on GitHub and it can be readily compiled on Windows and Linux. Pre-compiled binaries are also available.